Hi,
Sjoerd's patch
http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=92af99c05811
appears to work for me.
Thanks!
Stefan
----- On Aug 12, 2015, at 10:20 AM, Stefan Manegold Stefan.Manegold@cwi.nl wrote:
> Hi,
>
> while I so far failed to reproduce the behavior reliably,
> I just happen to have a Oct2014-generated database where upgrading to Jul2015
> show the reported error, not only with the initial upgrade, but with every
> start of Jul2015 on that database.
>
> Below you find a debug trace of the event, in case this helps some experts
> (Sjoerd?, Niels?) to understand / locate the problem ...
>
> Thanks!
>
> Stefan
>
> ========
> (gdb) r
> Starting program:
> /export/scratch1/manegold/Monet/HG/Jul2015/prefix/_/bin/mserver5
> nss-softokn-freebl-3.18.0-1.fc20.x86_64 nss-util-3.18.0-1.fc!> --dbpath=/export/scratch1/manegold/dbfarm
> --dbextra=/export/scratch2/manegold/dbfarm
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib64/libthread_db.so.1".
> # MonetDB 5 server v11.21.0
> # This is an unreleased version
> # Serving database 'dbfarm', using 8 threads
> # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs and 128bit
> integers dynamically linked
> # Found 15.590 GiB available main-memory.
> # Copyright (c) 1993-July 2008 CWI.
> # Copyright (c) August 2008-2015 MonetDB B.V., all rights reserved
> # Visit http://www.monetdb.org/ for further information
> [New Thread 0x7ffff047c700 (LWP 15521)]
> [New Thread 0x7fffec658700 (LWP 15549)]
> # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/
> # MonetDB/GIS module loaded
> # Start processing logs sql/sql_logs version 52200
>
> Breakpoint 1, GDKerror (format=0x7ffff778a8b8 "BATkey: void column cannot be
> unique.\n") at
> /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_utils.c:1498
> 1498 size_t len = strlen(GDKERROR);
> Missing separate debuginfos, use: debuginfo-install R-core-3.2.0-2.fc20.x86_64
> atlas-3.8.4-14.fc20.x86_64 blas-3.4.2-8.fc20.x86_64
> bzip2-libs-1.0.6-9.fc20.x86_64 cfitsio-3.350-2.fc20.x86_64
> cyrus-sasl-lib-2.1.26-14.fc20.x86_64 geos-3.4.2-1.fc20.x86_64
> glibc-2.18-19.fc20.x86_64 gsl-1.15-9.fc20.x86_64 hdf5-1.8.11-6.fc20.x86_64
> keyutils-libs-1.5.9-1.fc20.x86_64 krb5-libs-1.11.5-19.fc20.x86_64
> libcom_err-1.42.12-4.fc20.x86_64 libcurl-7.32.0-20.fc20.x86_64
> libgcc-4.8.3-7.fc20.x86_64 libgfortran-4.8.3-7.fc20.x86_64
> libgomp-4.8.3-7.fc20.x86_64 libicu-50.1.2-12.fc20.x86_64
> libidn-1.28-3.fc20.x86_64 libjpeg-turbo-1.3.1-3.fc20.x86_64
> libquadmath-4.8.3-7.fc20.x86_64 libselinux-2.2.1-6.fc20.x86_64
> libssh2-1.5.0-1.fc20.x86_64 libstdc++-4.8.3-7.fc20.x86_64
> libuuid-2.24.2-2.fc20.x86_64 libxml2-2.9.1-4.fc20.x86_64
> ncurses-libs-5.9-12.20130511.fc20.x86_64 netcdf-4.3.2-4.fc20.x86_64
> nspr-4.10.8-1.fc20.x86_64 nss-3.18.0-1.fc20.x86_64
> 20.x86_64
> openldap-2.4.39-4.fc20.x86_64 openssl-libs-1.0.1e-42.fc20.x86_64
> pcre-8.33-11.fc20.x86_64 readline-6.2-10.fc20.x86_64
> xz-libs-5.1.2-12alpha.fc20.x86_64 zlib-1.2.8-3.fc20.x86_64
> (gdb) bt
> #0 GDKerror (format=0x7ffff778a8b8 "BATkey: void column cannot be unique.\n")
> at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_utils.c:1498
> #1 0x00007ffff7540db5 in BATkey (b=0x1ecda50, flag=1) at
> /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_bat.c:2162
> #2 0x00007ffff73d97f4 in ALIGNsetH (b1=0x1ecda50, b2=0x60b810) at
> /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_align.c:117
> #3 0x00007ffff752f841 in BATcopy (b=0x60b810, ht=0, tt=5, writable=1, role=1)
> at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_bat.c:937
> #4 0x00007ffff75590ae in logger_new (debug=0, fn=0x7fffef1faaaa "sql",
> logdir=0x7fffef1d55b0 "sql_logs", version=52200, prefuncp=0x7fffef1a6a82
> <bl_preversion>, postfuncp=0x7fffef1a6b9f <bl_postversion>) at
> /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_logger.c:1546
> #5 0x00007ffff75598da in logger_create (debug=0, fn=0x7fffef1faaaa "sql",
> logdir=0x7fffef1d55b0 "sql_logs", version=52200, prefuncp=0x7fffef1a6a82
> <bl_preversion>, postfuncp=0x7fffef1a6b9f <bl_postversion>) at
> /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_logger.c:1720
> #6 0x00007fffef1a8028 in bl_create (debug=0, logdir=0x7fffef1d55b0 "sql_logs",
> cat_version=52200) at
> /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/storage/bat/bat_logger.c:156
> #7 0x00007fffef187ddc in store_init (debug=0, store=store_bat, readonly=0,
> singleuser=0, logdir=0x7fffef1d55b0 "sql_logs", stk=0) at
> /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/storage/store.c:1341
> #8 0x00007fffef0f925d in mvc_init (debug=0, store=store_bat, ro=0, su=0, stk=0)
> at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/server/sql_mvc.c:36
> #9 0x00007fffef041f91 in SQLinit () at
> /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/backends/monet5/sql_scenario.c:251
> #10 0x00007fffef041b39 in SQLprelude (ret=0x9bd8f0) at
> /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/backends/monet5/sql_scenario.c:178
> #11 0x00007ffff7b12834 in malCommandCall (stk=0x9bd690, pci=0x1dc39d0) at
> /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_interpreter.c:77
> #12 0x00007ffff7b15871 in runMALsequence (cntxt=0x7ffff047d020, mb=0x68d110,
> startpc=1, stoppc=0, stk=0x9bd690, env=0x0, pcicaller=0x0) at
> /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_interpreter.c:655
> #13 0x00007ffff7b142fb in runMAL (cntxt=0x7ffff047d020, mb=0x68d110,
> mbcaller=0x0, env=0x0) at
> /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_interpreter.c:365
> #14 0x00007ffff7b44c1e in MALengine (c=0x7ffff047d020) at
> /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_session.c:637
> #15 0x00007ffff7b4272e in malBootstrap () at
> /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_session.c:57
> #16 0x00007ffff7aee041 in mal_init () at
> /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal.c:95
> #17 0x0000000000403afc in main (argc=3, av=0x7fffffffda68) at
> /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/tools/mserver/mserver5.c:624
> (gdb) li
> 1493
> 1494 void
> 1495 GDKerror(const char *format, ...)
> 1496 {
> 1497 char message[GDKERRLEN];
> 1498 size_t len = strlen(GDKERROR);
> 1499 va_list ap;
> 1500
> 1501 if (!strncmp(format, GDKERROR, len)) {
> 1502 len = 0;
> (gdb) up
> #1 0x00007ffff7540db5 in BATkey (b=0x1ecda50, flag=1) at
> /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_bat.c:2162
> 2162 GDKerror("BATkey: void column cannot be unique.\n");
> (gdb) li
> 2157 if (b->hseqbase != oid_nil && flag == FALSE) {
> 2158 GDKerror("BATkey: dense column must be unique.\n");
> 2159 return GDK_FAIL;
> 2160 }
> 2161 if (b->hseqbase == oid_nil && flag == TRUE && b->batCount > 1) {
> 2162 GDKerror("BATkey: void column cannot be unique.\n");
> 2163 return GDK_FAIL;
> 2164 }
> 2165 }
> 2166 if (flag)
> (gdb) p flag
> $1 = 1
> (gdb) p b
> $2 = (BAT *) 0x1ecda50
> (gdb) p *b
> $3 = {batCacheid = 909, H = 0x1ecda90, T = 0x1ecdb30, S = 0x1ecdbd0}
> (gdb) p b->H
> $4 = (COLrec *) 0x1ecda90
> (gdb) p BATprint(b)
> #--------------------------#
> # h t # name
> # void int # type
> #--------------------------#
> [ nil, 1 ]
> [ nil, 0 ]
> $5 = GDK_SUCCEED
> (gdb) p *b->H
> $6 = {id = 0x7ffff77894e1 "h", width = 0, type = 0 '\000', shift = 0 '\000',
> varsized = 1, key = 0, dense = 0, nonil = 1, nil = 0, sorted = 1, revsorted =
> 1, align = 1028385, nokey = {0, 0}, nosorted = 0, norevsorted = 0, nodense = 0,
> seq = 9223372036854775808, heap = {free = 0, size = 0, base = 0x0, filename =
> 0x0, copied = 0, hashash = 0, forcemap = 0, storage = STORE_MEM, newstorage =
> STORE_MEM, dirty = 0 '\000', farmid = 1 '\001', parentid = 0}, vheap = 0x0,
> hash = 0x0,
> imprints = 0x0, props = 0x0}
> (gdb) p *b->T
> $7 = {id = 0x7ffff77894e3 "t", width = 4, type = 5 '\005', shift = 2 '\002',
> varsized = 0, key = 0, dense = 0, nonil = 1, nil = 0, sorted = 1, revsorted =
> 1, align = 1028386, nokey = {0, 0}, nosorted = 0, norevsorted = 0, nodense = 0,
> seq = 0, heap = {free = 8, size = 1024, base = 0x1ecdcf0 "\001", filename =
> 0x1ecdcc0 "16/1615.tail", copied = 0, hashash = 0, forcemap = 0, storage =
> STORE_MEM, newstorage = STORE_MEM, dirty = 0 '\000', farmid = 1 '\001',
> parentid = 0},
> vheap = 0x0, hash = 0x0, imprints = 0x0, props = 0x0}
> (gdb) p *b->S
> $8 = {tid = 140737333930113, stamp = 69, copiedtodisk = 0, dirty = 1,
> dirtyflushed = 0, descdirty = 1, restricted = 0, persistence = 1, role = 1,
> unused = 0, sharecnt = 0, map_head = 0 '\000', map_tail = 0 '\000', map_hheap =
> 0 '\000', map_theap = 0 '\000', deleted = 0, first = 0, inserted = 0, count =
> 2, capacity = 256}
> (gdb)
> ========
>
> ----- On Aug 11, 2015, at 9:32 AM, Stefan Manegold Stefan.Manegold@cwi.nl wrote:
>
>> Well, in fact the error says that we internally try to mark a void column
>> unique,
>> which is not allowed and hence not done.
>>
>> Thus, there seems to be a bug that we incorrectly try to mark a void column
>> unique,
>> but since that fails, this is likely more an "annoying" than a crucial bug ...
>>
>> Stefan
>>
>> ----- On Aug 11, 2015, at 9:29 AM, Stefan Manegold Stefan.Manegold@cwi.nl wrote:
>>
>>>> hi,
>>>> does the notification itself indicate anything important?
>>>
>>> Internally incorrectly set (internal) BAT property.
>>> That might be crucial as we (internally) use properties to "cut corners" (aka.
>>> "optimize").
>>>
>>>> if people do not flush the logs, will any data be lost?
>>>
>>> Should not --- the server flushes logs itself.
>>>
>>> I just suggested for the purpose of reproducing this problem,
>>> to trigger the flush instead of waiting for the server to do it "eventually"
>>> and transparently in the background ...
>>>
>>> Stefan
>>>
>>>> thanks!
>>>>
>>>> On Tue, Aug 11, 2015 at 2:42 AM, Stefan Manegold < Stefan.Manegold@cwi.nl >
>>>> wrote:
>>>>
>>>>
>>>> Hi,
>>>>
>>>> I've just seen similar behavior,
>>>> but failed so far to reliably reproduce it with a minimal test.
>>>>
>>>> It seems requires at least
>>>> creating a new DB with Oct2014,
>>>> creating a table, inserting a least 2 tuples,
>>>> stopping & restarting Oct2014 server to flush WAL logs,
>>>> stopping Oct2014 server
>>>> starting Jul2015 server to upgrade.
>>>>
>>>> Best,
>>>> Stefan
>>>>
>>>> ----- On Aug 10, 2015, at 7:55 PM, Anthony Damico ajdamico@gmail.com wrote:
>>>>
>>>>> hi, i've just upgraded on my windows 8 laptop. every time i open the demo
>>>>> server, i get this GDK error below. it doesn't appear to have any effect when i
>>>>> run some simple queries. the error below does not appear when i create a fresh
>>>>> dbfarm elsewhere, it only happened on a database that's been upgraded.
>>>>>
>>>>> any idea what's going on? thanks!
>>>>>
>>>>>
>>>>> # MonetDB 5 server v11.21.1 "Jul2015"
>>>>> # Serving database 'demo', using 8 threads
>>>>> # Compiled for x86_64-pc-winnt/64bit with 64bit OIDs dynamically linked
>>>>> # Found 15.914 GiB available main-memory.
>>>>> # Copyright (c) 1993-July 2008 CWI.
>>>>> # Copyright (c) August 2008-2015 MonetDB B.V., all rights reserved
>>>>> # Visit http://www.monetdb.org/ for further information
>>>>> # Listening for connection requests on mapi:monetdb:// 127.0.0.1:50000/
>>>>> # Start processing logs sql/sql_logs version 52200
>>>>> # Start reading the write-ahead log 'sql_logs\sql\log.14'
>>>>> # Finished reading the write-ahead log 'sql_logs\sql\log.14'
>>>>> # Finished processing logs sql/sql_logs
>>>>> # MonetDB/SQL module loaded
>>>>> !GDKerror:!ERROR: BATkey: void column cannot be unique.
>>>>> !ERROR: BATkey: void column cannot be unique.
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Aug 10, 2015 at 9:25 AM, Sjoerd Mullender < sjoerd@acm.org > wrote:
>>>>>
>>>>>
>>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>>> Hash: SHA256
>>>>>
>>>>> I have created a release candidate for the Jul2015 release. The
>>>>> candidate is available at http://dev.monetdb.org/downloads/testing/ .
>>>>> If all goes well, this candidate will morph into a release sometime
>>>>> soon(ish). In the mean time, you can use this release candidate and
>>>>> test to see whether it works. And if it doesn't work, I would really
>>>>> like to hear about it.
>>>>>
>>>>> - --
>>>>> Sjoerd Mullender
>>>>> -----BEGIN PGP SIGNATURE-----
>>>>> Version: GnuPG v2
>>>>>
>>>>> iQEcBAEBCAAGBQJVyKZgAAoJEISMxT6LrWYgE28H/2rudFXhcIfNgnX4yYJlpRT5
>>>>> eI2Dzp9kShEmRibvNkWvCEF1j/qyWgQhx/jlHvucuEF/0k+U+7NYuK0vhzA0llyJ
>>>>> hspMkQ73UDXeVO0/SivtNYb40UpMyUQoXC7nSaSneglVDe4JcX4S3Y5GvHPByiqz
>>>>> 3UNMGDARfWlqPcbTtW57XfLejK/sd8nNZ3mGQuerVqEvMcKZaZIBSXjg/16WQVsN
>>>>> DMbPacPpu3yFeWQQtP2er6SL3UdfwgOP5+VNifKyjW9i6ZtB2lHWLEBWQY8G/5Tb
>>>>> 5lwgHvD99SJeLRMzcW5TFWhwv0ROx8chKP3btxMofZxjHhqseswGRLNDQa4XCU0=
>>>>> =ktpg
>>>>> -----END PGP SIGNATURE-----
>>>>> _______________________________________________
>>>>> users-list mailing list
>>>>> users-list@monetdb.org
>>>>> https://www.monetdb.org/mailman/listinfo/users-list
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> users-list mailing list
>>>>> users-list@monetdb.org
>>>>> https://www.monetdb.org/mailman/listinfo/users-list
>>>>
>>>> --
>>>>| Stefan.Manegold@CWI.nl | DB Architectures (DA) |
>>>>| www.CWI.nl/~manegold/ | Science Park 123 (L321) |
>>>>| +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |
>>>> _______________________________________________
>>>> users-list mailing list
>>>> users-list@monetdb.org
>>>> https://www.monetdb.org/mailman/listinfo/users-list
>>>>
>>>>
>>>> _______________________________________________
>>>> users-list mailing list
>>>> users-list@monetdb.org
>>>> https://www.monetdb.org/mailman/listinfo/users-list
>>>
>>> --
>>>| Stefan.Manegold@CWI.nl | DB Architectures (DA) |
>>>| www.CWI.nl/~manegold/ | Science Park 123 (L321) |
>>> | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |
>>
>> --
>>| Stefan.Manegold@CWI.nl | DB Architectures (DA) |
>>| www.CWI.nl/~manegold/ | Science Park 123 (L321) |
>>| +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |
>> _______________________________________________
>> users-list mailing list
>> users-list@monetdb.org
>> https://www.monetdb.org/mailman/listinfo/users-list
>
> --
>| Stefan.Manegold@CWI.nl | DB Architectures (DA) |
>| www.CWI.nl/~manegold/ | Science Park 123 (L321) |
>| +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |
> _______________________________________________
> users-list mailing list
> users-list@monetdb.org
> https://www.monetdb.org/mailman/listinfo/users-list
--
| Stefan.Manegold@CWI.nl | DB Architectures (DA) |
| www.CWI.nl/~manegold/ | Science Park 123 (L321) |
| +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |
_______________________________________________
users-list mailing list
users-list@monetdb.org
https://www.monetdb.org/mailman/listinfo/users-list