Re: Jul2015 Release Candidate

12 Aug 2015

      Hi,

while I so far failed to reproduce the behavior reliably,
I just happen to have a Oct2014-generated database where upgrading to Jul2015
show the reported error, not only with the initial upgrade, but with every
start of Jul2015 on that database.

Below you find a debug trace of the event, in case this helps some experts
(Sjoerd?, Niels?) to understand / locate the problem ...

Thanks!

Stefan

========
(gdb) r
Starting program: /net/rome.ins.cwi.nl/export/scratch1/manegold/Monet/HG/Jul2015/prefix/_/bin/mserver5 --dbpath=/export/scratch1/manegold/dbfarm --dbextra=/export/scratch2/manegold/dbfarm
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
# MonetDB 5 server v11.21.0
# This is an unreleased version
# Serving database 'dbfarm', using 8 threads
# Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs and 128bit integers dynamically linked
# Found 15.590 GiB available main-memory.
# Copyright (c) 1993-July 2008 CWI.
# Copyright (c) August 2008-2015 MonetDB B.V., all rights reserved
# Visit http://www.monetdb.org/ for further information
[New Thread 0x7ffff047c700 (LWP 15521)]
[New Thread 0x7fffec658700 (LWP 15549)]
# Listening for connection requests on mapi:monetdb://127.0.0.1:50000/
# MonetDB/GIS module loaded
# Start processing logs sql/sql_logs version 52200

Breakpoint 1, GDKerror (format=0x7ffff778a8b8 "BATkey: void column cannot be unique.\n") at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_utils.c:1498
1498		size_t len = strlen(GDKERROR);
Missing separate debuginfos, use: debuginfo-install R-core-3.2.0-2.fc20.x86_64 atlas-3.8.4-14.fc20.x86_64 blas-3.4.2-8.fc20.x86_64 bzip2-libs-1.0.6-9.fc20.x86_64 cfitsio-3.350-2.fc20.x86_64 cyrus-sasl-lib-2.1.26-14.fc20.x86_64 geos-3.4.2-1.fc20.x86_64 glibc-2.18-19.fc20.x86_64 gsl-1.15-9.fc20.x86_64 hdf5-1.8.11-6.fc20.x86_64 keyutils-libs-1.5.9-1.fc20.x86_64 krb5-libs-1.11.5-19.fc20.x86_64 libcom_err-1.42.12-4.fc20.x86_64 libcurl-7.32.0-20.fc20.x86_64 libgcc-4.8.3-7.fc20.x86_64 libgfortran-4.8.3-7.fc20.x86_64 libgomp-4.8.3-7.fc20.x86_64 libicu-50.1.2-12.fc20.x86_64 libidn-1.28-3.fc20.x86_64 libjpeg-turbo-1.3.1-3.fc20.x86_64 libquadmath-4.8.3-7.fc20.x86_64 libselinux-2.2.1-6.fc20.x86_64 libssh2-1.5.0-1.fc20.x86_64 libstdc++-4.8.3-7.fc20.x86_64 libuuid-2.24.2-2.fc20.x86_64 libxml2-2.9.1-4.fc20.x86_64 ncurses-libs-5.9-12.20130511.fc20.x86_64 netcdf-4.3.2-4.fc20.x86_64 nspr-4.10.8-1.fc20.x86_64 nss-3.18.0-1.fc20.x86_64 nss-softokn-freebl-3.18.0-1.fc20.x86_64 nss-util-3.18.0-1.fc20.x86_64 openldap-2.4.39-4.fc20.x86_64 openssl-libs-1.0.1e-42.fc20.x86_64 pcre-8.33-11.fc20.x86_64 readline-6.2-10.fc20.x86_64 xz-libs-5.1.2-12alpha.fc20.x86_64 zlib-1.2.8-3.fc20.x86_64
(gdb) bt
#0  GDKerror (format=0x7ffff778a8b8 "BATkey: void column cannot be unique.\n") at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_utils.c:1498
#1  0x00007ffff7540db5 in BATkey (b=0x1ecda50, flag=1) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_bat.c:2162
#2  0x00007ffff73d97f4 in ALIGNsetH (b1=0x1ecda50, b2=0x60b810) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_align.c:117
#3  0x00007ffff752f841 in BATcopy (b=0x60b810, ht=0, tt=5, writable=1, role=1) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_bat.c:937
#4  0x00007ffff75590ae in logger_new (debug=0, fn=0x7fffef1faaaa "sql", logdir=0x7fffef1d55b0 "sql_logs", version=52200, prefuncp=0x7fffef1a6a82 , postfuncp=0x7fffef1a6b9f ) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_logger.c:1546
#5  0x00007ffff75598da in logger_create (debug=0, fn=0x7fffef1faaaa "sql", logdir=0x7fffef1d55b0 "sql_logs", version=52200, prefuncp=0x7fffef1a6a82 , postfuncp=0x7fffef1a6b9f ) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_logger.c:1720
#6  0x00007fffef1a8028 in bl_create (debug=0, logdir=0x7fffef1d55b0 "sql_logs", cat_version=52200) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/storage/bat/bat_logger.c:156
#7  0x00007fffef187ddc in store_init (debug=0, store=store_bat, readonly=0, singleuser=0, logdir=0x7fffef1d55b0 "sql_logs", stk=0) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/storage/store.c:1341
#8  0x00007fffef0f925d in mvc_init (debug=0, store=store_bat, ro=0, su=0, stk=0) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/server/sql_mvc.c:36
#9  0x00007fffef041f91 in SQLinit () at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/backends/monet5/sql_scenario.c:251
#10 0x00007fffef041b39 in SQLprelude (ret=0x9bd8f0) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/backends/monet5/sql_scenario.c:178
#11 0x00007ffff7b12834 in malCommandCall (stk=0x9bd690, pci=0x1dc39d0) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_interpreter.c:77
#12 0x00007ffff7b15871 in runMALsequence (cntxt=0x7ffff047d020, mb=0x68d110, startpc=1, stoppc=0, stk=0x9bd690, env=0x0, pcicaller=0x0) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_interpreter.c:655
#13 0x00007ffff7b142fb in runMAL (cntxt=0x7ffff047d020, mb=0x68d110, mbcaller=0x0, env=0x0) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_interpreter.c:365
#14 0x00007ffff7b44c1e in MALengine (c=0x7ffff047d020) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_session.c:637
#15 0x00007ffff7b4272e in malBootstrap () at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_session.c:57
#16 0x00007ffff7aee041 in mal_init () at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal.c:95
#17 0x0000000000403afc in main (argc=3, av=0x7fffffffda68) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/tools/mserver/mserver5.c:624
(gdb) li
1493	
1494	void
1495	GDKerror(const char *format, ...)
1496	{
1497		char message[GDKERRLEN];
1498		size_t len = strlen(GDKERROR);
1499		va_list ap;
1500	
1501		if (!strncmp(format, GDKERROR, len)) {
1502			len = 0;
(gdb) up
#1  0x00007ffff7540db5 in BATkey (b=0x1ecda50, flag=1) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_bat.c:2162
2162				GDKerror("BATkey: void column cannot be unique.\n");
(gdb) li
2157			if (b->hseqbase != oid_nil && flag == FALSE) {
2158				GDKerror("BATkey: dense column must be unique.\n");
2159				return GDK_FAIL;
2160			}
2161			if (b->hseqbase == oid_nil && flag == TRUE && b->batCount > 1) {
2162				GDKerror("BATkey: void column cannot be unique.\n");
2163				return GDK_FAIL;
2164			}
2165		}
2166		if (flag)
(gdb) p flag
$1 = 1
(gdb) p b
$2 = (BAT *) 0x1ecda50
(gdb) p *b
$3 = {batCacheid = 909, H = 0x1ecda90, T = 0x1ecdb30, S = 0x1ecdbd0}
(gdb) p b->H
$4 = (COLrec *) 0x1ecda90
(gdb) p BATprint(b)
#--------------------------#
# h	t  # name
# void	int  # type
#--------------------------#
[ nil,	1  ]
[ nil,	0  ]
$5 = GDK_SUCCEED
(gdb) p *b->H
$6 = {id = 0x7ffff77894e1 "h", width = 0, type = 0 '\000', shift = 0 '\000', varsized = 1, key = 0, dense = 0, nonil = 1, nil = 0, sorted = 1, revsorted = 1, align = 1028385, nokey = {0, 0}, nosorted = 0, norevsorted = 0, nodense = 0, seq = 9223372036854775808, heap = {free = 0, size = 0, base = 0x0, filename = 0x0, copied = 0, hashash = 0, forcemap = 0, storage = STORE_MEM, newstorage = STORE_MEM, dirty = 0 '\000', farmid = 1 '\001', parentid = 0}, vheap = 0x0, hash = 0x0, 
  imprints = 0x0, props = 0x0}
(gdb) p *b->T
$7 = {id = 0x7ffff77894e3 "t", width = 4, type = 5 '\005', shift = 2 '\002', varsized = 0, key = 0, dense = 0, nonil = 1, nil = 0, sorted = 1, revsorted = 1, align = 1028386, nokey = {0, 0}, nosorted = 0, norevsorted = 0, nodense = 0, seq = 0, heap = {free = 8, size = 1024, base = 0x1ecdcf0 "\001", filename = 0x1ecdcc0 "16/1615.tail", copied = 0, hashash = 0, forcemap = 0, storage = STORE_MEM, newstorage = STORE_MEM, dirty = 0 '\000', farmid = 1 '\001', parentid = 0}, 
  vheap = 0x0, hash = 0x0, imprints = 0x0, props = 0x0}
(gdb) p *b->S
$8 = {tid = 140737333930113, stamp = 69, copiedtodisk = 0, dirty = 1, dirtyflushed = 0, descdirty = 1, restricted = 0, persistence = 1, role = 1, unused = 0, sharecnt = 0, map_head = 0 '\000', map_tail = 0 '\000', map_hheap = 0 '\000', map_theap = 0 '\000', deleted = 0, first = 0, inserted = 0, count = 2, capacity = 256}
(gdb) 
========

----- On Aug 11, 2015, at 9:32 AM, Stefan Manegold Stefan.Manegold@cwi.nl wrote:
...
Well, in fact the error says that we internally try to mark a void column
unique,
which is not allowed and hence not done.
Thus, there seems to be a bug that we incorrectly try to mark a void column
unique,
but since that fails, this is likely more an "annoying" than a crucial bug ...
Stefan
----- On Aug 11, 2015, at 9:29 AM, Stefan Manegold Stefan.Manegold@cwi.nl wrote:
...
...
hi,
does the notification itself indicate anything important?
Internally incorrectly set (internal) BAT property.
That might be crucial as we (internally) use properties to "cut corners" (aka.
"optimize").
...
if people do not flush the logs, will any data be lost?
Should not --- the server flushes logs itself.
I just suggested for the purpose of reproducing this problem,
to trigger the flush instead of waiting for the server to do it "eventually"
and transparently in the background ...
Stefan
...
thanks!
On Tue, Aug 11, 2015 at 2:42 AM, Stefan Manegold < Stefan.Manegold@cwi.nl >
wrote:
Hi,
I've just seen similar behavior,
but failed so far to reliably reproduce it with a minimal test.
It seems requires at least
creating a new DB with Oct2014,
creating a table, inserting a least 2 tuples,
stopping & restarting Oct2014 server to flush WAL logs,
stopping Oct2014 server
starting Jul2015 server to upgrade.
Best,
Stefan
----- On Aug 10, 2015, at 7:55 PM, Anthony Damico ajdamico@gmail.com wrote:
...
hi, i've just upgraded on my windows 8 laptop. every time i open the demo
server, i get this GDK error below. it doesn't appear to have any effect when i
run some simple queries. the error below does not appear when i create a fresh
dbfarm elsewhere, it only happened on a database that's been upgraded.
any idea what's going on? thanks!
# MonetDB 5 server v11.21.1 "Jul2015"
# Serving database 'demo', using 8 threads
# Compiled for x86_64-pc-winnt/64bit with 64bit OIDs dynamically linked
# Found 15.914 GiB available main-memory.
# Copyright (c) 1993-July 2008 CWI.
# Copyright (c) August 2008-2015 MonetDB B.V., all rights reserved
# Visit http://www.monetdb.org/ for further information
# Listening for connection requests on mapi:monetdb:// 127.0.0.1:50000/
# Start processing logs sql/sql_logs version 52200
# Start reading the write-ahead log 'sql_logs\sql\log.14'
# Finished reading the write-ahead log 'sql_logs\sql\log.14'
# Finished processing logs sql/sql_logs
# MonetDB/SQL module loaded
!GDKerror:!ERROR: BATkey: void column cannot be unique.
!ERROR: BATkey: void column cannot be unique.
...
On Mon, Aug 10, 2015 at 9:25 AM, Sjoerd Mullender < sjoerd@acm.org > wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
I have created a release candidate for the Jul2015 release. The
candidate is available at http://dev.monetdb.org/downloads/testing/ .
If all goes well, this candidate will morph into a release sometime
soon(ish). In the mean time, you can use this release candidate and
test to see whether it works. And if it doesn't work, I would really
like to hear about it.
- --
Sjoerd Mullender
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQEcBAEBCAAGBQJVyKZgAAoJEISMxT6LrWYgE28H/2rudFXhcIfNgnX4yYJlpRT5
eI2Dzp9kShEmRibvNkWvCEF1j/qyWgQhx/jlHvucuEF/0k+U+7NYuK0vhzA0llyJ
hspMkQ73UDXeVO0/SivtNYb40UpMyUQoXC7nSaSneglVDe4JcX4S3Y5GvHPByiqz
3UNMGDARfWlqPcbTtW57XfLejK/sd8nNZ3mGQuerVqEvMcKZaZIBSXjg/16WQVsN
DMbPacPpu3yFeWQQtP2er6SL3UdfwgOP5+VNifKyjW9i6ZtB2lHWLEBWQY8G/5Tb
5lwgHvD99SJeLRMzcW5TFWhwv0ROx8chKP3btxMofZxjHhqseswGRLNDQa4XCU0=
=ktpg
-----END PGP SIGNATURE-----
_______________________________________________
users-list mailing list
users-list@monetdb.org
https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________
users-list mailing list
users-list@monetdb.org
https://www.monetdb.org/mailman/listinfo/users-list
--
| Stefan.Manegold@CWI.nl | DB Architectures (DA) |
| www.CWI.nl/~manegold/ | Science Park 123 (L321) |
| +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |
_______________________________________________
users-list mailing list
users-list@monetdb.org
https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________
users-list mailing list
users-list@monetdb.org
https://www.monetdb.org/mailman/listinfo/users-list
--
| Stefan.Manegold@CWI.nl | DB Architectures   (DA) |
| www.CWI.nl/~manegold/  | Science Park 123 (L321) |
| +31 (0)20 592-4212     | 1098 XG Amsterdam  (NL) |
--
| Stefan.Manegold@CWI.nl | DB Architectures   (DA) |
| www.CWI.nl/~manegold/  | Science Park 123 (L321) |
| +31 (0)20 592-4212     | 1098 XG Amsterdam  (NL) |
_______________________________________________
users-list mailing list
users-list@monetdb.org
https://www.monetdb.org/mailman/listinfo/users-list
-- 
| Stefan.Manegold@CWI.nl | DB Architectures   (DA) |
| www.CWI.nl/~manegold/  | Science Park 123 (L321) |
| +31 (0)20 592-4212     | 1098 XG Amsterdam  (NL) |