Jul2015 Release Candidate

Sjoerd Mullender

10 Aug 2015 10 Aug '15

1:25 p.m.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 I have created a release candidate for the Jul2015 release. The candidate is available at http://dev.monetdb.org/downloads/testing/. If all goes well, this candidate will morph into a release sometime soon(ish). In the mean time, you can use this release candidate and test to see whether it works. And if it doesn't work, I would really like to hear about it. - -- Sjoerd Mullender -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBCAAGBQJVyKZgAAoJEISMxT6LrWYgE28H/2rudFXhcIfNgnX4yYJlpRT5 eI2Dzp9kShEmRibvNkWvCEF1j/qyWgQhx/jlHvucuEF/0k+U+7NYuK0vhzA0llyJ hspMkQ73UDXeVO0/SivtNYb40UpMyUQoXC7nSaSneglVDe4JcX4S3Y5GvHPByiqz 3UNMGDARfWlqPcbTtW57XfLejK/sd8nNZ3mGQuerVqEvMcKZaZIBSXjg/16WQVsN DMbPacPpu3yFeWQQtP2er6SL3UdfwgOP5+VNifKyjW9i6ZtB2lHWLEBWQY8G/5Tb 5lwgHvD99SJeLRMzcW5TFWhwv0ROx8chKP3btxMofZxjHhqseswGRLNDQa4XCU0= =ktpg -----END PGP SIGNATURE-----

Show replies by date

Anthony Damico

10 Aug 10 Aug

5:55 p.m.

hi, i've just upgraded on my windows 8 laptop. every time i open the demo server, i get this GDK error below. it doesn't appear to have any effect when i run some simple queries. the error below does not appear when i create a fresh dbfarm elsewhere, it only happened on a database that's been upgraded. any idea what's going on? thanks! # MonetDB 5 server v11.21.1 "Jul2015" # Serving database 'demo', using 8 threads # Compiled for x86_64-pc-winnt/64bit with 64bit OIDs dynamically linked # Found 15.914 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2015 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # Start processing logs sql/sql_logs version 52200 # Start reading the write-ahead log 'sql_logs\sql\log.14' # Finished reading the write-ahead log 'sql_logs\sql\log.14' # Finished processing logs sql/sql_logs # MonetDB/SQL module loaded !GDKerror:!ERROR: BATkey: void column cannot be unique. !ERROR: BATkey: void column cannot be unique.

...

On Mon, Aug 10, 2015 at 9:25 AM, Sjoerd Mullender wrote:

...

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256

I have created a release candidate for the Jul2015 release. The candidate is available at http://dev.monetdb.org/downloads/testing/. If all goes well, this candidate will morph into a release sometime soon(ish). In the mean time, you can use this release candidate and test to see whether it works. And if it doesn't work, I would really like to hear about it.

- -- Sjoerd Mullender -----BEGIN PGP SIGNATURE----- Version: GnuPG v2

iQEcBAEBCAAGBQJVyKZgAAoJEISMxT6LrWYgE28H/2rudFXhcIfNgnX4yYJlpRT5 eI2Dzp9kShEmRibvNkWvCEF1j/qyWgQhx/jlHvucuEF/0k+U+7NYuK0vhzA0llyJ hspMkQ73UDXeVO0/SivtNYb40UpMyUQoXC7nSaSneglVDe4JcX4S3Y5GvHPByiqz 3UNMGDARfWlqPcbTtW57XfLejK/sd8nNZ3mGQuerVqEvMcKZaZIBSXjg/16WQVsN DMbPacPpu3yFeWQQtP2er6SL3UdfwgOP5+VNifKyjW9i6ZtB2lHWLEBWQY8G/5Tb 5lwgHvD99SJeLRMzcW5TFWhwv0ROx8chKP3btxMofZxjHhqseswGRLNDQa4XCU0= =ktpg -----END PGP SIGNATURE----- _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

Stefan Manegold

11 Aug 11 Aug

6:42 a.m.

Hi, I've just seen similar behavior, but failed so far to reliably reproduce it with a minimal test. It seems requires at least creating a new DB with Oct2014, creating a table, inserting a least 2 tuples, stopping & restarting Oct2014 server to flush WAL logs, stopping Oct2014 server starting Jul2015 server to upgrade. Best, Stefan ----- On Aug 10, 2015, at 7:55 PM, Anthony Damico ajdamico@gmail.com wrote:

...

hi, i've just upgraded on my windows 8 laptop. every time i open the demo server, i get this GDK error below. it doesn't appear to have any effect when i run some simple queries. the error below does not appear when i create a fresh dbfarm elsewhere, it only happened on a database that's been upgraded.

any idea what's going on? thanks!

# MonetDB 5 server v11.21.1 "Jul2015" # Serving database 'demo', using 8 threads # Compiled for x86_64-pc-winnt/64bit with 64bit OIDs dynamically linked # Found 15.914 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2015 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information # Listening for connection requests on mapi:monetdb:// 127.0.0.1:50000/ # Start processing logs sql/sql_logs version 52200 # Start reading the write-ahead log 'sql_logs\sql\log.14' # Finished reading the write-ahead log 'sql_logs\sql\log.14' # Finished processing logs sql/sql_logs # MonetDB/SQL module loaded !GDKerror:!ERROR: BATkey: void column cannot be unique. !ERROR: BATkey: void column cannot be unique.

...
On Mon, Aug 10, 2015 at 9:25 AM, Sjoerd Mullender < sjoerd@acm.org > wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256

I have created a release candidate for the Jul2015 release. The candidate is available at http://dev.monetdb.org/downloads/testing/ . If all goes well, this candidate will morph into a release sometime soon(ish). In the mean time, you can use this release candidate and test to see whether it works. And if it doesn't work, I would really like to hear about it.

- -- Sjoerd Mullender -----BEGIN PGP SIGNATURE----- Version: GnuPG v2

iQEcBAEBCAAGBQJVyKZgAAoJEISMxT6LrWYgE28H/2rudFXhcIfNgnX4yYJlpRT5 eI2Dzp9kShEmRibvNkWvCEF1j/qyWgQhx/jlHvucuEF/0k+U+7NYuK0vhzA0llyJ hspMkQ73UDXeVO0/SivtNYb40UpMyUQoXC7nSaSneglVDe4JcX4S3Y5GvHPByiqz 3UNMGDARfWlqPcbTtW57XfLejK/sd8nNZ3mGQuerVqEvMcKZaZIBSXjg/16WQVsN DMbPacPpu3yFeWQQtP2er6SL3UdfwgOP5+VNifKyjW9i6ZtB2lHWLEBWQY8G/5Tb 5lwgHvD99SJeLRMzcW5TFWhwv0ROx8chKP3btxMofZxjHhqseswGRLNDQa4XCU0= =ktpg -----END PGP SIGNATURE----- _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

Anthony Damico

7:15 a.m.

hi, does the notification itself indicate anything important? if people do not flush the logs, will any data be lost? thanks! On Tue, Aug 11, 2015 at 2:42 AM, Stefan Manegold wrote:

...

Hi,

I've just seen similar behavior, but failed so far to reliably reproduce it with a minimal test.

It seems requires at least creating a new DB with Oct2014, creating a table, inserting a least 2 tuples, stopping & restarting Oct2014 server to flush WAL logs, stopping Oct2014 server starting Jul2015 server to upgrade.

Best, Stefan

----- On Aug 10, 2015, at 7:55 PM, Anthony Damico ajdamico@gmail.com wrote:

...
hi, i've just upgraded on my windows 8 laptop. every time i open the demo server, i get this GDK error below. it doesn't appear to have any effect when i run some simple queries. the error below does not appear when i create a fresh dbfarm elsewhere, it only happened on a database that's been upgraded.

any idea what's going on? thanks!

# MonetDB 5 server v11.21.1 "Jul2015" # Serving database 'demo', using 8 threads # Compiled for x86_64-pc-winnt/64bit with 64bit OIDs dynamically linked # Found 15.914 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2015 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information # Listening for connection requests on mapi:monetdb:// 127.0.0.1:50000/ # Start processing logs sql/sql_logs version 52200 # Start reading the write-ahead log 'sql_logs\sql\log.14' # Finished reading the write-ahead log 'sql_logs\sql\log.14' # Finished processing logs sql/sql_logs # MonetDB/SQL module loaded !GDKerror:!ERROR: BATkey: void column cannot be unique. !ERROR: BATkey: void column cannot be unique.

...
On Mon, Aug 10, 2015 at 9:25 AM, Sjoerd Mullender < sjoerd@acm.org > wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256

I have created a release candidate for the Jul2015 release. The candidate is available at http://dev.monetdb.org/downloads/testing/ . If all goes well, this candidate will morph into a release sometime soon(ish). In the mean time, you can use this release candidate and test to see whether it works. And if it doesn't work, I would really like to hear about it.

- -- Sjoerd Mullender -----BEGIN PGP SIGNATURE----- Version: GnuPG v2

iQEcBAEBCAAGBQJVyKZgAAoJEISMxT6LrWYgE28H/2rudFXhcIfNgnX4yYJlpRT5 eI2Dzp9kShEmRibvNkWvCEF1j/qyWgQhx/jlHvucuEF/0k+U+7NYuK0vhzA0llyJ hspMkQ73UDXeVO0/SivtNYb40UpMyUQoXC7nSaSneglVDe4JcX4S3Y5GvHPByiqz 3UNMGDARfWlqPcbTtW57XfLejK/sd8nNZ3mGQuerVqEvMcKZaZIBSXjg/16WQVsN DMbPacPpu3yFeWQQtP2er6SL3UdfwgOP5+VNifKyjW9i6ZtB2lHWLEBWQY8G/5Tb 5lwgHvD99SJeLRMzcW5TFWhwv0ROx8chKP3btxMofZxjHhqseswGRLNDQa4XCU0= =ktpg -----END PGP SIGNATURE----- _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

-- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

Stefan Manegold

7:29 a.m.

...

hi, does the notification itself indicate anything important?

Internally incorrectly set (internal) BAT property. That might be crucial as we (internally) use properties to "cut corners" (aka. "optimize").

...

if people do not flush the logs, will any data be lost?

Should not --- the server flushes logs itself. I just suggested for the purpose of reproducing this problem, to trigger the flush instead of waiting for the server to do it "eventually" and transparently in the background ... Stefan

...

thanks!

On Tue, Aug 11, 2015 at 2:42 AM, Stefan Manegold < Stefan.Manegold@cwi.nl > wrote:

Hi,

I've just seen similar behavior, but failed so far to reliably reproduce it with a minimal test.

It seems requires at least creating a new DB with Oct2014, creating a table, inserting a least 2 tuples, stopping & restarting Oct2014 server to flush WAL logs, stopping Oct2014 server starting Jul2015 server to upgrade.

Best, Stefan

----- On Aug 10, 2015, at 7:55 PM, Anthony Damico ajdamico@gmail.com wrote:

...
hi, i've just upgraded on my windows 8 laptop. every time i open the demo server, i get this GDK error below. it doesn't appear to have any effect when i run some simple queries. the error below does not appear when i create a fresh dbfarm elsewhere, it only happened on a database that's been upgraded.

any idea what's going on? thanks!

# MonetDB 5 server v11.21.1 "Jul2015" # Serving database 'demo', using 8 threads # Compiled for x86_64-pc-winnt/64bit with 64bit OIDs dynamically linked # Found 15.914 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2015 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information # Listening for connection requests on mapi:monetdb:// 127.0.0.1:50000/ # Start processing logs sql/sql_logs version 52200 # Start reading the write-ahead log 'sql_logs\sql\log.14' # Finished reading the write-ahead log 'sql_logs\sql\log.14' # Finished processing logs sql/sql_logs # MonetDB/SQL module loaded !GDKerror:!ERROR: BATkey: void column cannot be unique. !ERROR: BATkey: void column cannot be unique.

...
On Mon, Aug 10, 2015 at 9:25 AM, Sjoerd Mullender < sjoerd@acm.org > wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256

I have created a release candidate for the Jul2015 release. The candidate is available at http://dev.monetdb.org/downloads/testing/ . If all goes well, this candidate will morph into a release sometime soon(ish). In the mean time, you can use this release candidate and test to see whether it works. And if it doesn't work, I would really like to hear about it.

- -- Sjoerd Mullender -----BEGIN PGP SIGNATURE----- Version: GnuPG v2

iQEcBAEBCAAGBQJVyKZgAAoJEISMxT6LrWYgE28H/2rudFXhcIfNgnX4yYJlpRT5 eI2Dzp9kShEmRibvNkWvCEF1j/qyWgQhx/jlHvucuEF/0k+U+7NYuK0vhzA0llyJ hspMkQ73UDXeVO0/SivtNYb40UpMyUQoXC7nSaSneglVDe4JcX4S3Y5GvHPByiqz 3UNMGDARfWlqPcbTtW57XfLejK/sd8nNZ3mGQuerVqEvMcKZaZIBSXjg/16WQVsN DMbPacPpu3yFeWQQtP2er6SL3UdfwgOP5+VNifKyjW9i6ZtB2lHWLEBWQY8G/5Tb 5lwgHvD99SJeLRMzcW5TFWhwv0ROx8chKP3btxMofZxjHhqseswGRLNDQa4XCU0= =ktpg -----END PGP SIGNATURE----- _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

-- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

Stefan Manegold

7:32 a.m.

Well, in fact the error says that we internally try to mark a void column unique, which is not allowed and hence not done. Thus, there seems to be a bug that we incorrectly try to mark a void column unique, but since that fails, this is likely more an "annoying" than a crucial bug ... Stefan ----- On Aug 11, 2015, at 9:29 AM, Stefan Manegold Stefan.Manegold@cwi.nl wrote:

...

...
hi, does the notification itself indicate anything important?

Internally incorrectly set (internal) BAT property. That might be crucial as we (internally) use properties to "cut corners" (aka. "optimize").

...
if people do not flush the logs, will any data be lost?

Should not --- the server flushes logs itself.

I just suggested for the purpose of reproducing this problem, to trigger the flush instead of waiting for the server to do it "eventually" and transparently in the background ...

Stefan

...
thanks!

On Tue, Aug 11, 2015 at 2:42 AM, Stefan Manegold < Stefan.Manegold@cwi.nl > wrote:

Hi,

I've just seen similar behavior, but failed so far to reliably reproduce it with a minimal test.

It seems requires at least creating a new DB with Oct2014, creating a table, inserting a least 2 tuples, stopping & restarting Oct2014 server to flush WAL logs, stopping Oct2014 server starting Jul2015 server to upgrade.

Best, Stefan

----- On Aug 10, 2015, at 7:55 PM, Anthony Damico ajdamico@gmail.com wrote:

...
hi, i've just upgraded on my windows 8 laptop. every time i open the demo server, i get this GDK error below. it doesn't appear to have any effect when i run some simple queries. the error below does not appear when i create a fresh dbfarm elsewhere, it only happened on a database that's been upgraded.

any idea what's going on? thanks!

# MonetDB 5 server v11.21.1 "Jul2015" # Serving database 'demo', using 8 threads # Compiled for x86_64-pc-winnt/64bit with 64bit OIDs dynamically linked # Found 15.914 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2015 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information # Listening for connection requests on mapi:monetdb:// 127.0.0.1:50000/ # Start processing logs sql/sql_logs version 52200 # Start reading the write-ahead log 'sql_logs\sql\log.14' # Finished reading the write-ahead log 'sql_logs\sql\log.14' # Finished processing logs sql/sql_logs # MonetDB/SQL module loaded !GDKerror:!ERROR: BATkey: void column cannot be unique. !ERROR: BATkey: void column cannot be unique.

...
On Mon, Aug 10, 2015 at 9:25 AM, Sjoerd Mullender < sjoerd@acm.org > wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256

I have created a release candidate for the Jul2015 release. The candidate is available at http://dev.monetdb.org/downloads/testing/ . If all goes well, this candidate will morph into a release sometime soon(ish). In the mean time, you can use this release candidate and test to see whether it works. And if it doesn't work, I would really like to hear about it.

- -- Sjoerd Mullender -----BEGIN PGP SIGNATURE----- Version: GnuPG v2

iQEcBAEBCAAGBQJVyKZgAAoJEISMxT6LrWYgE28H/2rudFXhcIfNgnX4yYJlpRT5 eI2Dzp9kShEmRibvNkWvCEF1j/qyWgQhx/jlHvucuEF/0k+U+7NYuK0vhzA0llyJ hspMkQ73UDXeVO0/SivtNYb40UpMyUQoXC7nSaSneglVDe4JcX4S3Y5GvHPByiqz 3UNMGDARfWlqPcbTtW57XfLejK/sd8nNZ3mGQuerVqEvMcKZaZIBSXjg/16WQVsN DMbPacPpu3yFeWQQtP2er6SL3UdfwgOP5+VNifKyjW9i6ZtB2lHWLEBWQY8G/5Tb 5lwgHvD99SJeLRMzcW5TFWhwv0ROx8chKP3btxMofZxjHhqseswGRLNDQa4XCU0= =ktpg -----END PGP SIGNATURE----- _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

-- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

-- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

Stefan Manegold

12 Aug 12 Aug

8:20 a.m.

Hi, while I so far failed to reproduce the behavior reliably, I just happen to have a Oct2014-generated database where upgrading to Jul2015 show the reported error, not only with the initial upgrade, but with every start of Jul2015 on that database. Below you find a debug trace of the event, in case this helps some experts (Sjoerd?, Niels?) to understand / locate the problem ... Thanks! Stefan ======== (gdb) r Starting program: /net/rome.ins.cwi.nl/export/scratch1/manegold/Monet/HG/Jul2015/prefix/_/bin/mserver5 --dbpath=/export/scratch1/manegold/dbfarm --dbextra=/export/scratch2/manegold/dbfarm [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". # MonetDB 5 server v11.21.0 # This is an unreleased version # Serving database 'dbfarm', using 8 threads # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs and 128bit integers dynamically linked # Found 15.590 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2015 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information [New Thread 0x7ffff047c700 (LWP 15521)] [New Thread 0x7fffec658700 (LWP 15549)] # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/GIS module loaded # Start processing logs sql/sql_logs version 52200 Breakpoint 1, GDKerror (format=0x7ffff778a8b8 "BATkey: void column cannot be unique.\n") at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_utils.c:1498 1498 size_t len = strlen(GDKERROR); Missing separate debuginfos, use: debuginfo-install R-core-3.2.0-2.fc20.x86_64 atlas-3.8.4-14.fc20.x86_64 blas-3.4.2-8.fc20.x86_64 bzip2-libs-1.0.6-9.fc20.x86_64 cfitsio-3.350-2.fc20.x86_64 cyrus-sasl-lib-2.1.26-14.fc20.x86_64 geos-3.4.2-1.fc20.x86_64 glibc-2.18-19.fc20.x86_64 gsl-1.15-9.fc20.x86_64 hdf5-1.8.11-6.fc20.x86_64 keyutils-libs-1.5.9-1.fc20.x86_64 krb5-libs-1.11.5-19.fc20.x86_64 libcom_err-1.42.12-4.fc20.x86_64 libcurl-7.32.0-20.fc20.x86_64 libgcc-4.8.3-7.fc20.x86_64 libgfortran-4.8.3-7.fc20.x86_64 libgomp-4.8.3-7.fc20.x86_64 libicu-50.1.2-12.fc20.x86_64 libidn-1.28-3.fc20.x86_64 libjpeg-turbo-1.3.1-3.fc20.x86_64 libquadmath-4.8.3-7.fc20.x86_64 libselinux-2.2.1-6.fc20.x86_64 libssh2-1.5.0-1.fc20.x86_64 libstdc++-4.8.3-7.fc20.x86_64 libuuid-2.24.2-2.fc20.x86_64 libxml2-2.9.1-4.fc20.x86_64 ncurses-libs-5.9-12.20130511.fc20.x86_64 netcdf-4.3.2-4.fc20.x86_64 nspr-4.10.8-1.fc20.x86_64 nss-3.18.0-1.fc20.x86_64 nss-softokn-freebl-3.18.0-1.fc20.x86_64 nss-util-3.18.0-1.fc20.x86_64 openldap-2.4.39-4.fc20.x86_64 openssl-libs-1.0.1e-42.fc20.x86_64 pcre-8.33-11.fc20.x86_64 readline-6.2-10.fc20.x86_64 xz-libs-5.1.2-12alpha.fc20.x86_64 zlib-1.2.8-3.fc20.x86_64 (gdb) bt #0 GDKerror (format=0x7ffff778a8b8 "BATkey: void column cannot be unique.\n") at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_utils.c:1498 #1 0x00007ffff7540db5 in BATkey (b=0x1ecda50, flag=1) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_bat.c:2162 #2 0x00007ffff73d97f4 in ALIGNsetH (b1=0x1ecda50, b2=0x60b810) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_align.c:117 #3 0x00007ffff752f841 in BATcopy (b=0x60b810, ht=0, tt=5, writable=1, role=1) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_bat.c:937 #4 0x00007ffff75590ae in logger_new (debug=0, fn=0x7fffef1faaaa "sql", logdir=0x7fffef1d55b0 "sql_logs", version=52200, prefuncp=0x7fffef1a6a82 , postfuncp=0x7fffef1a6b9f ) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_logger.c:1546 #5 0x00007ffff75598da in logger_create (debug=0, fn=0x7fffef1faaaa "sql", logdir=0x7fffef1d55b0 "sql_logs", version=52200, prefuncp=0x7fffef1a6a82 , postfuncp=0x7fffef1a6b9f ) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_logger.c:1720 #6 0x00007fffef1a8028 in bl_create (debug=0, logdir=0x7fffef1d55b0 "sql_logs", cat_version=52200) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/storage/bat/bat_logger.c:156 #7 0x00007fffef187ddc in store_init (debug=0, store=store_bat, readonly=0, singleuser=0, logdir=0x7fffef1d55b0 "sql_logs", stk=0) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/storage/store.c:1341 #8 0x00007fffef0f925d in mvc_init (debug=0, store=store_bat, ro=0, su=0, stk=0) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/server/sql_mvc.c:36 #9 0x00007fffef041f91 in SQLinit () at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/backends/monet5/sql_scenario.c:251 #10 0x00007fffef041b39 in SQLprelude (ret=0x9bd8f0) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/backends/monet5/sql_scenario.c:178 #11 0x00007ffff7b12834 in malCommandCall (stk=0x9bd690, pci=0x1dc39d0) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_interpreter.c:77 #12 0x00007ffff7b15871 in runMALsequence (cntxt=0x7ffff047d020, mb=0x68d110, startpc=1, stoppc=0, stk=0x9bd690, env=0x0, pcicaller=0x0) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_interpreter.c:655 #13 0x00007ffff7b142fb in runMAL (cntxt=0x7ffff047d020, mb=0x68d110, mbcaller=0x0, env=0x0) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_interpreter.c:365 #14 0x00007ffff7b44c1e in MALengine (c=0x7ffff047d020) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_session.c:637 #15 0x00007ffff7b4272e in malBootstrap () at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_session.c:57 #16 0x00007ffff7aee041 in mal_init () at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal.c:95 #17 0x0000000000403afc in main (argc=3, av=0x7fffffffda68) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/tools/mserver/mserver5.c:624 (gdb) li 1493 1494 void 1495 GDKerror(const char *format, ...) 1496 { 1497 char message[GDKERRLEN]; 1498 size_t len = strlen(GDKERROR); 1499 va_list ap; 1500 1501 if (!strncmp(format, GDKERROR, len)) { 1502 len = 0; (gdb) up #1 0x00007ffff7540db5 in BATkey (b=0x1ecda50, flag=1) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_bat.c:2162 2162 GDKerror("BATkey: void column cannot be unique.\n"); (gdb) li 2157 if (b->hseqbase != oid_nil && flag == FALSE) { 2158 GDKerror("BATkey: dense column must be unique.\n"); 2159 return GDK_FAIL; 2160 } 2161 if (b->hseqbase == oid_nil && flag == TRUE && b->batCount > 1) { 2162 GDKerror("BATkey: void column cannot be unique.\n"); 2163 return GDK_FAIL; 2164 } 2165 } 2166 if (flag) (gdb) p flag $1 = 1 (gdb) p b $2 = (BAT *) 0x1ecda50 (gdb) p *b $3 = {batCacheid = 909, H = 0x1ecda90, T = 0x1ecdb30, S = 0x1ecdbd0} (gdb) p b->H $4 = (COLrec *) 0x1ecda90 (gdb) p BATprint(b) #--------------------------# # h t # name # void int # type #--------------------------# [ nil, 1 ] [ nil, 0 ] $5 = GDK_SUCCEED (gdb) p *b->H $6 = {id = 0x7ffff77894e1 "h", width = 0, type = 0 '\000', shift = 0 '\000', varsized = 1, key = 0, dense = 0, nonil = 1, nil = 0, sorted = 1, revsorted = 1, align = 1028385, nokey = {0, 0}, nosorted = 0, norevsorted = 0, nodense = 0, seq = 9223372036854775808, heap = {free = 0, size = 0, base = 0x0, filename = 0x0, copied = 0, hashash = 0, forcemap = 0, storage = STORE_MEM, newstorage = STORE_MEM, dirty = 0 '\000', farmid = 1 '\001', parentid = 0}, vheap = 0x0, hash = 0x0, imprints = 0x0, props = 0x0} (gdb) p *b->T $7 = {id = 0x7ffff77894e3 "t", width = 4, type = 5 '\005', shift = 2 '\002', varsized = 0, key = 0, dense = 0, nonil = 1, nil = 0, sorted = 1, revsorted = 1, align = 1028386, nokey = {0, 0}, nosorted = 0, norevsorted = 0, nodense = 0, seq = 0, heap = {free = 8, size = 1024, base = 0x1ecdcf0 "\001", filename = 0x1ecdcc0 "16/1615.tail", copied = 0, hashash = 0, forcemap = 0, storage = STORE_MEM, newstorage = STORE_MEM, dirty = 0 '\000', farmid = 1 '\001', parentid = 0}, vheap = 0x0, hash = 0x0, imprints = 0x0, props = 0x0} (gdb) p *b->S $8 = {tid = 140737333930113, stamp = 69, copiedtodisk = 0, dirty = 1, dirtyflushed = 0, descdirty = 1, restricted = 0, persistence = 1, role = 1, unused = 0, sharecnt = 0, map_head = 0 '\000', map_tail = 0 '\000', map_hheap = 0 '\000', map_theap = 0 '\000', deleted = 0, first = 0, inserted = 0, count = 2, capacity = 256} (gdb) ======== ----- On Aug 11, 2015, at 9:32 AM, Stefan Manegold Stefan.Manegold@cwi.nl wrote:

...

Well, in fact the error says that we internally try to mark a void column unique, which is not allowed and hence not done.

Thus, there seems to be a bug that we incorrectly try to mark a void column unique, but since that fails, this is likely more an "annoying" than a crucial bug ...

Stefan

----- On Aug 11, 2015, at 9:29 AM, Stefan Manegold Stefan.Manegold@cwi.nl wrote:

...
...
hi, does the notification itself indicate anything important?

Internally incorrectly set (internal) BAT property. That might be crucial as we (internally) use properties to "cut corners" (aka. "optimize").

...
if people do not flush the logs, will any data be lost?

Should not --- the server flushes logs itself.

I just suggested for the purpose of reproducing this problem, to trigger the flush instead of waiting for the server to do it "eventually" and transparently in the background ...

Stefan

...
thanks!

On Tue, Aug 11, 2015 at 2:42 AM, Stefan Manegold < Stefan.Manegold@cwi.nl > wrote:

Hi,

I've just seen similar behavior, but failed so far to reliably reproduce it with a minimal test.

It seems requires at least creating a new DB with Oct2014, creating a table, inserting a least 2 tuples, stopping & restarting Oct2014 server to flush WAL logs, stopping Oct2014 server starting Jul2015 server to upgrade.

Best, Stefan

----- On Aug 10, 2015, at 7:55 PM, Anthony Damico ajdamico@gmail.com wrote:

...
hi, i've just upgraded on my windows 8 laptop. every time i open the demo server, i get this GDK error below. it doesn't appear to have any effect when i run some simple queries. the error below does not appear when i create a fresh dbfarm elsewhere, it only happened on a database that's been upgraded.

any idea what's going on? thanks!

# MonetDB 5 server v11.21.1 "Jul2015" # Serving database 'demo', using 8 threads # Compiled for x86_64-pc-winnt/64bit with 64bit OIDs dynamically linked # Found 15.914 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2015 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information # Listening for connection requests on mapi:monetdb:// 127.0.0.1:50000/ # Start processing logs sql/sql_logs version 52200 # Start reading the write-ahead log 'sql_logs\sql\log.14' # Finished reading the write-ahead log 'sql_logs\sql\log.14' # Finished processing logs sql/sql_logs # MonetDB/SQL module loaded !GDKerror:!ERROR: BATkey: void column cannot be unique. !ERROR: BATkey: void column cannot be unique.

...
On Mon, Aug 10, 2015 at 9:25 AM, Sjoerd Mullender < sjoerd@acm.org > wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256

I have created a release candidate for the Jul2015 release. The candidate is available at http://dev.monetdb.org/downloads/testing/ . If all goes well, this candidate will morph into a release sometime soon(ish). In the mean time, you can use this release candidate and test to see whether it works. And if it doesn't work, I would really like to hear about it.

- -- Sjoerd Mullender -----BEGIN PGP SIGNATURE----- Version: GnuPG v2

iQEcBAEBCAAGBQJVyKZgAAoJEISMxT6LrWYgE28H/2rudFXhcIfNgnX4yYJlpRT5 eI2Dzp9kShEmRibvNkWvCEF1j/qyWgQhx/jlHvucuEF/0k+U+7NYuK0vhzA0llyJ hspMkQ73UDXeVO0/SivtNYb40UpMyUQoXC7nSaSneglVDe4JcX4S3Y5GvHPByiqz 3UNMGDARfWlqPcbTtW57XfLejK/sd8nNZ3mGQuerVqEvMcKZaZIBSXjg/16WQVsN DMbPacPpu3yFeWQQtP2er6SL3UdfwgOP5+VNifKyjW9i6ZtB2lHWLEBWQY8G/5Tb 5lwgHvD99SJeLRMzcW5TFWhwv0ROx8chKP3btxMofZxjHhqseswGRLNDQa4XCU0= =ktpg -----END PGP SIGNATURE----- _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

-- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

-- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

-- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

Stefan Manegold

9:58 a.m.

Hi, Sjoerd's patch http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=92af99c05811 appears to work for me. Thanks! Stefan ----- On Aug 12, 2015, at 10:20 AM, Stefan Manegold Stefan.Manegold@cwi.nl wrote:

...

Hi,

while I so far failed to reproduce the behavior reliably, I just happen to have a Oct2014-generated database where upgrading to Jul2015 show the reported error, not only with the initial upgrade, but with every start of Jul2015 on that database.

Below you find a debug trace of the event, in case this helps some experts (Sjoerd?, Niels?) to understand / locate the problem ...

Thanks!

Stefan

======== (gdb) r Starting program: /export/scratch1/manegold/Monet/HG/Jul2015/prefix/_/bin/mserver5 --dbpath=/export/scratch1/manegold/dbfarm --dbextra=/export/scratch2/manegold/dbfarm [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". # MonetDB 5 server v11.21.0 # This is an unreleased version # Serving database 'dbfarm', using 8 threads # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs and 128bit integers dynamically linked # Found 15.590 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2015 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information [New Thread 0x7ffff047c700 (LWP 15521)] [New Thread 0x7fffec658700 (LWP 15549)] # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/GIS module loaded # Start processing logs sql/sql_logs version 52200

Breakpoint 1, GDKerror (format=0x7ffff778a8b8 "BATkey: void column cannot be unique.\n") at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_utils.c:1498 1498 size_t len = strlen(GDKERROR); Missing separate debuginfos, use: debuginfo-install R-core-3.2.0-2.fc20.x86_64 atlas-3.8.4-14.fc20.x86_64 blas-3.4.2-8.fc20.x86_64 bzip2-libs-1.0.6-9.fc20.x86_64 cfitsio-3.350-2.fc20.x86_64 cyrus-sasl-lib-2.1.26-14.fc20.x86_64 geos-3.4.2-1.fc20.x86_64 glibc-2.18-19.fc20.x86_64 gsl-1.15-9.fc20.x86_64 hdf5-1.8.11-6.fc20.x86_64 keyutils-libs-1.5.9-1.fc20.x86_64 krb5-libs-1.11.5-19.fc20.x86_64 libcom_err-1.42.12-4.fc20.x86_64 libcurl-7.32.0-20.fc20.x86_64 libgcc-4.8.3-7.fc20.x86_64 libgfortran-4.8.3-7.fc20.x86_64 libgomp-4.8.3-7.fc20.x86_64 libicu-50.1.2-12.fc20.x86_64 libidn-1.28-3.fc20.x86_64 libjpeg-turbo-1.3.1-3.fc20.x86_64 libquadmath-4.8.3-7.fc20.x86_64 libselinux-2.2.1-6.fc20.x86_64 libssh2-1.5.0-1.fc20.x86_64 libstdc++-4.8.3-7.fc20.x86_64 libuuid-2.24.2-2.fc20.x86_64 libxml2-2.9.1-4.fc20.x86_64 ncurses-libs-5.9-12.20130511.fc20.x86_64 netcdf-4.3.2-4.fc20.x86_64 nspr-4.10.8-1.fc20.x86_64 nss-3.18.0-1.fc20.x86_64 nss-softokn-freebl-3.18.0-1.fc20.x86_64 nss-util-3.18.0-1.fc! 20.x86_64 openldap-2.4.39-4.fc20.x86_64 openssl-libs-1.0.1e-42.fc20.x86_64 pcre-8.33-11.fc20.x86_64 readline-6.2-10.fc20.x86_64 xz-libs-5.1.2-12alpha.fc20.x86_64 zlib-1.2.8-3.fc20.x86_64 (gdb) bt #0 GDKerror (format=0x7ffff778a8b8 "BATkey: void column cannot be unique.\n") at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_utils.c:1498 #1 0x00007ffff7540db5 in BATkey (b=0x1ecda50, flag=1) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_bat.c:2162 #2 0x00007ffff73d97f4 in ALIGNsetH (b1=0x1ecda50, b2=0x60b810) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_align.c:117 #3 0x00007ffff752f841 in BATcopy (b=0x60b810, ht=0, tt=5, writable=1, role=1) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_bat.c:937 #4 0x00007ffff75590ae in logger_new (debug=0, fn=0x7fffef1faaaa "sql", logdir=0x7fffef1d55b0 "sql_logs", version=52200, prefuncp=0x7fffef1a6a82 , postfuncp=0x7fffef1a6b9f ) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_logger.c:1546 #5 0x00007ffff75598da in logger_create (debug=0, fn=0x7fffef1faaaa "sql", logdir=0x7fffef1d55b0 "sql_logs", version=52200, prefuncp=0x7fffef1a6a82 , postfuncp=0x7fffef1a6b9f ) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_logger.c:1720 #6 0x00007fffef1a8028 in bl_create (debug=0, logdir=0x7fffef1d55b0 "sql_logs", cat_version=52200) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/storage/bat/bat_logger.c:156 #7 0x00007fffef187ddc in store_init (debug=0, store=store_bat, readonly=0, singleuser=0, logdir=0x7fffef1d55b0 "sql_logs", stk=0) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/storage/store.c:1341 #8 0x00007fffef0f925d in mvc_init (debug=0, store=store_bat, ro=0, su=0, stk=0) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/server/sql_mvc.c:36 #9 0x00007fffef041f91 in SQLinit () at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/backends/monet5/sql_scenario.c:251 #10 0x00007fffef041b39 in SQLprelude (ret=0x9bd8f0) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/backends/monet5/sql_scenario.c:178 #11 0x00007ffff7b12834 in malCommandCall (stk=0x9bd690, pci=0x1dc39d0) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_interpreter.c:77 #12 0x00007ffff7b15871 in runMALsequence (cntxt=0x7ffff047d020, mb=0x68d110, startpc=1, stoppc=0, stk=0x9bd690, env=0x0, pcicaller=0x0) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_interpreter.c:655 #13 0x00007ffff7b142fb in runMAL (cntxt=0x7ffff047d020, mb=0x68d110, mbcaller=0x0, env=0x0) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_interpreter.c:365 #14 0x00007ffff7b44c1e in MALengine (c=0x7ffff047d020) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_session.c:637 #15 0x00007ffff7b4272e in malBootstrap () at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_session.c:57 #16 0x00007ffff7aee041 in mal_init () at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal.c:95 #17 0x0000000000403afc in main (argc=3, av=0x7fffffffda68) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/tools/mserver/mserver5.c:624 (gdb) li 1493 1494 void 1495 GDKerror(const char *format, ...) 1496 { 1497 char message[GDKERRLEN]; 1498 size_t len = strlen(GDKERROR); 1499 va_list ap; 1500 1501 if (!strncmp(format, GDKERROR, len)) { 1502 len = 0; (gdb) up #1 0x00007ffff7540db5 in BATkey (b=0x1ecda50, flag=1) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_bat.c:2162 2162 GDKerror("BATkey: void column cannot be unique.\n"); (gdb) li 2157 if (b->hseqbase != oid_nil && flag == FALSE) { 2158 GDKerror("BATkey: dense column must be unique.\n"); 2159 return GDK_FAIL; 2160 } 2161 if (b->hseqbase == oid_nil && flag == TRUE && b->batCount > 1) { 2162 GDKerror("BATkey: void column cannot be unique.\n"); 2163 return GDK_FAIL; 2164 } 2165 } 2166 if (flag) (gdb) p flag $1 = 1 (gdb) p b $2 = (BAT *) 0x1ecda50 (gdb) p *b $3 = {batCacheid = 909, H = 0x1ecda90, T = 0x1ecdb30, S = 0x1ecdbd0} (gdb) p b->H $4 = (COLrec *) 0x1ecda90 (gdb) p BATprint(b) #--------------------------# # h t # name # void int # type #--------------------------# [ nil, 1 ] [ nil, 0 ] $5 = GDK_SUCCEED (gdb) p *b->H $6 = {id = 0x7ffff77894e1 "h", width = 0, type = 0 '\000', shift = 0 '\000', varsized = 1, key = 0, dense = 0, nonil = 1, nil = 0, sorted = 1, revsorted = 1, align = 1028385, nokey = {0, 0}, nosorted = 0, norevsorted = 0, nodense = 0, seq = 9223372036854775808, heap = {free = 0, size = 0, base = 0x0, filename = 0x0, copied = 0, hashash = 0, forcemap = 0, storage = STORE_MEM, newstorage = STORE_MEM, dirty = 0 '\000', farmid = 1 '\001', parentid = 0}, vheap = 0x0, hash = 0x0, imprints = 0x0, props = 0x0} (gdb) p *b->T $7 = {id = 0x7ffff77894e3 "t", width = 4, type = 5 '\005', shift = 2 '\002', varsized = 0, key = 0, dense = 0, nonil = 1, nil = 0, sorted = 1, revsorted = 1, align = 1028386, nokey = {0, 0}, nosorted = 0, norevsorted = 0, nodense = 0, seq = 0, heap = {free = 8, size = 1024, base = 0x1ecdcf0 "\001", filename = 0x1ecdcc0 "16/1615.tail", copied = 0, hashash = 0, forcemap = 0, storage = STORE_MEM, newstorage = STORE_MEM, dirty = 0 '\000', farmid = 1 '\001', parentid = 0}, vheap = 0x0, hash = 0x0, imprints = 0x0, props = 0x0} (gdb) p *b->S $8 = {tid = 140737333930113, stamp = 69, copiedtodisk = 0, dirty = 1, dirtyflushed = 0, descdirty = 1, restricted = 0, persistence = 1, role = 1, unused = 0, sharecnt = 0, map_head = 0 '\000', map_tail = 0 '\000', map_hheap = 0 '\000', map_theap = 0 '\000', deleted = 0, first = 0, inserted = 0, count = 2, capacity = 256} (gdb) ========

----- On Aug 11, 2015, at 9:32 AM, Stefan Manegold Stefan.Manegold@cwi.nl wrote:

...
Well, in fact the error says that we internally try to mark a void column unique, which is not allowed and hence not done.

Thus, there seems to be a bug that we incorrectly try to mark a void column unique, but since that fails, this is likely more an "annoying" than a crucial bug ...

Stefan

----- On Aug 11, 2015, at 9:29 AM, Stefan Manegold Stefan.Manegold@cwi.nl wrote:

...
...
hi, does the notification itself indicate anything important?

Internally incorrectly set (internal) BAT property. That might be crucial as we (internally) use properties to "cut corners" (aka. "optimize").

...
if people do not flush the logs, will any data be lost?

Should not --- the server flushes logs itself.

I just suggested for the purpose of reproducing this problem, to trigger the flush instead of waiting for the server to do it "eventually" and transparently in the background ...

Stefan

...
thanks!

On Tue, Aug 11, 2015 at 2:42 AM, Stefan Manegold < Stefan.Manegold@cwi.nl > wrote:

Hi,

I've just seen similar behavior, but failed so far to reliably reproduce it with a minimal test.

It seems requires at least creating a new DB with Oct2014, creating a table, inserting a least 2 tuples, stopping & restarting Oct2014 server to flush WAL logs, stopping Oct2014 server starting Jul2015 server to upgrade.

Best, Stefan

----- On Aug 10, 2015, at 7:55 PM, Anthony Damico ajdamico@gmail.com wrote:

...
hi, i've just upgraded on my windows 8 laptop. every time i open the demo server, i get this GDK error below. it doesn't appear to have any effect when i run some simple queries. the error below does not appear when i create a fresh dbfarm elsewhere, it only happened on a database that's been upgraded.

any idea what's going on? thanks!

# MonetDB 5 server v11.21.1 "Jul2015" # Serving database 'demo', using 8 threads # Compiled for x86_64-pc-winnt/64bit with 64bit OIDs dynamically linked # Found 15.914 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2015 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information # Listening for connection requests on mapi:monetdb:// 127.0.0.1:50000/ # Start processing logs sql/sql_logs version 52200 # Start reading the write-ahead log 'sql_logs\sql\log.14' # Finished reading the write-ahead log 'sql_logs\sql\log.14' # Finished processing logs sql/sql_logs # MonetDB/SQL module loaded !GDKerror:!ERROR: BATkey: void column cannot be unique. !ERROR: BATkey: void column cannot be unique.

...
On Mon, Aug 10, 2015 at 9:25 AM, Sjoerd Mullender < sjoerd@acm.org > wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256

I have created a release candidate for the Jul2015 release. The candidate is available at http://dev.monetdb.org/downloads/testing/ . If all goes well, this candidate will morph into a release sometime soon(ish). In the mean time, you can use this release candidate and test to see whether it works. And if it doesn't work, I would really like to hear about it.

- -- Sjoerd Mullender -----BEGIN PGP SIGNATURE----- Version: GnuPG v2

iQEcBAEBCAAGBQJVyKZgAAoJEISMxT6LrWYgE28H/2rudFXhcIfNgnX4yYJlpRT5 eI2Dzp9kShEmRibvNkWvCEF1j/qyWgQhx/jlHvucuEF/0k+U+7NYuK0vhzA0llyJ hspMkQ73UDXeVO0/SivtNYb40UpMyUQoXC7nSaSneglVDe4JcX4S3Y5GvHPByiqz 3UNMGDARfWlqPcbTtW57XfLejK/sd8nNZ3mGQuerVqEvMcKZaZIBSXjg/16WQVsN DMbPacPpu3yFeWQQtP2er6SL3UdfwgOP5+VNifKyjW9i6ZtB2lHWLEBWQY8G/5Tb 5lwgHvD99SJeLRMzcW5TFWhwv0ROx8chKP3btxMofZxjHhqseswGRLNDQa4XCU0= =ktpg -----END PGP SIGNATURE----- _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

-- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

-- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

-- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

-- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

Anthony Damico

10:39 a.m.

thanks all! On Wed, Aug 12, 2015 at 5:58 AM, Stefan Manegold wrote:

...

Hi,

Sjoerd's patch http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=92af99c05811 appears to work for me.

Thanks!

Stefan

----- On Aug 12, 2015, at 10:20 AM, Stefan Manegold Stefan.Manegold@cwi.nl wrote:

...
Hi,

while I so far failed to reproduce the behavior reliably, I just happen to have a Oct2014-generated database where upgrading to Jul2015 show the reported error, not only with the initial upgrade, but with every start of Jul2015 on that database.

Below you find a debug trace of the event, in case this helps some experts (Sjoerd?, Niels?) to understand / locate the problem ...

Thanks!

Stefan

======== (gdb) r Starting program: /export/scratch1/manegold/Monet/HG/Jul2015/prefix/_/bin/mserver5 --dbpath=/export/scratch1/manegold/dbfarm --dbextra=/export/scratch2/manegold/dbfarm [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". # MonetDB 5 server v11.21.0 # This is an unreleased version # Serving database 'dbfarm', using 8 threads # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs and 128bit integers dynamically linked # Found 15.590 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2015 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information [New Thread 0x7ffff047c700 (LWP 15521)] [New Thread 0x7fffec658700 (LWP 15549)] # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/GIS module loaded # Start processing logs sql/sql_logs version 52200

Breakpoint 1, GDKerror (format=0x7ffff778a8b8 "BATkey: void column cannot be unique.\n") at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_utils.c:1498 1498 size_t len = strlen(GDKERROR); Missing separate debuginfos, use: debuginfo-install R-core-3.2.0-2.fc20.x86_64 atlas-3.8.4-14.fc20.x86_64 blas-3.4.2-8.fc20.x86_64 bzip2-libs-1.0.6-9.fc20.x86_64 cfitsio-3.350-2.fc20.x86_64 cyrus-sasl-lib-2.1.26-14.fc20.x86_64 geos-3.4.2-1.fc20.x86_64 glibc-2.18-19.fc20.x86_64 gsl-1.15-9.fc20.x86_64 hdf5-1.8.11-6.fc20.x86_64 keyutils-libs-1.5.9-1.fc20.x86_64 krb5-libs-1.11.5-19.fc20.x86_64 libcom_err-1.42.12-4.fc20.x86_64 libcurl-7.32.0-20.fc20.x86_64 libgcc-4.8.3-7.fc20.x86_64 libgfortran-4.8.3-7.fc20.x86_64 libgomp-4.8.3-7.fc20.x86_64 libicu-50.1.2-12.fc20.x86_64 libidn-1.28-3.fc20.x86_64 libjpeg-turbo-1.3.1-3.fc20.x86_64 libquadmath-4.8.3-7.fc20.x86_64 libselinux-2.2.1-6.fc20.x86_64 libssh2-1.5.0-1.fc20.x86_64 libstdc++-4.8.3-7.fc20.x86_64 libuuid-2.24.2-2.fc20.x86_64 libxml2-2.9.1-4.fc20.x86_64 ncurses-libs-5.9-12.20130511.fc20.x86_64 netcdf-4.3.2-4.fc20.x86_64 nspr-4.10.8-1.fc20.x86_64 nss-3.18.0-1.fc20.x86_64 nss-softokn-freebl-3.18.0-1.fc20.x86_64 nss-util-3.18.0-1.fc! 20.x86_64 openldap-2.4.39-4.fc20.x86_64 openssl-libs-1.0.1e-42.fc20.x86_64 pcre-8.33-11.fc20.x86_64 readline-6.2-10.fc20.x86_64 xz-libs-5.1.2-12alpha.fc20.x86_64 zlib-1.2.8-3.fc20.x86_64 (gdb) bt #0 GDKerror (format=0x7ffff778a8b8 "BATkey: void column cannot be unique.\n") at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_utils.c:1498 #1 0x00007ffff7540db5 in BATkey (b=0x1ecda50, flag=1) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_bat.c:2162 #2 0x00007ffff73d97f4 in ALIGNsetH (b1=0x1ecda50, b2=0x60b810) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_align.c:117 #3 0x00007ffff752f841 in BATcopy (b=0x60b810, ht=0, tt=5, writable=1, role=1) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_bat.c:937 #4 0x00007ffff75590ae in logger_new (debug=0, fn=0x7fffef1faaaa "sql", logdir=0x7fffef1d55b0 "sql_logs", version=52200, prefuncp=0x7fffef1a6a82 , postfuncp=0x7fffef1a6b9f ) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_logger.c:1546 #5 0x00007ffff75598da in logger_create (debug=0, fn=0x7fffef1faaaa "sql", logdir=0x7fffef1d55b0 "sql_logs", version=52200, prefuncp=0x7fffef1a6a82 , postfuncp=0x7fffef1a6b9f ) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_logger.c:1720 #6 0x00007fffef1a8028 in bl_create (debug=0, logdir=0x7fffef1d55b0 "sql_logs", cat_version=52200) at

/ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/storage/bat/bat_logger.c:156

...
#7 0x00007fffef187ddc in store_init (debug=0, store=store_bat, readonly=0, singleuser=0, logdir=0x7fffef1d55b0 "sql_logs", stk=0) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/storage/store.c:1341 #8 0x00007fffef0f925d in mvc_init (debug=0, store=store_bat, ro=0, su=0, stk=0) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/server/sql_mvc.c:36 #9 0x00007fffef041f91 in SQLinit () at

/ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/backends/monet5/sql_scenario.c:251

...
#10 0x00007fffef041b39 in SQLprelude (ret=0x9bd8f0) at

/ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/sql/backends/monet5/sql_scenario.c:178

...
#11 0x00007ffff7b12834 in malCommandCall (stk=0x9bd690, pci=0x1dc39d0) at

/ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_interpreter.c:77

...
#12 0x00007ffff7b15871 in runMALsequence (cntxt=0x7ffff047d020, mb=0x68d110, startpc=1, stoppc=0, stk=0x9bd690, env=0x0, pcicaller=0x0) at

/ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_interpreter.c:655

...
#13 0x00007ffff7b142fb in runMAL (cntxt=0x7ffff047d020, mb=0x68d110, mbcaller=0x0, env=0x0) at

/ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_interpreter.c:365

...
#14 0x00007ffff7b44c1e in MALengine (c=0x7ffff047d020) at

/ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_session.c:637

...
#15 0x00007ffff7b4272e in malBootstrap () at

/ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal_session.c:57

...
#16 0x00007ffff7aee041 in mal_init () at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/monetdb5/mal/mal.c:95 #17 0x0000000000403afc in main (argc=3, av=0x7fffffffda68) at

/ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/tools/mserver/mserver5.c:624

...
(gdb) li 1493 1494 void 1495 GDKerror(const char *format, ...) 1496 { 1497 char message[GDKERRLEN]; 1498 size_t len = strlen(GDKERROR); 1499 va_list ap; 1500 1501 if (!strncmp(format, GDKERROR, len)) { 1502 len = 0; (gdb) up #1 0x00007ffff7540db5 in BATkey (b=0x1ecda50, flag=1) at /ufs/manegold/_/Monet/HG/Jul2015/source/MonetDB/gdk/gdk_bat.c:2162 2162 GDKerror("BATkey: void column cannot be unique.\n"); (gdb) li 2157 if (b->hseqbase != oid_nil && flag == FALSE) { 2158 GDKerror("BATkey: dense column must be unique.\n"); 2159 return GDK_FAIL; 2160 } 2161 if (b->hseqbase == oid_nil && flag == TRUE && b->batCount > 1) { 2162 GDKerror("BATkey: void column cannot be unique.\n"); 2163 return GDK_FAIL; 2164 } 2165 } 2166 if (flag) (gdb) p flag $1 = 1 (gdb) p b $2 = (BAT *) 0x1ecda50 (gdb) p *b $3 = {batCacheid = 909, H = 0x1ecda90, T = 0x1ecdb30, S = 0x1ecdbd0} (gdb) p b->H $4 = (COLrec *) 0x1ecda90 (gdb) p BATprint(b) #--------------------------# # h t # name # void int # type #--------------------------# [ nil, 1 ] [ nil, 0 ] $5 = GDK_SUCCEED (gdb) p *b->H $6 = {id = 0x7ffff77894e1 "h", width = 0, type = 0 '\000', shift = 0 '\000', varsized = 1, key = 0, dense = 0, nonil = 1, nil = 0, sorted = 1, revsorted = 1, align = 1028385, nokey = {0, 0}, nosorted = 0, norevsorted = 0, nodense = 0, seq = 9223372036854775808, heap = {free = 0, size = 0, base = 0x0, filename = 0x0, copied = 0, hashash = 0, forcemap = 0, storage = STORE_MEM, newstorage = STORE_MEM, dirty = 0 '\000', farmid = 1 '\001', parentid = 0}, vheap = 0x0, hash = 0x0, imprints = 0x0, props = 0x0} (gdb) p *b->T $7 = {id = 0x7ffff77894e3 "t", width = 4, type = 5 '\005', shift = 2 '\002', varsized = 0, key = 0, dense = 0, nonil = 1, nil = 0, sorted = 1, revsorted = 1, align = 1028386, nokey = {0, 0}, nosorted = 0, norevsorted = 0, nodense = 0, seq = 0, heap = {free = 8, size = 1024, base = 0x1ecdcf0 "\001", filename = 0x1ecdcc0 "16/1615.tail", copied = 0, hashash = 0, forcemap = 0, storage = STORE_MEM, newstorage = STORE_MEM, dirty = 0 '\000', farmid = 1 '\001', parentid = 0}, vheap = 0x0, hash = 0x0, imprints = 0x0, props = 0x0} (gdb) p *b->S $8 = {tid = 140737333930113, stamp = 69, copiedtodisk = 0, dirty = 1, dirtyflushed = 0, descdirty = 1, restricted = 0, persistence = 1, role = 1, unused = 0, sharecnt = 0, map_head = 0 '\000', map_tail = 0 '\000', map_hheap = 0 '\000', map_theap = 0 '\000', deleted = 0, first = 0, inserted = 0, count = 2, capacity = 256} (gdb) ========

----- On Aug 11, 2015, at 9:32 AM, Stefan Manegold Stefan.Manegold@cwi.nl wrote:

...
Well, in fact the error says that we internally try to mark a void

column >> unique, >> which is not allowed and hence not done. >> >> Thus, there seems to be a bug that we incorrectly try to mark a void column >> unique, >> but since that fails, this is likely more an "annoying" than a crucial bug ... >> >> Stefan >> >> ----- On Aug 11, 2015, at 9:29 AM, Stefan Manegold Stefan.Manegold@cwi.nl wrote: >> >>>> hi, >>>> does the notification itself indicate anything important? >>> >>> Internally incorrectly set (internal) BAT property. >>> That might be crucial as we (internally) use properties to "cut corners" (aka. >>> "optimize"). >>> >>>> if people do not flush the logs, will any data be lost? >>> >>> Should not --- the server flushes logs itself. >>> >>> I just suggested for the purpose of reproducing this problem, >>> to trigger the flush instead of waiting for the server to do it "eventually" >>> and transparently in the background ... >>> >>> Stefan >>> >>>> thanks! >>>> >>>> On Tue, Aug 11, 2015 at 2:42 AM, Stefan Manegold < Stefan.Manegold@cwi.nl > >>>> wrote: >>>> >>>> >>>> Hi, >>>> >>>> I've just seen similar behavior, >>>> but failed so far to reliably reproduce it with a minimal test. >>>> >>>> It seems requires at least >>>> creating a new DB with Oct2014, >>>> creating a table, inserting a least 2 tuples, >>>> stopping & restarting Oct2014 server to flush WAL logs, >>>> stopping Oct2014 server >>>> starting Jul2015 server to upgrade. >>>> >>>> Best, >>>> Stefan >>>> >>>> ----- On Aug 10, 2015, at 7:55 PM, Anthony Damico ajdamico@gmail.com wrote: >>>> >>>>> hi, i've just upgraded on my windows 8 laptop. every time i open the demo >>>>> server, i get this GDK error below. it doesn't appear to have any effect when i >>>>> run some simple queries. the error below does not appear when i create a fresh >>>>> dbfarm elsewhere, it only happened on a database that's been upgraded. >>>>> >>>>> any idea what's going on? thanks! >>>>> >>>>> >>>>> # MonetDB 5 server v11.21.1 "Jul2015" >>>>> # Serving database 'demo', using 8 threads >>>>> # Compiled for x86_64-pc-winnt/64bit with 64bit OIDs dynamically linked >>>>> # Found 15.914 GiB available main-memory. >>>>> # Copyright (c) 1993-July 2008 CWI. >>>>> # Copyright (c) August 2008-2015 MonetDB B.V., all rights reserved >>>>> # Visit http://www.monetdb.org/ for further information >>>>> # Listening for connection requests on mapi:monetdb:// 127.0.0.1:50000/ >>>>> # Start processing logs sql/sql_logs version 52200 >>>>> # Start reading the write-ahead log 'sql_logs\sql\log.14' >>>>> # Finished reading the write-ahead log 'sql_logs\sql\log.14' >>>>> # Finished processing logs sql/sql_logs >>>>> # MonetDB/SQL module loaded >>>>> !GDKerror:!ERROR: BATkey: void column cannot be unique. >>>>> !ERROR: BATkey: void column cannot be unique. >>>>>> >>>>> >>>>> >>>>> >>>>> On Mon, Aug 10, 2015 at 9:25 AM, Sjoerd Mullender < sjoerd@acm.org > wrote: >>>>> >>>>> >>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>> Hash: SHA256 >>>>> >>>>> I have created a release candidate for the Jul2015 release. The >>>>> candidate is available at http://dev.monetdb.org/downloads/testing/ . >>>>> If all goes well, this candidate will morph into a release sometime >>>>> soon(ish). In the mean time, you can use this release candidate and >>>>> test to see whether it works. And if it doesn't work, I would really >>>>> like to hear about it. >>>>> >>>>> - -- >>>>> Sjoerd Mullender >>>>> -----BEGIN PGP SIGNATURE----- >>>>> Version: GnuPG v2 >>>>> >>>>> iQEcBAEBCAAGBQJVyKZgAAoJEISMxT6LrWYgE28H/2rudFXhcIfNgnX4yYJlpRT5 >>>>> eI2Dzp9kShEmRibvNkWvCEF1j/qyWgQhx/jlHvucuEF/0k+U+7NYuK0vhzA0llyJ >>>>> hspMkQ73UDXeVO0/SivtNYb40UpMyUQoXC7nSaSneglVDe4JcX4S3Y5GvHPByiqz >>>>> 3UNMGDARfWlqPcbTtW57XfLejK/sd8nNZ3mGQuerVqEvMcKZaZIBSXjg/16WQVsN >>>>> DMbPacPpu3yFeWQQtP2er6SL3UdfwgOP5+VNifKyjW9i6ZtB2lHWLEBWQY8G/5Tb >>>>> 5lwgHvD99SJeLRMzcW5TFWhwv0ROx8chKP3btxMofZxjHhqseswGRLNDQa4XCU0= >>>>> =ktpg >>>>> -----END PGP SIGNATURE----- >>>>> _______________________________________________ >>>>> users-list mailing list >>>>> users-list@monetdb.org >>>>> https://www.monetdb.org/mailman/listinfo/users-list >>>>> >>>>> >>>>> _______________________________________________ >>>>> users-list mailing list >>>>> users-list@monetdb.org >>>>> https://www.monetdb.org/mailman/listinfo/users-list >>>> >>>> -- >>>>| Stefan.Manegold@CWI.nl | DB Architectures (DA) | >>>>| www.CWI.nl/~manegold/ | Science Park 123 (L321) | >>>>| +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | >>>> _______________________________________________ >>>> users-list mailing list >>>> users-list@monetdb.org >>>> https://www.monetdb.org/mailman/listinfo/users-list >>>> >>>> >>>> _______________________________________________ >>>> users-list mailing list >>>> users-list@monetdb.org >>>> https://www.monetdb.org/mailman/listinfo/users-list >>> >>> -- >>>| Stefan.Manegold@CWI.nl | DB Architectures (DA) | >>>| www.CWI.nl/~manegold/ | Science Park 123 (L321) | >>> | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | >> >> -- >>| Stefan.Manegold@CWI.nl | DB Architectures (DA) | >>| www.CWI.nl/~manegold/ | Science Park 123 (L321) | >>| +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | >> _______________________________________________ >> users-list mailing list >> users-list@monetdb.org >> https://www.monetdb.org/mailman/listinfo/users-list > > -- >| Stefan.Manegold@CWI.nl | DB Architectures (DA) | >| www.CWI.nl/~manegold/ | Science Park 123 (L321) | >| +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | > _______________________________________________ > users-list mailing list > users-list@monetdb.org > https://www.monetdb.org/mailman/listinfo/users-list

-- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

Sjoerd Mullender

3:30 p.m.

...

I have created a release candidate for the Jul2015 release. The candidate is available at http://dev.monetdb.org/downloads/testing/. If all goes well, this candidate will morph into a release sometime soon(ish). In the mean time, you can use this release candidate and test to see whether it works. And if it doesn't work, I would really like to hear about it.

_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

- -- Sjoerd Mullender -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBCAAGBQJVy2ajAAoJEISMxT6LrWYgUMMH/2WicRL9H/7zO3l26dsV6NqO J3h8f985OX7iF7Vejz1J63AbKW9vvORD9h/TO/Y84gFDM5UBeCzewiyG8ZW+xXOX msbi7RbXCyXTefYm5e275V32j7aZWin2/1kNSm0ZrFM2QPJpSYqo1GhE1iAHOgdT 4twm0p2DF3cVvg9dWtC6sorBd09nnguiaym30dpBXiJBdVFXAtHdIfDuqJ7N4fv4 XMd09eV2d305O74GuQgFbD9DwZFL1d3HCLWOJK8oDstdL/Dc8IBf+n3YNlkhtQte m365AgWpjRvAdCB2HtmJnFBL5ZRHwvQeMncDikKNARSG5jeSaWRiSkuP9/A3SeY= =yTjC -----END PGP SIGNATURE-----

Richard Hughes

5:54 p.m.

On Mon, 10 Aug 2015 15:25:52 +0200, Sjoerd Mullender wrote:

...

I have created a release candidate for the Jul2015 release. The candidate is available at http://dev.monetdb.org/downloads/testing/. If all goes well, this candidate will morph into a release sometime soon(ish). In the mean time, you can use this release candidate and test to see whether it works. And if it doesn't work, I would really like to hear about it.

Do you have a preliminary list of new features in this release? I'm not going to test features that I don't know exist... I've had my eye on the 'merge tables' work. The documentation at https://www.monetdb.org/Documentation/Cookbooks/SQLrecipes/DataPartitioning is clear on the usage of the feature, but can somebody knowledgeable comment on the advantages/disadvantages relative to the existing approach of using a view of unions? Basically I want to know whether it's worth rewriting my well-tested existing code to make use of the new facility. Thanks, Richard.

Sjoerd Mullender

13 Aug 13 Aug

7:33 a.m.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 https://www.monetdb.org/downloads/testing/relnotes.html On 12/08/15 19:54, Richard Hughes wrote:

...

On Mon, 10 Aug 2015 15:25:52 +0200, Sjoerd Mullender wrote:

...
I have created a release candidate for the Jul2015 release. The candidate is available at http://dev.monetdb.org/downloads/testing/. If all goes well, this candidate will morph into a release sometime soon(ish). In the mean time, you can use this release candidate and test to see whether it works. And if it doesn't work, I would really like to hear about it.

Do you have a preliminary list of new features in this release? I'm not going to test features that I don't know exist...

I've had my eye on the 'merge tables' work. The documentation at https://www.monetdb.org/Documentation/Cookbooks/SQLrecipes/DataPartiti oning

is clear on the usage of the feature, but can somebody knowledgeable

...

comment on the advantages/disadvantages relative to the existing approach of using a view of unions? Basically I want to know whether it's worth rewriting my well-tested existing code to make use of the new facility.

Thanks,

Richard. _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

- -- Sjoerd Mullender -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBCAAGBQJVzEhVAAoJEISMxT6LrWYgb40H/2ek8V3iDa18ZqUygN1xWWXn nFlVe9YE8EkBY8NHmSHlora6WdCjUEV/llgGwLGfj5qDRlVUyiVuPtRBlnM6AZFg QNKx7zUdsd7M7wFu29qrOa8DugUf9jEABxwgAirh1cBvmUI2Ci+RCHmFgQlfYlaG B7cU4FS5lnSbqd2+/3Ef054Esszx3sGce5pXfbjUEBYAt/U61/v5jqVMAqQ0YrwE 1UcnYCmo8Jo03+B5zHgFT5ZdzKQ1kAmhd8vuz7BKTIoEnMdsoiwBv07mvMcFXYCw Ug5Vz5oBzyLqqOmwdaFaZQaA/YYNZs8ra2AKvu3w4YEyPCh3NyQcdNVyleeWR5M= =UTld -----END PGP SIGNATURE-----

Niels Nes

14 Aug 14 Aug

9:46 a.m.

On Thu, Aug 13, 2015 at 09:33:41AM +0200, Sjoerd Mullender wrote:

...

https://www.monetdb.org/downloads/testing/relnotes.html

On 12/08/15 19:54, Richard Hughes wrote:

...
On Mon, 10 Aug 2015 15:25:52 +0200, Sjoerd Mullender wrote:

...
I have created a release candidate for the Jul2015 release. The candidate is available at http://dev.monetdb.org/downloads/testing/. If all goes well, this candidate will morph into a release sometime soon(ish). In the mean time, you can use this release candidate and test to see whether it works. And if it doesn't work, I would really like to hear about it.

Do you have a preliminary list of new features in this release? I'm not going to test features that I don't know exist...

I've had my eye on the 'merge tables' work. The documentation at https://www.monetdb.org/Documentation/Cookbooks/SQLrecipes/DataPartiti oning

is clear on the usage of the feature, but can somebody knowledgeable

...
comment on the advantages/disadvantages relative to the existing approach of using a view of unions? Basically I want to know whether it's worth rewriting my well-tested existing code to make use of the new facility. Richard

I'd expect your application specific solution to always out perform a more general solution. But afcourse we are interested in more testing/comparisons, so if you have some time please try also with the new merge table support (possibly also with remote and replica tables). The merge tables are indeed just a union all kind of view, with the difference that we know the parts and therefor can more easily apply relational rewrites over the parts. For now these are limited too 'readonly' marked partitions. These partitions could be eliminated in the rewrite and could greatly reduce the size of the mal plans. We use statistics on those readonly parts (obtained via analyze) to do the rewrites. Niels

...

...
Thanks,

Richard. _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

-- Sjoerd Mullender _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

-- Niels Nes, Manager ITF, Centrum Wiskunde & Informatica (CWI) Science Park 123, 1098 XG Amsterdam, The Netherlands room L3.14, phone ++31 20 592-4098 sip:4098@sip.cwi.nl url: https://www.cwi.nl/people/niels e-mail: Niels.Nes@cwi.nl

Martin Schwitalla

19 Aug 19 Aug

11:33 a.m.

Hi, I didn`t tried the RC, but will it be able to just copy/paste generated MAL-Code and execute it, when it releases? Kind regards, Martin

...

Am 14.08.2015 um 11:46 schrieb Niels Nes :

On Thu, Aug 13, 2015 at 09:33:41AM +0200, Sjoerd Mullender wrote:

...
https://www.monetdb.org/downloads/testing/relnotes.html

On 12/08/15 19:54, Richard Hughes wrote:

...
On Mon, 10 Aug 2015 15:25:52 +0200, Sjoerd Mullender wrote:

...
I have created a release candidate for the Jul2015 release. The candidate is available at http://dev.monetdb.org/downloads/testing/. If all goes well, this candidate will morph into a release sometime soon(ish). In the mean time, you can use this release candidate and test to see whether it works. And if it doesn't work, I would really like to hear about it.

Do you have a preliminary list of new features in this release? I'm not going to test features that I don't know exist...

I've had my eye on the 'merge tables' work. The documentation at https://www.monetdb.org/Documentation/Cookbooks/SQLrecipes/DataPartiti oning

is clear on the usage of the feature, but can somebody knowledgeable

...
comment on the advantages/disadvantages relative to the existing approach of using a view of unions? Basically I want to know whether it's worth rewriting my well-tested existing code to make use of the new facility. Richard

I'd expect your application specific solution to always out perform a more general solution. But afcourse we are interested in more testing/comparisons, so if you have some time please try also with the new merge table support (possibly also with remote and replica tables).

The merge tables are indeed just a union all kind of view, with the difference that we know the parts and therefor can more easily apply relational rewrites over the parts. For now these are limited too 'readonly' marked partitions. These partitions could be eliminated in the rewrite and could greatly reduce the size of the mal plans. We use statistics on those readonly parts (obtained via analyze) to do the rewrites.

Niels

...
...
Thanks,

Richard. _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

-- Sjoerd Mullender _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

-- Niels Nes, Manager ITF, Centrum Wiskunde & Informatica (CWI) Science Park 123, 1098 XG Amsterdam, The Netherlands room L3.14, phone ++31 20 592-4098 sip:4098@sip.cwi.nl url: https://www.cwi.nl/people/niels e-mail: Niels.Nes@cwi.nl _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

3490

Age (days ago)

3499

Last active (days ago)

List overview

Download

13 comments

6 participants

participants (6)

Anthony Damico
Martin Schwitalla
Niels Nes
Richard Hughes
Sjoerd Mullender
Stefan Manegold