November 2007 - users-list

[MonetDB-users] Memory Leak in mserver5.exe
by Markus Gritsch 24 Nov '07

24 Nov '07

Hi, I have a medium sized dataset of about 50 MB text data. When I repeatedly call SELECT * FROM entry WHERE LOWER(note) LIKE '%hui%' I get at some time not my result, but the following error in DbVisualizer: protocol violation, unexpected line: VirtualAlloc(1a0a0000,29806592,MEM_COMMIT,PAGE_READWRITE): failed Repeatedly calling the query from the command line like this mclient.exe -lsql -s"SELECT * FROM entry WHERE LOWER(note) LIKE '%hui%'" > result.txt produces the attached memory usage graph. After 11 times calling the query the memory is full, and some kind of garbage collection seems to happen. However, after only 6 additional times the memory is again full and the garbage collection is quite less effective than before. Subsequent calls reach the memory limit every time. Markus P.S.: I have disabled the virtual memory on my Windows XP because when the OS starts paging, it becomes unbearably slow.

2 5

[MonetDB-users] New MonetDB Bug Fix Release
by Sjoerd Mullender 24 Nov '07

24 Nov '07

The MonetDB team at CWI is pleased to announce a new bug fix release of the MonetDB suite of programs. The following packages have been released: MonetDB Clients 1.20.2 MonetDB5 Server 5.2.2 MonetDB SQL 2.20.2 There is of course also a new "Super Source Ball", a tarball containing the combined sources. All this is available on our SourceForge project page. See https://sourceforge.net/project/showfiles.php?group_id=56967 for the downloadable files. See http://monetdb.cwi.nl/projects/monetdb/Development/Releases/Version4.20/ind… for a list of fixes. -- Sjoerd Mullender

1 0

[MonetDB-users] MonetSQLdb and Unicode
by Markus Gritsch 23 Nov '07

23 Nov '07

Hi, is the Python module MonetSQLdb expected to handle unicode? Currently when INSERTing I must make sure that Python unicode types get converted to strings using UTF-8 encoding. Likewise when SELECTing I get UTF-8 encoded strings back. BTW, is it possible to use another encoding than UTF-8 with MonetDB? The Python DB API for SQLite allows inserting Python unicode types and yields unicode when selecting. MySQLdb allows specifying a 'charset' parameter in the connection constructor and uses this charset to automatically encode unicode objects when inserting. It also returns unicode objects. The following script raises an exception when used with MonetDB: import MonetSQLdb x = MonetSQLdb.connect() c = x.cursor() c.execute('DROP TABLE test') c.execute('CREATE TABLE test(id int, name varchar(20))') c.execute('INSERT INTO test (id, name) VALUES (%s, %s)', (0, u'm')) c.execute("SELECT * FROM test") value = c.fetchone()[-1] print type(value) c.close() x.close() --- Traceback (most recent call last): File "test.py", line 8, in <module> c.execute('INSERT INTO test (id, name) VALUES (%s, %s)', (0, u'm')) File "C:\Python25\lib\site-packages\mdb\MonetSQLdb\cursors.py", line 95, in execute self.errorhandler(self, exc, value) File "C:\Python25\lib\site-packages\mdb\MonetSQLdb\__init__.py", line 77, in defaulterrorhandler raise errorclass, errorvalue RuntimeError: !SELECT: identifier 'm' unknown

2 1

[MonetDB-users] No useful error message on SQL syntax error
by Markus Gritsch 23 Nov '07

23 Nov '07

Hi, when using MonetSQLdb from Python and the executed SQL contains a syntax error, there is no useful information displayed in the traceback. The following script for example import MonetSQLdb x = MonetSQLdb.connect() c = x.cursor() c.execute('DROP TABLE test') c.execute('DROP TABLE test') yield this traceback: Traceback (most recent call last): File "testMonetDB.py", line 9, in <module> c.execute('DROP TABLE test') File "C:\Python25\lib\site-packages\mdb\MonetSQLdb\cursors.py", line 95, in execute self.errorhandler(self, exc, value) File "C:\Python25\lib\site-packages\mdb\MonetSQLdb\__init__.py", line 72, in defaulterrorhandler raise errorclass, errorvalue RuntimeError: None The Message is None whereas it should be DROP TABLE: no such table 'test' Kind regards, Markus

2 2

[MonetDB-users] bug 1834820 AUTO_INCREMENT failures...
by m h 22 Nov '07

22 Nov '07

Hello again - I've filed another bug here.... http://sourceforge.net/tracker/index.php?func=detail&aid=1834820&group_id=5… Similar to the NUMERIC not allowing initial NULLs, AUTO_INCREMENT won't allow skipping of Primary Keys (the first time after, the second time it works) after you've inserted with a Primary Key... -matt

2 3

[MonetDB-users] Merovingian
by Venks 21 Nov '07

21 Nov '07

Hi, I have been using some old scripts to START and STOP MonetDB instances. One of our requirements is to not to use default database and default directories for data/logs. So, I specify the data/log/checkpoint and the mapi_port explicitly when I bring up the instance using "mserver5". Please find enclosed the actual command from the START script below. #...Launch the database /root/countdb/bin/mserver5 --dbfarm=/mnt/countdb1/dbfarm --dbname=csdb --dbinit='include sql;' --set sql_logdir=/mnt/countdb1/sql_logs --set checkpoint_dir=/mnt/countdb1/chkpnt --set mapi_port=50001 --set daemon=yes --set monet_welcom=no --set MONETDB5_PREFIX=$MDIR >> $LOG 2>&1 & I understand the "merovingian" is the new and updated program to handle instance administration issues. Can you please provide more information on how I can use "merovingian" to START/STOP MonetDB instances? Is there online help for "merovingian" apart from what mentioned in the configuration file? Like "mserver5 --help" I was expecting "merovingian --help" but that doesn't work. Thanks, -Venks

4 5

[MonetDB-users] Running multiple instances on same machine
by Venks 21 Nov '07

21 Nov '07

Hi, Can you please tell me how to run three different MonetDB instances on one machine? I created three different binaries from the install and I tried to change the port from 50000 to 50001, 50002 and 50003. But for some reason I can only connect using 50000. Thanks, -Venks

3 3

[MonetDB-users] Bug 1833800 - python client not getting actual errors, only None
by m h 19 Nov '07

19 Nov '07

I've filed bug 1833800. It was reported in passing in a previous thread that the python client isn't getting the acutal errors. A fix for this wonderful! thanks much, matt

2 3

[MonetDB-users] python issues
by m h 17 Nov '07

17 Nov '07

Hello- The simple python example from the website is crashing [0]. I get the following output when I try to run it: matt@r52 ~/work/II/svn/trunk/2MonetDB $ python testmdb.py (981, 'schemas', 980, None, '0', True, '0', '0') *** glibc detected *** python: free(): invalid pointer: 0x080cf8d8 *** ======= Backtrace: ========= /lib/libc.so.6[0xb7cdea00] /lib/libc.so.6(cfree+0x89)[0xb7ce06f9] /usr/lib/libMapi.so.0(mapi_destroy+0x77)[0xb7b4a40f] /home/matt/work/vpython/lib/python2.4/site-packages/MonetDB/_MapiLib.so[0xb7fa25a5] /usr/lib/libpython2.4.so.1.0(PyCFunction_Call+0x10d)[0xb7eef420] /usr/lib/libpython2.4.so.1.0(PyEval_EvalFrame+0x3259)[0xb7f1c7b5] /usr/lib/libpython2.4.so.1.0(PyEval_EvalCodeEx+0x62e)[0xb7f1e17a] /usr/lib/libpython2.4.so.1.0[0xb7ee1d0c] /usr/lib/libpython2.4.so.1.0(PyObject_Call+0x27)[0xb7ece52e] /usr/lib/libpython2.4.so.1.0[0xb7ed378e] /usr/lib/libpython2.4.so.1.0(PyObject_Call+0x27)[0xb7ece52e] /usr/lib/libpython2.4.so.1.0(PyEval_CallObjectWithKeywords+0xcc)[0xb7f18c55] /usr/lib/libpython2.4.so.1.0[0xb7ed60b5] /usr/lib/libpython2.4.so.1.0(PyDict_DelItem+0xc8)[0xb7eee29a] /usr/lib/libpython2.4.so.1.0[0xb7ed5755] /usr/lib/libpython2.4.so.1.0(PyObject_SetAttr+0xbc)[0xb7ef098b] /usr/lib/libpython2.4.so.1.0(PyEval_EvalFrame+0x23b5)[0xb7f1b911] /usr/lib/libpython2.4.so.1.0(PyEval_EvalCodeEx+0x62e)[0xb7f1e17a] /usr/lib/libpython2.4.so.1.0[0xb7ee1d0c] /usr/lib/libpython2.4.so.1.0(PyObject_Call+0x27)[0xb7ece52e] /usr/lib/libpython2.4.so.1.0[0xb7ed378e] /usr/lib/libpython2.4.so.1.0(PyObject_Call+0x27)[0xb7ece52e] /usr/lib/libpython2.4.so.1.0(PyEval_CallObjectWithKeywords+0xcc)[0xb7f18c55] /usr/lib/libpython2.4.so.1.0[0xb7ed60b5] /usr/lib/libpython2.4.so.1.0[0xb7eec587] /usr/lib/libpython2.4.so.1.0(PyDict_SetItem+0xb0)[0xb7eee75d] /usr/lib/libpython2.4.so.1.0[0xb7ed579f] /usr/lib/libpython2.4.so.1.0(PyObject_SetAttr+0xbc)[0xb7ef098b] /usr/lib/libpython2.4.so.1.0(PyEval_EvalFrame+0x2372)[0xb7f1b8ce] /usr/lib/libpython2.4.so.1.0(PyEval_EvalFrame+0x3390)[0xb7f1c8ec] /usr/lib/libpython2.4.so.1.0(PyEval_EvalCodeEx+0x62e)[0xb7f1e17a] /usr/lib/libpython2.4.so.1.0[0xb7ee1d0c] /usr/lib/libpython2.4.so.1.0(PyObject_Call+0x27)[0xb7ece52e] /usr/lib/libpython2.4.so.1.0[0xb7ed378e] /usr/lib/libpython2.4.so.1.0(PyObject_Call+0x27)[0xb7ece52e] /usr/lib/libpython2.4.so.1.0(PyEval_CallObjectWithKeywords+0xcc)[0xb7f18c55] /usr/lib/libpython2.4.so.1.0[0xb7ed60b5] /usr/lib/libpython2.4.so.1.0[0xb7eec587] /usr/lib/libpython2.4.so.1.0(PyDict_SetItem+0xb0)[0xb7eee75d] /usr/lib/libpython2.4.so.1.0(_PyModule_Clear+0x148)[0xb7eef9c3] /usr/lib/libpython2.4.so.1.0(PyImport_Cleanup+0x1ee)[0xb7f2fc18] /usr/lib/libpython2.4.so.1.0(Py_Finalize+0xf9)[0xb7f35261] /usr/lib/libpython2.4.so.1.0(Py_Main+0x9ad)[0xb7f3b1e9] python(main+0x1a)[0x804866e] /lib/libc.so.6(__libc_start_main+0xdc)[0xb7c8efdc] python[0x80485b1] ======= Memory map: ======== 08048000-08049000 r-xp 00000000 08:03 1772622 /home/matt/work/vpython/bin/python 08049000-0804a000 rw-p 00000000 08:03 1772622 /home/matt/work/vpython/bin/python 0804a000-080f5000 rw-p 0804a000 00:00 0 [heap] b7800000-b7821000 rw-p b7800000 00:00 0 b7821000-b7900000 ---p b7821000 00:00 0 b79e1000-b79e9000 r-xp 00000000 08:03 4735072 /lib/libnss_files-2.6.1.so b79e9000-b79ea000 r--p 00007000 08:03 4735072 /lib/libnss_files-2.6.1.so b79ea000-b79eb000 rw-p 00008000 08:03 4735072 /lib/libnss_files-2.6.1.so b79eb000-b79f6000 r-xp 00000000 08:03 4669940 /lib/libbz2.so.1.0.4 b79f6000-b79f7000 r--p 0000a000 08:03 4669940 /lib/libbz2.so.1.0.4 b79f7000-b79f8000 rw-p 0000b000 08:03 4669940 /lib/libbz2.so.1.0.4 b79f8000-b7a06000 r-xp 00000000 08:03 2621464 /lib/libz.so.1.2.3 b7a06000-b7a07000 rw-p 0000d000 08:03 2621464 /lib/libz.so.1.2.3 b7a07000-b7a0d000 r-xp 00000000 08:03 541522 /usr/lib/libstream.so.0.0.0 b7a0d000-b7a0e000 r--p 00005000 08:03 541522 /usr/lib/libstream.so.0.0.0 b7a0e000-b7a0f000 rw-p 00006000 08:03 541522 /usr/lib/libstream.so.0.0.0 b7a0f000-b7af6000 r-xp 00000000 08:03 3432882 /usr/lib/libcrypto.so.0.9.8 b7af6000-b7b0b000 rw-p 000e6000 08:03 3432882 /usr/lib/libcrypto.so.0.9.8 b7b0b000-b7b0e000 rw-p b7b0b000 00:00 0 b7b0e000-b7b3d000 r-xp 00000000 08:03 3432885 /usr/lib/libssl.so.0.9.8 b7b3d000-b7b41000 rw-p 0002e000 08:03 3432885 /usr/lib/libssl.so.0.9.8 b7b41000-b7b4c000 r-xp 00000000 08:03 541530 /usr/lib/libMapi.so.0.0.0 b7b4c000-b7b4d000 r--p 0000a000 08:03 541530 /usr/lib/libMapi.so.0.0.0 b7b4d000-b7b4e000 rw-p 0000b000 08:03 541530 /usr/lib/libMapi.so.0.0.0 b7b73000-b7c79000 rw-p b7b73000 00:00 0 b7c79000-b7da3000 r-xp 00000000 08:03 4735105 /lib/libc-2.6.1.so b7da3000-b7da5000 r--p 0012a000 08:03 4735105 /lib/libc-2.6.1.so b7da5000-b7da6000 rw-p 0012c000 08:03 4735105 /lib/libc-2.6.1.so b7da6000-b7da9000 rw-p b7da6000 00:00 0 b7da9000-b7db2000 r-xp 00000000 08:03 1492902 /usr/lib/gcc/i686-pc-linux-gnu/4.1.2/libgcc_s.so.1 b7db2000-b7db3000 rw-p 00009000 08:03 1492902 /usr/lib/gcc/i686-pc-linux-gnu/4.1.2/libgcc_s.so.1 b7db3000-b7db4000 rw-p b7db3000 00:00 0 b7db4000-b7dd8000 r-xp 00000000 08:03 4735063 /lib/libm-2.6.1.so b7dd8000-b7dd9000 r--p 00023000 08:03 4735063 /lib/libm-2.6.1.so b7dd9000-b7dda000 rw-p 00024000 08:03 4735063 /lib/libm-2.6.1.so b7dda000-b7e85000 r-xp 00000000 08:03 1712627 /usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.6/libstdc++.so.5.0.7 b7e85000-b7e8a000 rw-p 000aa000 08:03 1712627 /usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.6/libstdc++.so.5.0.7 b7e8a000-b7e8f000 rw-p b7e8a000 00:00 0 b7e8f000-b7e91000 r-xp 00000000 08:03 4735083 /lib/libutil-2.6.1.so b7e91000-b7e92000 r--p 00001000 08:03 4735083 /lib/libutil-2.6.1.so b7e92000-b7e93000 rw-p 00002000 08:03 4735083 /lib/libutil-2.6.1.so b7e93000-b7e95000 r-xp 00000000 08:03 4735065 /lib/libdl-2.6.1.so b7e95000-b7e96000 r--p 00001000 08:03 4735065 /lib/libdl-2.6.1.so b7e96000-b7e97000 rw-p 00002000 08:03 4735065 /lib/libdl-2.6.1.so b7e97000-b7eaa000 r-xp 00000000 08:03 4735068 /lib/libpthread-2.6.1.so b7eaa000-b7eab000 r--p 00013000 08:03 4735068 /lib/libpthread-2.6.1.so b7eab000-b7eac000 rw-p 00014000 08:03 4735068 /lib/libpthread-2.6.1.so b7eac000-b7eae000 rw-p b7eac000 00:00 0 b7eae000-b7f62000 r-xp 00000000 08:03 2050159 /usr/lib/libpython2.4.so.1.0 b7f62000-b7f63000 r--p 000b3000 08:03 2050159 /usr/lib/libpython2.4.so.1.0 b7f63000-b7f85000 rw-p 000b4000 08:03 2050159 /usr/lib/libpython2.4.so.1.0 b7f85000-b7f8b000 rw-p b7f85000 00:00 0 b7f8b000-b7f8f000 r-xp 00000000 08:03 4166255 /usr/lib/python2.4/lib-dynload/strop.so b7f8f000-b7f90000 r--p 00003000 08:03 4166255 /usr/lib/python2.4/lib-dynload/strop.so b7f90000-b7f92000 rw-p 00004000 08:03 4166255 /usr/lib/python2.4/lib-dynload/strop.so b7f92000-b7f94000 r-xp 00000000 08:03 541562 /usr/lib/libmutils.so.0.0.0 b7f94000-b7f95000 r--p 00001000 08:03 541562 /usr/lib/libmutils.so.0.0.0 b7f95000-b7f96000 rw-p 00002000 08:03 541562 /usr/lib/libmutils.so.0.0.0 b7f96000-b7fab000 r-xp 00000000 08:03 606460 /usr/lib/python2.4/site-packages/MonetDB/_MapiLib.so.0.0.0 b7fab000-b7fac000 r--p 00014000 08:03 606460 /usr/lib/python2.4/site-packages/MonetDB/_MapiLib.so.0.0.0 b7fac000-b7fad000 rw-p 00015000 08:03 606460 /usr/lib/python2.4/site-packages/MonetDB/_MapiLib.so.0.0.0 b7fad000-b7fae000 rw-p b7fad000 00:00 0 b7fb1000-b7fb2000 r-xp b7fb1000 00:00 0 [vdso] b7fb2000-b7fcc000 r-xp 00000000 08:03 4735086 /lib/ld-2.6.1.so b7fcc000-b7fcd000 r--p 00019000 08:03 4735086 /lib/ld-2.6.1.so b7fcd000-b7fce000 rw-p 0001a000 08:03 4735086 /lib/ld-2.6.1.so bff36000-bff58000 rw-p bff36000 00:00 0 [stack] Aborted matt@r52 ~/work/II/svn/trunk/2MonetDB $ I'm running the latest ebuild for gentoo. Any suggestions to remove crashing would be great. thanks much, matt 0 -http://monetdb.cwi.nl/projects/monetdb/SQL/Documentation/The-Python-Library.html#The-Python-Library

5 17

[MonetDB-users] (some) Numeric columns don't allow NULL?
by m h 16 Nov '07

16 Nov '07

Hello again, I've got a column declared as such: foo NUMERIC(9, 2), When I try and enter NULL values into it, it gives the following error: !SQLException:int:"" is no decimal value (doesn't contain a '.') Now, the message itself had me confused (still does actually), but I figured out by trial that it was complaining that I was shoving a NULL value in that column. Can someone explain why a NULL value can't go it here? (There is no "NOT NULL" on the column) Also columns defined as "NUMERIC(7,0)" allow NULL values..... Confused..... -matt

2 4