- users-list - monetdb.org

Crashed DB - BBPextend: trying to extend BAT pool beyond the limit
by Pedro Salgueiro 08 Oct '13

08 Oct '13

Hi, We have a MonetDB instance which was working without problems and suddenly stopped working. The following error shows up in "merovingian.log" when the client(mclient) tries to connect to the database: 2013-10-07 10:35:27 ERR testbd[12111]: !FATAL: BBPextend: trying to extend > BAT pool beyond the limit (16384000) > 2013-10-07 10:35:27 ERR testbd[12111]: > 2013-10-07 10:35:27 MSG merovingian[12015]: database 'testbd' (12111) has > exited with exit status 1 > 2013-10-07 10:35:27 ERR merovingian[12015]: client error: database > 'testbd' has crashed after starting, manual intervention needed, check > monetdbd's logfile for details > 2013-10-07 10:35:27 MSG merovingian[12015]: database 'testbd' has crashed > after start on 2013-10-02 16:36:06, attempting restart, up min/avg/max: > 0s/0s/0s, crash average: 1.00 1.00 0.83 (25-0=25) The DB size is about 1.1 GB. I really don't know how many tables, columns or records this specific database has, as it was being used for a while and the data is created/uploaded daily by its users. I tried with Feb2013-SP2, Feb2013-SP3 and Feb2013-SP4 I have a few question about this issue: 1 - Is there any way to see a more detailed log about the error? 2 - Is it possible to recover the DB from this error? I saw somewhere in the mailing list that removing the sql_logs could help starting the DB, at the cost of loosing some data. Already tried that with no success. 3 - What is this "BAT pool limit" thing? Is this related in some way with the number of tables, number of columns per table, or total amount of columns that a DB can have? Thank you, Pedro Salgueiro

3 8

ORDER BY in sub-query
by Hadley Wickham 08 Oct '13

08 Oct '13

Currently, monetdb fails on this query: SELECT * FROM (SELECT * FROM "Batting" ORDER BY "playerID") with syntax error, unexpected ORDER, expecting INTERSECT or EXCEPT or UNION or ')' in: "select * from (select * (obviously that query isn't very useful, but it makes it harder to automatically generate sql) In case it matters, that query (with minor escaping variations) works in sqlite, postgres and mysql. Hadley -- Chief Scientist, RStudio http://had.co.nz/

5 17

Create database other then "demo" in windows 8
by Prashant Patel 07 Oct '13

07 Oct '13

Hi all, I am using MonetDB on my windows 8 platform every time open monetdb client and server , it automatically connects to "demo" database by default. I am very new to these technology I have searched but couldn't find how to create our own database in monetdb (other then "demo"). Can anyone tell me how to create database and connect that database in monetdb server and client. I know this is very dump question but help will be really appreciated. ( If i can get some documents that would also work) Thanks in advance, Prashant Rajkotiya Research Aspirant

2 2

How to take COLD RUNs Execution time
by Prashant Patel 07 Oct '13

07 Oct '13

as a part of my project, i have to check the performance of monetdb column-store database. I am very new to this technology, so i dont know where the monetdb caches its previous query results.. I want to know how can i take results for cold runs on monetdb ( i have tried closing monetdb client and server every time i fire a query , but doesnt seem to clear the cache and i always get hot runs result). I am using window 8 core i5 2.5GHz machine. so please can anyone help me regarding these.... suggestions are appreciated.. thanks in advance.. Prashant Rajkotiya M.tech. (ICT) DA-IICT ,Gandhinagar

2 2

Window functions
by Hadley Wickham 05 Oct '13

05 Oct '13

Hi all, Does monetdb support window functions? I found https://www.monetdb.org/pipermail/users-list/2011-June/004992.html, which claims that OVER is supported, but: * I can't see where support is listed on http://www.monetdb.org/print/book/export/html/63 * I get errors when I try and use window functions: sql> SELECT *, MAX("AB") OVER (PARTITION BY "playerID") AS "_W4", MAX("G") OVER (PARTITION BY "playerID") FROM "Batting"; SELECT: function 'max' not found But that might be because I don't have the syntax right (but that query works in Postgres) Are there are any plans to add standard window functions like rank, ntile, lead, lag etc to monetdb? They are pretty useful for analysis. Hadley -- Chief Scientist, RStudio http://had.co.nz/

2 1

new "too many digits" bug in SP-4
by Anthony Damico 03 Oct '13

03 Oct '13

this issue is new for SP-4 ..i have been using this syntax for a long time and never encountered this problem until the upgrade :( > dbGetQuery( db , "select ( sum( ( bene_sex_ident_cd = 1 ) ) ) / ( count( * ) ) as pct_male from x11" ) pct_male 1 0 > dbGetQuery( db , "select ( sum( ( bene_sex_ident_cd = 1 ) )*1.000 ) / ( count( * )*1.000 ) as pct_male from x11" ) Error in .local(conn, statement, ...) : Unable to execute statement 'select ( sum( ( bene_sex_ident_cd = 1 ) )*1.000 ) / ( count( * )*1.000 ) as pct_male from x11'. Server says 'too many digits (19 > 18)' [#22003]. > what diagnostics can i run for you to troubleshoot this? thanks!! :)

2 7

Bug with fetching EXPLAIN results in Python DBAPI driver.
by Pete Hollobon 03 Oct '13

03 Oct '13

Hi, I spotted a bug in the Python DBAPI driver when fetching results from an EXPLAIN. >>> import monetdb.sql >>> db = monetdb.sql.connect(database="demo") >>> cursor = db.cursor() >>> cursor.execute("explain select count(*) from sys.tables") 31 >>> cursor.fetchone() Traceback (most recent call last): File "<stdin>", line 1, in <module> File "monetdb/sql/cursors.py", line 203, in fetchone result = self.__rows[self.rownumber - self.__offset] IndexError: list index out of range >>> I've logged a bug with patches for Python 2 and 3 here: http://bugs.monetdb.org/show_bug.cgi?id=3380 thanks, Pete.

1 0

Static binary compilation
by Hassan, Ahmad 01 Oct '13

01 Oct '13

Hi, May be someone has tried compiling monetdb as a static binary please? I tried compiling the binary using -static flag but the resulting binary is still 'Dynamically linked' binary. I think it is due to glibc and dl libraries. If I pass libglibc.a and libdl.a statically then I get errors like: ============= /tmp/mserver5-rkIbiC.o: In function `mdlopen': /data/ahmad/test/MonetDB/gdk/gdk_posix.c:454: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/lib/x86_64-linux-gnu/libreadline.a(complete.o): In function `rl_username_completion_function': (.text+0x2bc9): warning: Using 'getpwent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking ... /data/ahmad/test/MonetDB/common/stream/stream.c:1765: warning: Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking ============= Does anyone know the workaround of creating static binary of monetdb executable please? Cheers, Ahmad

2 1

new gdk bug (on windows) that might've been introduced by Feb2013-SP4
by Anthony Damico 30 Sep '13

30 Sep '13

if you run this R code (with MonetDB.R), you will automatically load a series of brazilian census bureau data sets onto your computer. after all tables have been downloaded and imported, the R script will run a very simple CREATE TABLE command that will cause mserver to shut down when it should not. library(downloader) setwd( "C:/My Directory/CENSO/" ) source_url( " https://raw.github.com/ajdamico/usgsd/master/Censo%20Demografico/download%2…" , prompt = FALSE , echo = TRUE ) after mserver.exe crashes, even if i re-load mserver.exe manually and then connect to it with mclient.exe, and run this block of code: create table c10_dom_pre_fpc as (SELECT * FROM ac_dom) UNION ALL (SELECT * FROM al_dom) UNION ALL (SELECT * FROM am_dom) UNION ALL (SELECT * FROM ap_dom) UNION ALL (SELECT * FROM ba_dom) UNION ALL (SELECT * FROM ce_dom) UNION ALL (SELECT * FROM df_dom) UNION ALL (SELECT * FROM es_dom) UNION ALL (SELECT * FROM go_dom) UNION ALL (SELECT * FROM ma_dom) UNION ALL (SELECT * FROM mg_dom) UNION ALL (SELECT * FROM ms_dom) UNION ALL (SELECT * FROM mt_dom) UNION ALL (SELECT * FROM pa_dom) UNION ALL (SELECT * FROM pb_dom) UNION ALL (SELECT * FROM pe_dom) UNION ALL (SELECT * FROM pi_dom) UNION ALL (SELECT * FROM pr_dom) UNION ALL (SELECT * FROM rj_dom) UNION ALL (SELECT * FROM rn_dom) UNION ALL (SELECT * FROM ro_dom) UNION ALL (SELECT * FROM rr_dom) UNION ALL (SELECT * FROM rs_dom) UNION ALL (SELECT * FROM sc_dom) UNION ALL (SELECT * FROM se_dom) UNION ALL (SELECT * FROM sp1_dom) UNION ALL (SELECT * FROM sp2_rm_dom) UNION ALL (SELECT * FROM to_dom ) WITH DATA; the exact same crash occurs, so this is a new mserver.exe bug not a MonetDB.R bug. should i file a bug report? thanks!!! in mserver---- # MonetDB 5 server v11.15.15 "Feb2013-SP4" # Serving database 'censo_demografico', using 8 threads # Compiled for x86_64-pc-winnt/64bit with 64bit OIDs dynamically linked # Found 7.860 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2013 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information # Listening for connection requests on mapi:monetdb://127.0.0.1:50011/ # MonetDB/JAQL module loaded # MonetDB/SQL module loaded >= .\..\..\gdk\gdk_posix.c:800: MT_mremap(bat\02\65\26501.tail,468c50000,1889785 61024,755914244096): GDKextend() failed in mclient---- C:\Windows\System32>cd .. C:\Windows>cd .. C:\>cd "Program Files" C:\Program Files>cd MonetDB C:\Program Files\MonetDB>mclient censo_demografico -p 50011 'mclient' is not recognized as an internal or external command, operable program or batch file. C:\Program Files\MonetDB>mclient 'mclient' is not recognized as an internal or external command, operable program or batch file. C:\Program Files\MonetDB>ls 'ls' is not recognized as an internal or external command, operable program or batch file. C:\Program Files\MonetDB>dir Volume in drive C has no label. Volume Serial Number is 882E-FD8A Directory of C:\Program Files\MonetDB 09/06/2013 04:34 PM <DIR> . 09/06/2013 04:34 PM <DIR> .. 09/27/2013 12:27 PM <DIR> MonetDB5 0 File(s) 0 bytes 3 Dir(s) 451,568,218,112 bytes free C:\Program Files\MonetDB>cd MonetDB5 C:\Program Files\MonetDB\MonetDB5>mclient censo_demografico -p 50011 user(win32):monetdb password: Welcome to mclient, the MonetDB/SQL interactive terminal (Feb2013-SP4) Database: MonetDB v11.15.15 (Feb2013-SP4), 'censo_demografico' Type \q to quit, \? for a list of available commands auto commit mode: on sql>create table c10_dom_pre_fpc as (SELECT * FROM ac_dom) UNION ALL (SELECT * F ROM al_dom) UNION ALL (SELECT * FROM am_dom) UNION ALL (SELECT * FROM ap_dom) UN ION ALL (SELECT * FROM ba_dom) UNION ALL (SELECT * FROM ce_dom) UNION ALL (SELEC T * FROM df_dom) UNION ALL (SELECT * FROM es_dom) UNION ALL (SELECT * FROM go_do m) UNION ALL (SELECT * FROM ma_dom) UNION ALL (SELECT * FROM mg_dom) UNION ALL ( SELECT * FROM ms_dom) UNION ALL (SELECT * FROM mt_dom) UNION ALL (SELECT * FROM pa_dom) UNION ALL (SELECT * FROM pb_dom) UNION ALL (SELECT * FROM pe_dom) UNION ALL (SELECT * FROM pi_dom) UNION ALL (SELECT * FROM pr_dom) UNION ALL (SELECT * FROM rj_dom) UNION ALL (SELECT * FROM rn_dom) UNION ALL (SELECT * FROM ro_dom) U NION ALL (SELECT * FROM rr_dom) UNION ALL (SELECT * FROM rs_dom) UNION ALL (SELE CT * FROM sc_dom) UNION ALL (SELECT * FROM se_dom) UNION ALL (SELECT * FROM sp1_ dom) UNION ALL (SELECT * FROM sp2_rm_dom) UNION ALL (SELECT * FROM to_dom ) WITH DATA; GDK reported error. HEAPextend: failed to extend to 755914244096 for 02\65\26501.tail: MT_mremap() f ailed sql> sql>

3 4

MonetDB compile error 11.15.15
by Brian Hood 28 Sep '13

28 Sep '13

Hi Guys, I just trying to update my Arch Linux package and when building came up with this compile error the package passed the sha1sum after download. Hope someone can help. In file included from /usr/include/string.h:25:0, from /var/abs/AUR/monetdb/src/MonetDB-11.15.15/monetdb5/extras/jaql/parser/jaql.y:23: /usr/include/features.h:327:4: warning: #warning _FORTIFY_SOURCE requires compiling with optimization (-O) [-Wcpp] # warning _FORTIFY_SOURCE requires compiling with optimization (-O) ^ y.tab.c: In function 'jaqlparse': y.tab.c:1665:32: error: 'scanner' undeclared (first use in this function) y.tab.c:1665:32: note: each undeclared identifier is reported only once for each function it appears in make[9]: *** [libjaqlp_la-jaql.tab.lo] Error 1 Regards, Brian Hood

1 0