September 2012 - users-list

[MonetDB-users] invalid tuple received from server, got 138 columns, expected 476, ignoring
by Anthony Damico 26 Sep '12

26 Sep '12

Hello, I'm pretty new to MonetDB and I'm trying to perform what seems like it should be a relatively straightforward inner join between two tables. I would like to merge the two Alaska files from the American Community Survey -- http://www2.census.gov/acs2010_1yr/pums/csv_hak.zip contains a CSV file with one record per household http://www2.census.gov/acs2010_1yr/pums/csv_pak.zip contains a CSV file with one record per personThese two files should be merged on the field "serialno" I am working on Windows 7 x64. Here's the contents of my mserver5.exe window: # MonetDB 5 server v11.11.11 "Jul2012-SP2" # Serving database 'demo', using 2 threads # Compiled for x86_64-pc-winnt/64bit with 64bit OIDs dynamically linked # Found 3.905 GiB available main-memory. # Copyright (c) 1993-July 2008 CWI. # Copyright (c) August 2008-2012 MonetDB B.V., all rights reserved # Visit http://www.monetdb.org/ for further information # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/ # MonetDB/SQL module loaded > The following commands.. 1) create a person-level table 2) create a household-level table 3) read the person-level CSV into the person table 4) read the household-level CSV into the household table 5) attempt to print the first five records with "select * from tablename limit 5" -- sometimes these work, sometimes they don't. 6) merge the two tables together 7) attempt to print the first five records of the merged table Unfortunately, I'm getting an error that I can't find with web searches.. The only place I can even find this error is http://bugs.monetdb.org/show_bug.cgi?id=2963 -- and that seems both time-related and fixed long ago.. I doubt it's a bug, I'm probably just making a beginner's mistake. invalid tuple received from server, got 138 columns, expected 476, ignoring +------+------+------+------+-------+------+---------+---------+------+------+ 1 tuple (417.984ms) !466 columns dropped! note: to disable dropping columns and/or truncating fields use \w-1 Even after setting \w to -1, the number of columns returned gets limited, crashing my program. I am ultimately trying to work with the new R/MonetDB package (which uses Java and is available from http://sqlsurvey.r-forge.r-project.org/) but none of the table interactivity will work until I'm able to return all of the columns properly. Any advice would be appreciated! Thanks!!! Anthony Damico Kaiser Family Foundation Here are the SQL commands: CREATE TABLE person (RT VARCHAR(255), SERIALNO INT, SPORDER INT, PUMA INT, ST INT, ADJINC INT, PWGTP INT, AGEP INT, CIT INT, CITWP INT, COW INT, DDRS INT, DEAR INT, DEYE INT, DOUT INT, DPHY INT, DRAT INT, DRATX INT, DREM INT, ENG INT, FER INT, GCL INT, GCM INT, GCR INT, HINS1 INT, HINS2 INT, HINS3 INT, HINS4 INT, HINS5 INT, HINS6 INT, HINS7 INT, INTP INT, JWMNP INT, JWRIP INT, JWTR INT, LANX INT, MAR INT, MARHD INT, MARHM INT, MARHT INT, MARHW INT, MARHYP INT, MIG INT, MIL INT, MLPA INT, MLPB INT, MLPC INT, MLPD INT, MLPE INT, MLPF INT, MLPG INT, MLPH INT, MLPI INT, MLPJ INT, MLPK INT, NWAB INT, NWAV INT, NWLA INT, NWLK INT, NWRE INT, OIP INT, PAP INT, RELP INT, RETP INT, SCH INT, SCHG INT, SCHL INT, SEMP INT, SEX INT, SSIP INT, SSP INT, WAGP INT, WKHP INT, WKL INT, WKW INT, WRK INT, YOEP INT, ANC INT, ANC1P INT, ANC2P INT, DECADE INT, DIS INT, DRIVESP INT, ESP INT, ESR INT, FOD1P INT, FOD2P INT, HICOV INT, HISP INT, INDP INT, JWAP INT, JWDP INT, LANP INT, MIGPUMA INT, MIGSP INT, MSP INT, NAICSP VARCHAR(255), NATIVITY INT, NOP INT, OC INT, OCCP INT, PAOC INT, PERNP INT, PINCP INT, POBP INT, POVPIP INT, POWPUMA INT, POWSP INT, PRIVCOV INT, PUBCOV INT, QTRBIR INT, RAC1P INT, RAC2P INT, RAC3P INT, RACAIAN INT, RACASN INT, RACBLK INT, RACNHPI INT, RACNUM INT, RACSOR INT, RACWHT INT, RC INT, SCIENGP INT, SCIENGRLP INT, SFN INT, SFR INT, SOCP VARCHAR(255), VPS INT, WAOB INT, FAGEP INT, FANCP INT, FCITP INT, FCITWP INT, FCOWP INT, FDDRSP INT, FDEARP INT, FDEYEP INT, FDOUTP INT, FDPHYP INT, FDRATP INT, FDRATXP INT, FDREMP INT, FENGP INT, FESRP INT, FFERP INT, FFODP INT, FGCLP INT, FGCMP INT, FGCRP INT, FHINS1P INT, FHINS2P INT, FHINS3C INT, FHINS3P INT, FHINS4C INT, FHINS4P INT, FHINS5C INT, FHINS5P INT, FHINS6P INT, FHINS7P INT, FHISP INT, FINDP INT, FINTP INT, FJWDP INT, FJWMNP INT, FJWRIP INT, FJWTRP INT, FLANP INT, FLANXP INT, FMARHDP INT, FMARHMP INT, FMARHTP INT, FMARHWP INT, FMARHYP INT, FMARP INT, FMIGP INT, FMIGSP INT, FMILPP INT, FMILSP INT, FOCCP INT, FOIP INT, FPAP INT, FPOBP INT, FPOWSP INT, FRACP INT, FRELP INT, FRETP INT, FSCHGP INT, FSCHLP INT, FSCHP INT, FSEMP INT, FSEXP INT, FSSIP INT, FSSP INT, FWAGP INT, FWKHP INT, FWKLP INT, FWKWP INT, FWRKP INT, FYOEP INT, pwgtp1 INT, pwgtp2 INT, pwgtp3 INT, pwgtp4 INT, pwgtp5 INT, pwgtp6 INT, pwgtp7 INT, pwgtp8 INT, pwgtp9 INT, pwgtp10 INT, pwgtp11 INT, pwgtp12 INT, pwgtp13 INT, pwgtp14 INT, pwgtp15 INT, pwgtp16 INT, pwgtp17 INT, pwgtp18 INT, pwgtp19 INT, pwgtp20 INT, pwgtp21 INT, pwgtp22 INT, pwgtp23 INT, pwgtp24 INT, pwgtp25 INT, pwgtp26 INT, pwgtp27 INT, pwgtp28 INT, pwgtp29 INT, pwgtp30 INT, pwgtp31 INT, pwgtp32 INT, pwgtp33 INT, pwgtp34 INT, pwgtp35 INT, pwgtp36 INT, pwgtp37 INT, pwgtp38 INT, pwgtp39 INT, pwgtp40 INT, pwgtp41 INT, pwgtp42 INT, pwgtp43 INT, pwgtp44 INT, pwgtp45 INT, pwgtp46 INT, pwgtp47 INT, pwgtp48 INT, pwgtp49 INT, pwgtp50 INT, pwgtp51 INT, pwgtp52 INT, pwgtp53 INT, pwgtp54 INT, pwgtp55 INT, pwgtp56 INT, pwgtp57 INT, pwgtp58 INT, pwgtp59 INT, pwgtp60 INT, pwgtp61 INT, pwgtp62 INT, pwgtp63 INT, pwgtp64 INT, pwgtp65 INT, pwgtp66 INT, pwgtp67 INT, pwgtp68 INT, pwgtp69 INT, pwgtp70 INT, pwgtp71 INT, pwgtp72 INT, pwgtp73 INT, pwgtp74 INT, pwgtp75 INT, pwgtp76 INT, pwgtp77 INT, pwgtp78 INT, pwgtp79 INT, pwgtp80 INT); CREATE TABLE household (RT VARCHAR(255), SERIALNO INT, DIVISION INT, PUMA INT, REGION INT, ST INT, ADJHSG INT, ADJINC INT, WGTP INT, NP INT, TYPE INT, ACR INT, AGS INT, BATH INT, BDSP INT, BLD INT, BUS INT, CONP INT, ELEP INT, FS INT, FULP INT, GASP INT, HFL INT, INSP INT, MHP INT, MRGI INT, MRGP INT, MRGT INT, MRGX INT, REFR INT, RMSP INT, RNTM INT, RNTP INT, RWAT INT, SINK INT, SMP INT, STOV INT, TEL INT, TEN INT, TOIL INT, VACS INT, VALP INT, VEH INT, WATP INT, YBL INT, FES INT, FINCP INT, FPARC INT, GRNTP INT, GRPIP INT, HHL INT, HHT INT, HINCP INT, HUGCL INT, HUPAC INT, HUPAOC INT, HUPARC INT, KIT INT, LNGI INT, MULTG INT, MV INT, NOC INT, NPF INT, NPP INT, NR INT, NRC INT, OCPIP INT, PARTNER INT, PLM INT, PSF INT, R18 INT, R60 INT, R65 INT, RESMODE INT, SMOCP INT, SMX INT, SRNT INT, SVAL INT, TAXP INT, WIF INT, WKEXREL INT, WORKSTAT INT, FACRP INT, FAGSP INT, FBATHP INT, FBDSP INT, FBLDP INT, FBUSP INT, FCONP INT, FELEP INT, FFSP INT, FFULP INT, FGASP INT, FHFLP INT, FINSP INT, FKITP INT, FMHP INT, FMRGIP INT, FMRGP INT, FMRGTP INT, FMRGXP INT, FMVP INT, FPLMP INT, FREFRP INT, FRMSP INT, FRNTMP INT, FRNTP INT, FRWATP INT, FSINKP INT, FSMP INT, FSMXHP INT, FSMXSP INT, FSTOVP INT, FTAXP INT, FTELP INT, FTENP INT, FTOILP INT, FVACSP INT, FVALP INT, FVEHP INT, FWATP INT, FYBLP INT, wgtp1 INT, wgtp2 INT, wgtp3 INT, wgtp4 INT, wgtp5 INT, wgtp6 INT, wgtp7 INT, wgtp8 INT, wgtp9 INT, wgtp10 INT, wgtp11 INT, wgtp12 INT, wgtp13 INT, wgtp14 INT, wgtp15 INT, wgtp16 INT, wgtp17 INT, wgtp18 INT, wgtp19 INT, wgtp20 INT, wgtp21 INT, wgtp22 INT, wgtp23 INT, wgtp24 INT, wgtp25 INT, wgtp26 INT, wgtp27 INT, wgtp28 INT, wgtp29 INT, wgtp30 INT, wgtp31 INT, wgtp32 INT, wgtp33 INT, wgtp34 INT, wgtp35 INT, wgtp36 INT, wgtp37 INT, wgtp38 INT, wgtp39 INT, wgtp40 INT, wgtp41 INT, wgtp42 INT, wgtp43 INT, wgtp44 INT, wgtp45 INT, wgtp46 INT, wgtp47 INT, wgtp48 INT, wgtp49 INT, wgtp50 INT, wgtp51 INT, wgtp52 INT, wgtp53 INT, wgtp54 INT, wgtp55 INT, wgtp56 INT, wgtp57 INT, wgtp58 INT, wgtp59 INT, wgtp60 INT, wgtp61 INT, wgtp62 INT, wgtp63 INT, wgtp64 INT, wgtp65 INT, wgtp66 INT, wgtp67 INT, wgtp68 INT, wgtp69 INT, wgtp70 INT, wgtp71 INT, wgtp72 INT, wgtp73 INT, wgtp74 INT, wgtp75 INT, wgtp76 INT, wgtp77 INT, wgtp78 INT, wgtp79 INT, wgtp80 INT); copy 50000 offset 2 records into person from 'C:\Users\user\desktop\ss10pak.csv' using delimiters ',','\n','"' NULL as '' ; copy 25000 offset 2 records into household from 'C:\Users\user\desktop\ss10hak.csv' using delimiters ',','\n','"' NULL as '' ; select * from person limit 5; select * from household limit 5; create table merged as select 'M' as rt, a.serialno, a.division, a.puma, a.region, a.st, a.adjhsg, a.adjinc, a.wgtp, a.np, a.type, a.acr, a.ags, a.bath, a.bdsp, a.bld, a.bus, a.conp, a.elep, a.fs, a.fulp, a.gasp, a.hfl, a.insp, a.mhp, a.mrgi, a.mrgp, a.mrgt, a.mrgx, a.refr, a.rmsp, a.rntm, a.rntp, a.rwat, a.sink, a.smp, a.stov, a.tel, a.ten, a.toil, a.vacs, a.valp, a.veh, a.watp, a.ybl, a.fes, a.fincp, a.fparc, a.grntp, a.grpip, a.hhl, a.hht, a.hincp, a.hugcl, a.hupac, a.hupaoc, a.huparc, a.kit, a.lngi, a.multg, a.mv, a.noc, a.npf, a.npp, a.nr, a.nrc, a.ocpip, a.partner, a.plm, a.psf, a.r18, a.r60, a.r65, a.resmode, a.smocp, a.smx, a.srnt, a.sval, a.taxp, a.wif, a.wkexrel, a.workstat, a.facrp, a.fagsp, a.fbathp, a.fbdsp, a.fbldp, a.fbusp, a.fconp, a.felep, a.ffsp, a.ffulp, a.fgasp, a.fhflp, a.finsp, a.fkitp, a.fmhp, a.fmrgip, a.fmrgp, a.fmrgtp, a.fmrgxp, a.fmvp, a.fplmp, a.frefrp, a.frmsp, a.frntmp, a.frntp, a.frwatp, a.fsinkp, a.fsmp, a.fsmxhp, a.fsmxsp, a.fstovp, a.ftaxp, a.ftelp, a.ftenp, a.ftoilp, a.fvacsp, a.fvalp, a.fvehp, a.fwatp, a.fyblp, a.wgtp1, a.wgtp2, a.wgtp3, a.wgtp4, a.wgtp5, a.wgtp6, a.wgtp7, a.wgtp8, a.wgtp9, a.wgtp10, a.wgtp11, a.wgtp12, a.wgtp13, a.wgtp14, a.wgtp15, a.wgtp16, a.wgtp17, a.wgtp18, a.wgtp19, a.wgtp20, a.wgtp21, a.wgtp22, a.wgtp23, a.wgtp24, a.wgtp25, a.wgtp26, a.wgtp27, a.wgtp28, a.wgtp29, a.wgtp30, a.wgtp31, a.wgtp32, a.wgtp33, a.wgtp34, a.wgtp35, a.wgtp36, a.wgtp37, a.wgtp38, a.wgtp39, a.wgtp40, a.wgtp41, a.wgtp42, a.wgtp43, a.wgtp44, a.wgtp45, a.wgtp46, a.wgtp47, a.wgtp48, a.wgtp49, a.wgtp50, a.wgtp51, a.wgtp52, a.wgtp53, a.wgtp54, a.wgtp55, a.wgtp56, a.wgtp57, a.wgtp58, a.wgtp59, a.wgtp60, a.wgtp61, a.wgtp62, a.wgtp63, a.wgtp64, a.wgtp65, a.wgtp66, a.wgtp67, a.wgtp68, a.wgtp69, a.wgtp70, a.wgtp71, a.wgtp72, a.wgtp73, a.wgtp74, a.wgtp75, a.wgtp76, a.wgtp77, a.wgtp78, a.wgtp79, a.wgtp80, sporder, pwgtp, agep, cit, citwp, cow, ddrs, dear, deye, dout, dphy, drat, dratx, drem, eng, fer, gcl, gcm, gcr, hins1, hins2, hins3, hins4, hins5, hins6, hins7, intp, jwmnp, jwrip, jwtr, lanx, mar, marhd, marhm, marht, marhw, marhyp, mig, mil, mlpa, mlpb, mlpc, mlpd, mlpe, mlpf, mlpg, mlph, mlpi, mlpj, mlpk, nwab, nwav, nwla, nwlk, nwre, oip, pap, relp, retp, sch, schg, schl, semp, sex, ssip, ssp, wagp, wkhp, wkl, wkw, wrk, yoep, anc, anc1p, anc2p, decade, dis, drivesp, esp, esr, fod1p, fod2p, hicov, hisp, indp, jwap, jwdp, lanp, migpuma, migsp, msp, naicsp, nativity, nop, oc, occp, paoc, pernp, pincp, pobp, povpip, powpuma, powsp, privcov, pubcov, qtrbir, rac1p, rac2p, rac3p, racaian, racasn, racblk, racnhpi, racnum, racsor, racwht, rc, sciengp, sciengrlp, sfn, sfr, socp, vps, waob, fagep, fancp, fcitp, fcitwp, fcowp, fddrsp, fdearp, fdeyep, fdoutp, fdphyp, fdratp, fdratxp, fdremp, fengp, fesrp, fferp, ffodp, fgclp, fgcmp, fgcrp, fhins1p, fhins2p, fhins3c, fhins3p, fhins4c, fhins4p, fhins5c, fhins5p, fhins6p, fhins7p, fhisp, findp, fintp, fjwdp, fjwmnp, fjwrip, fjwtrp, flanp, flanxp, fmarhdp, fmarhmp, fmarhtp, fmarhwp, fmarhyp, fmarp, fmigp, fmigsp, fmilpp, fmilsp, foccp, foip, fpap, fpobp, fpowsp, fracp, frelp, fretp, fschgp, fschlp, fschp, fsemp, fsexp, fssip, fssp, fwagp, fwkhp, fwklp, fwkwp, fwrkp, fyoep, pwgtp1, pwgtp2, pwgtp3, pwgtp4, pwgtp5, pwgtp6, pwgtp7, pwgtp8, pwgtp9, pwgtp10, pwgtp11, pwgtp12, pwgtp13, pwgtp14, pwgtp15, pwgtp16, pwgtp17, pwgtp18, pwgtp19, pwgtp20, pwgtp21, pwgtp22, pwgtp23, pwgtp24, pwgtp25, pwgtp26, pwgtp27, pwgtp28, pwgtp29, pwgtp30, pwgtp31, pwgtp32, pwgtp33, pwgtp34, pwgtp35, pwgtp36, pwgtp37, pwgtp38, pwgtp39, pwgtp40, pwgtp41, pwgtp42, pwgtp43, pwgtp44, pwgtp45, pwgtp46, pwgtp47, pwgtp48, pwgtp49, pwgtp50, pwgtp51, pwgtp52, pwgtp53, pwgtp54, pwgtp55, pwgtp56, pwgtp57, pwgtp58, pwgtp59, pwgtp60, pwgtp61, pwgtp62, pwgtp63, pwgtp64, pwgtp65, pwgtp66, pwgtp67, pwgtp68, pwgtp69, pwgtp70, pwgtp71, pwgtp72, pwgtp73, pwgtp74, pwgtp75, pwgtp76, pwgtp77, pwgtp78, pwgtp79, pwgtp80 from household as a inner join person as b on a.serialno = b.serialno with data; select * from merged limit 5; invalid tuple received from server, got 138 columns, expected 476, ignoring

2 1

[MonetDB-users] Debug sqoop-monetdb
by Jeremy O'Sullivan 25 Sep '12

25 Sep '12

Hi, I am trying to insert data into monetdb using sqoop, via the monet jdbc interface. It keeps crashing 20 odd rows into the import, with a monetdb ROLLBACK. The debug info from Sqoop is poor. Does anyone know if the monetdb ROLLBACK can be disabled, so that I can see the last row imported. Has anyone tried Sqoop to Monetdb, are there any debugging tips that you can give ? Regards, Jeremy sqoop export --connect jdbc:monetdb://cluster-03/ngpm -driver nl.cwi.monetdb.jdbc.MonetDriver --num-mappers 1 --password monetdb --username monetdb --table query_start --export-dir /test/dash_test/QueryStart.Log --fields-terminated-by ',' 12/09/24 17:27:49 INFO mapreduce.Job: Task Id : attempt_1347971665109_0247_m_000000_1, Status : FAILED Error: java.io.IOException: java.sql.SQLException: EXEC: wrong number of arguments for prepared statement: 100, expected 102 at org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.j ava:192) at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java :610) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:730) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.ja va:1232) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147) Caused by: java.sql.SQLException: EXEC: wrong number of arguments for prepared statement: 100, expected 102 at nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.executeQuery(MonetConnectio n.java:2524) at nl.cwi.monetdb.jdbc.MonetConnection$ResponseList.processQuery(MonetConnectio n.java:2274) at nl.cwi.monetdb.jdbc.MonetStatement.internalExecute(MonetStatement.java:508) at nl.cwi.monetdb.jdbc.MonetStatement.execute(MonetStatement.java:349) at nl.cwi.monetdb.jdbc.MonetPreparedStatement.execute(MonetPreparedStatement.ja va:247) at org.apache.sqoop.mapreduce.AsyncSqlOutputFormat$AsyncSqlExecThread.run(Async SqlOutputFormat.java:230)

1 0

[MonetDB-users] Announcement: New Jul2012-SP2 Bugfix release of MonetDB suite
by Sjoerd Mullender 21 Sep '12

21 Sep '12

The MonetDB team at CWI/MonetDB BV is pleased to announce the Jul2012-SP2 bugfix release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <http://www.monetdb.org/>. For details on this release, please see the release notes at <http://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <http://dev.monetdb.org/downloads/>. Jul 2012-SP2 bugfix release Java Module * Fixed a bug where DatabaseMetaData.getURL() did return null:0 for hostname:port. MonetDB5 Server * Changed the way nclients maximum was calculated to avoid 'out of client slots' errors way before the maximum was reached. Merovingian * Resolved a problem where monetdb could fail to start a database with the message "database 'X' started up, but failed to open up a communication channel". Bug #3134, comment #7. MonetDB Common * Fixed a bug in BATantijoin when either side is a singleton BAT. This fixes bug 3139. Bug Fixes * 3117: test sql/test/BugTracker-2011/Tests/history.Bug-2909.sql produces extra profiling output since changeset 817328b74e2f * 3134: Database gets killed by timeout during startup * 3139: Not equal evaluation does not work * 3144: Memleak when connecting to a database (mclient and mapi) * 3145: mclient error when enabling history * 3150: Multiple string equality tests fail to match (Test_JdbcClient)

1 0

[MonetDB-users] time for ssbm
by guangliang peng 18 Sep '12

18 Sep '12

hi, my query schema is SSBM.My database size is 65G.There are five tables : lineorder 600038145 rows part 1400000 date 2556 supplier 1000000 customer 3000000 and my host have enough memory space up to 500G when i run query select sum(lo_extendedprice*lo_discount) as revenue from voc.lineorder, voc.dates where lo_orderdate = d_datekey and d_weeknuminyear = 6 and d_year = 1994 and lo_discount between 5 and 7 and lo_quantity between 26 and 35; the result is below | revenue | +===============+ | 2605756481323 | +---------------+ 1 tuple (6.2s) but if i delete the condition of " lo_discount between 5 and 7" the run result is more faster than the original one. query select sum(lo_extendedprice*lo_discount) as revenue from voc.lineorder, voc.dates where lo_orderdate = d_datekey and d_weeknuminyear = 6 and d_year = 1994 and lo_quantity between 26 and 35; +---------------+ | revenue | +===============+ | 7990711777299 | +---------------+ 1 tuple (410.451ms) at the same time ,if i delete the condition of "lo_quantity between 26 and 35" the run result is more faster than the original one too. query select sum(lo_extendedprice*lo_discount) as revenue from voc.lineorder, voc.dates where lo_orderdate = d_datekey and d_weeknuminyear = 6 and d_year = 1994 and lo_quantity between 26 and 35; +---------------+ | revenue | +===============+ | 7990711777299 | +---------------+ 1 tuple (463.617ms) i am confused and if the semijoin costs a lot of execution time ? Thank you for your help. best reguards, guangliang peng

1 0

[MonetDB-users] Disk space cleanup policy
by Jukka 17 Sep '12

17 Sep '12

Hi, What is the disk space cleanup policy of MonetDb? If I remove all rows of table with delete command, the disk utilization of dbfarm is still the same. The storage() function is still reporting the same size for the table as before the delete command only count of rows was decreased to zero. Also executing drop table sql statement does not seem to release the disk space. -- Jukka

1 0

[MonetDB-users] MonetDb and too many open files
by Jukka 17 Sep '12

17 Sep '12

Hi, How many files MonetDb is keeping open during runtime? I have running some test on MonetDb on RedHat 6 environment and faced the problem with the too many open files error during the loading phase after having several tables created to the database. I managed to avoid the current error with increasing the file handle limit: vi /etc/security/limits.conf #add for file handle #see lsof -p pid monetdb soft nofile 65536 monetdb hard nofile 65536 Now the system is running, but after 500 tables in MonetDb, I see about 17000 file handle listed with lsof -p command. The most of those are pointing to files in dbfarm folder. Is the files of all columns of all tables kept all the time? Is there formula to estimate the number file handles needed? Is there any limit how many tables or columns we can have in one MonetDb instance? The version used is "MonetDB Database Server v1.6 (Jul2012-SP1)" compiled from sources for RedHat 6. -- Jukka jukkakar(a)iki.fi

1 0

[MonetDB-users] MonetDb slowing down
by Cory Isaacson 07 Sep '12

07 Sep '12

We have MonetDb running in the AWS cloud, with about 32GB of physical RAM. Our process uses COPY INTO every few minutes to load small chunks of data (usually 1000 rows at a time). That seems to work fine. There are a few users running queries against MonetDb, and now at least once/day we have to start/stop MonetDb. We see very high disk I/O (mainly I/O wait), so it appears that memory usage is high and does not get released. We are using the April 2012 release. The JDBC client is used for these queries. Overall MonetDb is doing a very good job, but this slowdown makes it appear unstable to end users. If there are any suggestions on how to improve this please let us know. cmi --

2 1

[MonetDB-users] Connecting from DBLite.com
by Medha Atre 06 Sep '12

06 Sep '12

Hello, I have successfully installed and used MonetDB in the past as a standalone server, by connecting/querying through "mclient" running on the same machine where server is running. I was trying out web-based access, in which I was trying DBLite's simple web interface http://user.dblite.com (running on the browser on the same machine). My MonetDB server was compiled with "enable-sql", "enable-mserver5", and other optimization options. Not compiled with JDBC/ODBC though (but DBLite is a PHP based interface). "Control" is set to YES and "passphrase" is set to "monetdb" too. While the server is running, I can very well connect and query through command line mclient, but if I give "localhost", port as 50000 and default user/passwd "monetdb/monetdb" on the http://user.dblite.com interface, it keeps giving me "access denied" error. I am wondering what I am missing! Thanks. Medha

2 1

[MonetDB-users] source code structure
by guangliang peng 06 Sep '12

06 Sep '12

hi, i prepare to look at the source code of monnetdb.Can you guys give me some advices which can help reduce the time on the source code .And what's the source code structure? thanks, guangliang

1 0

[MonetDB-users] costmodel
by guangliang peng 05 Sep '12

05 Sep '12

hello, how do i show costmodel where i run a query ?because i want to know the internal execution of a query . thanks, guangliang

2 1