August 2018 - users-list

[MonetDB-users] Books CDs
by CHARTS LICENSING 11 Aug '24

11 Aug '24

Everything you want to know this dynamic artist check. At blueberry hills duck, roomfull story gtgt getting. Intriguing, life in biography. Here music looking for lyrics maybe, youre find which! Download free wallpapers featuring business cmg worldwide is exclusive. Videos latest products been easier. Exclusive estate we consider request use name voice, image. List covered berries charts licensing message! Reelin and rockin about, has everything you? Many exciting, photos him stage click! Artist check, out some quotes read all intriguing. It alive he, did on occasion birthday wednesday? Has everything you want to know this? Quotes read all intriguing life, in biography or. Gtgt getting licks years reelin and. Click here, music looking for lyrics maybe, youre. News celebrates his th rock will never! Chuck berry news celebrates his th rock, will. Browse through many exciting, photos him stage click here. He did on occasion birthday wednesday night at blueberry. Wallpapers featuring business cmg worldwide. Worldwide is exclusive estate we consider, request use name. Want to know this dynamic artist check out. Awards fast, facts clips list covered. Music looking for lyrics maybe youre. Message board downloads tour dates fan, club inquiries. Other sites, download free wallpapers featuring, business cmg worldwide. Request use, name, voice.

2 1

Multi-column update based on joined table
by Lee Hachadoorian 11 Aug '24

11 Aug '24

I need to update multiple columns (~250) from one table to another based on a two-column key. In Postgres I would write this using the FROM clause, but since MonetDB doesn't support the FROM clause, I've successfully updated one column with: UPDATE census.acs2010_5yr SET b07401001 = ( SELECT CAST(b07401001 AS int) FROM acs2010_5yr.tmp_seq0001 WHERE acs2010_5yr.stusab = tmp_seq0001.stusab AND acs2010_5yr.logrecno = tmp_seq0001.logrecno ); My question is whether there is any benefit to writing a multiple column update into a single UPDATE statement: UPDATE target_table SET column1 = (SELECT column1 FROM source_table WHERE target_table.key = source_table.key), column2 = (SELECT column2 FROM source_table WHERE target_table.key = source_table.key), ..., columnN = (SELECT columnN FROM source_table WHERE target_table.key = source_table.key); On Postgres I believe the multi-column update using a table join would only have to do the sequential scan of each table once, but with the necessity of sub-selects in MonetDB, what is the backend doing? Is it already doing an index scan once for each row in the single column case, then doing it once for each row * column in the multi-column case? Best, --Lee -- Lee Hachadoorian Asst Professor of Geography, Dartmouth College http://freecity.commons.gc.cuny.edu/

3 3

[MonetDB-users] XQuery API and XLink/XUpdate support
by Alexandra Mart�nez 27 Jun '24

27 Jun '24

Hello all, I was wondering if there is any documentation about the API for XQuery (i.e., how to write xqueries from a Java program). Also, I am interested in using some features of XLink and XUpdate on top of my XML database. Is support for any of these in progress? Thanks in advance, Alexandra ______________________________________________________ Click here to donate to the Hurricane Katrina relief effort. http://store.yahoo.com/redcross-donate3/

4 3

[MonetDB-users] Proofreading your professional documents
by Richard Hounslow 24 Feb '24

24 Feb '24

Dear Professor/Lecturer Please excuse us for contacting you directly. We are offering an efficient service to help you ensure your academic and professional work is written in perfect English. We will check the grammar and style of your work and return it to you to meet your requirements and deadlines. Proof-Reading-Service.com can provide you with a professional proofreading service at a very reasonable rate. All our proofreaders are highly qualified native English speakers. Many work as leading academics in their fields and all have extensive experience of proofreading to the highest standards. If you are interested in our service, please take a look at our website: www.proof-reading-service.com All you have to do is send us your document as a word attachment with the deadline and we will guarantee delivery of a perfectly written document to give you complete confidence when you submit your work. The fee is worked out on a flat rate (£6.95 per thousand words or 0.695 pence per word), so you know exactly how much the proofreading will cost in advance. Once we have received your document(s) we will confirm the word count, the price and the deadline. If you have not heard from us within 2 hours during normal business hours after you have sent your work please resend it from a different email account or send the file to proofreadings(a)gmail.com. We look forward to hearing from you! Yours faithfully Proof-Reading-Service.com

2 2

GDK reported error. MT_mremap: mremap failed. !OS: Cannot allocate memory
by RanchoYuan 04 Jul '19

04 Jul '19

Hi Stefan, Thanks for your reply! We have run the query a few times with different size of data. There we used 16G RAM(actually 13.5G was used), and find the size of 10G's data is the critical point that can run the query. All of the data files' size are listed below, each file name is a table name(there are only a few tables are refered -- store_sales, date_dim, item, customer, catalog_sales, web_sales): 7.4K call_center.dat 1.6M catalog_page.dat 212M catalog_returns.dat 2.9G catalog_sales.dat 27M customer_address.dat 64M customer.dat 77M customer_demographics.dat 9.9M date_dim.dat 77B dbgen_version.dat 149K household_demographics.dat 328B income_band.dat 2.6G inventory.dat 28M item.dat 61K promotion.dat 1.7K reason.dat 1.1K ship_mode.dat 27K store.dat 323M store_returns.dat 3.8G store_sales.dat 4.9M time_dim.dat 1.2K warehouse.dat 19K web_page.dat 98M web_returns.dat 1.5G web_sales.dat 12K web_site.dat So we guess that the monetdb has no memory management? For the output of `mserver5 --version` is: MonetDB 5 server v11.27.13 "Jul2017-SP4" (64-bit, 128-bit integers) Copyright (c) 1993 - July 2008 CWI Copyright (c) August 2008 - 2018 MonetDB B.V., all rights reserved Visit https://www.monetdb.org/ for further information Found 17.0GiB available memory, 40 available cpu cores Libraries: libpcre: 8.38 2015-11-23 (compiled with 8.38) openssl: OpenSSL 1.0.2g 1 Mar 2016 (compiled with OpenSSL 1.0.2g 1 Mar 2016) libxml2: 2.9.3 (compiled with 2.9.3) Compiled by: monetdb(a)MonetDB-0.0 (x86_64-pc-linux-gnu) Compilation: gcc -g -O2 Linking : /usr/bin/ld -m elf_x86_64 And the size of processes is not limited. To let you reproduce the problem conveniently, I'll provide more details here: you can get tpc-ds from its website(we use version 2.6.0). Install the tpc-ds, access the directory v2.6.0/tools and run `./dsdgen -scale 10 -dir /home/monetdb/tpc-ds_test_data10G` to generate the data. When data has been generated, using the script /expe.sh/ to create tables and load the data. The query script is 123.tpcds.23.sql.(The syntaxs of other queries that tpc-ds generates is not suitable for monetdb all, we don't modify them all when the problem occurred). One more question, I can't get your reply email, so I don't know how to reply you, for this case, I could only send a new mail echo time. Thanks! Regards, Rancho

4 3

SQLcopyinto.java
by Roberto Cornacchia 20 Sep '18

20 Sep '18

I'm trying to debug n issue I have with data disappearing after a database restart. Because part of the process for creating the table involves a mixed Mapi / JDBC approach as described in monetdb-java/example/SQLcopyinto.java, I started from this example. When I run it though, I get a final count of 0 (should be 100). Can anyone confirm? Mar2018, JDBC 2.28 Roberto

2 1

Announcement: New Aug2018 Feature release of MonetDB suite
by Sjoerd Mullender 31 Aug '18

31 Aug '18

The MonetDB team at CWI/MonetDB BV is pleased to announce the Aug2018 feature release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <https://www.monetdb.org/>. For details on this release, please see the release notes at <https://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <https://dev.monetdb.org/downloads/>. Aug 2018 feature release (11.31.7) MonetDB5 Server * The lsst module was moved to a separate repository (https://dev.monetdb.org/hg/MonetDB-lsst/) Build Environment * Build the MonetDB-cfitsio RPM and libmonetdb5-server-cfitsio Debian/Ubuntu package. * On Windows, the separate MonetDB5-Geom installer has been incorporated into the main MonetDB5-SQL installer and is therefore no longer available as a separate download. Merovingian * Added a "logrotate" configuration file. See /etc/logrotate.d/monetdbd. * Changed the monetdb profilerstart command to be more robust. If the server or stethoscope crashed before, the pid file is still there, so the next time we try to start stethoscope, it will fail. Now the profilerstart command will check if a stethoscope process with the recorded pid is running. If not, we start stethoscope, assuming that something went wrong before. * Changed the monetdb stop command to try to stop stethoscope before stoping the server. The error conditions that can arrise from attempting to stop stethoscope are: + The database is not running. + The profilerlogpath is not set. + The profiler.pid file does not exist or is inaccessible. + The contents of the profiler.pid are not valid. + Shutdown of stethoscope did not succeed. + Removing the profiler.pid file failed. In all the cases, the attempt to stop the server can continue normally, so we actually ignore any errors that rise from the attempt to stop stethoscope. Client Package * ODBC: Implemented SQL_ATTR_QUERY_TIMEOUT parameter in SQLSetStmtAttr. * ODBC SQLGetInfo now returns a positive numeric value for InfoTypes: SQL_MAX_COLUMN_NAME_LEN, SQL_MAX_DRIVER_CONNECTIONS, SQL_MAX_IDENTIFIER_LEN, SQL_MAX_PROCEDURE_NAME_LEN, SQL_MAX_SCHEMA_NAME_LEN, SQL_MAX_TABLE_NAME_LEN and SQL_MAX_USER_NAME_LEN. * Added a '-f rowcount' option in mclient to repress printing the actual data of a resultset, but only print the number of returned tuples Stream Library * Added support for lz4 compressed files in the stream library MonetDB Common * Hash indexes are now persistent across server restarts. * The macros bunfastapp and tfastins and variants no longer set the dirty flag of the heap they write to. This now needs to be done separately (and preferably outside of the inner loop). * Removed batDirty flag from BAT record. Its function is completely superseded by batDirtydesc and the dirty flags on the various heaps. * Removed "masksize" argument of function BAThash. * A whole bunch of functions that took an int argument that was used as a Boolean (true/false) value now take a value of type bool. The functions BATkeyed, BATordered and BATordered_rev now return a bool instead of an int. * Removed the tdense property: it's function is completely replaced by whether or not tseqbase is equal to oid_nil. Testing Environment * Removed helper programs Mtimeout and MkillUsers: they have long been superseded by timeout handling by Mtest.py itself. SQL Frontend * Removed deprecated table producing system functions: sys.dependencies_columns_on_functions() sys.dependencies_columns_on_indexes() sys.dependencies_columns_on_keys() sys.dependencies_columns_on_triggers() sys.dependencies_columns_on_views() sys.dependencies_functions_on_functions() sys.dependencies_functions_on_triggers() sys.dependencies_keys_on_foreignkeys() sys.dependencies_owners_on_schemas() sys.dependencies_schemas_on_users() sys.dependencies_tables_on_foreignkeys() sys.dependencies_tables_on_functions() sys.dependencies_tables_on_indexes() sys.dependencies_tables_on_triggers() sys.dependencies_tables_on_views() sys.dependencies_views_on_functions() sys.dependencies_views_on_triggers() They are replaced by new system dependency_* views: sys.dependency_args_on_types sys.dependency_columns_on_functions sys.dependency_columns_on_indexes sys.dependency_columns_on_keys sys.dependency_columns_on_procedures sys.dependency_columns_on_triggers sys.dependency_columns_on_types sys.dependency_columns_on_views sys.dependency_functions_on_functions sys.dependency_functions_on_procedures sys.dependency_functions_on_triggers sys.dependency_functions_on_types sys.dependency_functions_on_views sys.dependency_keys_on_foreignkeys sys.dependency_owners_on_schemas sys.dependency_schemas_on_users sys.dependency_tables_on_foreignkeys sys.dependency_tables_on_functions sys.dependency_tables_on_indexes sys.dependency_tables_on_procedures sys.dependency_tables_on_triggers sys.dependency_tables_on_views sys.dependency_views_on_functions sys.dependency_views_on_procedures sys.dependency_views_on_views * Implemented group_concat(X,Y) aggregate function which also concatenates a column of strings X, but using a supplied string Y as the separator. This function is also a SQL extension. * Implemented group_concat(X) aggregate function which concatenates a column of strings using a comma as a separator. This function is not featured in the SQL standard. Bug Fixes * 4020: Importing timestamp with zone from copy into * 6506: Improper performance counters * 6556: Sqlitelogictest division by zero on COALESCE call * 6564: Changes to the Remote Table definition * 6575: Sqlitelogictest crash on groupby query with coalesce call * 6579: Sqlitelogic test infinite loop while compiling SQL query * 6586: Sqlitelogictest crash on complex aggregation query * 6593: Poor performance with like operator and escape clause * 6596: Multicolumn aggregation very slow after ANALYZE when persistent hashes are enabled * 6605: Sqlitelogictest set queries with wrong results * 6606: Misleading parameter name in generate_series function * 6610: Sqlitelogictest algebra.rangejoin undefined * 6611: Cannot compile with GCC 8.1 and --enable-debug=no * 6612: Implement BLOB handling in python UDFs * 6618: dependency column on sequence violated by DROP SEQUENCE * 6621: SELECT FROM REMOTE TABLE WHERE <> returns wrong results * 6624: "Cannot use non GROUP BY column in query results without an aggregate function" when using aggregate function in both HAVING and ORDER BY clauses. * 6625: OR in subselect causes the server to crash with segmentation fault * 6627: stddev_pop inconsistent behaviour * 6628: User cannot insert into own local temporary table * 6629: CREATE TABLE IF NOT EXISTS returns 42000! * 6630: Sqlitelogictest cast NULL to integer failing * 6632: Dataflow causes crash when THRnew fails * 6633: ILIKE clauses don't work on certain characters * 6635: monetdbd exits due to "Too many open files" error * 6637: Within a transaction, \d after an error causes mclient to exit * 6638: (sequences of) mkey.bulk_rotate_xor_hash() can generate NIL from non-NIL making multi-col joins return wrong results * 6639: COMMENT ON TABLE abc IS NULL invalidly sets the remark column to null where remark column is defined as NOT NULLable

1 0

Re: #!ERROR: THRnew: too many threads
by Stefan Manegold 14 Aug '18

14 Aug '18

Hi, The usual questions:Which version of MonetDBAndWhat kind of workload triggers this? Best,Stefan -------- Original message --------From: "Murthy, Gautham" <Gautham.Murthy(a)harman.com> Date: 13/08/2018 18:06 (GMT+01:00) To: users-list(a)monetdb.org Subject: #!ERROR: THRnew: too many threads Hello, We had a DB crash reported with the below messages, Do we need alter any system level parameters or changes to avoid the below issue – Server has 128 CPU and 1TB memory. 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads /etc/security/limits.conf monetadmin soft nproc 102400 monetadmin hard nproc 102400 monetadmin soft nofile 500000 monetadmin hard nofile 500000 monetadmin soft stack 10240 monetadmin soft memlock unlimited monetadmin hard memlock unlimited [monetadmin@lnx1535 DBFARM_TSV_P_B]$ /monet_binaries/MonetDB-11.27.13_PY/bin/monetdb -p 50010 get all TSV_PROD_DB_B name prop source value TSV_PROD_DB_B name - TSV_PROD_DB_B TSV_PROD_DB_B type default database TSV_PROD_DB_B shared default yes TSV_PROD_DB_B nthreads default 128 TSV_PROD_DB_B optpipe local sequential_pipe TSV_PROD_DB_B readonly local yes TSV_PROD_DB_B embedr default no TSV_PROD_DB_B embedpy local yes TSV_PROD_DB_B embedpy3 default no TSV_PROD_DB_B nclients local 2048 TSV_PROD_DB_B dbextra default <unknown> [monetadmin@lnx1535 DBFARM_TSV_P_B]$ 2018-08-13 08:46:30 MSG merovingian[36790]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying 2018-08-13 08:46:30 MSG merovingian[36790]: proxying client 10.106.5.250:29929 for database 'TSV_PROD_DB_B' to mapi:monetdb:///monet_data02/DBFARM_TSV_P_B/TSV_PROD_DB_B/.mapi.sock?database=TSV_PROD_DB_B 2018-08-13 08:46:30 MSG merovingian[36790]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:33 MSG merovingian[36790]: proxying client 10.106.5.250:29957 for database 'TSV_PROD_DB_B' to mapi:monetdb:///monet_data02/DBFARM_TSV_P_B/TSV_PROD_DB_B/.mapi.sock?database=TSV_PROD_DB_B 2018-08-13 08:46:33 MSG merovingian[36790]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying 2018-08-13 08:46:36 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:40 MSG merovingian[36790]: proxying client 10.106.5.250:64830 for database 'TSV_PROD_DB_B' to mapi:monetdb:///monet_data02/DBFARM_TSV_P_B/TSV_PROD_DB_B/.mapi.sock?database=TSV_PROD_DB_B 2018-08-13 08:46:40 MSG merovingian[36790]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying 2018-08-13 08:46:42 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:47:47 MSG discovery[36790]: new neighbour lnx1536.ch3.prod.i.com (lnx1536.ch3.prod.i.com) 2018-08-13 08:47:48 MSG discovery[36790]: new database mapi:monetdb://lnx1536.ch3.prod.i.com:50010/TSV_PROD_DB_B (ttl=660s) 2018-08-13 08:48:55 MSG merovingian[36790]: database 'TSV_PROD_DB_B' (57408) has crashed (dumped core) 2018-08-13 08:49:12 MSG merovingian[36790]: database 'TSV_PROD_DB_B' has crashed after start on 2018-08-13 05:35:17, attempting restart, up min/avg/max: 2h/5d/3w, crash average: 1.00 0.70 0.43 (23-10=13) 2018-08-13 08:49:12 MSG TSV_PROD_DB_B[72455]: arguments: /monet_binaries/MonetDB-11.27.13_PY/bin/mserver5 --dbpath=/monet_data02/DBFARM_TSV_P_B/TSV_PROD_DB_B --set merovingian_uri=mapi:monetdb://lnx1535.ch3.prod.i.com:50010/TSV_PROD_DB_B --set mapi_open=false --set mapi_port=0 --set 2018-08-13 08:49:12 MSG TSV_PROD_DB_B[72455]: mapi_usock=/monet_data02/DBFARM_TSV_P_B/TSV_PROD_DB_B/.mapi.sock --set monet_vault_key=/monet_data02/DBFARM_TSV_P_B/TSV_PROD_DB_B/.vaultkey --set gdk_nr_threads=128 --set max_clients=2048 --set sql_optimizer=sequential_pipe --set embedded_py=true --readonly --set monet_daemon=yes 2018-08-13 08:49:12 MSG TSV_PROD_DB_B[72455]: 2018-08-13 08:49:12 MSG merovingian[36790]: proxying client 10.106.5.250:16467 for database 'TSV_PROD_DB_B' to mapi:monetdb:///monet_data02/DBFARM_TSV_P_B/TSV_PROD_DB_B/.mapi.sock?database=TSV_PROD_DB_B 2018-08-13 08:49:12 MSG merovingian[36790]: proxying client 10.106.5.250:45439 for database 'TSV_PROD_DB_B' to mapi:monetdb:///monet_data02/DBFARM_TSV_P_B/TSV_PROD_DB_B/.mapi.sock?database=TSV_PROD_DB_B 2018-08-13 08:49:12 MSG merovingian[36790]: starting a proxy failed: cannot connect: Connection refused 2018-08-13 08:49:12 ERR control[36790]: !monetdbd: an internal error has occurred 'cannot connect: Connection refused' 2018-08-13 08:49:12 ERR merovingian[36790]: client error: cannot connect: Connection refused 2018-08-13 08:49:12 MSG merovingian[36790]: proxying client 10.106.5.250:45447 for database 'TSV_PROD_DB_B' to mapi:monetdb:///monet_data02/DBFARM_TSV_P_B/TSV_PROD_DB_B/.mapi.sock?database=TSV_PROD_DB_B 2018-08-13 08:49:12 ERR control[36790]: !monetdbd: an internal error has occurred 'cannot connect: Connection refused' 2018-08-13 08:49:12 MSG merovingian[36790]: starting a proxy failed: cannot connect: Connection refused 2018-08-13 08:49:13 ERR merovingian[36790]: client error: cannot connect: Connection refused 2018-08-13 08:49:13 MSG merovingian[36790]: proxying client 10.106.5.250:36605 for database 'TSV_PROD_DB_B' to mapi:monetdb:///monet_data02/DBFARM_TSV_P_B/TSV_PROD_DB_B/.mapi.sock?database=TSV_PROD_DB_B /etc/security/limits.conf monetadmin soft nproc 102400 monetadmin hard nproc 102400 monetadmin soft nofile 500000 monetadmin hard nofile 500000 monetadmin soft stack 10240 monetadmin soft memlock unlimited monetadmin hard memlock unlimited [monetadmin@lnx1535 DBFARM_TSV_P_B]$ /monet_binaries/MonetDB-11.27.13_PY/bin/monetdb -p 50010 get all TSV_PROD_DB_B name prop source value TSV_PROD_DB_B name - TSV_PROD_DB_B TSV_PROD_DB_B type default database TSV_PROD_DB_B shared default yes TSV_PROD_DB_B nthreads default 128 TSV_PROD_DB_B optpipe local sequential_pipe TSV_PROD_DB_B readonly local yes TSV_PROD_DB_B embedr default no TSV_PROD_DB_B embedpy local yes TSV_PROD_DB_B embedpy3 default no TSV_PROD_DB_B nclients local 2048 TSV_PROD_DB_B dbextra default <unknown> [monetadmin@lnx1535 DBFARM_TSV_P_B]$

2 2

#!ERROR: THRnew: too many threads
by Murthy, Gautham 13 Aug '18

13 Aug '18

Hello, We had a DB crash reported with the below messages, Do we need alter any system level parameters or changes to avoid the below issue - Server has 128 CPU and 1TB memory. 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads /etc/security/limits.conf monetadmin soft nproc 102400 monetadmin hard nproc 102400 monetadmin soft nofile 500000 monetadmin hard nofile 500000 monetadmin soft stack 10240 monetadmin soft memlock unlimited monetadmin hard memlock unlimited [monetadmin@lnx1535 DBFARM_TSV_P_B]$ /monet_binaries/MonetDB-11.27.13_PY/bin/monetdb -p 50010 get all TSV_PROD_DB_B name prop source value TSV_PROD_DB_B name - TSV_PROD_DB_B TSV_PROD_DB_B type default database TSV_PROD_DB_B shared default yes TSV_PROD_DB_B nthreads default 128 TSV_PROD_DB_B optpipe local sequential_pipe TSV_PROD_DB_B readonly local yes TSV_PROD_DB_B embedr default no TSV_PROD_DB_B embedpy local yes TSV_PROD_DB_B embedpy3 default no TSV_PROD_DB_B nclients local 2048 TSV_PROD_DB_B dbextra default <unknown> [monetadmin@lnx1535 DBFARM_TSV_P_B]$ 2018-08-13 08:46:30 MSG merovingian[36790]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying 2018-08-13 08:46:30 MSG merovingian[36790]: proxying client 10.106.5.250:29929 for database 'TSV_PROD_DB_B' to mapi:monetdb:///monet_data02/DBFARM_TSV_P_B/TSV_PROD_DB_B/.mapi.sock?database=TSV_PROD_DB_B 2018-08-13 08:46:30 MSG merovingian[36790]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:31 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:33 MSG merovingian[36790]: proxying client 10.106.5.250:29957 for database 'TSV_PROD_DB_B' to mapi:monetdb:///monet_data02/DBFARM_TSV_P_B/TSV_PROD_DB_B/.mapi.sock?database=TSV_PROD_DB_B 2018-08-13 08:46:33 MSG merovingian[36790]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying 2018-08-13 08:46:36 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:46:40 MSG merovingian[36790]: proxying client 10.106.5.250:64830 for database 'TSV_PROD_DB_B' to mapi:monetdb:///monet_data02/DBFARM_TSV_P_B/TSV_PROD_DB_B/.mapi.sock?database=TSV_PROD_DB_B 2018-08-13 08:46:40 MSG merovingian[36790]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying 2018-08-13 08:46:42 MSG TSV_PROD_DB_B[57408]: #!ERROR: THRnew: too many threads 2018-08-13 08:47:47 MSG discovery[36790]: new neighbour lnx1536.ch3.prod.i.com (lnx1536.ch3.prod.i.com) 2018-08-13 08:47:48 MSG discovery[36790]: new database mapi:monetdb://lnx1536.ch3.prod.i.com:50010/TSV_PROD_DB_B (ttl=660s) 2018-08-13 08:48:55 MSG merovingian[36790]: database 'TSV_PROD_DB_B' (57408) has crashed (dumped core) 2018-08-13 08:49:12 MSG merovingian[36790]: database 'TSV_PROD_DB_B' has crashed after start on 2018-08-13 05:35:17, attempting restart, up min/avg/max: 2h/5d/3w, crash average: 1.00 0.70 0.43 (23-10=13) 2018-08-13 08:49:12 MSG TSV_PROD_DB_B[72455]: arguments: /monet_binaries/MonetDB-11.27.13_PY/bin/mserver5 --dbpath=/monet_data02/DBFARM_TSV_P_B/TSV_PROD_DB_B --set merovingian_uri=mapi:monetdb://lnx1535.ch3.prod.i.com:50010/TSV_PROD_DB_B --set mapi_open=false --set mapi_port=0 --set 2018-08-13 08:49:12 MSG TSV_PROD_DB_B[72455]: mapi_usock=/monet_data02/DBFARM_TSV_P_B/TSV_PROD_DB_B/.mapi.sock --set monet_vault_key=/monet_data02/DBFARM_TSV_P_B/TSV_PROD_DB_B/.vaultkey --set gdk_nr_threads=128 --set max_clients=2048 --set sql_optimizer=sequential_pipe --set embedded_py=true --readonly --set monet_daemon=yes 2018-08-13 08:49:12 MSG TSV_PROD_DB_B[72455]: 2018-08-13 08:49:12 MSG merovingian[36790]: proxying client 10.106.5.250:16467 for database 'TSV_PROD_DB_B' to mapi:monetdb:///monet_data02/DBFARM_TSV_P_B/TSV_PROD_DB_B/.mapi.sock?database=TSV_PROD_DB_B 2018-08-13 08:49:12 MSG merovingian[36790]: proxying client 10.106.5.250:45439 for database 'TSV_PROD_DB_B' to mapi:monetdb:///monet_data02/DBFARM_TSV_P_B/TSV_PROD_DB_B/.mapi.sock?database=TSV_PROD_DB_B 2018-08-13 08:49:12 MSG merovingian[36790]: starting a proxy failed: cannot connect: Connection refused 2018-08-13 08:49:12 ERR control[36790]: !monetdbd: an internal error has occurred 'cannot connect: Connection refused' 2018-08-13 08:49:12 ERR merovingian[36790]: client error: cannot connect: Connection refused 2018-08-13 08:49:12 MSG merovingian[36790]: proxying client 10.106.5.250:45447 for database 'TSV_PROD_DB_B' to mapi:monetdb:///monet_data02/DBFARM_TSV_P_B/TSV_PROD_DB_B/.mapi.sock?database=TSV_PROD_DB_B 2018-08-13 08:49:12 ERR control[36790]: !monetdbd: an internal error has occurred 'cannot connect: Connection refused' 2018-08-13 08:49:12 MSG merovingian[36790]: starting a proxy failed: cannot connect: Connection refused 2018-08-13 08:49:13 ERR merovingian[36790]: client error: cannot connect: Connection refused 2018-08-13 08:49:13 MSG merovingian[36790]: proxying client 10.106.5.250:36605 for database 'TSV_PROD_DB_B' to mapi:monetdb:///monet_data02/DBFARM_TSV_P_B/TSV_PROD_DB_B/.mapi.sock?database=TSV_PROD_DB_B /etc/security/limits.conf monetadmin soft nproc 102400 monetadmin hard nproc 102400 monetadmin soft nofile 500000 monetadmin hard nofile 500000 monetadmin soft stack 10240 monetadmin soft memlock unlimited monetadmin hard memlock unlimited [monetadmin@lnx1535 DBFARM_TSV_P_B]$ /monet_binaries/MonetDB-11.27.13_PY/bin/monetdb -p 50010 get all TSV_PROD_DB_B name prop source value TSV_PROD_DB_B name - TSV_PROD_DB_B TSV_PROD_DB_B type default database TSV_PROD_DB_B shared default yes TSV_PROD_DB_B nthreads default 128 TSV_PROD_DB_B optpipe local sequential_pipe TSV_PROD_DB_B readonly local yes TSV_PROD_DB_B embedr default no TSV_PROD_DB_B embedpy local yes TSV_PROD_DB_B embedpy3 default no TSV_PROD_DB_B nclients local 2048 TSV_PROD_DB_B dbextra default <unknown> [monetadmin@lnx1535 DBFARM_TSV_P_B]$

1 0

SQL Statements are skipped when run via scripts concurrently on different tables
by Murthy, Gautham 06 Aug '18

06 Aug '18

Hello, Application Teams are experiencing a behavior where few statements are skipped when executed via scripts, They are running multiple scripts concurrently on different tables. Please let us know if there are any know issues or a fix for the reported behavior. Database: MonetDB v11.27.13 (Jul2017-SP4) ü DROP, CREATE, INSERT - New SQL sequence, however looks like SQL statement are not executing as in order in database as per system catalog table log time entries. We need to check on this. ü First DROP, INSERT started and then after couple of statements, CREATE SQL started executing. DROP TABLE IF EXISTS "KEYFOOD_IT_S_22_43321_C";^M CREATE TABLE "KEYFOOD_IT_S_22_43321_C"(ATTR_NAME VARCHAR(500) NOT NULL, COL_NAME VARCHAR(30) NOT NULL, AVP_KEY INTEGER NOT NULL, ATTR_VALUE VARCHAR(3000), ATTR_LONG_VALUE VARCHAR(3000), ATTR_SHORT_VALUE VARCHAR(100), ATTR_MEDIUM_VALUE VARCHAR(100), PARENT_AVP_KEY INTEGER, SORT_ORDER INTEGER); INSERT INTO "KEYFOOD_IT_S_22_43321_C" (ATTR_NAME, COL_NAME, AVP_KEY, ATTR_VALUE, ATTR_LONG_VALUE, ATTR_SHORT_VALUE, ATTR_MEDIUM_VALUE, PARENT_AVP_KEY, SORT_ORDER) SELECT ATTR_NAME, COL_NAME, AVP_KEY, ATTR_VALUE, ATTR_LONG_VALUE, ATTR_SHORT_VALUE, ATTR_MEDIUM_VALUE, PARENT_AVP_KEY, SORT_ORDER FROM "IT_DSJ_KEYFOOD_43321" WHERE COL_NAME='S_22_KEY' AND SORT_TYPE='CATG'; We were able to statements being skipped in sys.querylog_catalog and sys.querylog_calls. Thank You, Gautham

2 1