September 2017 - users-list

Embedded Python is enabled but an error was thrown during initialization.
by Anton Kravchenko 15 Sep '17

15 Sep '17

Hi there, sql>create or replace function f1() returns table(v1 INT) more>LANGUAGE PYTHON { more> return [1,2,3] more>}; operation successful (3.658ms) sql>select * from f1(); Embedded Python is enabled but an error was thrown during initialization. Does one have to setup another python env variables (like it's done on MonetDB for Windows: PYTHONHOME, PYTHONPATH) on Linux? p.s. Here are is how I setup environment 1) installed MonetDB v11.27.5 (Jul2017-SP1) binaries on centos-release-7-4.1708.el7.centos.x86_64: sudo yum install https://dev.monetdb.org/downloads/epel/MonetDB-release-epel.noarch.rpm sudo yum install https://www.monetdb.org/downloads/epel/7/x86_64/MonetDB-stream-11.27.5-2017… sudo yum install https://www.monetdb.org/downloads/epel/7/x86_64/MonetDB-client-11.27.5-2017… sudo yum install https://www.monetdb.org/downloads/epel/7/x86_64/MonetDB5-server-11.27.5-201… sudo yum install https://www.monetdb.org/downloads/epel/7/x86_64/MonetDB-SQL-server5-11.27.5… sudo yum install https://www.monetdb.org/downloads/epel/7/x86_64/MonetDB-python2-11.27.5-201… 2) installed Anaconda2 Python 2.7.13 |Anaconda 4.4.0 (64-bit)| (default, Dec 20 2016, 23:09:15) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2 export PATH="/home/akravchenko/anaconda2/bin:$PATH" 3) created and started Monet database: monetdbd create hnode1_dbfarm monetdbd set port=50001 hnode1_dbfarm monetdbd set control=yes hnode1_dbfarm monetdbd set passphrase=monetdb hnode1_dbfarm monetdbd start hnode1_dbfarm monetdb create hnode1_dbfarm_db1 *monetdb set embedpy=true hnode1_dbfarm_db1* monetdb start hnode1_dbfarm_db1 monetdb release hnode1_dbfarm_db1 4)monetdbd get all hnode1_dbfarm property value hostname somehostname dbfarm hnode1_dbfarm status monetdbd[31089] 1.7 (Jul2017-SP1) is serving this dbfarm mserver /usr/bin/mserver5 logfile hnode1_dbfarm/merovingian.log pidfile hnode1_dbfarm/merovingian.pid sockdir /tmp listenaddr localhost port 50001 exittimeout 60 forward proxy discovery yes discoveryttl 600 control yes passphrase {SHA512}a73f1d86383446438ac64f56e15ada38b41fbb18f029d2181723aeb2acac6a831f60e5fdbd64ac2c8c70e035dd44cbbe3b45565ef2d58feb2821a2078c7fad35 mapisock /tmp/.s.monetdb.50001 controlsock /tmp/.s.merovingian.50001 5) monetdb get all hnode1_dbfarm_db1 name prop source value hnode1_dbfarm_db1 name - hnode1_dbfarm_db1 hnode1_dbfarm_db1 type default database hnode1_dbfarm_db1 shared default yes hnode1_dbfarm_db1 nthreads default 32 hnode1_dbfarm_db1 optpipe default default_pipe hnode1_dbfarm_db1 readonly default no hnode1_dbfarm_db1 embedr default no hnode1_dbfarm_db1 embedpy local yes hnode1_dbfarm_db1 embedpy3 default no hnode1_dbfarm_db1 nclients default 64 hnode1_dbfarm_db1 dbextra default <unknown> Thank you, Anton

1 2

SQL_CALC_FOUND_ROWS equivalent?
by Alberto Ferrari 15 Sep '17

15 Sep '17

Hi, is there any function or trick in Monet equivalent to mysql "SELECT SQL_CALC_FOUND_ROWS ..." to get de total rows when limit is present ? ((pagination) So I could get de TOTAL rows/pages number without exec another query everytime I need paginate results. Thanks!

2 2

Re: Column rename issue
by Stefan Manegold 15 Sep '17

15 Sep '17

Hi, Add new column, copy/cast data from old to new column, drop old column. Stefan -------- Original message --------From: Amit Ambekar <amit(a)g-square.in> Date: 9/15/17 12:14 (GMT+01:00) To: users-list(a)monetdb.org Subject: Column rename issue Hi, how to update already created column name and type in monetdb ? _______________________________________________ users-list mailing list users-list(a)monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

1 0

Column rename issue
by Amit Ambekar 15 Sep '17

15 Sep '17

Hi, how to update already created column name and type in monetdb ?

1 0

large scale hardware configuration for MonetDB
by Anton Kravchenko 06 Sep '17

06 Sep '17

Hi there, >From https://www.monetdbsolutions.com/news/monetdb-numascale : "The system is fine-tuned to exploit the large main memories of modern computer systems effectively and has proven to be highly efficient in data warehouse query processing. NumaQ can easily scale to thousands of cores and terabytes of memory for data-intensive." I wonder if there is a public documentation on how to configure large scale hardware for optimal MonetDB performance. So one could try setup NumaQ-like machine by using what you already learned. Also from https://stackoverflow.com/questions/7435235/is-it-worth-trying-monetdb : "MonetDB is developed/tuned under two main assumptions: 1. Your workload is analytical, i.e., you have lots of (grouped) aggregations and the like. 2. Even more important: your hot dataset (the data that you actually work with) fits into the main memory of your system. MonetDB does not have it's own Buffer Manager but relies on the OS to handle disk I/O. Since the OS (especially windows but Linux too) is sometimes very dumb about disk swapping that may become a problem (especially for joins that run out of memory)." Are there any official benchmarks for Windows vs Linux MonetDB performance (especially for queries that require disk swapping) ? Thank you, Anton

2 1

default_pipe optimizer slowness during concurrency
by Sharma, Sreejith 06 Sep '17

06 Sep '17

Hi, I have a scenario where if I run the a SQL with 10 user concurrency in a recursive mode, the execution time is significantly slower. The SQL which completes in .8sec will end up in completing 5-10hrs during concurrency. This behavior is seen when the optimizer is set to "default_pipe" , however when set to "sequential_pipe" the timing has been drastically reduced and it end ups in seconds. How is "sequential_pipe" optimizer works here, is it to do anything with concurrency. NOTE - this happens when there is DISTINCT pattern in SQL. Regards, Sreejith

2 1

Autoload scripts from MonetDB/sql/scripts
by Lefteris 05 Sep '17

05 Sep '17

After playing around with some udfs and such, it seems to me that the autoload feature of the scripts in directory MonetDB/sql/scripts only run if I start monetdb with the following command mserver5 --dbpath=... and then when I connect with mclient all these messages about loading the scripts appear at the mserver5 console. However, if I create a dbfarm with monetdbd create path/ monetdbd start path/ and then a database monetdb create dbname monetdb release dbname and connect with mclient -lsql -d dbname all the goodies from the script dir do not appear to be there. Is there another trigger for this case that runs autoload? thanx

3 5

How to get Memory Dump. Is there any way to eliminate Memory leaks.
by Sharma, Sreejith 05 Sep '17

05 Sep '17

All, Is there any way to get the memory dump from MonetDB? The memory consumed by merver5 post concurrent query execution are not coming down. Need to investigate why memory leak is happening here. Regards, Sreejith

2 2

Monetdb on windows with embedded python 3.6
by imad hajj chahine 05 Sep '17

05 Sep '17

Hi, I have been successfully using monetdb (11.25.3) on windows with embedded python 2.7, recently I needed a library that is only supported on python 3.6, is it possible somehow to point monetdb to the python 3.6 folder? I know this is possible on linux, but my dev environment is on windows. Thank you.

3 6

Defining a table with a sorted column
by Cota, Luis 05 Sep '17

05 Sep '17

Is it possible to define a table such that the server maintains the table sorted with each new row? I'm dealing with observations on market data, and while market data for a specific stock symbol will be sorted, I will be adding multiple symbols to the same table in chunks, so that each symbol is contained within its own load file, and is sorted within that file. How can I enforce this constraint? - Luis ________________________________ IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.

2 1