November 2015 - users-list

回复：Re: 回复：Range query in MonetDB
by chenkaijiang＠bigknow.com.cn 26 Nov '15

26 Nov '15

Hi, Martin, great to hear it, thank you very much! Do you have a schedule, when to release this feature? We're looking forward to running MonetDB in the production environment!  ----- 原始邮件 ----- 发件人：Martin Kersten <martin.kersten(a)cwi.nl> 收件人：Communication channel for MonetDB users <users-list(a)monetdb.org> 主题：Re: 回复：Range query in MonetDB 日期：2015-11-26 04:02:19 On 25/11/15 17:08, chenkaijiang(a)bigknow.com.cn wrote: > Sometimes the traditional DB is 10 to 100 times faster with the help of the index. Yes. It is on the roadmap. > > ----- 原始邮件 ----- > 发件人：<chenkaijiang(a)bigknow.com.cn> > 收件人：users-list<users-list(a)monetdb.org>,developers-list<developers-list(a)monetdb.org>, > 主题：Range query in MonetDB > 日期：2015-11-25 23:11:07 > > Since MonetDB actually ignore SQL statement CREATE INDEX, the performance sometimes is quite poor comparing to traditional DB. Such as: https://www.monetdb.org/pipermail/users-list/2008-April/001839.html, > http://stackoverflow.com/questions/22430284/monetdb-slow-query-with-index > > In reality, queries with ranges (such as date ranges) are very common and thus index is critical. Sometimes, it gives 100 times performance boost, especially when the queries are relatively simple. > > We debug into MonetDB source codes and found that CREATE INDEX only creates an entry in the system catalog. > > Is MonetDB going to support index in the coming future, or in some roadmap? > > > > > > > > > _______________________________________________ > users-list mailing list > users-list(a)monetdb.org > https://www.monetdb.org/mailman/listinfo/users-list > _______________________________________________ users-list mailing list users-list(a)monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

1 0

回复：Range query in MonetDB
by chenkaijiang＠bigknow.com.cn 25 Nov '15

25 Nov '15

Sometimes the traditional DB is 10 to 100 times faster with the help of the index. ----- 原始邮件 ----- 发件人：<chenkaijiang(a)bigknow.com.cn> 收件人：users-list<users-list(a)monetdb.org>,developers-list<developers-list(a)monetdb.org>, 主题：Range query in MonetDB 日期：2015-11-25 23:11:07 Since MonetDB actually ignore SQL statement CREATE INDEX, the performance sometimes is quite poor comparing to traditional DB. Such as: https://www.monetdb.org/pipermail/users-list/2008-April/001839.html, http://stackoverflow.com/questions/22430284/monetdb-slow-query-with-index In reality, queries with ranges (such as date ranges) are very common and thus index is critical. Sometimes, it gives 100 times performance boost, especially when the queries are relatively simple. We debug into MonetDB source codes and found that CREATE INDEX only creates an entry in the system catalog. Is MonetDB going to support index in the coming future, or in some roadmap?

2 1

Driver JDBC 2.8 dont work in JUL2015SP1
by Luciano Sasso 25 Nov '15

25 Nov '15

Driver JDBC 2.8 dont work in JUL2015SP1? I use 2.8 for ETL (run work with decimal, 2.18 dont work), and 2.18 for dashboards access. Att, -- Luciano Sasso Vieira Data Scientist & Solutions Architect luciano(a)gsgroup.com.br <http://www.gsgroup.com.br> | tel: 17 3353-0833 | cel: 17 99706-9335 www.gsgroup.com.br <http://www.gsgroup.com.br> --- Este email foi escaneado pelo Avast antivírus. https://www.avast.com/antivirus

1 0

Range query in MonetDB
by chenkaijiang＠bigknow.com.cn 25 Nov '15

25 Nov '15

Since MonetDB actually ignore SQL statement CREATE INDEX, the performance sometimes is quite poor comparing to traditional DB. Such as: https://www.monetdb.org/pipermail/users-list/2008-April/001839.htm… In reality, queries with ranges (such as date ranges) are very common and thus index is critical. Sometimes, it gives 100 times performance boost, especially when the queries are relatively simple. We debug into MonetDB source codes and found that CREATE INDEX only creates an entry in the system catalog.  Is MonetDB going to support index in the coming future, or in some roadmap? 

1 0

Serious error in data recording of type numeric
by Luciano Sasso 25 Nov '15

25 Nov '15

Serious error in data recording of type numeric. Using the jdbc driver 2.8, recording both base OUT2014SP4 when JUL2015SP1, records normally. But when using any jdbc driver above 2.8, the recording is wrong, for example, the value 845 is recorded as 850. Is it a bug? Att, -- Luciano Sasso Vieira Data Scientist & Solutions Architect luciano(a)gsgroup.com.br <http://www.gsgroup.com.br> | tel: 17 3353-0833 | cel: 17 99706-9335 www.gsgroup.com.br <http://www.gsgroup.com.br> --- Este email foi escaneado pelo Avast antivírus. https://www.avast.com/antivirus

1 0

MonetDB with dynamic columns
by Niklas Bivald 25 Nov '15

25 Nov '15

Hi, I've had my eyes on MonetDB for quite some time and the Numpy implementation really pushed me over the edge (as in this is the right solution for us). A big shout out to Mark Raasveldt for both the function and support and bug-fixes along the way. We have a fairly small dataset which probably won't exceed 10 million rows, 500 columns. We do flight data analysis so each row can be though of as a flight, and all the columns are data-points connected to it. We then use Numpy to prepare the data for graphs and limit our selection using standard SQL. We generally SELECT between 1-5 columns and use the WHERE statement on *another* 1-5 columns to select our rows. However, as we develop more analyses we connect more and more data-points (columns) to each row. Now there is a few ways we could deal with this: 1. Store the data in a column-less format (such as JSON) and re-create the MonetDB database when we need to add or remove columns (since MonetDB is really fast to read data into it, and we have no problem automating this process) - This is what we've done so far in our proof of concept 2. Store everything in MonetDB and use Schema alterations as new columns are added (requires altering in a live database) 3. Split the data into multiple tables (one for each analysis) and use JOIN to select them. This makes it not so much one huge table, but several small. Still need schema alterations when doing anything besides adding analysis. (4. Use a lookup table syntax, where we store three columns: ID, NAME, VALUE and instead of having 10 million rows and 500 columns use 10*500 rows - one row for each data point) - Since MonetDB excels at being a columnar storage putting everything into one table with only three columns seems like a very bad fit. Nevertheless it's a though. Before I spend more time doing proof-of-concepts on scenario 2 and 3 I just wanted to ask: Am I missing something? Is there any obvious solution that I am not covering? Is there any way to have "open" columns (i.e dynamic columns instead of fixed schema where the columns are created upon insert) Again, thank you so much for the work everyone of you is doing on MonetDB. I hope to start contributing as I learn more (probably just adding documentation short-term, possibly committing fixes in the long-term) Regards, Niklas

2 1

about replica table
by chenkaijiang＠bigknow.com.cn 25 Nov '15

25 Nov '15

Hi, from https://www.monetdb.org/Documentation/Manuals/SQLreference/Tables, there 4 types of tables: normal, merge, remote, replica. For merge and remote table, I found explaining on https://www.monetdb.org/Documentation/Cookbooks/SQLrecipes/Distribu…. But what is replica table? Any documents? Could somebody help?

2 1

Driver JDBC, error rounding numeric value
by Luciano Sasso 25 Nov '15

25 Nov '15

Any idea, about error driver JDBC rounding error. Driver 2.18 run error, driver 2.1 run work ok. -- Luciano Sasso Vieira Data Scientist & Solutions Architect luciano(a)gsgroup.com.br <http://www.gsgroup.com.br> | tel: 17 3353-0833 | cel: 17 99706-9335 www.gsgroup.com.br <http://www.gsgroup.com.br> --- Este email foi escaneado pelo Avast antivírus. https://www.avast.com/antivirus

1 0

Error driver JDBC
by Luciano Sasso 24 Nov '15

24 Nov '15

Hi, Iam use MonetDB JUL 2015-SPI, and driver JDBC 2.18, but error on load data, in rooud value, exampla, value 854 in source, load 850 in target. I am test with driver JDBC 2.1 and run word, value 845, load in target with 845. any idea !!! -- Luciano Sasso Vieira Data Scientist & Solutions Architect luciano(a)gsgroup.com.br <http://www.gsgroup.com.br> | tel: 17 3353-0833 | cel: 17 99706-9335 www.gsgroup.com.br <http://www.gsgroup.com.br> --- Este email foi escaneado pelo Avast antivírus. https://www.avast.com/antivirus

1 0

Vacuum limitations
by Sébastien RAILLARD (PASSMAN) 24 Nov '15

24 Nov '15

Dear all, I have tried to call the vacuum function using this command: call sys.vacuum('my_schema','my_table'); I have a message telling me "vacuum not allowed on tables with indices". So, if I understand correctly, the table on which vacuum can be done must not have any index, even a primary key? Best regards, Sebastien -- Parc d'activité Tolstoï 4 rue Edouard Aynard 69100 Villeurbanne Tél. +33 (0)4 78 95 05 80 Fax +33 (0)4 78 95 00 17 www.passman.fr www.passman-hotels.com www.passman-camping.com www.passman-sante.com <https://www.facebook.com/PASSMAN-187787814053/>

2 2