April 2018 - developers-list

Vakantie
by Martin Kersten 22 Apr '18

22 Apr '18

wanneer was je op vakantie? Sent from my iPad > On 22 Apr 2018, at 11:54, Niels Nes <commits+niels=cwi.nl(a)monetdb.org> wrote: > > Changeset: 354f2d85e58b for MonetDB > URL: https://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=354f2d85e58b > Added Files: > sql/test/BugTracker-2018/Tests/sqlitelogictest-complex-expression.Bug-6589.stable.err > sql/test/BugTracker-2018/Tests/sqlitelogictest-complex-expression.Bug-6589.stable.out > Modified Files: > sql/server/rel_select.c > Branch: Mar2018 > Log Message: > > fixes for bug 6589 > > > diffs (99 lines): > > diff --git a/sql/server/rel_select.c b/sql/server/rel_select.c > --- a/sql/server/rel_select.c > +++ b/sql/server/rel_select.c > @@ -2098,7 +2098,10 @@ rel_logical_value_exp(mvc *sql, sql_rel > > r = rel_value_exp(sql, &z, sval, f, ek); > if (l && r && IS_ANY(st->type->eclass)){ > - l = rel_check_type(sql, exp_subtype(r), l, type_equal); > + sql_exp *nl = rel_check_type(sql, exp_subtype(r), l, type_equal); > + if (nl != l && is_new) > + left = rel_project_exp(sql->sa, nl); > + l = nl; > if (l) > st = exp_subtype(l); > } > diff --git a/sql/test/BugTracker-2018/Tests/sqlitelogictest-complex-expression.Bug-6589.stable.err b/sql/test/BugTracker-2018/Tests/sqlitelogictest-complex-expression.Bug-6589.stable.err > new file mode 100644 > --- /dev/null > +++ b/sql/test/BugTracker-2018/Tests/sqlitelogictest-complex-expression.Bug-6589.stable.err > @@ -0,0 +1,35 @@ > +stderr of test 'sqlitelogictest-complex-expression.Bug-6589` in directory 'sql/test/BugTracker-2018` itself: > + > + > +# 11:47:28 > > +# 11:47:28 > "mserver5" "--debug=10" "--set" "gdk_nr_threads=0" "--set" "mapi_open=true" "--set" "mapi_port=31863" "--set" "mapi_usock=/var/tmp/mtest-18504/.s.monetdb.31863" "--set" "monet_prompt=" "--forcemito" "--dbpath=/home/niels/scratch/rc-old/Linux-x86_64/var/MonetDB/mTests_sql_test_BugTracker-2018" "--set" "embedded_c=true" > +# 11:47:28 > > + > +# builtin opt gdk_dbpath = /home/niels/scratch/rc-old/Linux-x86_64/var/monetdb5/dbfarm/demo > +# builtin opt gdk_debug = 0 > +# builtin opt gdk_vmtrim = no > +# builtin opt monet_prompt = > > +# builtin opt monet_daemon = no > +# builtin opt mapi_port = 50000 > +# builtin opt mapi_open = false > +# builtin opt mapi_autosense = false > +# builtin opt sql_optimizer = default_pipe > +# builtin opt sql_debug = 0 > +# cmdline opt gdk_nr_threads = 0 > +# cmdline opt mapi_open = true > +# cmdline opt mapi_port = 31863 > +# cmdline opt mapi_usock = /var/tmp/mtest-18504/.s.monetdb.31863 > +# cmdline opt monet_prompt = > +# cmdline opt gdk_dbpath = /home/niels/scratch/rc-old/Linux-x86_64/var/MonetDB/mTests_sql_test_BugTracker-2018 > +# cmdline opt embedded_c = true > +# cmdline opt gdk_debug = 553648138 > + > +# 11:47:28 > > +# 11:47:28 > "mclient" "-lsql" "-ftest" "-tnone" "-Eutf-8" "-i" "-e" "--host=/var/tmp/mtest-18504" "--port=31863" > +# 11:47:28 > > + > + > +# 11:47:28 > > +# 11:47:28 > "Done." > +# 11:47:28 > > + > diff --git a/sql/test/BugTracker-2018/Tests/sqlitelogictest-complex-expression.Bug-6589.stable.out b/sql/test/BugTracker-2018/Tests/sqlitelogictest-complex-expression.Bug-6589.stable.out > new file mode 100644 > --- /dev/null > +++ b/sql/test/BugTracker-2018/Tests/sqlitelogictest-complex-expression.Bug-6589.stable.out > @@ -0,0 +1,39 @@ > +stdout of test 'sqlitelogictest-complex-expression.Bug-6589` in directory 'sql/test/BugTracker-2018` itself: > + > + > +# 11:47:28 > > +# 11:47:28 > "mserver5" "--debug=10" "--set" "gdk_nr_threads=0" "--set" "mapi_open=true" "--set" "mapi_port=31863" "--set" "mapi_usock=/var/tmp/mtest-18504/.s.monetdb.31863" "--set" "monet_prompt=" "--forcemito" "--dbpath=/home/niels/scratch/rc-old/Linux-x86_64/var/MonetDB/mTests_sql_test_BugTracker-2018" "--set" "embedded_c=true" > +# 11:47:28 > > + > +# MonetDB 5 server v11.29.2 > +# This is an unreleased version > +# Serving database 'mTests_sql_test_BugTracker-2018', using 8 threads > +# Compiled for x86_64-unknown-linux-gnu/64bit > +# Found 15.387 GiB available main-memory. > +# Copyright (c) 1993 - July 2008 CWI. > +# Copyright (c) August 2008 - 2018 MonetDB B.V., all rights reserved > +# Visit https://www.monetdb.org/ for further information > +# Listening for connection requests on mapi:monetdb://xps13:31863/ > +# Listening for UNIX domain connection requests on mapi:monetdb:///var/tmp/mtest-18504/.s.monetdb.31863 > +# MonetDB/GIS module loaded > +# MonetDB/SQL module loaded > + > +Ready. > + > +# 11:47:28 > > +# 11:47:28 > "mclient" "-lsql" "-ftest" "-tnone" "-Eutf-8" "-i" "-e" "--host=/var/tmp/mtest-18504" "--port=31863" > +# 11:47:28 > > + > +#SELECT DISTINCT + - 82 * + 26 - 12 * + - 27 * + + CASE WHEN NOT NULL > +#IN ( - 69, ( + SUM ( DISTINCT CAST ( NULL AS INTEGER ) ) ) + + MAX ( - 6 ) + + - COUNT ( * ) / - - ( + 77 ) + 37, + 44, > +#- - 0 * + + 23, - - 86, 3, 12 ) THEN + 41 WHEN NOT - 21 / - 95 IS NOT NULL THEN 63 END AS col0; > +% .L42 # table_name > +% col0 # name > +% int # type > +% 1 # length > +[ NULL ] > + > +# 11:47:28 > > +# 11:47:28 > "Done." > +# 11:47:28 > > + > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > https://www.monetdb.org/mailman/listinfo/checkin-list

1 1

INSERT using multiple cores (inspired by COPY from CSV in LOCKED mode)
by Anton Kravchenko 12 Apr '18

12 Apr '18

Hi there, 1) It looks like INSERTing into tables is bounded by a single CPU. And it kills performance of a query that normally uses multiple CPUs... CREATE TEMP TABLE t1 AS (some 'select query' that uses multiple cores when run standalone) ON COMMIT PRESERVE ROWS; or INSERT into t1 (some 'select query' that uses multiple cores when run standalone); 2) Do you think you could implement INSERT with LOCKED option? (like it was done for COPY from CSV in LOCKED mode) Please let me know whether you think it's doable. Thank you, Anton

1 1

ordered index Error in optimizer garbageCollector
by Anton Kravchenko 06 Apr '18

06 Apr '18

Here is a reproducible example. create table t1(v0 int, v1 char(1)); insert into t1 values (1,'a'),(2,'b'),(3,'c'); create ordered index index_t1_v1 on t1(v1); create table t2(v1 char(1)); insert into t2 values ('a'); 1) gives "Error in optimizer garbageCollector" create temp table t3 as (select t1.v0 from t1,t2 where trim(t2.v1)=t1.v1) on commit preserve rows; 2) works as expected create temp table t3 as (select t1.v0 from t1,t2 where t2.v1=t1.v1) on commit preserve rows; 3) works as expected select t1.v0 from t1,t2 where trim(t2.v1)=t1.v1; 4) works as expected drop index index_t1_v1; create temp table t3 as (select t1.v0 from t1,t2 where trim(t2.v1)=t1.v1) on commit preserve rows; I am using the latest MonetDB version: MonetDB 5 server v11.29.3 "Mar2018" (64-bit, 128-bit integers) Copyright (c) 1993 - July 2008 CWI Copyright (c) August 2008 - 2018 MonetDB B.V., all rights reserved Visit https://www.monetdb.org/ for further information Found 31.3GiB available memory, 4 available cpu cores Libraries: libpcre: 8.32 2012-11-30 (compiled with 8.32) openssl: OpenSSL 1.0.2k 26 Jan 2017 (compiled with ) libxml2: 2.9.1 (compiled with 2.9.1) Compiled by: mockbuild@ (x86_64-redhat-linux-gnu) Compilation: gcc -std=gnu99 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic Linking : /usr/bin/ld -m elf_x86_64 -Wl,-z,relro -Wl,-Bsymbolic-functions Thanks, Anton

1 0