- developers-list - monetdb.org

Re: Question about : LOAD DATA
by Stefan Manegold 27 Mar '13

27 Mar '13

Please see my answer I sent to your identical question yesterday. Stefan -- | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) | | http://CWI.nl/~manegold/ | Science Park 123 (L321) | | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | ----- Reply message ----- From: "Bouslah Ahmed" <a.bouslah(a)yahoo.fr> To: "developers-list(a)monetdb.org" <developers-list(a)monetdb.org> Subject: Question about : LOAD DATA Date: Wed, Mar 27, 2013 19:44 Good afternoon, I have a csv file with 300000000 million lines. when I load it in a MonetDB table, it takes more than 16 hours, whereas in Oracle it only takes 15 minutes. Where is the problem? I use mclient as follows :? mclient -u user1 -d demo -ims -s " COPY 300000009 RECORDS INTO LINEORDER FROM STDIN USING DELIMITERS '|','\n'" - < D:\SSB\LINEORDER.tbl for information I use : MonetDB5 Windows Memory : 16 Go There are two constraints : primary key and foreign key on the table prior to loading data? cordialy.

1 0

Question about : LOAD DATA
by Bouslah Ahmed 27 Mar '13

27 Mar '13

Good afternoon, I have a csv file with 300000000 million lines. when I load it in a MonetDB table, it takes more than 16 hours, whereas in Oracle it only takes 15 minutes. Where is the problem? I use mclient as follows :? mclient -u user1 -d demo -ims -s " COPY 300000009 RECORDS INTO LINEORDER FROM STDIN USING DELIMITERS '|','\n'" - < D:\SSB\LINEORDER.tbl for information I use : MonetDB5 Windows Memory : 16 Go There are two constraints : primary key and foreign key on the table prior to loading data? cordialy.

1 0

Question in re remote.exec()
by Stephen P. Morgan 27 Mar '13

27 Mar '13

I have a question about remote.exec(). Let's say I have a user function, s1_1 that takes a BAT as a parameter and returns a BAT. Its signature might be: function user.s1_1(in:bat[:oid,:int]):bat[:oid,:int]; How can I call s1_1 using remote.exec to pass in a BAT and to retrieve the result? I want to do something like X_2 := remote.exec(conn, "user", "s1_1", X_1); where X_1 and X_2 are both :bat[:oid,:int]. But that doesn't seem to work. Thanks, Steve Morgan

3 3

Question
by Bouslah Ahmed 26 Mar '13

26 Mar '13

I have a csv file with 300000000 million lines. when I load it in a MonetDB table, it takes more than 16 hours, whereas in Oracle it only takes 15 minutes. Where is the problem? I use mclient as follows : mclient -u user1 -d demo -ims -s " COPY 300000009 RECORDS INTO LINEORDER FROM STDIN USING DELIMITERS '|','\n'" - < D:\SSB\LINEORDER.tbl cordially.

2 1

reply soon
by Meenesh sharma 24 Mar '13

24 Mar '13

sir, please tell can i add coding for variant stochastic cracking ....so, that i can analysis it with work done previously . . ... do help me -- " HAVE A NICE DAY"

1 0

do reply
by Meenesh sharma 22 Mar '13

22 Mar '13

i am doing my dissertation work on variant stochastic cracking .....so, i want to know which tool should i use to implement and then analysis it for various workload . do reply sir . . . i will very very thank ful to you .. . .please guide me ............. -- " HAVE A NICE DAY"

2 1

Re: MonetDB: default - approved output
by Sjoerd Mullender 20 Mar '13

20 Mar '13

Shouldn't this go to the Feb2013 branch? On 2013-03-20 15:55, Niels Nes wrote: > Changeset: 59ae06013de6 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=59ae06013de6 > Modified Files: > sql/backends/monet5/rel_bin.c > sql/test/BugTracker-2012/Tests/rewrite_like_into_likesubselect.Bug-3179.stable.out > Branch: default > Log Message: > > approved output > fixed bug view-view, bug 3245 > > > diffs (281 lines): > > diff --git a/sql/backends/monet5/rel_bin.c b/sql/backends/monet5/rel_bin.c > --- a/sql/backends/monet5/rel_bin.c > +++ b/sql/backends/monet5/rel_bin.c > @@ -2340,14 +2340,21 @@ rel2bin_select( mvc *sql, sql_rel *rel, > if (!sub) > return NULL; > sub = row2cols(sql, sub); > - } else { > + } > + if (!sub && !predicate) > predicate = rel2bin_predicate(sql); > - } > + else if (!predicate) > + predicate = stmt_const(sql->sa, bin_first_column(sql->sa, sub), stmt_bool(sql->sa, 1)); > if (!rel->exps->h) { > if (sub) > return sub; > return predicate; > } > + if (!sub && predicate) { > + list *l = sa_list(sql->sa); > + append(l, predicate); > + sub = stmt_list(sql->sa, l); > + } > /* handle possible index lookups */ > /* expressions are in index order ! */ > if (sub && (en = rel->exps->h) != NULL) { > @@ -2368,10 +2375,8 @@ rel2bin_select( mvc *sql, sql_rel *rel, > assert(0); > return NULL; > } > - if (s->nrcols == 0){ > - if (!predicate) > - predicate = rel2bin_predicate(sql); > - predicate = stmt_uselect(sql->sa, predicate, s, cmp_equal, NULL); > + if (s->nrcols == 0){ > + sel = stmt_uselect(sql->sa, predicate, s, cmp_equal, sel); > } else if (e->type != e_cmp) { > sel = stmt_uselect(sql->sa, s, stmt_bool(sql->sa, 1), cmp_equal, NULL); > } else { > @@ -2379,14 +2384,6 @@ rel2bin_select( mvc *sql, sql_rel *rel, > } > } > > - if (predicate && sel) { > - sel = stmt_reverse(sql->sa, sel); > - sel = stmt_join(sql->sa, sel, predicate, cmp_all); > - sel = stmt_result(sql->sa, sel, 0); > - predicate = NULL; > - if (!sub) > - predicate = sel; > - } > /* construct relation */ > l = sa_list(sql->sa); > if (sub && sel) { > @@ -2399,22 +2396,6 @@ rel2bin_select( mvc *sql, sql_rel *rel, > col = stmt_project(sql->sa, sel, col); > list_append(l, col); > } > - } else if (sub && predicate) { > - stmt *h = NULL; > - n = sub->op4.lval->h; > - h = stmt_join(sql->sa, column(sql->sa, n->data), predicate, cmp_all); > - h = stmt_result(sql->sa, h, 0); > - for( n = sub->op4.lval->h; n; n = n->next ) { > - stmt *col = n->data; > - > - if (col->nrcols == 0) /* constant */ > - col = stmt_const(sql->sa, h, col); > - else > - col = stmt_project(sql->sa, h, col); > - list_append(l, col); > - } > - } else if (predicate) { > - list_append(l, predicate); > } > return stmt_list(sql->sa, l); > } > diff --git a/sql/test/BugTracker-2012/Tests/rewrite_like_into_likesubselect.Bug-3179.stable.out b/sql/test/BugTracker-2012/Tests/rewrite_like_into_likesubselect.Bug-3179.stable.out > --- a/sql/test/BugTracker-2012/Tests/rewrite_like_into_likesubselect.Bug-3179.stable.out > +++ b/sql/test/BugTracker-2012/Tests/rewrite_like_into_likesubselect.Bug-3179.stable.out > @@ -29,17 +29,17 @@ Ready. > % .explain # table_name > % mal # name > % clob # type > -% 117 # length > +% 111 # length > function user.s0_1(A0:str):void; > X_2 := sql.mvc(); > X_6 := sql.bind(X_2,"sys","functions","name",0); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","functions"); > - X_49 := algebra.likesubselect(X_6,X_3,A0,"",false); > + X_48 := algebra.likesubselect(X_6,X_3,A0,"",false); > (X_9,r1_9) := sql.bind(X_2,"sys","functions","name",2); > - X_50 := algebra.likesubselect(r1_9,A0,"",false); > + X_49 := algebra.likesubselect(r1_9,A0,"",false); > X_12 := sql.bind(X_2,"sys","functions","name",1); > - X_51 := algebra.likesubselect(X_12,X_3,A0,"",false); > - X_15 := sql.subdelta(X_49,X_9,X_50,X_51); > + X_50 := algebra.likesubselect(X_12,X_3,A0,"",false); > + X_15 := sql.subdelta(X_48,X_9,X_49,X_50); > X_17 := sql.projectdelta(X_15,X_6,X_9,r1_9,X_12); > X_18 := sql.bind(X_2,"sys","functions","func",0); > (X_20,r1_28) := sql.bind(X_2,"sys","functions","func",2); > @@ -55,17 +55,17 @@ end s0_1; > % .explain # table_name > % mal # name > % clob # type > -% 121 # length > +% 115 # length > function user.s1_1(A0:str):void; > X_2 := sql.mvc(); > X_6 := sql.bind(X_2,"sys","functions","name",0); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","functions"); > - X_49 := algebra.likesubselect(X_6,X_3,A0,"",true); > + X_48 := algebra.likesubselect(X_6,X_3,A0,"",true); > (X_9,r1_9) := sql.bind(X_2,"sys","functions","name",2); > - X_50 := algebra.likesubselect(r1_9,A0,"",true); > + X_49 := algebra.likesubselect(r1_9,A0,"",true); > X_12 := sql.bind(X_2,"sys","functions","name",1); > - X_51 := algebra.likesubselect(X_12,X_3,A0,"",true); > - X_15 := sql.subdelta(X_49,X_9,X_50,X_51); > + X_50 := algebra.likesubselect(X_12,X_3,A0,"",true); > + X_15 := sql.subdelta(X_48,X_9,X_49,X_50); > X_17 := sql.projectdelta(X_15,X_6,X_9,r1_9,X_12); > X_18 := sql.bind(X_2,"sys","functions","func",0); > (X_20,r1_28) := sql.bind(X_2,"sys","functions","func",2); > @@ -81,17 +81,17 @@ end s1_1; > % .explain # table_name > % mal # name > % clob # type > -% 118 # length > +% 112 # length > function user.s2_1(A0:str):void; > X_2 := sql.mvc(); > X_6 := sql.bind(X_2,"sys","functions","name",0); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","functions"); > - X_49 := algebra.ilikesubselect(X_6,X_3,A0,"",false); > + X_48 := algebra.ilikesubselect(X_6,X_3,A0,"",false); > (X_9,r1_9) := sql.bind(X_2,"sys","functions","name",2); > - X_50 := algebra.ilikesubselect(r1_9,A0,"",false); > + X_49 := algebra.ilikesubselect(r1_9,A0,"",false); > X_12 := sql.bind(X_2,"sys","functions","name",1); > - X_51 := algebra.ilikesubselect(X_12,X_3,A0,"",false); > - X_15 := sql.subdelta(X_49,X_9,X_50,X_51); > + X_50 := algebra.ilikesubselect(X_12,X_3,A0,"",false); > + X_15 := sql.subdelta(X_48,X_9,X_49,X_50); > X_17 := sql.projectdelta(X_15,X_6,X_9,r1_9,X_12); > X_18 := sql.bind(X_2,"sys","functions","func",0); > (X_20,r1_28) := sql.bind(X_2,"sys","functions","func",2); > @@ -107,17 +107,17 @@ end s2_1; > % .explain # table_name > % mal # name > % clob # type > -% 122 # length > +% 116 # length > function user.s3_1(A0:str):void; > X_2 := sql.mvc(); > X_6 := sql.bind(X_2,"sys","functions","name",0); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","functions"); > - X_49 := algebra.ilikesubselect(X_6,X_3,A0,"",true); > + X_48 := algebra.ilikesubselect(X_6,X_3,A0,"",true); > (X_9,r1_9) := sql.bind(X_2,"sys","functions","name",2); > - X_50 := algebra.ilikesubselect(r1_9,A0,"",true); > + X_49 := algebra.ilikesubselect(r1_9,A0,"",true); > X_12 := sql.bind(X_2,"sys","functions","name",1); > - X_51 := algebra.ilikesubselect(X_12,X_3,A0,"",true); > - X_15 := sql.subdelta(X_49,X_9,X_50,X_51); > + X_50 := algebra.ilikesubselect(X_12,X_3,A0,"",true); > + X_15 := sql.subdelta(X_48,X_9,X_49,X_50); > X_17 := sql.projectdelta(X_15,X_6,X_9,r1_9,X_12); > X_18 := sql.bind(X_2,"sys","functions","func",0); > (X_20,r1_28) := sql.bind(X_2,"sys","functions","func",2); > @@ -153,19 +153,19 @@ end s3_1; > % .explain # table_name > % mal # name > % clob # type > -% 121 # length > +% 115 # length > function user.s4_1(A0:str):void; > X_2 := sql.mvc(); > X_6 := sql.bind(X_2,"sys","functions","name",0); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","functions"); > - X_52 := calc.+("%",A0); > - X_53 := calc.+(X_52,"%"); > - X_56 := algebra.likesubselect(X_6,X_3,X_53,"",false,false); > + X_51 := calc.+("%",A0); > + X_52 := calc.+(X_51,"%"); > + X_55 := algebra.likesubselect(X_6,X_3,X_52,"",false,false); > (X_9,r1_9) := sql.bind(X_2,"sys","functions","name",2); > - X_57 := algebra.likesubselect(r1_9,X_53,"",false,false); > + X_56 := algebra.likesubselect(r1_9,X_52,"",false,false); > X_12 := sql.bind(X_2,"sys","functions","name",1); > - X_58 := algebra.likesubselect(X_12,X_3,X_53,"",false,false); > - X_14 := sql.subdelta(X_56,X_9,X_57,X_58); > + X_57 := algebra.likesubselect(X_12,X_3,X_52,"",false,false); > + X_14 := sql.subdelta(X_55,X_9,X_56,X_57); > X_15 := sql.projectdelta(X_14,X_6,X_9,r1_9,X_12); > X_16 := sql.bind(X_2,"sys","functions","func",0); > (X_18,r1_35) := sql.bind(X_2,"sys","functions","func",2); > @@ -181,19 +181,19 @@ end s4_1; > % .explain # table_name > % mal # name > % clob # type > -% 125 # length > +% 119 # length > function user.s5_1(A0:str):void; > X_2 := sql.mvc(); > X_6 := sql.bind(X_2,"sys","functions","name",0); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","functions"); > - X_52 := calc.+("%",A0); > - X_53 := calc.+(X_52,"%"); > - X_57 := algebra.likesubselect(X_6,X_3,X_53,"",false,true); > + X_51 := calc.+("%",A0); > + X_52 := calc.+(X_51,"%"); > + X_56 := algebra.likesubselect(X_6,X_3,X_52,"",false,true); > (X_9,r1_9) := sql.bind(X_2,"sys","functions","name",2); > - X_58 := algebra.likesubselect(r1_9,X_53,"",false,true); > + X_57 := algebra.likesubselect(r1_9,X_52,"",false,true); > X_12 := sql.bind(X_2,"sys","functions","name",1); > - X_59 := algebra.likesubselect(X_12,X_3,X_53,"",false,true); > - X_14 := sql.subdelta(X_57,X_9,X_58,X_59); > + X_58 := algebra.likesubselect(X_12,X_3,X_52,"",false,true); > + X_14 := sql.subdelta(X_56,X_9,X_57,X_58); > X_15 := sql.projectdelta(X_14,X_6,X_9,r1_9,X_12); > X_16 := sql.bind(X_2,"sys","functions","func",0); > (X_18,r1_35) := sql.bind(X_2,"sys","functions","func",2); > @@ -209,19 +209,19 @@ end s5_1; > % .explain # table_name > % mal # name > % clob # type > -% 122 # length > +% 116 # length > function user.s6_1(A0:str):void; > X_2 := sql.mvc(); > X_6 := sql.bind(X_2,"sys","functions","name",0); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","functions"); > - X_52 := calc.+("%",A0); > - X_53 := calc.+(X_52,"%"); > - X_57 := algebra.likesubselect(X_6,X_3,X_53,"",true,false); > + X_51 := calc.+("%",A0); > + X_52 := calc.+(X_51,"%"); > + X_56 := algebra.likesubselect(X_6,X_3,X_52,"",true,false); > (X_9,r1_9) := sql.bind(X_2,"sys","functions","name",2); > - X_58 := algebra.likesubselect(r1_9,X_53,"",true,false); > + X_57 := algebra.likesubselect(r1_9,X_52,"",true,false); > X_12 := sql.bind(X_2,"sys","functions","name",1); > - X_59 := algebra.likesubselect(X_12,X_3,X_53,"",true,false); > - X_14 := sql.subdelta(X_57,X_9,X_58,X_59); > + X_58 := algebra.likesubselect(X_12,X_3,X_52,"",true,false); > + X_14 := sql.subdelta(X_56,X_9,X_57,X_58); > X_15 := sql.projectdelta(X_14,X_6,X_9,r1_9,X_12); > X_16 := sql.bind(X_2,"sys","functions","func",0); > (X_18,r1_35) := sql.bind(X_2,"sys","functions","func",2); > @@ -237,19 +237,19 @@ end s6_1; > % .explain # table_name > % mal # name > % clob # type > -% 126 # length > +% 120 # length > function user.s7_1(A0:str):void; > X_2 := sql.mvc(); > X_6 := sql.bind(X_2,"sys","functions","name",0); > X_3:bat[:oid,:oid] := sql.tid(X_2,"sys","functions"); > - X_52 := calc.+("%",A0); > - X_53 := calc.+(X_52,"%"); > - X_56 := algebra.likesubselect(X_6,X_3,X_53,"",true,true); > + X_51 := calc.+("%",A0); > + X_52 := calc.+(X_51,"%"); > + X_55 := algebra.likesubselect(X_6,X_3,X_52,"",true,true); > (X_9,r1_9) := sql.bind(X_2,"sys","functions","name",2); > - X_57 := algebra.likesubselect(r1_9,X_53,"",true,true); > + X_56 := algebra.likesubselect(r1_9,X_52,"",true,true); > X_12 := sql.bind(X_2,"sys","functions","name",1); > - X_58 := algebra.likesubselect(X_12,X_3,X_53,"",true,true); > - X_14 := sql.subdelta(X_56,X_9,X_57,X_58); > + X_57 := algebra.likesubselect(X_12,X_3,X_52,"",true,true); > + X_14 := sql.subdelta(X_55,X_9,X_56,X_57); > X_15 := sql.projectdelta(X_14,X_6,X_9,r1_9,X_12); > X_16 := sql.bind(X_2,"sys","functions","func",0); > (X_18,r1_35) := sql.bind(X_2,"sys","functions","func",2); > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > -- Sjoerd Mullender

1 0

Re: MonetDB: default - Toptimize doesn't measure anything usefull, i...
by Martin Kersten 20 Mar '13

20 Mar '13

I agree, was looking at it as well. Idea was take the time of all optimizers together. sql>call querylog_enable(); sql>select * from querylog_catalog(); +----+-------+---------+-------+------+-----+----------+ | id | owner | defined | query | pipe | mal | optimize | +====+=======+=========+=======+======+=====+==========+ +----+-------+---------+-------+------+-----+----------+ 0 tuples (1.752ms) sql>call querylog_enable(); sql>select count(*) from tables; +------+ | L1 | +======+ | 33 | +------+ 1 tuple (3.112ms) sql>select * from querylog_catalog(); +-----------+---------+----------------------------+------------------------------+--------------+------+----------+ | id | owner | defined | query | pipe | mal | optimize | +===========+=========+============================+==============================+==============+======+==========+ | 1359324@0 | monetdb | 2013-03-20 13:22:14.000000 | select count(*) from tables; | default_pipe | 54 | 69124831 | +-----------+---------+----------------------------+------------------------------+--------------+------+----------+ 1 tuple (1.740ms) sql>select * from querylog_calls(); +-----------+----------------------------+----------------------------+--------------+--------+------+------+------+------+-------+ | id | start | stop | arguments | tuples | run | ship | cpu | io | space | +===========+============================+============================+==============+========+======+======+======+======+=======+ | 1359324@0 | 2013-03-20 13:22:14.000000 | 2013-03-20 13:22:14.000000 | user.s3_1(); | 1 | 742 | 14 | 0 | 0 | 0 | +-----------+----------------------------+----------------------------+--------------+--------+------+------+------+------+-------+ On 3/20/13 2:21 PM, Niels Nes wrote: > Changeset: 6bd79fe72615 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=6bd79fe72615 > Modified Files: > monetdb5/mal/mal_runtime.c > sql/backends/monet5/sql_gencode.c > Branch: default > Log Message: > > Toptimize doesn't measure anything usefull, ie just store 0 for the time > being (solves all problems with changing output). We need a different > solution to store the time spend in the optimizer stack. > > > diffs (34 lines): > > diff --git a/monetdb5/mal/mal_runtime.c b/monetdb5/mal/mal_runtime.c > --- a/monetdb5/mal/mal_runtime.c > +++ b/monetdb5/mal/mal_runtime.c > @@ -172,8 +172,8 @@ runtimeProfileExit(Client cntxt, MalBlkP > if (malProfileMode == 0) > return; /* mostly true */ > > - if (stk != NULL && prof->stkpc >= 0 && mb->profiler != NULL && mb->profiler[stkpc].trace && mb->profiler[stkpc].clk) > - { > + if (stk != NULL && prof->stkpc >= 0 && mb->profiler != NULL && > + mb->profiler[stkpc].trace && mb->profiler[stkpc].clk) { > MT_lock_set(&mal_contextLock, "sysmon"); > gettimeofday(&mb->profiler[stkpc].clock, NULL); > mb->profiler[stkpc].counter++; > diff --git a/sql/backends/monet5/sql_gencode.c b/sql/backends/monet5/sql_gencode.c > --- a/sql/backends/monet5/sql_gencode.c > +++ b/sql/backends/monet5/sql_gencode.c > @@ -2408,7 +2408,7 @@ backend_dumpproc(backend *be, Client c, > { > char *t; > InstrPtr q; > - lng Toptimize = GDKusec(); > + lng Toptimize = 0; > > if ( be->q && be->q->codestring) { > t = GDKstrdup( be->q->codestring); > @@ -2421,7 +2421,6 @@ backend_dumpproc(backend *be, Client c, > q->token = REMsymbol; // will be patched > q = pushStr(mb, q, t); > q = pushStr(mb, q, pipe= initSQLoptimizer()); > - Toptimize = GDKusec() - Toptimize; > (void) pushLng(mb, q, Toptimize); > m->Tparse = 0; > GDKfree(pipe); > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list >

1 0

Re: MonetDB: default - Reduce critical section in mal_resource
by Sjoerd Mullender 19 Mar '13

19 Mar '13

Why are you messing with this code? What was wrong? The ATOMIC_CAS was the initialization of the variable running. It needs to happen. Why did you move it? The ATOMIC_GET that you first commented out and now removed actually already was just reading the variable without locks on most systems. Why did you replace it? It doesn't make *any* difference on x86_64 systems. Obviously you didn't measure the impact since there cannot have been any. On 2013-03-19 21:27, Martin Kersten wrote: > Changeset: a43f806ebd08 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=a43f806ebd08 > Modified Files: > monetdb5/mal/mal_resource.c > Branch: default > Log Message: > > Reduce critical section in mal_resource > > > diffs (31 lines): > > diff --git a/monetdb5/mal/mal_resource.c b/monetdb5/mal/mal_resource.c > --- a/monetdb5/mal/mal_resource.c > +++ b/monetdb5/mal/mal_resource.c > @@ -198,7 +198,6 @@ MALresourceFairness(lng usec) > if ( usec > 0 && ( (usec = GDKusec()-usec)) <= TIMESLICE ) > return; > threads = GDKnr_threads > 0 ? GDKnr_threads : 1; > - ATOMIC_CAS_int(running, 0, threads, runningLock, "MALresourceFairness"); > > /* use GDKmem_cursize as MT_getrss(); is to expensive */ > rss = GDKmem_cursize(); > @@ -210,16 +209,14 @@ MALresourceFairness(lng usec) > clk = usec / 1000; > > if ( clk > DELAYUNIT ) { > + ATOMIC_CAS_int(running, 0, threads, runningLock, "MALresourceFairness"); > PARDEBUG mnstr_printf(GDKstdout, "#delay initial "LLFMT"n", clk); > ATOMIC_DEC_int(running, runningLock, "MALresourceFairness"); > - while (clk > 0) { > + /* always keep one running to avoid all waiting */ > + while (clk > 0 && running >= 2) { > /* speed up wake up when we have memory */ > if (rss < MEMORY_THRESHOLD * monet_memory) > break; > - /* always keep one running to avoid all waiting */ > - if ( running < 2) /* dirty read of shared variable is safe here */ > - //if ( (r =ATOMIC_GET_int(running, runningLock, "MALresourceFairness")) < 2) > - break; > delay = (unsigned int) ( ((double)DELAYUNIT * running) / threads); > if (delay) { > if ( delayed++ == 0){ > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list > -- Sjoerd Mullender

1 0

Questions About Generated MAL Code
by Stephen P. Morgan 13 Mar '13

13 Mar '13

Hi, My colleagues and I from Huawei have been experimenting with the MAL code generated by SQL explain. We noticed big differences between the code generated by version 15 and the present top-of-tree. For example, where the level-15 code tended to generate just one barrier-exit block for a simple "select *," the new code generates several. (That might have to do with views--that's my guess, anyway.) Unfortunately, whereas the "old" (level 15) code is manually executable by the MAL interpreter, the new code doesn't run directly. For example, the interpreter complains about uninitialized variables. Of course, this is straightforward to fix manually, though somewhat tedious. But we also find when running multiple-barrier code, we get many complaints about undefined things, e.g., sql.mvc(), sql.bind(), among others. We don't (yet?) know how to overcome this issue. Is this just a temporary situation, or will it be possible at some point again to manually run MAL code? (Is it a policy of the MonetDB project that explained MAL code should be manually executable?) Would it be possible to explain (briefly) the reasons for the differences between the two versions of generated MAL code? We're curious what would prompt such a big change in the generated code. Thank you, Steve Morgan P.S. I've created a version of the mapi.connect() and mapi.reconnect() code that accepts a database name as another parameter. We've used it to send RPCs between systems (Linux containers, actually). We would be happy to donate this code to your organization, if you would like it. I added a new MAL routine for mapi.connect(), i.e., SERVERconnectWithDBname() and similar things for the mapi.reconnect() calls that detects the extra parameter and uses it if it's there. It shouldn't break any existing code, and it allows one to run RPCs while using monetdbd (not just mserver5).

1 0