- developers-list - monetdb.org

[Monetdb-developers] trace in SQL....
by Romulo Goncalves 17 Nov '08

17 Nov '08

If I use the latest stable version of M5-SQL and I trace a query I can't see the execution time for each MAL instruction. What I get now is the same output as the one output by the explain plus the mal code ends with #end s0_1; # mdb.setTrace(false); Was it a decision or a mistake done during the implementation of the new logging tool called History? Romulo

1 0

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/storage store.mx, Nov2008, 1.22.2.1, 1.22.2.2
by Sjoerd Mullender 14 Nov '08

14 Nov '08

Romulo Goncalves wrote: > Update of /cvsroot/monetdb/sql/src/storage > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv25673 > > Modified Files: > Tag: Nov2008 > store.mx > Log Message: > I got the single thing.... > Sorry the mistake... This is exactly what I mean with being *very* conservative and being *absolutely* sure. This error makes me very unhappy. > > U store.mx > Index: store.mx > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/storage/store.mx,v > retrieving revision 1.22.2.1 > retrieving revision 1.22.2.2 > diff -u -d -r1.22.2.1 -r1.22.2.2 > --- store.mx 14 Nov 2008 13:46:52 -0000 1.22.2.1 > +++ store.mx 14 Nov 2008 14:04:09 -0000 1.22.2.2 > @@ -110,7 +110,7 @@ > } > list_destroy(k->columns); > base_destroy(&k->base); > - if ((k->type == pkey) && (k->t->pkey = (sql_ukey *) k)) > + if ((k->type == pkey) && (k->t->pkey == (sql_ukey *) k)) > k->t->pkey = NULL; > _DELETE(k); > } > > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins -- Sjoerd Mullender

2 1

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/storage store.mx, Nov2008, 1.22, 1.22.2.1
by Sjoerd Mullender 14 Nov '08

14 Nov '08

Romulo Goncalves wrote: > Update of /cvsroot/monetdb/sql/src/storage > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv24421 > > Modified Files: > Tag: Nov2008 > store.mx > Log Message: > Set the pkey from the table structure to NULL to avoid garbage when we are checking if the table > already contain a specific pkey... > > > U store.mx > Index: store.mx > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/storage/store.mx,v > retrieving revision 1.22 > retrieving revision 1.22.2.1 > diff -u -d -r1.22 -r1.22.2.1 > --- store.mx 30 Oct 2008 08:48:54 -0000 1.22 > +++ store.mx 14 Nov 2008 13:46:52 -0000 1.22.2.1 > @@ -110,6 +110,8 @@ > } > list_destroy(k->columns); > base_destroy(&k->base); > + if ((k->type == pkey) && (k->t->pkey = (sql_ukey *) k)) Do you really mean a single = here? ^ > + k->t->pkey = NULL; > _DELETE(k); > } > > > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins -- Sjoerd Mullender

2 2

[Monetdb-developers] closing the stable branch
by Sjoerd Mullender 14 Nov '08

14 Nov '08

I am closing the stable branch for further checkins. We need to converge to a release. Today I will only still accept fixes that make that things compile on all platforms. I do not want anything that could possibly break tonight's compilation or testing. Note, this is only for the Nov2008 stable branch. -- Sjoerd Mullender

2 4

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/tests/BugTracker/Tests All, , 1.119, 1.120
by Lefteris 14 Nov '08

14 Nov '08

I jsut runned the fn_collection_after_delete.SF-2141000 test and did not produced any errors. I was not aware that it was disabled. lefteris On Fri, Nov 14, 2008 at 12:53 PM, Stefan Manegold <stmane(a)users.sourceforge.net> wrote: > Update of /cvsroot/monetdb/pathfinder/tests/BugTracker/Tests > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv18290/pathfinder/tests/BugTracker/Tests > > Modified Files: > All > Log Message: > > Since Lefteris implemented fn:collection() (also) for the Algebra back-end, > we can re-enable the fn_collection_after_delete.SF-2141000 test (also) for the Algebra back-end. > > Wrapped changes in Lefteris's "fn_collection_before" & "fn_collection_after" tags. > > > U All > Index: All > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/tests/BugTracker/Tests/All,v > retrieving revision 1.119 > retrieving revision 1.120 > diff -u -d -r1.119 -r1.120 > --- All 15 Oct 2008 10:32:18 -0000 1.119 > +++ All 14 Nov 2008 11:53:03 -0000 1.120 > @@ -164,7 +164,7 @@ > duplicates_error.SF-2142522 > order_by.SF-2140992 > combination_of_constructed_and_doc_elements.SF-2059394 > -NOT_ALGEBRA?fn_collection_after_delete.SF-2141000 > +fn_collection_after_delete.SF-2141000 > XML_attribute_generation.SF-2038906-unpack_doc > XML_attribute_generation.SF-2038906 > empty_file.SF-2017862 > > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins >

1 0

[Monetdb-developers] Transactions
by Eric M. Hielscher 13 Nov '08

13 Nov '08

Hi all, My name is Eric Hielscher, and I'm a PhD student working on a database project. The goal will be to replace the transaction processing subsystem of a dbms, and I've decided to use MonetDB. I've yet to really look much at the source. If anyone has the time, I'd appreciate some tips about where to look for: - The transaction processing code - The scheduling of transaction execution This is of course something I could do on my own, but would be more efficient if someone with expert knowledge could just quickly walk me through things. Thanks in advance, -- Eric Hielscher

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime/xrpc/admin admin.xq, , 1.10, 1.11 xrpcwebclient.js, , 1.12, 1.13
by Ying Zhang 12 Nov '08

12 Nov '08

On Fri, Oct 31, 2008 at 03:26:46PM +0000, Sjoerd Mullender wrote: > Update of /cvsroot/monetdb/pathfinder/runtime/xrpc/admin > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv25251/runtime/xrpc/admin > > Modified Files: > admin.xq xrpcwebclient.js > Log Message: > Fixed administration console: > > - Reverted back the merge from the xrpcdemo branch since it did not > work at all: the function XRPC was called with too few parameters, > and the callback function was never called. Hi Sjoerd, Thanks for trying fix the admin gui. However, simply revert the changes does not seem to help to improve the situation, i.e., executing an admin function just returns error. Since the JS script before the merge is out-dated, I would prefer to fix things based on the latest version. Hence, is it ok if I undo your reverting of xrpcwebclient.js, while keeping your bonus (see below)? Jennie > - Reintroduced the logger_base variable in pf_logmanager since it is > used by the statistics call. > - Added a CATCH in the statistics implementation so that the lock is > freed even if an error occurs. > - If there are no documents, the sum() in the statistics > implementation returns nil which causes serialization of the answer > to fail. > > As a bonus, replaced the non-ASCII characters in xrpcwebclient.js with > octal escapes so that the file can also be viewed in a UTF-8 > environment. > > > U admin.xq > Index: admin.xq > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/xrpc/admin/admin.xq,v > retrieving revision 1.10 > retrieving revision 1.11 > diff -u -d -r1.10 -r1.11 > --- admin.xq 6 Jul 2008 11:20:06 -0000 1.10 > +++ admin.xq 31 Oct 2008 15:26:44 -0000 1.11 > @@ -86,7 +86,7 @@ > (: =================== server management =================== :) > > declare function admin:db-stats() > -{ pf:mil('lock_set(pf_short); var ret := bat(str,str); var tot := 0LL; colname_runtime@batloop() tot :+= sum([batsize]($t)); ret.insert("xquery_index_curMB", str(tot/1048576LL)); tot := doc_timestamp.select(timestamp_nil,timestamp_nil).reverse(); ret.insert("xquery_cache_curdocs", str(count(tot))); tot := tot.join(doc_collection).tunique().reverse().join(collection_size).sum(); ret.insert("xquery_cache_curMB", str(tot/1048576LL)); ret.insert("xquery_log_curMB", str((logger_changes(pf_logger) - logger_base)/131072)); ret.insert("gdk_vm_cursize", str(vm_cursize())); ret.insert("gdk_mem_cursize", str(mem_cursize())); lock_unset(pf_short); return ret;') }; > +{ pf:mil('var ret; var err := CATCH({ lock_set(pf_short); ret := bat(str,str); var tot := 0LL; colname_runtime@batloop() tot :+= sum([batsize]($t)); ret.insert("xquery_index_curMB", str(tot/1048576LL)); tot := doc_timestamp.select(timestamp_nil,timestamp_nil).reverse(); ret.insert("xquery_cache_curdocs", str(count(tot))); tot := tot.join(doc_collection).tunique().reverse().join(collection_size).sum(); if (isnil(tot)) tot := 0LL; ret.insert("xquery_cache_curMB", str(tot/1048576LL)); ret.insert("xquery_log_curMB", str((logger_changes(pf_logger) - logger_base)/131072)); ret.insert("gdk_vm_cursize", str(vm_cursize())); ret.insert("gdk_mem_cursize", str(mem_cursize())); }); lock_unset(pf_short); if (not(isnil(err))) ERROR(err); return ret;') }; > > declare function admin:db-env() > { pf:mil('var dels := new(void, str, 20).append("exec_prefix").append("prefix").append("gdk_debug").append("gdk_embedded").append("gdk_vm_minsize").append("mapi_debug").append("mapi_noheaders").append("monet_daemon").append("monet_mod_path").append("monet_pid").append("monet_prompt").append("monet_welcome").append("sql_debug").append("sql_logdir").reverse(); return monet_environment.kdiff(dels).access(BAT_WRITE).insert("gdk_mem_maxsize", str(mem_maxsize())).sort();') }; > > U xrpcwebclient.js > Index: xrpcwebclient.js > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/xrpc/admin/xrpcwebclient.js,v > retrieving revision 1.12 > retrieving revision 1.13 > diff -u -d -r1.12 -r1.13 > --- xrpcwebclient.js 3 Jul 2008 15:10:37 -0000 1.12 > +++ xrpcwebclient.js 31 Oct 2008 15:26:44 -0000 1.13 > @@ -4,28 +4,18 @@ > moduleurl, /* module (physical) at-hint URL. Module file must be here! */ > method, /* method name (matches function name in module) */ > arity, /* arity of the method */ > - updating, /* whether the function is an updating function */ > call, /* one or more XRPC_CALL() parameter specs (concatenated strings) */ > - callback, /* callback function to call with the XML response */ > - timeout, /* timeout value, when > 0 repeatable isolation level is presumed */ > - mode) /* (none | repeatable) [-iterative][-trace] */ > -{ > - clnt.sendReceive(posturl, method, XRPC_REQUEST(module,moduleurl,method,arity,updating,call,timeout,mode), callback); > -} > - > -function XRPC_PART(geturl, /* Your XRPC server. Usually: "http://yourhost:yourport/xrpc" */ > callback) /* callback function to call with the XML response */ > { > - clntPart.sendReceivePart(geturl, callback); > + clnt.sendReceive(posturl, method, XRPC_REQUEST(module,moduleurl,method,arity,call), callback); > } > > /********************************************************************** > functions to construct valid XRPC soap requests > ***********************************************************************/ > > -function XRPC_REQUEST(module, moduleurl, method, arity, updating, body, timeout, mode) > -{ > - return '<?xml version="1.0" encoding="utf-8"?>\n' + > +function XRPC_REQUEST(module, moduleurl, method, arity, body) { > + var r = '<?xml version="1.0" encoding="utf-8"?>\n' + > '<env:Envelope ' + > 'xmlns:env="http://www.w3.org/2003/05/soap-envelope" ' + > 'xmlns:xrpc="http://monetdb.cwi.nl/XQuery" ' + > @@ -36,11 +26,10 @@ > '<xrpc:request xrpc:module="' + module + '" ' + > 'xrpc:location="' + moduleurl + '" ' + > 'xrpc:method="' + method + '" ' + > - 'xrpc:mode="' + mode + '" ' + > - 'xrpc:updCall="' + (updating?"true":"false") + '" ' + > 'xrpc:arity="' + arity + '">' + > body > + '</xrpc:request></env:Body></env:Envelope>'; > + return r; > } > > /* a body consists of one or more calls */ > @@ -83,25 +72,6 @@ > } > } > > -function string2XML(text) { > - try //Internet Explorer > - { > - xmlDoc=new ActiveXObject("Microsoft.XMLDOM"); > - xmlDoc.async="false"; > - xmlDoc.loadXML(text); > - } > - catch(e) > - { > - try //Firefox, Mozilla, Opera, etc. > - { > - parser=new DOMParser(); > - xmlDoc=parser.parseFromString(text,"text/xml"); > - } > - catch(e) {alert(e.message)} > - } > - return xmlDoc; > -} > - > function getnodesXRPC(node,tagname) { > try { > return node.getElementsByTagNameNS("http://monetdb.cwi.nl/XQuery",tagname); > @@ -122,75 +92,21 @@ > } > } > > -XRPCWebClientPart = function () { > - if (window.XMLHttpRequest) { > - this.xmlhttp = new XMLHttpRequest(); > - } else if (window.ActiveXObject) { > - try { > - this.xmlhttp = new ActiveXObject("Msxml2.XMLHTTP"); > - } catch(e) { > - this.xmlhttp = new ActiveXObject("Microsoft.XMLHTTP"); > - } > - } > -} > - > XRPCWebClient.prototype.sendReceive = function(posturl, method, request, callback) { > try { > - this.xmlhttp.open("POST", posturl, true); > - //alert(request); > - if (XRPCDEBUG && method != 'getdoc') { > - //document.getElementById("messreq").value = request; > - messreqChanged(string2XML(request)); > - } > - this.xmlhttp.send(request); > - var app = this; > - > - this.xmlhttp.onreadystatechange = function() { > - if (app.xmlhttp.readyState == 4 ) { > - if (app.xmlhttp.status == 200 && > - app.xmlhttp.responseText.indexOf("!ERROR") < 0 && > - app.xmlhttp.responseText.indexOf("<env:Fault>") < 0) > - { > - if (XRPCDEBUG) { > - if (app.xmlhttp.responseText) { > - > - > - if(method != 'getdoc') messresChanged(app.xmlhttp.responseXML? app.xmlhttp.responseXML: string2XML(app.xmlhttp.responseText)); > - callback(app.xmlhttp.responseXML? app.xmlhttp.responseXML: string2XML(app.xmlhttp.responseText)); > - } > - } > - } else { > - var errmsg = > - '!ERROR: "' + method + ' execution failed at the remote side"\n\n' + > - '!ERROR: HTTP/1.1 ' + app.xmlhttp.status + '\n' + > - '!ERROR: HTTP Response:\n\n\t' + app.xmlhttp.responseText; > - alert(errmsg); > - return null; > - } > - } > - }; > - } catch (e) { > - alert('sendRequest('+posturl,','+method+'): '+e); > - } > -} > + this.xmlhttp.open("POST", posturl, true); > + if (XRPCDEBUG) alert(request); > + this.xmlhttp.send(request); > > -XRPCWebClientPart.prototype.sendReceivePart = function(geturl, callback) { > - try { > - //alert("get " + geturl); > - this.xmlhttp.open("GET", geturl, true); > - this.xmlhttp.send(""); > var app = this; > - > - this.xmlhttp.onreadystatechange = function() { > + this.xmlhttp.onreadystatechange = function() { > if (app.xmlhttp.readyState == 4 ) { > if (app.xmlhttp.status == 200 && > app.xmlhttp.responseText.indexOf("!ERROR") < 0 && > app.xmlhttp.responseText.indexOf("<env:Fault>") < 0) > { > - if (XRPCDEBUG) { > - if (app.xmlhttp.responseText) > - callback(app.xmlhttp.responseXML? app.xmlhttp.responseXML: string2XML(app.xmlhttp.responseText)); > - } > + if (XRPCDEBUG) alert(serializeXML(app.xmlhttp.responseXML)); > + callback(app.xmlhttp.responseXML); > } else { > var errmsg = > '!ERROR: "' + method + ' execution failed at the remote side"\n\n' + > @@ -202,7 +118,7 @@ > } > }; > } catch (e) { > - alert('sendRequest('+geturl+'): '+e); > + alert('sendRequest('+method+'): '+e); > } > } > > @@ -222,15 +138,15 @@ > '\\030', '\\031', '\\032', '\\033', '\\034', '\\035', '\\036', '\\037'); > > > - var chars = new Array ('&','�','�','�','�','�','�','�','�','�','�', > - '�','�','�','�','�','�','�','�','�','�','�', > - '�','�','�','�','�','�','�','�','�','�','�', > - '�','�','�','�','�','�','�','�','�','�','�', > - '�','�','�','�','�','�','�','�','�','�','�', > - '�','�','�','�','�','�','�','�','\"','�','<', > - '>','�','�','�','�','�','�','�','�','�','�', > - '�','�','�','�','�','�','�','�','�','�','�', > - '�','�','�','�','�','�','�','�'); > + var chars = new Array ('&','\340','\341','\342','\343','\344','\345','\346','\347','\350','\351', > + '\352','\353','\354','\355','\356','\357','\360','\361','\362','\363','\364', > + '\365','\366','\370','\371','\372','\373','\374','\375','\376','\377','\300', > + '\301','\302','\303','\304','\305','\306','\307','\310','\311','\312','\313', > + '\314','\315','\316','\317','\320','\321','\322','\323','\324','\325','\326', > + '\330','\331','\332','\333','\334','\335','\336','\200','\"','\337','<', > + '>','\242','\243','\244','\245','\246','\247','\250','\251','\252','\253', > + '\254','\255','\256','\257','\260','\261','\262','\263','\264','\265','\266', > + '\267','\270','\271','\272','\273','\274','\275','\276'); > > var entities = new Array ('amp','agrave','aacute','acirc','atilde','auml','aring', > 'aelig','ccedil','egrave','eacute','ecirc','euml','igrave', > > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins

2 1

[Monetdb-developers] Mtest: remove M5 warning about modules
by Fabian Groffen 11 Nov '08

11 Nov '08

Currently Mtest.py spits out the following warning when using M5: Mtest.py Warning: MonetDB/5: Check, whether required modules do exist, is not possible, yet! It's in there for years. I think the functionality of Mtest to be able to check whether required modules are available before running a test come at the price of administering which modules are necessary to run the test. Implementing the module check in M5 hence would also mean adding the required module information to each test. Hence, I would suggest to remove the warning from Mtest.py and just deal with the error messages that appear when modules don't exist. Opinions?

2 1

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/server sql_env.mx, Nov2008, 1.74, 1.74.2.1
by Martin Kersten 05 Nov '08

05 Nov '08

It will change again ;) see in my local dir Niels Nes wrote: > Update of /cvsroot/monetdb/sql/src/server > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv17857/src/server > > Modified Files: > Tag: Nov2008 > sql_env.mx > Log Message: > keepCall now has 12 arguments > > > U sql_env.mx > Index: sql_env.mx > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/server/sql_env.mx,v > retrieving revision 1.74 > retrieving revision 1.74.2.1 > diff -u -d -r1.74 -r1.74.2.1 > --- sql_env.mx 4 Nov 2008 11:38:16 -0000 1.74 > +++ sql_env.mx 5 Nov 2008 21:16:00 -0000 1.74.2.1 > @@ -114,7 +114,7 @@ > sgn = stack_get_number(m, "history"); > assert((lng) GDK_int_min <= sgn && sgn <= (lng) GDK_int_max); > if (!sql_find_func(sys, "keepquery", 4) || > - !sql_find_func(sys, "keepcall", 6)) > + !sql_find_func(sys, "keepcall", 12)) > return sql_message( "Cannot activate history because the keepQuery and keepCall procedures are not available\n" ); > m->history = (int) (sgn)?1:0; > } > > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins >

1 0

[Monetdb-developers] Release schedule
by Sjoerd Mullender 04 Nov '08

04 Nov '08

Today we decided that we want to try to do releases more often and more regularly. I have been appointed release manager for all upcoming releases. I propose that we keep to the following schedule: - Do a major (feature) release once every 4 months. - Do a minor (bug fix) release once every month inbetween the major releases. This should solve the problem that we get bug reports for problems in the release for bugs that were already fixed months ago but not yet released. In the worst case the submitter has to wait a month for the fix, but usually of course much less than that. It also solves the problem that developers always want to add just one more feature to the next release and thereby forcing the release to be postponed. If a feature doesn't make it in a release, it will be released in four months time. If users (and developers) had to wait years for this feature, they can wait a little longer. I also propose to create a new stable branch right after releasing the last of the bug fix releases in a cycle. This gives us one month to stabilize and test a new release. This plan is effective as of now. -- Sjoerd Mullender

4 7