- developers-list - monetdb.org

Re: [Monetdb-developers] Data type of the text() nodes
by Peter Boncz 04 Jun '07

04 Jun '07

Hi Jennie + Jens, Jennie, the idea in XRPC is to put the burden of function resolution and type promotion at the client -- server does nothing, in order to make it easy to implement XRPC compliant web services in environments that lack any XPath/XQuery (type analysis) support, such as simple wrapper code. The client side (i.e. pf) already performs type promotion, otherwise the function call would have given the resolution error at client side. So, the only thing that needs to be done is *use* the types of the function signature when generating the XRPC request. ..Fix around line 1000 in xrpc_client.mx.. 1067 cur_kind = XTRACT_KIND(cmbn_cont_kind); 1068 1069 switch (cur_kind) { But, Jens, there is also static vs. dynamic typing issue here. Of course, dynamic types are often more specific than static types. An example that excploits this is: declare function isInt($ctx as item()* as xs:boolean { typeswitch($ctx) case xs:integer: return true() default: return false() } For this reason my proposed fix is to use the function signature type *only* if it is a subtype of the runtime type (= if it is more specific). While the above probably fixes Jennie's issue, I do think there is a problem in pathfinder with function parameter promotion. This can surface when the typeswitch() is used inside the function body. Basically, when types are promoted, the runtime representation should be forced to reflect this. AFAIK this is not done in mps -- maybe the new type annotation system (in progress) handles this? With such a system in place, the runtime type would already be xs:string and the hack I propose above would not be needed. As an aside, I also think that there is something wrong in pathfinder with type promotion in general comparisons. Specifically, while the spec says that in case of a comparision between untyped data and a numeric, the untyped should be cast to xs:double. Pathfinder seems to apply a direct cast to the specific numeric data type (e.g. xs:integer) instead. Example: <a t='1.1'/>/@t = 1 I stubled on this in my Friday exercise to implement selection pushdown (not yet finished). Peter ------------------------------ Message: 4 Date: Fri, 1 Jun 2007 17:14:46 +0200 From: Ying Zhang <Y.Zhang(a)cwi.nl> Subject: [Monetdb-developers] Data type of the text() nodes To: monetdb-developers(a)lists.sourceforge.net Message-ID: <20070601151446.GV11079(a)cwi.nl> Content-Type: text/plain; charset=us-ascii Hello, I'm having a problem with the data type of text() nodes in XRPC implementation. --------------------------- module function definition: --------------------------- module namespace foo = "xrpc-test-function"; declare function foo:q2($strs as xs:string*) as node()* { for $s in $strs return <bar>{$s}</bar> }; ----------- test query: ----------- import module namespace test = "xrpc-test-function" at "/ufs/zhang/max-nr-params/xrpc-mod.xq"; let $hello := <hello> <lang>en</lang> <lang>nl</lang> </hello> let $lang := $hello/lang/text() return execute at {"localhost"} {test:q2($lang)} ------------------------------------------------------------- I got the error "function could not be resolved" when I run the XRPC query above. The reason is that all items in the sequence $lang have the type U_A (xs:untypedAtomic), and they are marshalled in the XRPC req. msg as, for example: <xrpc:atomic-value xsi:type="xs:untypedAtomic">en</xrpc:atomic-value> At the server side, the values are unmarshalled and assigned the type U_A again. So far, the values are handled correctly, IMHO. But then problem happens, namely, when the server calls the function 'xquery_sig_match' (in pathfinder.mx), it concludes that xs:untypedAtomic is not a sub-type of xs:string, hence, the function signature can not be resolved. This error does not occur if the function 'q2' is called without XRPC. So, what is the trick of pathfinder to match the signature? Is xs:untypedAtomic just treated the same as xs:string? The easiest solution of my problem is to just marshall an xs:untypedAtomic as an xs:string, but if someone has better suggestions, I would like to know. Thanks a lot! Regards, Jennie ------------------------------ Message: 5 Date: Fri, 1 Jun 2007 17:27:03 +0200 From: Jens Teubner <jens.teubner(a)in.tum.de> Subject: Re: [Monetdb-developers] Data type of the text() nodes To: monetdb-developers(a)lists.sourceforge.net Message-ID: <20070601152703.GB8539(a)notekemper14.informatik.tu-muenchen.de> Content-Type: text/plain; charset=us-ascii On Fri, Jun 01, 2007 at 05:14:46PM +0200, Ying Zhang wrote: > [...] > > But then problem happens, namely, when the server calls the function > 'xquery_sig_match' (in pathfinder.mx), it concludes that > xs:untypedAtomic is not a sub-type of xs:string, hence, the function > signature can not be resolved. > > This error does not occur if the function 'q2' is called without XRPC. > So, what is the trick of pathfinder to match the signature? Is > xs:untypedAtomic just treated the same as xs:string? Hi Jennie, the keywords to look for in the Formal Semantics are "type promotion" and "function call". And there are the function conversion rules as implemented in core/fs.brg:function_conversion(). Sorry for my very brief answer, but these pointers should bring you quite far. Jens -- Jens Teubner Technische Universitaet Muenchen, Department of Informatics D-85748 Garching, Germany Tel: +49 89 289-17259 Fax: +49 89 289-17263 I invented <Ctrl><Alt><Delete>, but Bill Gates made it famous. -- David Bradley, member of the original IBM PC design team ------------------------------ Message: 6 Date: Fri, 1 Jun 2007 21:00:27 +0200 From: Jens Teubner <jens.teubner(a)in.tum.de> Subject: Re: [Monetdb-developers] [MonetDB-users] XRPC problem: too many parameters To: Ying Zhang <Y.Zhang(a)cwi.nl> Cc: monetdb-developers(a)lists.sourceforge.net Message-ID: <20070601190027.GF8539(a)notekemper14.informatik.tu-muenchen.de> Content-Type: text/plain; charset=us-ascii On Fri, Jun 01, 2007 at 06:50:41PM +0200, Ying Zhang wrote: > [...] > > I forgot to say that you need to wrap "$gb-hit/Hit_gb/text()" into the > fn:data() function before those values can be passed to "test:q1". This > is a restriction of the current implementation of XRPC. > > The values you get by selecting the "text()" child node are of the type > "untypedAtomic". XRPC currently does not cast eash untypedAtomic to the > expected atomic type, hence, you need to add a fn:data() to do this. I > will add this cast later in XRPC's implementation. This sounds odd. Applied to (untyped) nodes, fn:data() returns xs:untypedAtomic. Only if a value is used in a function call, the items are cast to the respective atomic type ("function conversion rules"). (Actually, I just saw that fn:data() is not allowed at all on untyped nodes. Maybe this changed in the latest revisions of the specs.) Jens -- Jens Teubner Technische Universitaet Muenchen, Department of Informatics D-85748 Garching, Germany Tel: +49 89 289-17259 Fax: +49 89 289-17263 Linus Torvalds is a lot like Bill Gates. Both are about the same height. -- USA Today ------------------------------ Message: 8 Date: Sat, 2 Jun 2007 22:14:44 +0200 From: Ying Zhang <Y.Zhang(a)cwi.nl> Subject: Re: [Monetdb-developers] [MonetDB-users] XRPC problem: too many parameters To: monetdb-developers(a)lists.sourceforge.net Message-ID: <20070602201444.GC11079(a)cwi.nl> Content-Type: text/plain; charset=us-ascii On Fri, Jun 01, 2007 at 09:00:27PM +0200, Jens Teubner wrote: > On Fri, Jun 01, 2007 at 06:50:41PM +0200, Ying Zhang wrote: > > > [...] > > > > I forgot to say that you need to wrap "$gb-hit/Hit_gb/text()" into the > > fn:data() function before those values can be passed to "test:q1". This > > is a restriction of the current implementation of XRPC. > > > > The values you get by selecting the "text()" child node are of the type > > "untypedAtomic". XRPC currently does not cast eash untypedAtomic to the > > expected atomic type, hence, you need to add a fn:data() to do this. I > > will add this cast later in XRPC's implementation. > > This sounds odd. Applied to (untyped) nodes, fn:data() returns > xs:untypedAtomic. Only if a value is used in a function call, the items > are cast to the respective atomic type ("function conversion rules"). > > (Actually, I just saw that fn:data() is not allowed at all on untyped > nodes. Maybe this changed in the latest revisions of the specs.) > > Jens Hi Jens, You are right about that fn:data() does not do what I need, namely take an untypedAtomic and return an atomic value in the right type. But according to the latest revision, I think fn:data() does allow untypedAtomic as its parameter: fn:data($arg as item()*) as xs:anyAtomicType* But I need to find another way to solve this problem. Jennie > > -- > Jens Teubner > Technische Universitaet Muenchen, Department of Informatics > D-85748 Garching, Germany > Tel: +49 89 289-17259 Fax: +49 89 289-17263 > > Linus Torvalds is a lot like Bill Gates. Both are > about the same height. > -- USA Today

2 1

Re: [Monetdb-developers] xs:docmgmt in XQuery user defined functions?
by Peter Boncz 04 Jun '07

04 Jun '07

Hi Djoerd, All document management functions return the docmgt type, as a mechanism to ensure that such queries do not perform any document updates. For MonetDB, this separation is handy, as document management and XQUF updates rely on two different mechanisms (checkpointing vs WAL) and if queries would rely on both, it would be hard to make them atomic. Pathfinder supports the 'declare document management function' syntax extension (imilar to the xQUF notation 'declare updating function') to allow user-defined functions that perform document management. Peter

1 0

[Monetdb-developers] Feature freeze & new release branches
by Stefan Manegold 04 Jun '07

04 Jun '07

Dear all, as decided and announced last week, I'll "enforce" the feature freeze for the upcoming *.18 release today, Sunday June 3 2007 --- well, the origin plan was for Friday June 1 2007 but other obligations and lack of time forced my to post-pone this action by two days --- hopefully, SF will be less busy on a weekend... Hence, from now on, please do NOT checkin to ANY CVS modules releated to MonetDB until I'm done later today. I'll let you know, once I'm done and you can checkin, again. Thank you very much for your cooperation. Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 1

Re: [Monetdb-developers] [MonetDB-users] XRPC problem: too many parameters
by Jens Teubner 02 Jun '07

02 Jun '07

On Fri, Jun 01, 2007 at 06:50:41PM +0200, Ying Zhang wrote: > [...] > > I forgot to say that you need to wrap "$gb-hit/Hit_gb/text()" into the > fn:data() function before those values can be passed to "test:q1". This > is a restriction of the current implementation of XRPC. > > The values you get by selecting the "text()" child node are of the type > "untypedAtomic". XRPC currently does not cast eash untypedAtomic to the > expected atomic type, hence, you need to add a fn:data() to do this. I > will add this cast later in XRPC's implementation. This sounds odd. Applied to (untyped) nodes, fn:data() returns xs:untypedAtomic. Only if a value is used in a function call, the items are cast to the respective atomic type ("function conversion rules"). (Actually, I just saw that fn:data() is not allowed at all on untyped nodes. Maybe this changed in the latest revisions of the specs.) Jens -- Jens Teubner Technische Universitaet Muenchen, Department of Informatics D-85748 Garching, Germany Tel: +49 89 289-17259 Fax: +49 89 289-17263 Linus Torvalds is a lot like Bill Gates. Both are about the same height. -- USA Today

2 1

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/mal mal.mx, 1.125, 1.126
by Martin Kersten 01 Jun '07

01 Jun '07

Niels Nes wrote: > Update of /cvsroot/monetdb/MonetDB5/src/mal > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv1997/src/mal > > Modified Files: > mal.mx > Log Message: > corrected checks for busy clients when exiting Nice bug. > > > Index: mal.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/mal/mal.mx,v > retrieving revision 1.125 > retrieving revision 1.126 > diff -u -d -r1.125 -r1.126 > --- mal.mx 1 Jun 2007 06:48:28 -0000 1.125 > +++ mal.mx 1 Jun 2007 09:48:20 -0000 1.126 > @@ -391,10 +391,10 @@ > do{ > freeclient=0; finishing=0; claimed=0; awaiting=0; > for(cntxt= mal_clients+1; cntxt<mal_clients+MAL_MAXCLIENTS; cntxt++){ > - freeclient += cntxt->mode & FREECLIENT; > - finishing += cntxt->mode & FINISHING; > - claimed += cntxt->mode & CLAIMED; > - awaiting += cntxt->mode & AWAITING; > + freeclient += (cntxt->mode == FREECLIENT); > + finishing += (cntxt->mode == FINISHING); > + claimed += (cntxt->mode == CLAIMED); > + awaiting += (cntxt->mode == AWAITING); > if( cntxt->mode & FINISHING) > printf("#CLient %d %d\n",(int)(cntxt - mal_clients), cntxt->idx); > } > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins

1 0

[Monetdb-developers] Data type of the text() nodes
by Ying Zhang 01 Jun '07

01 Jun '07

Hello, I'm having a problem with the data type of text() nodes in XRPC implementation. --------------------------- module function definition: --------------------------- module namespace foo = "xrpc-test-function"; declare function foo:q2($strs as xs:string*) as node()* { for $s in $strs return <bar>{$s}</bar> }; ----------- test query: ----------- import module namespace test = "xrpc-test-function" at "/ufs/zhang/max-nr-params/xrpc-mod.xq"; let $hello := <hello> <lang>en</lang> <lang>nl</lang> </hello> let $lang := $hello/lang/text() return execute at {"localhost"} {test:q2($lang)} ------------------------------------------------------------- I got the error "function could not be resolved" when I run the XRPC query above. The reason is that all items in the sequence $lang have the type U_A (xs:untypedAtomic), and they are marshalled in the XRPC req. msg as, for example: <xrpc:atomic-value xsi:type="xs:untypedAtomic">en</xrpc:atomic-value> At the server side, the values are unmarshalled and assigned the type U_A again. So far, the values are handled correctly, IMHO. But then problem happens, namely, when the server calls the function 'xquery_sig_match' (in pathfinder.mx), it concludes that xs:untypedAtomic is not a sub-type of xs:string, hence, the function signature can not be resolved. This error does not occur if the function 'q2' is called without XRPC. So, what is the trick of pathfinder to match the signature? Is xs:untypedAtomic just treated the same as xs:string? The easiest solution of my problem is to just marshall an xs:untypedAtomic as an xs:string, but if someone has better suggestions, I would like to know. Thanks a lot! Regards, Jennie

2 1

[Monetdb-developers] xs:docmgmt in XQuery user defined functions?
by Djoerd Hiemstra 01 Jun '07

01 Jun '07

Dear all, Just a small suggestion for the release version. New functions like pf:add-doc() return an item of type docmgmt. However, this type does not seem to exist if I try to use user defined functions such as declare function local:weird( ) as xs:docmgmt { pf:add-doc("weird.xml", "weird.xml") }; gives > use of undefined type: at (14,38-14,47): ``xs:docmgmt'' Similarly with "docmgmt" or "pf:docmgmt", etc. Or if I use "item()" > type error: body of local:weird() evaluates to `docmgmt' which is not > a subtype of `item' Best, Djoerd.

1 0

[Monetdb-developers] MonetDB Release Preparation
by Martin Kersten 29 May '07

29 May '07

Dear fellow MonetDB developers/users, we plan to have a new release of the MonetDB software family within the next two weeks, i.e., Feature Freeze: Friday Jun 01 2007 Release: Monday Jun 11 2007 This release cycle is used to reduce the component dependencies, such that we are better prepared to release individual components more frequently. A synopsis of what to expect: MonetDB 5 beta status is dropped and it becomes the sole back-end for the SQL front-end. SQL now supports Persistent Stored Modules. XQuery Update Facility has undergone a large series of stress tests to improve correctness, performance and robustness against concurrent updates. Many bug-fixes based on highly appreciated feedback from our user community. The priorities set for post-release actions as agreed upon by the development team are: shortterm:(summer) SQL GIS geometry model testing and release SQL/XML support development and release PF/SQL exploration (research focus) Skyserver demo website (target 2.7TB online demo) midterm: (fall) Remote SQL execution in a cluster setting SQL adaptive partitioning of large databases XQuery Algebra -> MIL compiler Date/time support in XQuery MonetDB Private BATs policy (kernel modification) lngterm: (fall/spring) XQuery Algebra -> MAL compiler public release of StreetTivo for collaborative media analysis Of course, as always, progress in each of these areas depend on availability of resources and their contribution to our research agenda. We are looking forward to messages from projects based on/exploitation of the MonetDB platform. We plan to add a section in the website to make this work more broadly visible. regards, Martin

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/modules/pftijah pftijah.mx, 1.123, 1.124
by Stefan Manegold 29 May '07

29 May '07

On Tue, May 29, 2007 at 12:20:21PM +0000, Jan Flokstra wrote: > Update of /cvsroot/monetdb/pathfinder/modules/pftijah > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv10231 > > Modified Files: > pftijah.mx > Log Message: > - try to repair pftijah "testcoll2" bug on 64bit archs > > > > Index: pftijah.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/modules/pftijah/pftijah.mx,v > retrieving revision 1.123 > retrieving revision 1.124 > diff -u -d -r1.123 -r1.124 > --- pftijah.mx 25 May 2007 12:34:24 -0000 1.123 > +++ pftijah.mx 29 May 2007 12:20:18 -0000 1.124 > @@ -3267,15 +3267,16 @@ > return GDK_FAIL; > } > > - doc_start = *(oid*)BUNtail(doc_firstpre,r); > - oid tj_nextIndex = tj_docIndex + 1; > + // 64bit ERROR?? doc_start = *(oid*)BUNtail(doc_firstpre,r); ^^^^^^^^^^^^^ No, a bug in the code: BAT doc_firstpre is [oid,int]., i.e., its tail is int, not oid, and int != oid; ======== pathfinder/modules/pftijah/pftijah.mx-73-.COMMAND pf2tijah_node( pathfinder/modules/pftijah/pftijah.mx-74- BAT[oid,str] doc_name, pathfinder/modules/pftijah/pftijah.mx:75: BAT[oid,int] doc_firstpre, ^^^^^^^^^^^^^^^^^^^^^^^^^ pathfinder/modules/pftijah/pftijah.mx-76- BAT[oid,oid] pfpre, pathfinder/modules/pftijah/pftijah.mx-77- BAT[oid,oid] item, pathfinder/modules/pftijah/pftijah.mx-78- BAT[oid,int] kind, pathfinder/modules/pftijah/pftijah.mx-79- BAT[oid,str] doc_loaded) pathfinder/modules/pftijah/pftijah.mx-80- : BAT[void,oid] = CMDpf2tijah_node; pathfinder/modules/pftijah/pftijah.mx-81- "Translate Pathfinder node sequence to tijah node sequence" ======== hence, to access the tail in C, you (obviously!) need to use int doc_start = *(int*) BUNtail(doc_firstpre,r); ^^^ ^^^ if (for what ever reason) doc_start needs to be of type oid instead of int, you need to use oid doc_start = (oid) *(int*) BUNtail(doc_firstpre,r); ^^^ ^^^ ^^^ (as you do below); in fact, you should first check, whether the respective tail value of doc_firstpre is not negative, before you caat it to oid! > + doc_start = (oid)*(int*)BUNtail(doc_firstpre,r); > + oid tj_nextIndex = tj_docIndex + (oid)1; > if ( BATcount(doc_firstpre) > tj_nextIndex ) { > r = BUNfnd(doc_firstpre,&tj_nextIndex); > if ( !r ) { > stream_printf(GDKout,"Cannot do range for tijah-firstpre @ %d.\n",tj_docIndex); > return GDK_FAIL; > } > - doc_end = *(oid*)BUNtail(doc_firstpre,r) - 1; > + doc_end = *(oid*)BUNtail(doc_firstpre,r) - (oid)1; NOPE! either int doc_end = *(int*) BUNtail(doc_firstpre,r) - 1; or oid doc_end = (oid) (*(int*) BUNtail(doc_firstpre,r) - 1); plus a check, whether *(int*) BUNtail(doc_firstpre,r) is > 0 ! Stefan > } else { > doc_end = oid_nil; > } > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 2

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/conf monetdb5.conf.in, 1.11, 1.12
by Stefan Manegold 29 May '07

29 May '07

Another M4 -> M5 "incompatibility" that should be documented properly in the M4 -> M5 transition documentation: ` While with M4, "mapi_open=true"/"mapi_open=false" works fine, M5 (seems to) require(s) "mapi_open=1"/"mapi_open=0". ` Stefan On Sun, May 27, 2007 at 05:27:33PM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/conf > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv31984 > > Modified Files: > monetdb5.conf.in > Log Message: > Use the proper values for mapi_open > > > Index: monetdb5.conf.in > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/conf/monetdb5.conf.in,v > retrieving revision 1.11 > retrieving revision 1.12 > diff -u -d -r1.11 -r1.12 > --- monetdb5.conf.in 17 May 2007 09:51:09 -0000 1.11 > +++ monetdb5.conf.in 27 May 2007 17:27:31 -0000 1.12 > @@ -126,7 +126,7 @@ > # Monet Application Interface Section > #==================================== > mapi_port=0 #default port to address a mserver > -mapi_open=false #should be set to 'true' to allow for > +mapi_open=0 #should be set to '1' to allow for > #remote access to a server > > # SQL Interface Section > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0