- users-list - monetdb.org

Re: [MonetDB-users] MonetDB XQuery Module Crash
by Stefan Manegold 11 Jul '06

11 Jul '06

On Sun, Jul 09, 2006 at 05:27:17PM +0200, Michael Schmidt wrote: > Hi Stefan, > > 2 more things: > > - I suppose you already posted my mail at MonetDB-users. I can't see them in the archives but I guess they are updated each night, am I right? I honestly don't know, when SF updates the mailinglist archives, but my mail (quoting your original mail) is (now) in there: http://sourceforge.net/mailarchive/forum.php?thread_id=21306336&forum_id=42… > - I just tried loading … [View More]

1 0

Re: [MonetDB-users] MonetDB XQuery Module Crash
by Stefan Manegold 11 Jul '06

11 Jul '06

Hi Michael, I just double-checked your problem on my AMD Athlon 64 X2 running 64-bit Fedora Core 4, and everything works fine. I can shred and query XMarks documents up to 1.1 GB without problems (didn't try larger ones ;-)). We also have an Athlon 64 running 64-bit SuSE Linux 9.3 --- but I haven't had time to check that one, yet; I'll try to check it the coming days (a 500kB document works fine on that machine in our nightly testing...). In the meantime, I have some more questions for you: … [View More]Which version of SuSE Linux and which version of gcc do you use? Which exact version of the MonetDB source do you use? - The released tar-balls from SourceForge (MonetDB-4.12.0.tar.gz & MonetDB-XQuery-0.12.0.tar.gz)? - The release source RPMs from SourceForge (MonetDB-4.12.0-1.src.rpm & MonetDB-XQuery-0.12.0-1.src.rpm)? - Source tar-balls or source RPMs from our daily-builds site (http://monetdb.cwi.nl/testing/projects/monetdb/Stable/.DailyBuilds./)? - A CVS checkout? (Which branch/tag?) Cheers, Stefan ps: I felt free to get let the Monetdb-users mailing list participate in the discussion, again... On Sun, Jul 09, 2006 at 05:14:23PM +0200, Michael Schmidt wrote: > Hi Stefan, > > > Sorry for the inconvenience! > > We'll update the info on the web site. > > And I feel free to propagate this discussion to the Monetdb-users list. > > Thank you very much, I will also register there... > > > Wrt. your actual problem, I have a couple of questions, maily requesting > > more information to analyse the problem: > > > > What kind of system are you running MonetDB/XQuery on? > > CPU (type, 32- or 64-bit)? > > amount of memory? > > free disk space? > > operating system (type, 32- or 64-bit)? > > I'm running SuSe Linux 64-bit, here my /proc/cpuinfo... If there are any more details you need to know feel free to ask me again. > > processor : 0 > vendor_id : GenuineIntel > cpu family : 15 > model : 4 > model name : Intel(R) Pentium(R) 4 CPU 3.00GHz > stepping : 3 > cpu MHz : 2992.716 > cache size : 2048 KB > physical id : 0 > siblings : 2 > core id : 0 > cpu cores : 1 > fpu : yes > fpu_exception : yes > cpuid level : 5 > wp : yes > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl est cid cx16 xtpr > bogomips : 5994.87 > clflush size : 64 > cache_alignment : 128 > address sizes : 36 bits physical, 48 bits virtual > power management: > > > > How did you install MonetDB/XQuery? Using binary (.rpm or .msi) packages or > > compiling from source (tarball or from CVS)? > > I compiled them from source. > > > Which queries are you running? Which of them are working with up to 50MB but > > fail with 100MB or more? > > It's the same for all queries. I'm running the XMark test Queries #1, #6, #8, #13 and #20. Here one example Query: (XMark #1): > > query1> > { for $b in doc("/home/.../xmark.xml")/site/people/person > where $b/person_id = 'person0' > return > <result> {$b/name} </result> } > </query1> > > > Did you load/shred your document explicitely via the "shred_doc" MIL command > > on the Mserver console, or did you have them loaded/shreded implicitely on > > the fly via the fn:doc() XQuery function? > > As running benchmarks where loading time should be included, explicit console loading is unsuitable. As seen above I load them from the query. > > > In the latter case, you could try whether the failing queries do work in > > larger documents once you increase the XML document cache limit by staring > > Mserver with "--set xquery_cacheMB=<size>" where size is large than the > > default 100, e.g., just larger than your largest document. > > (Admittedly, there is only very little "well hidden" documentation about the > > "--set" option(s) of Mserver; we're working on that...) > > I tested "--set xquery_cacheMB=500" without success. > > Kind regards > Michael > _____________________________________________________________________ > Der WEB.DE SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! > http://smartsurfer.web.de/?mc=100071&distributionid=000000000071 > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 | [View Less]

1 0

Re: [MonetDB-users] MonetDB XQuery Module Crash
by Stefan Manegold 09 Jul '06

09 Jul '06

Hi Michael, thank you very much for being interested in and using MonetDB/XQuery, as well as for reporting your problem! Apparently, the mailinglist information on the FAQ page of the MonetDB/XQUery web site is indeed a bit misleading. There are actually no specific MonetDB/*XQuery* mailing lists, but "only" general mailing lists for MonetDB that are supposed to be used for all communication about/around MonetDB and all of it's companions and front-ends, o.a., MonetDB/SQL & MonetDB/… [View More]XQuery; see http://monetdb.cwi.nl/News/MailChannels/ for details. Sorry for the inconvenience! We'll update the info on the web site. And I feel free to propagate this discussion to the Monetdb-users list. Wrt. your actual problem, I have a couple of questions, maily requesting more information to analyse the problem: What kind of system are you running MonetDB/XQuery on? CPU (type, 32- or 64-bit)? amount of memory? free disk space? operating system (type, 32- or 64-bit)? How did you install MonetDB/XQuery? Using binary (.rpm or .msi) packages or compiling from source (tarball or from CVS)? Which queries are you running? Which of them are working with up to 50MB but fail with 100MB or more? Did you load/shred your document explicitely via the "shred_doc" MIL command on the Mserver console, or did you have them loaded/shreded implicitely on the fly via the fn:doc() XQuery function? In the latter case, you could try whether the failing queries do work in larger documents once you increase the XML document cache limit by staring Mserver with "--set xquery_cacheMB=<size>" where size is large than the default 100, e.g., just larger than your largest document. (Admittedly, there is only very little "well hidden" documentation about the "--set" option(s) of Mserver; we're working on that...) If this helps, then there might be a bug somewhere in our document caching code... Kind regards, Stefan On Sun, Jul 09, 2006 at 03:54:30PM +0200, Michael Schmidt wrote: > Hi all, > > I'm confused as I expected a MonetDB XQuery users mailing list instead of > a developer list. However, this list was linked in the MonetDB/XQuery FAQ > section so I thought it would be worth trying... > > Here is my problem: I'm just running some XQuery benchmarks using MonetDB > together with the XQuery module. Installation was ok and I have chosen a > set of queries to be evaluated against data generated with the xmlgen > tool. Query processing works fine for 10MB and 50MB documents, but when > processing large documents (100MB and 200MB) the client reports an error > message. I even tried to figure out the source of the problem using trace > option -t but do not understand the listed error messages. Here is the > final error message reported by the client: > > ERROR: [rename]: 29 times inserted nil due to errors at tuples "pre_size", "pre_level", "pre_prop", "pre_kind", "qn_uri", "qn_prefix". > ERROR: [rename]: first error was: > ERROR: rename(<tmp_351>,pre_size2): operation failed > ERROR: interpret_unpin: [rename] bat=171,stamp=-1285 OVERWRITTEN > ERROR: BBPdecref: tmp_253 does not have pointer fixes. > ERROR: interpret_params: +(param 2): evaluation error. > > Once again: evaluation of the same queries works fine on small streams but > fails on large streams. By the way, I've installed the latest offcial > release of both the server and the XQuery module. > > Is this a bug or are there any default memory limits causing the crash? I > tried to figure out what the --set option is good for but was not able to > detect any documentation on supported options. > > Kind regards > Michael > _____________________________________________________________________ > Der WEB.DE SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! > http://smartsurfer.web.de/?mc=100071&distributionid=000000000071 > > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Monetdb-developers mailing list > Monetdb-developers(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-developers > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 | [View Less]

1 0

[MonetDB-users] MonetDB as a transaction processing XML database?
by Sander Kruseman 06 Jul '06

06 Jul '06

Hello, Is my conclusion correct that I should not use MonetDB (yet) as a transaction processing XML database? (inserts and deletes of lots of XML snippets, xqueries on the complete database content, by multiple processes) Could you perhaps suggest an alternative solution? Any familiarity with Berkely DB XML by Sleepycat for example? Btw. we need a C++ API for this. Thanks in advance. Sander

2 1

[MonetDB-users] MonetDB 4.12.0 Release Announcement
by Fabian Groffen 23 Jun '06

23 Jun '06

It is my great pleasure to announce the availability of MonetDB 4.12.0 and its accompanying SQL 2.12.0 and XQuery 0.12.0. This release is the next step in delivering the high performance database system that has been under development for several years now. Some of the many changes since the 4.10.2 release include: MonetDB/XQuery 0.12.0 (nick-named "Ceres") Various improvments of functionality and performance. Addition of XPath extensions for XML standoff annotations. "Ceres" is a small … [View More]detour on our trip to "Mars" [1], as update functionality is unfortunately still not robust enough for consumption by the majority. MonetDB/SQL 2.12.0 Many bugfixes. Support for local and global temporary tables. Preliminary support for triggers. For a complete list of new features and known problems, please see the release notes and errata list, available at: http://monetdb.cwi.nl/Development/Roadmap/Version4.12/index.html MonetDB/Five Alpha Release 1 ============================ The efforts on the next generation of MonetDB, version Five, are released now. This snapshot of version Five of MonetDB is the very first alpha release of this ongoing development. Beware: alpha is alpha! This software is only suitable as sneak-preview into the MonetDB developers team's kitchen! Come and taste some of the many delicious spices we prepare. Unlike the previous generation, Five comes with the MAL language, instead of MIL, which makes it completely incompatible with old MIL scripts. However, the SQL sources work fine with this alpha release. Comparing the performance of both backends via SQL should give you an indication of what our cooks will serve for dinner. This first alpha "appetiser" release of MonetDB/Five is nick-named "Tomato Soup". Interested users are invited to visit our MonetDB/Five page for more details: http://monetdb.cwi.nl/Development/Five/index.html Availability ============ The MonetDB family products are supported on i686, x86_64 and ia64 Linux, i686 Solaris 10, ppc and ppc64 Mac OS X 10.4 and i686 Windows XP. Binary distributions are available for Linux and Windows. The new release is available for download from our SourceForge page: http://sourceforge.net/project/showfiles.php?group_id=56967 Acknowledgements ================ Many man-hours were donated by various people to make this release happening. I want to thank al MonetDB developers for their efforts spent into this piece of software. All of our contributors and bug reporters also deserve the credits for the time they put into their contribution. Further Information =================== Further information about MonetDB, MonetDB/SQL and MonetDB/XQuery is available on our web site at: http://monetdb.cwi.nl/ http://monetdb.cwi.nl/SQL/ http://monetdb.cwi.nl/XQuery/ For details about recent changes, migration from earlier versions, and future plans, please visit our roadmap page: http://monetdb.cwi.nl/Development/Roadmap/ User CVS Information ==================== If you use a CVS checkout, you might have to change some things in order to keep on working with the latest sources. With every release some new tags are set and numbers changed. Please check your source trees that you use the right sticky tags on them. The sources that are put in this release are tagged as follows: MonetDB MonetDB_4-12-0 sql SQL_2-12-0 pathfinder XQuery_0-12-0 The new stable branches are now available through the following tags: MonetDB MonetDB_4-12 sql SQL_2-12 pathfinder XQuery_0-12 To update to the new stable branches use ``cvs update -r <branch tag>`` for the repositories you work with. Feedback or Questions ===================== If you have any questions or feedback regarding this release, feel free to contact us through this list: monetdb-users(a)lists.sourceforge.net We hope you will enjoy this new release of MonetDB as much as we do. [1] See http://en.wikipedia.org/wiki/1_Ceres and http://www.nasa.gov/multimedia/imagegallery/image_feature_404.html [View Less]

1 0

Re: [MonetDB-users] Reminder: MonetDB Bug-Etiquette
by Stefan Manegold 19 Jun '06

19 Jun '06

Dear all, two more small pleas that will make bug reporting and fixing much simpler and smoother: - When submitting a bug, please *always* set both the "Category:" and the "Group:"! This helps the developer(s) to identify which version/part of the system your bug occured with. - When changing the "Status:", "Resolution:", and/or "Priority:", please *always* provide a comment that explans the motivation and/or reason for your action. E.g., when closing a bug report, please mention … [View More]whether, how and where you fixed it. Thanks again! Stefan On Sun, Jun 18, 2006 at 10:09:13PM +0200, Stefan Manegold wrote: > Dear all, > > since the last MonetDB "BugDay" on Dec 19 2005, many bugs have been reported > and fixed. > > Unfortunately, not all bug reports that have been closed since then have > been backed-up by a test script in CVS to verify and monitor to validity of > the respective bug fix, as described in the "MonetDB Bug-Etiquette" > (cf. one of > http://sourceforge.net/mailarchive/forum.php?thread_id=9021727&forum_id=398… > http://sourceforge.net/mailarchive/forum.php?thread_id=9021726&forum_id=425… > ) > > According to the "MonetDB Bug-Etiquette", a bug report should (may/must) not > be closed without adding a respective test to our test system in CVS. > > While both the user/submitter that reports the bug and the > developer/assignee that fixes it should share the task of "designing" the > test and adding it to the test system in CVS, we agreed that the > developer/assignee is finally "responsible" that the test is indeed added > *before* the bug report is closed. > > > To ensure the stability and correctness of MonetDB and its companions and to > avoid that supposedly fixed bug sneak in again unnoticed, I'd like to ask > all of you that reported and/or fix any bug(s) since Dec 19 2005 > the following favour: > > Please check all "your" bugs (see * below) since Dec 19 2005, whether a test > has been added or not. > + If the comments in the bug report are not clear, please check the CVS > repository. > + If you're a user/submitter and don't know where to look or what to do, > please remind the respective developer/assignee of your bug by added a > respective comment and re-opening the bug report. > + If there is a test, or no test is necessary or feasible, but the > bug-report does not mention this clearly, please add a new comment that > explains the situation. > + If there is no test although there should be one, please re-open the bug > report with a respective comment to indicate that a test must be added. > As said before, "users/submitters" are welcome to help adding tests, but > developers/assignees are finally responsible that the tests are indeed > added. > > Please recall, we plan to release MonetDB 4.12, MonetDB/SQL 2.12 and > MonetDB/XQuery 0.12 soon (i.e., before the end of this month), > and we all want them to be as stable and "bug-free" as possible...! > > > Feel free to use these channels to comment and/or ask questions. > > > Thank you all very much for your contributions to make MonetDB even better > than it already is! > > > Kind regards, > > Stefan > > > *) To find "your" bugs, just > - go to http://sourceforge.net/tracker/?group_id=56967&atid=482468 , > - select your username from the list "Assignee:", > - or enter your username as "Submitter username (show mine):" > (and set "Assignee:" to "Any"), > - set "Status:", "Category:" & "Group:" to "Any", > - set "Sort By:" to "Open Date" and "Descending", > - and click on "Browse". > > -- > | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | > | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | > | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | > | The Netherlands | Fax : +31 (20) 592-4312 | -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 | [View Less]

1 0

[MonetDB-users] Reminder: MonetDB Bug-Etiquette
by Stefan Manegold 18 Jun '06

18 Jun '06

Dear all, since the last MonetDB "BugDay" on Dec 19 2005, many bugs have been reported and fixed. Unfortunately, not all bug reports that have been closed since then have been backed-up by a test script in CVS to verify and monitor to validity of the respective bug fix, as described in the "MonetDB Bug-Etiquette" (cf. one of http://sourceforge.net/mailarchive/forum.php?thread_id=9021727&forum_id=398… http://sourceforge.net/mailarchive/forum.php?thread_id=9021726&forum_id=425… ) … [View More]According to the "MonetDB Bug-Etiquette", a bug report should (may/must) not be closed without adding a respective test to our test system in CVS. While both the user/submitter that reports the bug and the developer/assignee that fixes it should share the task of "designing" the test and adding it to the test system in CVS, we agreed that the developer/assignee is finally "responsible" that the test is indeed added *before* the bug report is closed. To ensure the stability and correctness of MonetDB and its companions and to avoid that supposedly fixed bug sneak in again unnoticed, I'd like to ask all of you that reported and/or fix any bug(s) since Dec 19 2005 the following favour: Please check all "your" bugs (see * below) since Dec 19 2005, whether a test has been added or not. + If the comments in the bug report are not clear, please check the CVS repository. + If you're a user/submitter and don't know where to look or what to do, please remind the respective developer/assignee of your bug by added a respective comment and re-opening the bug report. + If there is a test, or no test is necessary or feasible, but the bug-report does not mention this clearly, please add a new comment that explains the situation. + If there is no test although there should be one, please re-open the bug report with a respective comment to indicate that a test must be added. As said before, "users/submitters" are welcome to help adding tests, but developers/assignees are finally responsible that the tests are indeed added. Please recall, we plan to release MonetDB 4.12, MonetDB/SQL 2.12 and MonetDB/XQuery 0.12 soon (i.e., before the end of this month), and we all want them to be as stable and "bug-free" as possible...! Feel free to use these channels to comment and/or ask questions. Thank you all very much for your contributions to make MonetDB even better than it already is! Kind regards, Stefan *) To find "your" bugs, just - go to http://sourceforge.net/tracker/?group_id=56967&atid=482468 , - select your username from the list "Assignee:", - or enter your username as "Submitter username (show mine):" (and set "Assignee:" to "Any"), - set "Status:", "Category:" & "Group:" to "Any", - set "Sort By:" to "Open Date" and "Descending", - and click on "Browse". -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 | [View Less]

1 0

[MonetDB-users] join on 2 attributes
by Henning Rode 07 Jun '06

07 Jun '06

hej monet-db-useres, i maybe forgot a very basic thing here, but how do you perform in monet an equi-join on two attributes. it seems such a normal task that there must be something easier than what i did now (see attachment: generating first example instances, then do the join with 'intersect') you should have the same problem when you select on a qname (namespace and tagname), but i could not find that example. if i underspecified my problem, just let me know in advance ;-) best -henning … [View More]

1 0

[MonetDB-users] MEL Question: Creating a New Search Accelerator
by John Cieslewicz 05 Jun '06

05 Jun '06

I am trying to build a new search accelerator as described in Section 2.5 of the Monet Extension Language (MEL) User Guide (http://monetdb.cwi.nl/TechDocs/Core/mel/index.html) When I execute mel on my module definition it does not seem to recognize the ".ACCELERATOR" key word. In Section 3.5, "Implementing Accelerators" the 'mean' accelerator implementation is described. Is there an example of the module definition of 'mean' in MEL? I also looked at the mel source code and found that in … [View More]

1 0

[MonetDB-users] python client library for Windows
by Max 10 May '06

10 May '06

Hello! I'm just installed Python 2.4 and MonetDB-Installer-4.10.2.msi, but can't find precompiled python Mapi modules. Generally, I want to try SQL interface to MonetDB from Python. After some dig around tar.gz sources i've found Mapi.py, MapiClient.py, CMapi.py, MapiLib.py and monet_options.py. Still need binary _MapiLib... Also please clarify meaning of monet_options.py file.

2 1