August 2008 - users-list

[MonetDB-users] Newbie question about XML / SQL capabilities
by Bill Walton 14 Aug '08

14 Aug '08

Greetings all, I just became aware of MonetDB via a post to the Ruby on Rails list about a new connection adapter for MonetDB (thanks Michalis). I'm intrigued by what looks like a very impressive capability. If I'm reading the site right, MonetDB can import an XML file using the XQuery client and then query that same data using the SQL client. Is that a correct reading? TIA, Bill

2 1

[MonetDB-users] Order of WHERE clause has significant performance impact
by Ross Bates 13 Aug '08

13 Aug '08

Hi All - I have 2 queries which run very different based on how I order the WHERE clause. When I put the larger filter first, it runs for 3.5 seconds. Query 1: select c5, c6, sum(sale) from sobi.sample where c5 in (5253,5110,5375,5268,5257,5266,5243,5256,5252,5361,5379,5363,5386,5367,5240,5241,5261,5249,5245,5260,5247,5264,5354,5250,5373,5366,5246,5356,5365,5255,5262,5259,5251,5369,5357,5269,5364,5380,5244,5248,5242,5254,5358,5360,5263,5362,5374,5378,5370,5294,5339,5272,5277,5275,5295,5340,5273,5276,5270,5061,5081,5079,5080,5066,5056,5073,5060,5070,5099,5101,5100,5067,5063,5077,5072,5062,5075,5083,5086,5071,5095,5058,5087,5090,5102,5084,5098,5076,5059,5074,5094,5091,5065,5069,5068,5104,5109,5107,5108,5013,5019,5022,5010,5009,5023,5011,5024,5012,5021,5007,5008,5025,5020,5055,5006,5036,5038,5054,5032,5033,5031,5039,5035,5026,5027,5030,5200,5043,5005,5041,5049,5042,5052,5040,5053,5045,5047,5048,5050,5051,5044,5046) and c6 in (5182) group by c5, c6; When I reverse the order of the WHERE clause, it runs in about .4 seconds. Query 2: c5, c6, sum(sale) from sobi.sample where c6 in (5182) and c5 in (5253,5110,5375,5268,5257,5266,5243,5256,5252,5361,5379,5363,5386,5367,5240,5241,5261,5249,5245,5260,5247,5264,5354,5250,5373,5366,5246,5356,5365,5255,5262,5259,5251,5369,5357,5269,5364,5380,5244,5248,5242,5254,5358,5360,5263,5362,5374,5378,5370,5294,5339,5272,5277,5275,5295,5340,5273,5276,5270,5061,5081,5079,5080,5066,5056,5073,5060,5070,5099,5101,5100,5067,5063,5077,5072,5062,5075,5083,5086,5071,5095,5058,5087,5090,5102,5084,5098,5076,5059,5074,5094,5091,5065,5069,5068,5104,5109,5107,5108,5013,5019,5022,5010,5009,5023,5011,5024,5012,5021,5007,5008,5025,5020,5055,5006,5036,5038,5054,5032,5033,5031,5039,5035,5026,5027,5030,5200,5043,5005,5041,5049,5042,5052,5040,5053,5045,5047,5048,5050,5051,5044,5046) group by c5, c6; This in on MonetDB server v5.6.0, based on kernel v1.24.0 which I built from the super tarball. I can attach the entire trace files for each query if appropriate for the list. While I am still learning to read the trace, it looks like most of the time spent in the first query on a series of algebra.semijoin operations. Here are the excerpted lines: Query 1: [2320345 usec # _461 := algebra.semijoin(_167=<~tmp_2116>bat[:str,:oid]{8280020}, _460=<~tmp_2016>bat[:str,:oid]{142}) ] [776017 usec # _463 := algebra.semijoin(_158=<tmp_2130>bat[:oid,:str]{8280020}, _462=<~tmp_1572>bat[:oid,:str]{5135708}) ] [253551 usec # _470 := algebra.semijoin(_464=<~tmp_2116>bat[:str,:oid]{5135708}, _469=<~tmp_2016>bat[:str,:oid]{1}) ] Query 2: [412122 usec # _179 := algebra.semijoin(_167=<~tmp_2076>bat[:str,:oid]{8280020}, _178=<~tmp_2016>bat[:str,:oid]{1}) ] [ 194 usec # _181 := algebra.semijoin(_158=<tmp_2070>bat[:oid,:str]{8280020}, _180=<~tmp_1572>bat[:oid,:str]{304}) ] [ 0 usec # _470 := algebra.semijoin(_182=<~tmp_2076>bat[:str,:oid]{304}, _469=<~tmp_2016>bat[:str,:oid]{142}) ] Any assistance would be greatly appreciated. -Ross

2 1

[MonetDB-users] Announcement: new minor release of MonetDB Clients package
by Sjoerd Mullender 12 Aug '08

12 Aug '08

The MonetDB team at CWI is pleased to announve a bug fix release for the MonetDB Clients package. A number of bugs in the ODBC driver have been fixed: - fixes for the handling of UTF-8 encoded queries and query parameters; - fixes for dealing with wide characters. There is also a fix for mclient: - the timer (-t/--time option) could overflow. There are new Windows installers for the ODBC driver, and there are new RPMs for Fedora 8 and 9. All of this for both 32 and 64 bit architectures. There is also a new Super Source Ball. -- Sjoerd Mullender

1 0

Re: [MonetDB-users] MonetDB rpm issues
by Stefan Manegold 11 Aug '08

11 Aug '08

On Mon, Aug 11, 2008 at 10:52:34AM +0100, John Baxter wrote: > Stefan, > > Sorry about the html format, it is just the default, i've changed it to > plain text now. Thanks! > cenos == centos, my mistake. > > CentOS is basically the same as Red Hat Enterprise server but rebranded. > I feel it is more stable than fedora core and as it is also open source > it is just as usable. > > I'm sorry, i didn't realise i would have to compile it first, i'll do > that today, thank you for the tips about what order to do it in. > > One last thing, i'm sure i read somewhere that the sql implementation is > the same as postgresql. Is this still correct, and is the implementation > exactly the same or are there some exceptions? Well, we do not follow or copy any other SQL implementation (in particular not their modification and/or extensions of the standard), but rather (try to) stick to teh standard. For details see http://monetdb.cwi.nl/SQL/Documentation/index.html and in particular http://monetdb.cwi.nl/SQL/Documentation/SQL-Reference.html Stefan > Thanks again for your assistance. > > Kind Regards > > John > > > Stefan Manegold wrote: >> Hi John, >> >> On Fri, Aug 08, 2008 at 04:26:09PM +0100, John Baxter wrote: >> >>> <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> >>> <html> >>> <head> >>> <meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type"> >>> </head> >>> <body bgcolor="#ffffff" text="#000000"> >>> Stefan, >>> >>> Thanks very much for replying, i really appreciate it. >>> >>> These are the RPM's i have tried: >>> >>> MonetDB5-server-5.6.0-4.src.rpm >>> MonetDB-SQL-2.24.0-4.src.rpm >>> >>> I'm only installing this on a local centos install. I downloaded the >>> RPM's to the root of the filesystem and ran them from there. >>> I'm running cenos from it's GUI at the moment so it would be the GUI >>> >> ^^^^^ >> Typo (i.e., "cenos" == "centos") or is that the application/tools you use to >> install RPMs? >> I don't know CentOS and/or it's adminstrative tools at all. Sorry. >> >> However, the "platform independent" RPMs are *source* RPMs that cannot be >> installed directly using, e.g., `rpm -i`. Instead, they have to be compiled >> (into platform-specific binary RPMs, first, e.g., via `rpmbuild --rebuild` >> or `rpmbuild --recompile`. >> >> See your systems RPM documentation (e.g., `man rpm` / `man rpmbuild` and/or >> http://www.rpm.org/ for a detailed documentation of RPM. >> >> >> Before compiling/building and installing >> MonetDB5-server-5.6.0-4.src.rpm >> and MonetDB-SQL-2.24.0-4.src.rpm >> >> you first need to compile/build and install >> MonetDB-1.24.0-4.src.rpm >> and MonetDB-client-1.24.0-4.src.rpm >> >> >> Stefan >> >> >> ps: We would greatly apprechiate if you would not sedn HTML(-only) >> emails/postings --- plain text is more than sufficient (and much easier >> to handle) ... >> >> >> >>> that gave me the error messages. >>> >>> Thanks >>> >>> John >>> >>> Stefan Manegold wrote: >>> <blockquote cite="mid:20080808151929.GA10101@cwi.nl" type="cite"> >>> <pre wrap="">Hi John, >>> >>> which "platform independent RPM's" are you refering to? >>> >>> On our download page on SF we only provide Fedora 8 & 9 (binary) RPMs as >>> well as source-RPMs --- only the latter are platform independent, except >>> that the target OS need to obviously provide RPM support in general. >>> >>> How exactly do you try to install the (source?) RPMs? >>> >>> And who (i.e., which program) give the error messages? >>> >>> Stefan >>> >>> On Fri, Aug 08, 2008 at 03:49:45PM +0100, John Baxter wrote: >>> </pre> >>> <blockquote type="cite"> >>> <pre wrap="">Stefan, >>> >>> I hope you don't mind me e-mailing you. You were kind enough to help >>> me once before when i was having trouble installing MonetDB and i'm >>> having trouble again. >>> >>> I'm trying to install the platform independent RPM's on centos 5.2 >>> but it is saying that it "cannot install source packages" and then >>> "no packages were given for installation" >>> >>> Are the RPM's usable at the moment? >>> >>> If you can offer any advice that would be great. >>> >>> Thank you >>> >>> Kind Regards >>> >>> John Baxter >>> </pre> >>> </blockquote> >>> <pre wrap=""> >>> </pre> >>> </blockquote> >>> >>> <pre class="moz-signature" cols="72">-- Co: Baxter Designs >>> >>> Web: <a class="moz-txt-link-freetext" href="http://www.baxterdesigns.co.uk">http://www.baxterdesigns.co.uk</a> >>> >>> Mail: <a class="moz-txt-link-abbreviated" href="mailto:john@baxterdesigns.co.uk">john(a)baxterdesigns.co.uk</a> >>> >>> Tel: 07968754215</pre> >>> </body> >>> </html> >>> >>> >> >> >>> ------------------------------------------------------------------------- >>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge >>> Build the coolest Linux based applications with Moblin SDK & win great prizes >>> Grand prize is a trip for two to an Open Source event anywhere in the world >>> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >>> _______________________________________________ >>> MonetDB-users mailing list >>> MonetDB-users(a)lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/monetdb-users >>> >> >> >> > > -- > Co: Baxter Designs > > Web: http://www.baxterdesigns.co.uk > > Mail: john(a)baxterdesigns.co.uk > > Tel: 07968754215 -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

3 5

[MonetDB-users] (no subject)
by Matt Magoffin 11 Aug '08

11 Aug '08

Hello, I am interested in evaluating MonetDB/XQuery. I have some XML collections for a project I work on that are of 1-2 million individual XML documents for which I use XQuery to access for OLAP style reporting operations. This is a Java-based project, and while working on this project I developed a thin XQuery-centric API so that I could evaluate/use many different XQuery implementations, which my company released as an open source project called Xaj (http://sourceforge.net/projects/xaj) Currently I am using a Berkeley DB XML implementation, but am interested in exploring other options. I'd like to implement a MonetDB version of this API now, but I am having some difficulty finding out how to effectively access MonetDB through Java. So far I was able to use the MonetDB JDBC driver to construct XQuery statements and get XML results, and also I was able to construct MonetDB pf:add-doc() statements to add documents, but I am wondering if this is the best approach (and if so, how to add documents to a remote MonetDB server using the pf:add-doc() mechanism). The API I am trying to implement you can view here, to give you an idea what I'm trying to do: http://xaj.svn.sourceforge.net/viewvc/xaj/xaj/src/net/sf/xaj/XmlDb.java?rev… It was somewhat modeled after the defunct XML:DB API, but focused just on add/store/XQuery operations. Any help/advise would be greatly appreciated! -- m@

2 3

Re: [MonetDB-users] MonetDB rpm issues
by Stefan Manegold 11 Aug '08

11 Aug '08

Hi John, which "platform independent RPM's" are you refering to? On our download page on SF we only provide Fedora 8 & 9 (binary) RPMs as well as source-RPMs --- only the latter are platform independent, except that the target OS need to obviously provide RPM support in general. How exactly do you try to install the (source?) RPMs? And who (i.e., which program) give the error messages? Stefan On Fri, Aug 08, 2008 at 03:49:45PM +0100, John Baxter wrote: > Stefan, > > I hope you don't mind me e-mailing you. You were kind enough to help me > once before when i was having trouble installing MonetDB and i'm having > trouble again. > > I'm trying to install the platform independent RPM's on centos 5.2 but > it is saying that it "cannot install source packages" and then "no > packages were given for installation" > > Are the RPM's usable at the moment? > > If you can offer any advice that would be great. > > Thank you > > Kind Regards > > John Baxter -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 2

[MonetDB-users] Announcing MonetDB support for Ruby on Rails
by Michalis Polakis 10 Aug '08

10 Aug '08

Hello folks, I developed the MonetDB adapter for Rail's ActiveRecord. It will enable you to work with MonetDB databases in Ruby on Rails. You are more than welcome to try the code, give me some feedback, and maybe help me maintain it. To get the code visit : http://rubyforge.org/projects/monetdb-ror Thanks, Michalis Polakis

1 0

[MonetDB-users] MonetDB Indexes and Data Types
by Ross Bates 09 Aug '08

09 Aug '08

Hi All - I'm new to MonetDB and just getting my feet wet. Quick question for the experts. Does the data type of the column impact the performance of the indexes created on the fly by MonetDB? In other words, should I expect WHERE clauses on INT columns to perform faster than VARCHAR like I would in a tradtional row store? Many Thanks, Ross

2 1

[MonetDB-users] full text search?
by mark 09 Aug '08

09 Aug '08

hi, i am beginning with monetdb. is there any support for full text search on monet-db via plugin or natively? thanks a lot

2 1

[MonetDB-users] COPY INTO on Ubuntu 64bit is slow
by Cronje Fourie 08 Aug '08

08 Aug '08

Hi guys.... I've got to machines running Monet. 1. Quad core Intel with 4GB RAM, Ubuntu 8.04 32BIT (server ed) 2. AMD64 Quad Core, 4GB RAM, KUbuntu 8.04 64BIT (kde 3.5.9) On system 1 I can do a "copy into" of 6million records in no time at all :) On system 2 the same function, with the exact same data takes forever. Any thoughts on the reason? Is this a memory related issue? Cronje

7 12