January 2010 - users-list

[MonetDB-users] OCaml-MonetDB is working.
by Guillaume Yziquel 22 Jan '10

22 Jan '10

Hello. I've managed to pull together, in my scarce free time, an OCaml binding for MonetDB. It is now working and I have written a very simple MAL client for an embedded database: > yziquel@seldon:~/git/ocaml-monetdb5$ utils/malclient.native > io.print(1); > 1 > io.print(2); > 2 I've got a small question concerning MAL. As I see it, MAL is currently a human-readable language. Is there somewhere in the MonetDB system a place where it is compiled to a more space-efficient representation? Or is interpreted as such? All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/

2 1

[MonetDB-users] MonetDB XQuery Bulk Load / Bulkload
by Alexander Koglin 21 Jan '10

21 Jan '10

I have currently installed a version of MonetDB XQuery for Linux and want to bulkload data like this: http://monetdb.cwi.nl/XQuery/Documentation/Bulk-Loading-a-Collection.html#B… For that purpose, I used the following command inside the Client: for $d in doc("/monetdbfiles.xml")//doc return pf:add-doc(fn:concat($d/@path,$d/@name), fn:string($d/@name), "abap10k", 0) <> It references the following, definitely existing XML file: <dir> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CCDEF"> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CCIMP"> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CCMAC"> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CI"> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CM001"> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CM002"> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CM003"> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CM004"> <doc path="/home/epic/tarari_svn/benchmarks/abap/ABAP/" name="ACL_AUNIT_RESULT==============CM005"> </dir> But unfortunately an error is thrown (I also tried it unter Windows and it was the same): MAPI = root@localhost:50000 QUERY = for $d in doc("/monetdbfiles.xml")//doc ERROR = !ERROR: err:FODC0002, Error retrieving resource (no such document "/monetdbfiles.xml"). What could be the reason for that?

3 2

[MonetDB-users] compression
by Jason Kinzer 20 Jan '10

20 Jan '10

Hi, Are there any plans on the horizon to roll compression into Monet? The X-100 project looked really interesting in this regard, but as I understand it, that work has been transferred into VectorWise. If there are no plans, is this because it's completely antithetical to the monet architecture (from the papers it seems like X-100 was, to some degree at least, 'integrated' in), or more due to lack of resources? My motivating example here is OLAP: I frequently have 1 relatively large fact table and then many much smaller dimensional tables. If optional compression were available, it we be nice to compress all or some of the BATs for the fact table columns and then have the others work as usual. (Well, at least this sounds good, maybe it makes no sense). Another motivation is there seems to be a lot of anecdotal evidence for companies moving from larger big iron servers to more numerous, smaller machines - so it would be really nice to have this capability for more memory constrained settings. I understand on a basic level how compression conflicts with the relatively simple approach monet uses to load BATs (e.g. memory map), but, dwelling in ignorance, I blithely assume there could be some solution not as complex as X-100 if one were to accept a significant performance cost. For example: decompressing BAT data on the fly as part of a BATiterator. I probably don't have the skills to implement even a basic on-the-fly decompression approach like this, but just wondering aloud: how hard a problem is this? Thanks, Jason

2 3

[MonetDB-users] manually setting gdk_nr_threads to N?
by Nafees Ur Rehman 19 Jan '10

19 Jan '10

Hi, I am using Windows 2003 with two core processors configures, and the gdk_nr_threads does not show the 2nd processor. Can you tell me how can I set that manually? Regards Nafees Ur Rehman

2 1

[MonetDB-users] Standard Deviation tips and hints request
by Dennis Forbes 17 Jan '10

17 Jan '10

Hi all. While it's possible to calculate the stddev with an aggregated cross join, is there perhaps a better way? Any tips or hints on where I should look? Can this be done in MIL or some other tactic to provide a more straightforward, better performing experience? I searched the web before asking and all I've found are unanswered questions, however I would really like to solve this even if it includes extending the system. Dennis W. Forbes Vastardis Capital Services 41 Madison Avenue, 30th Floor New York, NY 10010 P: 646-230-0933 F: 212-243-5251

2 2

[MonetDB-users] mapi_query with MAL code?
by Guillaume Yziquel 17 Jan '10

17 Jan '10

Hello. I've been using an embedded SQL with moderate success with my OCaml binding. Essentially passing the SQL statement through the mapi_query() function call. I would now like to try out some MAL code in a similar setting: embedded MonetDB, code passed via mapi_query(). Could someone provide me with a simple "Hello world" MAL code to try? For now, all I've been able to achieve with MAL are segfaults... All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/

2 1

[MonetDB-users] native socket protocol vs mapi
by Andy Chambers 16 Jan '10

16 Jan '10

Hi, I want to be able to use monetdb from common lisp and was looking at the other language bindings for clues on how to write one for cl. Python and Ruby seem to communicate using a socket rather than use the C api. However, in common lisp it's relatively easy to use C bindings (though CFFI) so I'd rather use them if I thought they were going to continue to be supported. Are there any other factors I should consider in deciding which API to use? Many Thanks, Andy

4 9

[MonetDB-users] Star Schema Benchmark: Can't load SF 5 ou SELECT, LINE_READ Crash
by Rui Anastácio 12 Jan '10

12 Jan '10

Hello, I Have Ubunto 9.10 Karmik Koala installed. On top of this I have installed MonetDB latest version (same problem with previous one). I'm working with Star Schema Benchmark (SSBM). For Scale Factor 1 and 3 I can load the data. For SF 5 our above I get READ_LINE error when loading LINEORDER table. Also when running select Q4.1 I also get the same error and the server crashes. Anyone as a clue? Thanks Rui A.

2 3

Re: [MonetDB-users] monetdb_sql/mal() ?
by Guillaume Yziquel 12 Jan '10

12 Jan '10

Jason Kinzer a écrit : >> >> Besides, It would seem to overlap with Mauricio's work which already >> provides some parsing from his OCaml algebra to SQL. But the thing is, >> you've got the SQL string created at OCaml compile time. I may be wrong about this compile time assertion. It now seems to me that this is runtime. In which case, it would make sense to map Mauricio's relational algebra to MAL's relational algebra. >> What would be nice >> is to have the SQL -> MAL compilation also happen at the same time you >> compile the OCaml program. >> >> (Moreover, it seems to me that MAL comes in two flavour: a human-readable >> one and a machine readable one. Is that correct? If yes, I'm aiming at the >> machine readable one.) >> >> How easy would that be? Do you need a running embedded server for that, or >> simply a subset of the functionalities of the MonetDB shared libraries? > Ah, I see now, so you're talking about actually writing a SQL->MAL compiler > as part of that existing Ocaml SQL frontend (sort of cut out the > "middle-man"). Yes. > That sounds neat, but bear in mind that could be tantamount > to re-writing the existing Monet SQL frontend, depending on your goals In fact, I would like to do an OCaml/C binding to the SQL -> MAL compilation that is already done in MonetDB. This seems to me the least effort path. But I'd like to know where in the code is this compilation / conversion from SQL to MAL done. > - so I let you draw your own conclusions. Try doing an "explain" on various SQL > statements to see the generated MAL. Sorry, I'm quite new to MonetDB and still having a hard time with the documentation. What is this "explain" thing I should be doing? > I would say writing a very naive, > minimal SQL translation is not easy but certainly within raeach. Here > someone other than I should comment, but I think most of the complexity of > the existing frontend lies in the optimization passes, tx handling, > approaches to handle datasets that won't fit in memory, etc. Rewriting an SQL -> MAL translation is not my goal. Reusing an existing one would be one my goals. All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/

2 1

[MonetDB-users] monetdb_sql/mal() ?
by Guillaume Yziquel 11 Jan '10

11 Jan '10

Hello. I've been looking at Debian packages, and the followig things struck me: -1- You have two shared libraries: libembeddedsql.so and libembeddedmal.so which are distributed, but there's only headers for the *sql.so one. You have to write your own for the *mal.so one. -2- In the source, there are two libembeddedclient.c.in files: one for sql, one for mal. The one for sql contains a monetdb_sql() function call. The one for mal does not. Is there a good reason for this? Or is it simply the way it currently is? All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/

2 4