Hi, again ;-)
Side notes: - I had to make some mods to get the monetdb.pm Perl module to work. In particular, in MIL mode you need to use double-quotes ("") rather than single quotes ('') when wrapping a parameter so that it can be translated (ie, it is a string, not a character).
Since I'm not a perl expert, I feel free to forward this to Steffen Goeldner, who is generously taking care of the Perl interface of MonetDB. Steffen, I hope, you don't mind, if I put his on your desk, do you? I would be great, I you could have a look at this problem. You can find the whole thread at http://sourceforge.net/mailarchive/forum.php?thread_id=6536996&forum_id=42505 If necessary, you can probabaly ask Ed for his patch... Thank you very much in advance! [ The remainder is not Perl-related, but I didn't want to split this mail in two. Sorry for the inconvenience...]
- I was able to rename a BAT to a name with a "." in it, but then was unable to refer to the BAT using the new name.
Thanks. This is actually also a bug that hasn't been detected, yet. Niels is also working on this one. It will be fixed in the final release. We will then disallow "strange characters" such as '.' in BAT names.
- I tried a load where I committed after every 100 rows. Noticed _huge_ I/O surges. Looking at the subdirectory, it looked like the X.theap file was being rewritten over and over from scratch. If this is a memory-mapped file, why would this occur? When you start playing with 55MB heap files, this kills performance.
For now, we cannot do much about that, and I suppose, there'll never be any change. Basically, we have to write the whole file to make sure that all changes are properly committed. But why do you consider having commits after every 100 rows, if you do load 250k rows in one go? Isn't a single commit at the end enough?
- Is it better to use "BAT.reverse().find(X)" than "BAT.search(X)"?
Well, this are two "different pairs of shoes": Actually, there is no "BAT.search(X)" in MIL, only a "str.search(str)" (or "str.search(chr)") for seaching in a single string: ======== MonetDB>help("search"); COMMAND: search(str, str) : int MODULE: str COMPILED: by adm on Thu Feb 10 20:16:51 2005 Search for a substring. Returns position, -1 if not found. COMMAND: search(str, chr) : int MODULE: str COMPILED: by adm on Thu Feb 10 20:16:51 2005 Search for a character. Returns position, -1 if not found. DEPRECATED: does not work for non-ASCII characters, use: search(str,str) ======== "BAT.find(X)", however, looks-up a value in the head-column of a BAT, and returns the respective tail value: ======== MonetDB>help("find"); COMMAND: find(BAT[any::1,any::2], any::1) : any::2 MODULE: algebra COMPILED: by adm on Thu Feb 10 20:16:52 2005 Returns the tail value 't' for which some [h,t] BUN exists in b. If no such BUN exists, an error occurs. ======== Note, find return only a single tail value. If the requested head-value occurs in multiple BUNs (tuples), ind choses a random one and return its tail value. Maybe, you actually ment "select" instead of "search": ======== MonetDB>help("select"); COMMAND: select(BAT[any::1,any::2], any::2) : BAT[any::1,any::2] MODULE: algebra COMPILED: by adm on Thu Feb 10 20:16:52 2005 Select all BUNs of a BAT with a certain tail value. Selection on NIL is also possible (it should be properly casted, e.g.: int(nil)). ======== Hence, using "find" only makes sense, if you only need the tail-value, and - you are sure that the looked-up value does exit exactly once in the head-column, - or, it occurs more than once, but you do not care, which tail-value you get. Note, "head" and "tail" in the above discussion can be swapped "at no cost", since "BAT.reverse()" does not cost anything (i.e., no CPU, no IO, no memory) in MonetDB. I hope, this answers your question. Please let us know, if you have more questions, problems, comments, etc. Regards, Stefan
Regards! Ed
-- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |