[Monetdb-developers] Problem editing the .mx files.
Hello. I need to pipe data into a monetdb database (something along the lines of 'myprocess | MapiClient -s "COPY ...."'. It doesn't seem to be possible out-of-the-box, so I wanted to poke around in the code to find out why and how to fix it (if needed), and hopefully submit a patch. I've been having a problem with that (playing with the code). I don't know how to 'compile' the modified .mx files in the sql module. I have both bison and flex installed, but the makefile uses the commands 'x' and 'l' to process the .mx, and I don't have any idea of what x and l are. So, in a nutshell, my question is: "How do I recompile the sql module after I modify the .mx files? What are 'x' and 'l'?" Regards, Zarrabeitia.
On 07/19/2006 05:02 PM, Luis Zarrabeitia wrote:
Hello. I need to pipe data into a monetdb database (something along the lines of 'myprocess | MapiClient -s "COPY ...."'. It doesn't seem to be possible out-of-the-box, so I wanted to poke around in the code to find out why and how to fix it (if needed), and hopefully submit a patch.
I've been having a problem with that (playing with the code). I don't know how to 'compile' the modified .mx files in the sql module. I have both bison and flex installed, but the makefile uses the commands 'x' and 'l' to process the .mx, and I don't have any idea of what x and l are.
So, in a nutshell, my question is: "How do I recompile the sql module after I modify the .mx files? What are 'x' and 'l'?"
You will definitely need the buildtools package and possibly the MonetDB package, both from the MonetDB repository at SourceForge. In MonetDB there is a file HowToStart which explains how to get buildtools and then refers you to the README file in buildtools on how to compile that. -- Sjoerd Mullender
Hi! On 19-07-2006 11:02:02 -0400, Luis Zarrabeitia wrote:
I need to pipe data into a monetdb database (something along the lines of 'myprocess | MapiClient -s "COPY ...."'. It doesn't seem to be possible
If you do, it would be great if you could implement it in such a way that matches our TODO on this process. Sjoerd already gave you pointers on how to get started coding. Enjoy! :)
On Wednesday 19 July 2006 12:22, Fabian Groffen wrote:
If you do, it would be great if you could implement it in such a way that matches our TODO on this process.
Sjoerd already gave you pointers on how to get started coding. Enjoy!
:)
I began coding already :D It doesn't seem too hard, in fact, I already have a preliminary version where I don't have to specify the number of records beforehand (and only a few lines of code). I don't like it much, though, so I think its better to consult the list before even trying to clean it up. On my cvs checkout Niels Nes' suggestion: ========== The 'copy into' should work but it should be part of your generated output. The stream should generated COPY opt_nr INTO qname FROM STDIN opt_seps data empty_line ========== didn't work, perhaps I checked out a wrong branch. However, it was easy to make it work, and that is what I coded (make the copy end if either an empty line is found, or the opt_nr records have been inserted). The reason I don't like it much is that I don't know if empty_line had some semantics before that and I'm changing it. I had been also thinking of adding a third (optional) opt_seps, with a sensible default (empty_line?), that would let me specify what will be the <<end of input>> marker, but it seems I cannot do that without getting deep (changing the mvc_create_table function's signature even). So, what would be better? Leave it with 'empty_line' (easy and almost there), make up another marker (postgres uses \.), or try to add the third opt_seps? What branch should I checkout? Regards, Zarrabeitia (eager to feed my 2 million row database to Monetdb).
Hi! Congrats on your first (positive) experience with MonetDB, hehehe. On 24-07-2006 16:02:22 -0400, Luis Zarrabeitia wrote: [snip]
I had been also thinking of adding a third (optional) opt_seps, with a sensible default (empty_line?), that would let me specify what will be the <<end of input>> marker, but it seems I cannot do that without getting deep (changing the mvc_create_table function's signature even).
So, what would be better? Leave it with 'empty_line' (easy and almost there), make up another marker (postgres uses \.), or try to add the third opt_seps?
I like the way PostgreSQL does their dumps. Compatibility with them would be a plus for me, and is on my wish-list.
What branch should I checkout?
Just the CVS HEAD (i.e. no branch), imho.
participants (3)
-
Fabian Groffen -
Luis Zarrabeitia -
Sjoerd Mullender