[MonetDB-users] bogus SQL in example + bad Mapi error reporting
Hi there, I just spent several hours scratching my head over why a piece of code didn't work. I was experimenting with Mapi and tried the example listed on the top of this page: http://monetdb.cwi.nl/TechDocs/APIs/Mapi/C/index.html This contains the following line: if ((hdl = mapi_query(dbh, "create table emp(name varchar, age int)")) == NULL) die(dbh); Unfortunately, it needs to be 'varchar(20)', not just plain 'varchar'. Just 'varchar' apparently does *not* result in an error, but instead all subsequent insert operations plus the select operation do nothing whatsoever; the fetching returns no results at all. The combination of this very bad error reporting and the mistake in the very first example given on the C API page cost me quite a bit of time, so I'm slightly grumpy. :) The bad example given is not good, but the way the error appears to be completely ignored is worse; this is not good for debuggability. The combination is terrible and not particularly conductive to the uptake of MonetDB. It may be of course that this bug is already fixed in CVS; I'm working off the latest release. Anyway, now that my test code finally works, I can hopefully make some progress hooking up XQuery to a Python API. (this is an experiment, and I realize the existing Python Mapi client code already ought to work, but we'll see..). Regards, Martijn
Hi Martijn, Sorry for the inconvenience created with this error and thanks for pointing it out. We will look into this area to see if there have been more bugs come alive over the years this code is being used. Please, don;t hesitate to identify problematic areas, because feedback is what we need to also prioritize our activities regarding bug-fixes and feature improvements (alongside our normal daily duties ;-)). regards, Martin Martijn Faassen wrote:
Hi there,
I just spent several hours scratching my head over why a piece of code didn't work. I was experimenting with Mapi and tried the example listed on the top of this page:
http://monetdb.cwi.nl/TechDocs/APIs/Mapi/C/index.html
This contains the following line:
if ((hdl = mapi_query(dbh, "create table emp(name varchar, age int)")) == NULL) die(dbh);
Unfortunately, it needs to be 'varchar(20)', not just plain 'varchar'. Just 'varchar' apparently does *not* result in an error, but instead all subsequent insert operations plus the select operation do nothing whatsoever; the fetching returns no results at all.
The combination of this very bad error reporting and the mistake in the very first example given on the C API page cost me quite a bit of time, so I'm slightly grumpy. :)
The bad example given is not good, but the way the error appears to be completely ignored is worse; this is not good for debuggability. The combination is terrible and not particularly conductive to the uptake of MonetDB.
It may be of course that this bug is already fixed in CVS; I'm working off the latest release.
Anyway, now that my test code finally works, I can hopefully make some progress hooking up XQuery to a Python API. (this is an experiment, and I realize the existing Python Mapi client code already ought to work, but we'll see..).
Regards,
Martijn
------------------------------------------------------- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
Hi Martin, Martin Kersten wrote:
Sorry for the inconvenience created with this error and thanks for pointing it out. We will look into this area to see if there have been more bugs come alive over the years this code is being used.
Please, don;t hesitate to identify problematic areas, because feedback is what we need to also prioritize our activities regarding bug-fixes and feature improvements (alongside our normal daily duties ;-)).
Thanks for the encouraging reply. Note that I'm not at all unhappy with the kind of feedback I've been getting so far; you guys are actually very quick. I am not discouraged easily and pretty outspoken, so don't worry, you'll get your feedback. I was just somewhat frustrated last night. I'm looking at MonetDB primarily with an interest in the XQuery implementation, which I'm hoping to eventually be able to apply in open source XML content management applications, using Python. (that said, the SQL implementation and the underlying relational layer are also interesting) If I have to struggle a bit to make it safe for Python, so be it; I have plenty of experience in that area, for instance here: http://codespeak.net/lxml which is an alternative binding to the libxml2 library for Python, with an aim to make it easy to use for Python programmers. I'm now speculating about a Python API for the XML database facilities MonetDB has (at a higher level than mapi). I glanced at MonetDB several times over the last years and found it interesting. Now that you have the XQuery implementation I believe you have the potential to attract a lot of open source developers in using this, as there isn't much else out there has this. Regards, Martijn
Martijn Faassen wrote:
If I have to struggle a bit to make it safe for Python, so be it; I have plenty of experience in that area, for instance here:
which is an alternative binding to the libxml2 library for Python, with an aim to make it easy to use for Python programmers. I'm now speculating about a Python API for the XML database facilities MonetDB has (at a higher level than mapi).
We have an XML:DB driver. That is Java, but it was the best I could find. It's mainly used by eXist, which has a Python binding too, I thinkg. We're looking for something better though, since it doesn't really cover all XQuery needs. Are there existing API's for Python, or are you up to write one yourself?
I glanced at MonetDB several times over the last years and found it interesting. Now that you have the XQuery implementation I believe you have the potential to attract a lot of open source developers in using this, as there isn't much else out there has this.
We hope your are right :)
Martijn Faassen wrote:
Hi there,
I just spent several hours scratching my head over why a piece of code didn't work. I was experimenting with Mapi and tried the example listed on the top of this page:
http://monetdb.cwi.nl/TechDocs/APIs/Mapi/C/index.html
This contains the following line:
if ((hdl = mapi_query(dbh, "create table emp(name varchar, age int)")) == NULL) die(dbh);
Unfortunately, it needs to be 'varchar(20)', not just plain 'varchar'. Just 'varchar' apparently does *not* result in an error, but instead all subsequent insert operations plus the select operation do nothing whatsoever; the fetching returns no results at all.
This is weird, because I once added a very explanatory error message for this situation. Using JDBC it seems this message is indeed generated: % JdbcClient -hlocalhost Connection warning: database specifier not supported on this server (mserver_sql), protocol version 4 Welcome to the MonetDB interactive JDBC terminal! Database: MonetDB 4.8.0_rc1 Driver: MonetDB Native Driver 1.2 (Blunt) Type \q to quit, \h for a list of available commands auto commit mode: on monetdb-> create table emp(name varchar, age int); Error: CHARACTER VARYING needs a mandatory length specification in: "create table emp(name varchar," monetdb-> So, I suspect the code sample to be faulty or the error reporting not to be optimal in this case for MAPI. Additionally, all subsequent inserts are bound to fail, but not silently. If you use auto commit mode, then you would get an error: monetdb-> insert into emp values ('bla', 300); Error: Inserting into non existing table emp otherwise, if you would have a transaction, the following error would arise: monetdb-> start transaction; auto commit mode: off monetdb-> create table emp(name varchar, age int); Error: CHARACTER VARYING needs a mandatory length specification in: "create table emp(name varchar," current transaction is aborted (please ROLLBACK) monetdb-> insert into emp values ('bla', 300); Error: current transaction is aborted (please ROLLBACK) monetdb-> rollback; auto commit mode: on monetdb-> There is always an error, but it simply doesn't reach you. I'm no MAPI expert. Sjoerd might see what's the problem here for this example. I included Sjoerd in the CC of this message.
The combination of this very bad error reporting and the mistake in the very first example given on the C API page cost me quite a bit of time, so I'm slightly grumpy. :)
I will fix the example on the website in CVS. Thanks.
The bad example given is not good, but the way the error appears to be completely ignored is worse; this is not good for debuggability. The combination is terrible and not particularly conductive to the uptake of MonetDB.
It may be of course that this bug is already fixed in CVS; I'm working off the latest release.
Anyway, now that my test code finally works, I can hopefully make some progress hooking up XQuery to a Python API. (this is an experiment, and I realize the existing Python Mapi client code already ought to work, but we'll see..).
Good luck!
Fabian wrote:
Martijn Faassen wrote:
[snip]
Unfortunately, it needs to be 'varchar(20)', not just plain 'varchar'. Just 'varchar' apparently does *not* result in an error, but instead all subsequent insert operations plus the select operation do nothing whatsoever; the fetching returns no results at all.
This is weird, because I once added a very explanatory error message for this situation. Using JDBC it seems this message is indeed generated:
% JdbcClient -hlocalhost Connection warning: database specifier not supported on this server (mserver_sql), protocol version 4 Welcome to the MonetDB interactive JDBC terminal! Database: MonetDB 4.8.0_rc1 Driver: MonetDB Native Driver 1.2 (Blunt) Type \q to quit, \h for a list of available commands auto commit mode: on monetdb-> create table emp(name varchar, age int); Error: CHARACTER VARYING needs a mandatory length specification in: "create table emp(name varchar,"
Right, this would've been a great error message indeed to have seen. :)
monetdb->
So, I suspect the code sample to be faulty or the error reporting not to be optimal in this case for MAPI.
The code sample is quite thorough in catching any error that Mapi may return (NULL value) and then to display it. Unfortunately NULL does not appear to be returned, so no error condition is noticed. Perhaps this is a case of: Almost every call can fail since the connection with the database server can fail at any time. Functions that return a handle (either Mapi or MapiHdl) may return NULL on failure, or they may return the handle with the error flag set. If the function returns a non-NULL handle, always check for errors with mapi_error. The sample code on the page is not doing this 'always checking for errors'. I shall try to modify it to do so, as it'd be good to have the sample do the right thing.
Additionally, all subsequent inserts are bound to fail, but not silently. If you use auto commit mode, then you would get an error:
monetdb-> insert into emp values ('bla', 300); Error: Inserting into non existing table emp
I tried auto commit mode, but that didn't seem to have any effect. Anyway, perhaps the errors are indeed there and just not caught by the C code. [snip]
There is always an error, but it simply doesn't reach you. I'm no MAPI expert. Sjoerd might see what's the problem here for this example. I included Sjoerd in the CC of this message.
Okay, thanks. I will try to see whether mapi_error does the trick. Regards, Martijn
Martijn Faassen wrote: [snip]
The code sample is quite thorough in catching any error that Mapi may return (NULL value) and then to display it. Unfortunately NULL does not appear to be returned, so no error condition is noticed.
Perhaps this is a case of:
Almost every call can fail since the connection with the database server can fail at any time. Functions that return a handle (either Mapi or MapiHdl) may return NULL on failure, or they may return the handle with the error flag set. If the function returns a non-NULL handle, always check for errors with mapi_error.
The sample code on the page is not doing this 'always checking for errors'. I shall try to modify it to do so, as it'd be good to have the sample do the right thing.
I've just done a simple modification, adding the following: if ((error = mapi_result_error(hdl)) != NULL) { mapi_explain_result(hdl, stderr); return 0; } and this indeed gives me the expected: MAPI = monetdb@localhost:45123 QUERY = create table emp(name varchar, age int) ERROR = !CHARACTER VARYING needs a mandatory length specification in: "create table emp(name varchar," !current transaction is aborted (please ROLLBACK) Because the text says: if the function returns a non-NULL handle, always check for errors with mapi_error. and later on says: To check for error messages from the server, call mapi_result_error(). This function returns NULL if there was no error, or the error message if there was. I would suggest changing the example so that indeed takes place. I'd also suggest altering the text a little so that it's clear you also should always check for mapi_result_error. Programmers like me will base their code off examples, instead of reading and understanding the text completely. :) Thanks for the feedback! Regards, Martijn
Martijn, thanks for digging this out!!! Sjoerd, would you please take care of fixing the samples and sample code in the documentation of Mapi.mx? I hope that now this is resolved you can actually start using MonetDB. Regards, Fabian Martijn Faassen wrote:
Martijn Faassen wrote: [snip]
The code sample is quite thorough in catching any error that Mapi may return (NULL value) and then to display it. Unfortunately NULL does not appear to be returned, so no error condition is noticed.
Perhaps this is a case of:
Almost every call can fail since the connection with the database server can fail at any time. Functions that return a handle (either Mapi or MapiHdl) may return NULL on failure, or they may return the handle with the error flag set. If the function returns a non-NULL handle, always check for errors with mapi_error.
The sample code on the page is not doing this 'always checking for errors'. I shall try to modify it to do so, as it'd be good to have the sample do the right thing.
I've just done a simple modification, adding the following:
if ((error = mapi_result_error(hdl)) != NULL) { mapi_explain_result(hdl, stderr); return 0; }
and this indeed gives me the expected:
MAPI = monetdb@localhost:45123 QUERY = create table emp(name varchar, age int) ERROR = !CHARACTER VARYING needs a mandatory length specification in: "create table emp(name varchar," !current transaction is aborted (please ROLLBACK)
Because the text says:
if the function returns a non-NULL handle, always check for errors with mapi_error.
and later on says:
To check for error messages from the server, call mapi_result_error(). This function returns NULL if there was no error, or the error message if there was.
I would suggest changing the example so that indeed takes place. I'd also suggest altering the text a little so that it's clear you also should always check for mapi_result_error. Programmers like me will base their code off examples, instead of reading and understanding the text completely. :)
Thanks for the feedback!
Regards,
Martijn
Fabian wrote:
thanks for digging this out!!! Sjoerd, would you please take care of fixing the samples and sample code in the documentation of Mapi.mx?
While am at it, there's another problem with the example code on: http://monetdb.cwi.nl/TechDocs/APIs/Mapi/C/index.html that I ran into and solved by finding the examples directory and looking at the code there. It uses "guest" for username and 0 for password. This doesn't work with the out of the box MonetDB install, while "monetdb", "monetdb" for username and password does work. The text: The user identification is the ubiquitous anonymous guest, for whom we do not require a password. ought also to be changed. There also appears to be a lack of error messages when you log in with the wrong username/password combination, though the program does appear to bail early in such cases.
I hope that now this is resolved you can actually start using MonetDB.
Yup, looking forward to playing with it more now. Happy to give feedback. Regards, Martijn
participants (3)
-
Fabian
-
Martijn Faassen
-
Martin Kersten