[MonetDB-users] segfault with monetdb_sql().
Hello. I've been having segfaults when using the following wrapper code. (Do not pay attention to the CAML* macros, and String_val is a macro that converts a string in OCaml to a C string, and Val_ptr is a function that wraps up a pointer into OCaml.)
CAMLprim value ml_monetdb_sql (value ml_dbfarm, value ml_dbname) { CAMLparam2(ml_dbfarm, ml_dbname); CAMLlocal1(result); char * dbfarm; char * dbname; dbfarm = String_val(ml_dbfarm); dbname = String_val(ml_dbname); Mapi mapi = monetdb_sql(dbfarm, dbname); result = Val_ptr(mapi); CAMLreturn(result); }
I'm calling this code with dbfarm and dbname set to a place I can write to, and that does not contain folders, files, or data at the time the function is called from OCaml code. When executing the following piece of code from the OCaml toplevel
#use "topfind";; #thread;; #load "monetDB5.cma";;
open MonetDB5;; let mapi = monetdb_sql "/home/yziquel/sandbox/monetdb" "testdb";;
I get a segfault arising from monetdb's shared libraries (I'm using the Debian packages on a 64 bits platform):
yziquel@seldon:~/git/ocaml-monetdb5$ ocaml -init ocamlinit Objective Caml version 3.11.1
Findlib has been successfully loaded. Additional directives: #require "package";; to load a package #list;; to list the available packages #camlp4o;; to load camlp4 (standard syntax) #camlp4r;; to load camlp4 (revised syntax) #predicates "p,q,...";; to set these predicates Topfind.reset();; to force that packages will be reloaded #thread;; to enable threads
/usr/lib/ocaml/threads: added to search path /usr/lib/ocaml/unix.cma: loaded /usr/lib/ocaml/threads/threads.cma: loaded Erreur de segmentation
Here at the last line... Looking at the code in monetdb-sql-2.34.2/src/backends/monet5/embeddedclient.c.in I'm unable to determine if the segault comes from the invocation of pthread_create or from the invocation of mapi_start_talking.
Mapi monetdb_sql(char *dbfarm, char *dbname) { Mapi mid; pthread_t sqlthread; stream **server;
int len = mo_builtin_settings(&embedded_set);
/* needed, to prevent the MonetDB config file to be used */ len = mo_add_option(&embedded_set, len, opt_config, "prefix", MONETDB5_PREFIX); len = mo_add_option(&embedded_set, len, opt_config, "config", MONETDB5_CONFFILE); len = mo_add_option(&embedded_set, len, opt_config, "monet_mod_path", "@QXlibdir@@QDIRSEP@MonetDB5@PATHSEP@@QXlibdir@@QDIRSEP@MonetDB5@QDIRSEP@lib@PATHSEP@@QXlibdir@@QDIRSEP@MonetDB5@QDIRSEP@bin");
embedded_len = mo_system_config(&embedded_set, len); embedded_len = mo_add_option(&embedded_set, embedded_len, opt_cmdline, "gdk_dbfarm", dbfarm); embedded_len = mo_add_option(&embedded_set, embedded_len, opt_cmdline, "gdk_dbname", dbname);
server = mapi_embedded_init(&mid,"sql");
pthread_create(&sqlthread, NULL, start_sql_server, (void *) server);
mapi_start_talking(mid);
return mid; }
Could someone give me hints as to what is going wrong here? All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/
On 26-12-2009 01:26:03 +0100, Guillaume Yziquel wrote:
Could someone give me hints as to what is going wrong here?
Is it possible to hook up a debugger to the ocaml process? e.g. GDB? I'm affraid to be able to give more useful information you'd have to compile monetdb yourself using debugging symbols (-g) and without stripping symbols.
Fabian Groffen a écrit :
On 26-12-2009 01:26:03 +0100, Guillaume Yziquel wrote:
Could someone give me hints as to what is going wrong here?
Is it possible to hook up a debugger to the ocaml process? e.g. GDB? I'm affraid to be able to give more useful information you'd have to compile monetdb yourself using debugging symbols (-g) and without stripping symbols.
There is a debugger, but scarcely used in the OCaml community. Unfortunately, I do not believe that it digs down to wrapped C stuff. I guess I'll need to have a closer look at the use of gdb with a standalone C program. Thanks. All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/
pthread_create(&sqlthread, NULL, start_sql_server, (void *) server);
I would start by being more defensive. What is the return value of
pthread_create? did it work?
mapi_start_talking(mid);
This leads to a potential race condition, when this function is called there is no proof that the server thread has actually be started. This means that all kind of variables will not have been initialized and which easily generates a segfault.
return mid; }
Could someone give me hints as to what is going wrong here?
Martin Kersten a écrit :
pthread_create(&sqlthread, NULL, start_sql_server, (void *) server);
I would start by being more defensive. What is the return value of pthread_create? did it work?
That is something I'd like to know. My problem is that the start_sql_server symbol is not exported from the shared library libembeddedsql5.so. This code is in fact what is available in the libembeddedsql5.so provided by the Debian package. The exported symbol is monetdb_sql:
yziquel@seldon:/usr/lib$ nm -D libembeddedsql5.so | grep monetdb_sql 0000000000017d70 T monetdb_sql
My issue is that start_sql_server is not exported...
mapi_start_talking(mid);
This leads to a potential race condition, when this function is called there is no proof that the server thread has actually be started. This means that all kind of variables will not have been initialized and which easily generates a segfault.
If there is a race condition in monetdb_sql and embedded_sql in the libembeddedsql5.so library, then this is likely a bug...
return mid; }
For the record, the full list of options used to start the server is the following:
# s;; - : MonetDB5.Opt.opt list = [{kind = Builtin; name = "gdk_arch"; value = "64bitx86_64-pc-linux-gnu"}; {kind = Builtin; name = "gdk_version"; value = "1.34.2"}; {kind = Builtin; name = "prefix"; value = "/usr"}; {kind = Builtin; name = "exec_prefix"; value = "${prefix}"}; {kind = Builtin; name = "gdk_dbname"; value = "tst"}; {kind = Builtin; name = "gdk_dbfarm"; value = "/var/MonetDB"}; {kind = Builtin; name = "gdk_debug"; value = "8"}; {kind = Builtin; name = "gdk_alloc_map"; value = "yes"}; {kind = Builtin; name = "gdk_vmtrim"; value = "yes"}; {kind = Builtin; name = "monet_admin"; value = "adm"}; {kind = Builtin; name = "monet_prompt"; value = ">"}; {kind = Builtin; name = "monet_welcome"; value = "yes"}; {kind = Builtin; name = "monet_mod_path"; value = "${exec_prefix}/lib/MonetDB"}; {kind = Builtin; name = "monet_daemon"; value = "yes"}; {kind = Builtin; name = "host"; value = "localhost"}; {kind = Builtin; name = "mapi_port"; value = "50000"}; {kind = Builtin; name = "mapi_noheaders"; value = "no"}; {kind = Builtin; name = "mapi_debug"; value = "0"}; {kind = Builtin; name = "mapi_clients"; value = "2"}; {kind = Builtin; name = "sql_debug"; value = "0"}; {kind = Builtin; name = "standoff_ns"; value = "http://monetdb.cwi.nl/standoff"}; {kind = Builtin; name = "standoff_start"; value = "start"}; {kind = Builtin; name = "standoff_end"; value = "end"}; {kind = Config; name = "prefix"; value = "/usr"}; {kind = Config; name = "config"; value = "/etc/monetdb5.conf"}; {kind = Config; name = "monet_mod_path"; value = "${exec_prefix}/lib/MonetDB5:${exec_prefix}/lib/MonetDB5/lib:${exec_prefix}/lib/MonetDB5/bin"}; {kind = Config; name = "prefix"; value = "/usr"}; {kind = Config; name = "exec_prefix"; value = "${prefix}"}; {kind = Config; name = "gdk_dbfarm"; value = "/var/MonetDB5/dbfarm"}; {kind = Config; name = "gdk_dbname"; value = "demo"}; {kind = Config; name = "gdk_alloc_map"; value = "no"}; {kind = Config; name = "gdk_embedded"; value = "no"}; {kind = Config; name = "gdk_debug"; value = "0"}; {kind = Config; name = "monet_mod_path"; value = "${exec_prefix}/lib/MonetDB5:${exec_prefix}/lib/MonetDB5/lib:${exec_prefix}/lib/MonetDB5/bin"}; {kind = Config; name = "monet_daemon"; value = "no"}; {kind = Config; name = "monet_welcome"; value = "yes"}; {kind = Config; name = "mero_msglog"; value = "/var/log/MonetDB/merovingian.log"}; {kind = Config; name = "mero_errlog"; value = "/var/log/MonetDB/merovingian.log"}; {kind = Config; name = "mero_pidfile"; value = "/var/run/MonetDB/merovingian.pid"}; {kind = Config; name = "mero_controlport"; value = "50001"}; {kind = Config; name = "mal_init"; value = "${exec_prefix}/lib/MonetDB5/mal_init.mal"}; {kind = Config; name = "mal_listing"; value = "2"}; {kind = Config; name = "mapi_port"; value = "50000"}; {kind = Config; name = "mapi_autosense"; value = "false"}; {kind = Config; name = "mapi_open"; value = "false"}; {kind = Config; name = "sql_optimizer"; value = "default_pipe"}; {kind = Config; name = "default_pipe"; value = "inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,garbageCollector,dataflow,history,multiplex"}; {kind = Config; name = "replication_pipe"; value = "inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,garbageCollector,dataflow,history,replication,multiplex"}; {kind = Config; name = "accumulator_pipe"; value = "inline,remap,evaluate,costModel,coercions,emptySet,aliases,mergetable,deadcode,constants,commonTerms,joinPath,deadcode,reduce,accumulators,garbageCollector,dataflow,history,multiplex"}; {kind = Config; name = "recycler_pipe"; value = "inline,remap,evaluate,costModel,coercions,emptySet,aliases,deadcode,constants,commonTerms,joinPath,deadcode,recycle,reduce,garbageCollector,dataflow,history,multiplex"}; {kind = Config; name = "cracker_pipe"; value = "inline,remap,evaluate,costModel,coercions,emptySet,aliases,selcrack,deadcode,constants,commonTerms,joinPath,deadcode,reduce,garbageCollector,dataflow,history,multiplex"}; {kind = Config; name = "datacell_pipe"; value = "inline,remap,evaluate,costModel,coercions,emptySet,aliases,deadcode,constants,commonTerms,joinPath,datacell,deadcode,reduce,garbageCollector,dataflow,history,multiplex"}; {kind = Config; name = "octopus_pipe"; value = "inline,remap,evaluate,costModel,coercions,emptySet,mitosis,aliases,mergetable,deadcode,constants,commonTerms,joinPath,octopus,deadcode,reduce,garbageCollector,dataflow,history,multiplex"}; {kind = Config; name = "mitosis_pipe"; value = "inline,remap,evaluate,costModel,coercions,emptySet,mitosis,aliases,mergetable,deadcode,constants,commonTerms,joinPath,reorder,deadcode,reduce,garbageCollector,dataflow,history,multiplex"}; {kind = Command_line; name = "gdk_dbfarm"; value = "/home/yziquel/sandbox/monetdb5"}; {kind = Command_line; name = "gdk_dbname"; value = "testdb"}]
Please tell me if there is something obviously wrong here. I'll try to check if the thread started successfully. -- Guillaume Yziquel http://yziquel.homelinux.org/
Martin Kersten a écrit :
pthread_create(&sqlthread, NULL, start_sql_server, (void *) server);
I would start by being more defensive. What is the return value of pthread_create? did it work?
I have been progressively wrapping stuff at a very low-level to see where the segfault comes from. I have reached the conclusion that the segfault occurs at the line 71 of embeddedclient.c.in, in the monetdb-sql debian package:
if (mal_init())
It's the call to mal_init() that gives rise to a segfault. That's quite a pain... All the best, -- Guillaume Yziquel http://yziquel.homelinux.org/
Guillaume Yziquel wrote:
Martin Kersten a écrit :
pthread_create(&sqlthread, NULL, start_sql_server, (void *) server);
I would start by being more defensive. What is the return value of pthread_create? did it work?
I have been progressively wrapping stuff at a very low-level to see where the segfault comes from. I have reached the conclusion that the segfault occurs at the line 71 of embeddedclient.c.in, in the monetdb-sql debian package:
if (mal_init())
It's the call to mal_init() that gives rise to a segfault.
As said before, there is no guarantee that your snippet will work as shown. It just contains basic programming errors, namely properly handling return values, and dealing with parallel processing. You have to wait for the sql-server thread to finish it initialization work, which may contain a lot of actions. Alternative, run it with gdb to be able to inspect the stack trace of all running threads to identify your programming misconception. regards, Martin
That's quite a pain...
All the best,
participants (3)
-
Fabian Groffen -
Guillaume Yziquel -
Martin Kersten