- developers-list - monetdb.org

[Monetdb-developers] Large XML instances with MonetDB/XQuery 4.38.5
by Christian Grün 02 Oct '10

02 Oct '10

Hi everyone, is it still possible to shred large XML instances with MonetDB/XQuery, or do I need more memory for shredding? I was just trying to the 1GB instance of XMark with WinXP 32Bit and 2GB RAM, and this is what I got: MonetDB>shred_doc("1gb.xml", "xmark"); #GDKmmap(536870912) fails, try to free up space [memory in use=36143780,virtual memory in use=1015480320] #GDKmmap(536870912) result [mem=26016940,vm=1005387776] !ERROR: [shred_url]: 1 times inserted nil due to errors at tuples 0@0. !ERROR: [shred_url]: first error was: !ERROR: HEAPextend: failed to extend to 536870912 for 06\632theap !ERROR: shredBAT_append_str: APPEND-STR[_prop_text]( follow murders prov shoes p resent eager delivered defy bend viol here marquis powers complexion pillar cran ts agate ), BUNappend fails !ERROR: CMDshred_url: operation failed. pf:add-doc(...) produces the same output; the XML document itself is well-formed. Thanks, Christian

2 6

[Monetdb-developers] How to commit into MonetDB repo.
by Alexandr Kulik 23 Sep '10

23 Sep '10

Hi All, I had implemented a few statistical function for MonetDB SQL. Standard deviation and variance for sample and population. I'm interesting is it possible to commit this code into repo, so everyone who need them could use it. This changes will partially cover bug # 2454. -- With Best Regards Alexandr Kulik Skype: kulya_u Yahoo: kulya_u LinkedIn: http://www.linkedin.com/in/alexandrkulik Mobile: +3(806)615-40511

3 3

Re: [Monetdb-developers] How to enable functions via sql?
by Stefan de Konink 23 Sep '10

23 Sep '10

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Op 23-09-10 20:54, Alexandr Kulik schreef: > I implement few functions (standard deviation sample and population, > variance sample ad population). Is it possible to commit them into > MonetDB repo? Again, use the monetdb-dev list. I guess your best shot is to log a bug for it at: <http://bugs.monetdb.org/> and mark it as enhancement. Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAkybq/kACgkQYH1+F2Rqwn1m/QCfXje9sE6hNYXhkp9IfUnGPo/n dw0An1d64ZiGZ07NCeJY+BgxaPVkyLLW =1hPu -----END PGP SIGNATURE-----

1 0

Re: [Monetdb-developers] How to enable functions via sql?
by Stefan de Konink 20 Sep '10

20 Sep '10

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Op 20-09-10 19:57, Alexandr Kulik schreef: > I find how could i do it. Now i fighting with another problem. Is it > possible from monetdb sql side create aggregation function that will > receive 2 arguments? Probably e-mailing the monetdb-dev list is a better place to ask such questions. But yes this must be possible. Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAkyXv3wACgkQYH1+F2Rqwn3Q3QCfQLHj8lz/osJxGu/nPL96a+0G YQwAoIWMLBQd4SpaQ3sjNia7kOHwDfy6 =8YsW -----END PGP SIGNATURE-----

1 0

[Monetdb-developers] About MonetDB query optimize
by xixi 17 Sep '10

17 Sep '10

Dear all, Hello,Recently I am intersting in query optimize, and have know that most database's query optimizing methods contain two ：1. Rule Base Optimizer; 2.Cost Base Optimizer. I also know that MonetDB has two-level optimizing ,one is relation algebra optimize and the other is MAL optimize when interprets a sql. I understand relation algebra optimize is no relationship with the column-oriented special query optimize.MAL optimizer may be some relationship with the column-oriented special query optimize. So I want to ask :MAL optimize is also contain Rule Base Optimizer and Cost Base Optimizer? which optimizers is the special query optimizer for the column-oriented and not for tuple-oriented database? Best regards!

1 0

Re: [Monetdb-developers] How to enable functions via sql?
by Alexandr Kulik 16 Sep '10

16 Sep '10

Stefan, I understand how to create function from sql that receive value as input param(s). But how could i write sql function that receive array of values as input? For example i want to enable stdev function at algebra module, is it possible to do from sql? Alex Op 06-09-10 18:13, Alexandr Kulik schreef: > I need to develop some extra statistical functions for MonetDB. I > spent about a week to understand how to use function via sql, but > still didn't have any succes results. May be someone could point me to > appropriate documentation or source revision, where i could look on > it? You might be interested in these two files it shows how to add a module to SQL written in the mx language: http://kinkrsoftware.nl/contrib/monetdb/sphinx.mx http://kinkrsoftware.nl/contrib/monetdb/sphinx.patch After that you will still have to 'create function', but that is relatively simple. Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAkyFFBUACgkQYH1+F2Rqwn1lRwCfe02Bjv3YozRMu3jaUug1bMgD lXYAnAn3ktCoAEZGq7CEr0aCAfKRCWNs =FNsy -----END PGP SIGNATURE----- -- With Best Regards Alexandr Kulik Skype: kulya_u Yahoo: kulya_u LinkedIn: http://www.linkedin.com/in/alexandrkulik Mobile: +3(806)615-40511

1 0

Re: [Monetdb-developers] MonetDB: Oct2010 - When you have MAL plans with 500k lines of co...
by Romulo Goncalves 15 Sep '10

15 Sep '10

Hello, I fixed the alloca problem, but I am wondering if the newStack function is correct. On 15/09/10 15:56, Romulo Goncalves wrote: > Changeset: 2727702e548b for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=2727702e548b > Modified Files: > MonetDB5/src/mal/mal_interpreter.mx > MonetDB5/src/mal/mal_stack.mx > Branch: Oct2010 > Log Message: > > When you have MAL plans with 500k lines of code mserver crashes at the memset after newStack. > This function is only called in one place and followed by a memset. Therefore, the alloca + memset > can be replaced by a zalloc. > > The structure is free in the end of the function call before the return. > > > diffs (32 lines): > ... > #define stackSize(CNT) (sizeof(ValRecord)*(CNT) + sizeof(MalStack)) > -#define newStack(S,CNT) S= (MalStkPtr) alloca(stackSize(CNT));\ > +#define newStack(S,CNT) S= (MalStkPtr) GDKzalloc(stackSize(CNT));\ > (S)->stksize=CNT; I do not know the context, but shouldn't the stack size be stackSize(CNT) instead of CNT? Romulo > > > _______________________________________________ > Checkin-list mailing list > Checkin-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/checkin-list

1 0

Re: [Monetdb-developers] Restart mclient
by Romulo Antonio Goncalves 07 Sep '10

07 Sep '10

Indeed, The problem is gone. Thanks a lot. Regards, Romulo ----- Original Message ----- From: "Sjoerd Mullender" <sjoerd(a)acm.org> To: monetdb-developers(a)lists.sourceforge.net Sent: Tuesday, September 7, 2010 3:22:47 PM Subject: Re: [Monetdb-developers] Restart mclient The culprit seems to be changeset f2ceaaf24796: changeset: 37471:f2ceaaf24796 parent: 37469:3733beae1496 user: Fabian Groffen <fabian(a)cwi.nl> date: Fri Sep 03 15:27:26 2010 +0200 files: MonetDB5/src/mal/mal_session.mx description: bugzie says that bug #1939 is fixed, so disable the guard On 2010-09-07 16:02, Romulo Antonio Goncalves wrote: > Hello, > > I updated my default branch and I do the following: > > mclient -lmal >> \q > mclient -lmal >> \q > > After the second exit I get a crash in the mserver: >> [New Thread 0x7fffdf7ac710 (LWP 19441)] > Program received signal SIGSEGV, Segmentation fault. > [Switching to Thread 0x7fffdf7ac710 (LWP 19441)] > 0x00007ffff7cd7198 in MALexitClient (c=0x605b38) at > /ufs/goncalve/MonetDB/current/MonetDB5/src/mal/mal_session.mx:522 522 > if (c->glb && c->curprg->def->errors == 0) > (gdb) p c->curprg->def > $1 = (struct MALBLK *) 0xdbdbdbdbdbdbdbdb > > Similar problem if you do > mclient -lsql >> \q > mclient -lsql > > The mserver crashes due to the same problem: > Program received signal SIGSEGV, Segmentation fault. > [Switching to Thread 0x7fffdf7ac710 (LWP 19561)] > 0x00007fffdfe22a60 in SQLparser (c=0x605b38) at > /ufs/goncalve/MonetDB/current/sql/src/backends/monet5/sql_scenario.mx:1155 > 1155 oldvtop = c->curprg->def->vtop; > (gdb) p c->curprg > $5 = (Symbol) 0x7fffd8112198 > (gdb) p *c->curprg > $6 = {peer = 0xdbdbdbdbdbdbdbdb, skip = 0xdbdbdbdbdbdbdbdb, > name = 0xdbdbdbdbdbdbdbdb <Address 0xdbdbdbdbdbdbdbdb out of > bounds>, kind = -606348325, def = 0xdbdbdbdbdbdbdbdb} > (gdb) > > > Were there any changes in the client structure or in the > initialization of the of the client threads? > > Regards, > Romulo > > PS: It does not happen in the Oct2010 branch. > > ------------------------------------------------------------------------------ > This SF.net Dev2Dev email is sponsored by: > > Show off your parallel programming skills. > Enter the Intel(R) Threading Challenge 2010. > http://p.sf.net/sfu/intel-thread-sfd > _______________________________________________ Monetdb-developers > mailing list > Monetdb-developers(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-developers -- Sjoerd Mullender ------------------------------------------------------------------------------ This SF.net Dev2Dev email is sponsored by: Show off your parallel programming skills. Enter the Intel(R) Threading Challenge 2010. http://p.sf.net/sfu/intel-thread-sfd _______________________________________________ Monetdb-developers mailing list Monetdb-developers(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers

1 0

[Monetdb-developers] Restart mclient
by Romulo Antonio Goncalves 07 Sep '10

07 Sep '10

Hello, I updated my default branch and I do the following: mclient -lmal >\q mclient -lmal >\q After the second exit I get a crash in the mserver: >[New Thread 0x7fffdf7ac710 (LWP 19441)] Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fffdf7ac710 (LWP 19441)] 0x00007ffff7cd7198 in MALexitClient (c=0x605b38) at /ufs/goncalve/MonetDB/current/MonetDB5/src/mal/mal_session.mx:522 522 if (c->glb && c->curprg->def->errors == 0) (gdb) p c->curprg->def $1 = (struct MALBLK *) 0xdbdbdbdbdbdbdbdb Similar problem if you do mclient -lsql >\q mclient -lsql The mserver crashes due to the same problem: Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fffdf7ac710 (LWP 19561)] 0x00007fffdfe22a60 in SQLparser (c=0x605b38) at /ufs/goncalve/MonetDB/current/sql/src/backends/monet5/sql_scenario.mx:1155 1155 oldvtop = c->curprg->def->vtop; (gdb) p c->curprg $5 = (Symbol) 0x7fffd8112198 (gdb) p *c->curprg $6 = {peer = 0xdbdbdbdbdbdbdbdb, skip = 0xdbdbdbdbdbdbdbdb, name = 0xdbdbdbdbdbdbdbdb <Address 0xdbdbdbdbdbdbdbdb out of bounds>, kind = -606348325, def = 0xdbdbdbdbdbdbdbdb} (gdb) Were there any changes in the client structure or in the initialization of the of the client threads? Regards, Romulo PS: It does not happen in the Oct2010 branch.

3 2

[Monetdb-developers] How to enable functions via sql?
by Alexandr Kulik 06 Sep '10

06 Sep '10

Hi, I need to develop some extra statistical functions for MonetDB. I spent about a week to understand how to use function via sql, but still didn't have any succes results. May be someone could point me to appropriate documentation or source revision, where i could look on it? -- With Best Regards Alexandr Kulik Skype: kulya_u Yahoo: kulya_u LinkedIn: http://www.linkedin.com/in/alexandrkulik Mobile: +3(806)615-40511

2 1