[MonetDB-users] Fatal error: aborted too deep recursion
During the development of our tool, using Xquery on MonetDB, we ran onto the following error: ERROR = !fatal error: aborted too deep recursion
From previous posts (I guess) I can conclude that this relates to an Xquery that simply is too big (too many operators) to create a good plan for it.
I tried several MonetDB releases (May2009-SP2, Aug2009-SP2, Nov2009-DB0) on Red Hat Enterprise Linux Server release 5.4 (Tikanga), all showing the same result. Does anyone have any directions on how to solve this? Can we reconfigure the compiler to fix this (or recompile it with options)? Thanks in advance! Harm van Beek Netherlands Forensic Institute
During the development of our tool, using Xquery on MonetDB, we ran onto the following error:
ERROR = !fatal error: aborted too deep recursion
From previous posts (I guess) I can conclude that this relates to an Xquery that simply is too big (too many operators) to create a good plan for it. All systems have limits that either cause an emergency stop. In this case,
Harm van Beek(DT) wrote: the operating system has such a limit.
I tried several MonetDB releases (May2009-SP2, Aug2009-SP2, Nov2009-DB0) on Red Hat Enterprise Linux Server release 5.4 (Tikanga), all showing the same result.
Does anyone have any directions on how to solve this?
Yes. Programming cleverness comes into play. Especially if you use a tool to generate queries, then that tool should be a little more conservative in the kind of queries generated.
Can we reconfigure the compiler to fix this (or recompile it with options)? No. At some point the optimization space becomes just too big. People that can spare the time to find the optimal settings for a larger application space are more then welcome to try it out on the source code made available.
regards, Martin
Thanks in advance!
Harm van Beek Netherlands Forensic Institute
------------------------------------------------------------------------------ Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
Hi Harm, The 'too deep recursion' error is typically triggered by very large queries that send the pathfinder compiler into a very deep (and time consuming) search. The error is given to prevent a crash due to lack of stack space. Recently, I do recall that the limits were tightened as there are some operating systems in our nightly test farm that give their processes little stack space, and could still cause pathfinder to crash due to lack of stack space. So, it depends. The limit when to break off search is arbitrary and based currently on the lowest common denominator among the systems we used. We could potentially increase it (and possibly fix the settings on our testing machines to allow more stack space). We could even introduce a user-controller limit. However, this phenomenon typically happens only on strange queries (e.fg doing tens or hundreds XPath steps after each other). Could you share what query produced such an error for you? Best, Peter Boncz
-----Original Message----- From: Martin Kersten [mailto:Martin.Kersten@cwi.nl] Sent: Thursday, November 26, 2009 4:28 PM To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
During the development of our tool, using Xquery on MonetDB, we ran onto the following error:
ERROR = !fatal error: aborted too deep recursion
From previous posts (I guess) I can conclude that this relates to an Xquery that simply is too big (too many operators) to create a good plan for it. All systems have limits that either cause an emergency stop. In this case,
Harm van Beek(DT) wrote: the operating system has such a limit.
I tried several MonetDB releases (May2009-SP2, Aug2009-SP2, Nov2009-DB0) on Red Hat Enterprise Linux Server release 5.4 (Tikanga), all showing the same result.
Does anyone have any directions on how to solve this?
Yes. Programming cleverness comes into play. Especially if you use a tool to generate queries, then that tool should be a little more conservative in the kind of queries generated.
Can we reconfigure the compiler to fix this (or recompile it with options)? No. At some point the optimization space becomes just too big. People that can spare the time to find the optimal settings for a larger application space are more then welcome to try it out on the source code made available.
regards, Martin
Thanks in advance!
Harm van Beek Netherlands Forensic Institute
-------------------------------------------------------------------------
-----
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30- Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
--------------------------------------------------------------------------- --- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.426 / Virus Database: 270.14.83/2526 - Release Date: 11/25/09 19:43:00
Hi Peter, We indeed execute a large query (which is constructed by progra mcode, so hard to change). You can find the query at the bottom of my reply. I understand that you introduced the limits to not run out of stack space. However, since we are in full control of the system on which the query is executed, we might be able to adapt the limits to our environment. Can you give us directions on where to set this? Thanks again, Harm -- query: element result { for $name in distinct-values( ( let $var1 := let $var2 := let $var3 := let $var4 := let $var5 := let $var6 := let $var7 := let $var8 := let $var9 := let $var10 := let $var11 := let $var12 := doc('document.xml')//properties/.. return $var12[ (self::registry) ] return $var11/ancestor-or-self::file[1]/properties/.. return $var10[ properties/(name)[ contains( lower-case(.), 'system' ) ] ] return $var9/child::registry/properties/.. return $var8/child::key/properties/.. return $var7/child::key/properties/.. return $var6/child::key/properties/.. return $var5[ properties/(name)[ . = 'Enum' ] ] return $var4/descendant-or-self::key/properties/.. return $var3[ properties/(name)[ contains( lower-case(.), 'usb' ) ] ] return $var2/descendant-or-self::value/properties/.. return $var1[ properties/(name)[ . = 'LocationInformation' ] ] )/properties/*/name() ) return element name { $name } }
-----Original Message----- From: Peter Boncz [mailto:P.Boncz@cwi.nl] Sent: Friday, November 27, 2009 10:26 AM To: 'Communication channel for MonetDB users' Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
Hi Harm,
The 'too deep recursion' error is typically triggered by very large queries that send the pathfinder compiler into a very deep (and time consuming) search. The error is given to prevent a crash due to lack of stack space. Recently, I do recall that the limits were tightened as there are some operating systems in our nightly test farm that give their processes little stack space, and could still cause pathfinder to crash due to lack of stack space.
So, it depends. The limit when to break off search is arbitrary and based currently on the lowest common denominator among the systems we used. We could potentially increase it (and possibly fix the settings on our testing machines to allow more stack space). We could even introduce a user-controller limit.
However, this phenomenon typically happens only on strange queries (e.fg doing tens or hundreds XPath steps after each other).
Could you share what query produced such an error for you?
Best,
Peter Boncz
-----Original Message----- From: Martin Kersten [mailto:Martin.Kersten@cwi.nl] Sent: Thursday, November 26, 2009 4:28 PM To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
During the development of our tool, using Xquery on MonetDB, we ran onto the following error:
ERROR = !fatal error: aborted too deep recursion
From previous posts (I guess) I can conclude that this relates to an Xquery that simply is too big (too many operators) to create a good plan for it. All systems have limits that either cause an emergency stop. In this case,
Harm van Beek(DT) wrote: the operating system has such a limit.
I tried several MonetDB releases (May2009-SP2,
Aug2009-SP2, Nov2009-DB0)
on Red Hat Enterprise Linux Server release 5.4 (Tikanga), all showing the same result.
Does anyone have any directions on how to solve this? Yes. Programming cleverness comes into play. Especially if you use a tool to generate queries, then that tool should be a little more conservative in the kind of queries generated.
Can we reconfigure the compiler to fix this (or recompile it with options)? No. At some point the optimization space becomes just too big. People that can spare the time to find the optimal settings for a larger application space are more then welcome to try it out on the source code made available.
regards, Martin
Thanks in advance!
Harm van Beek Netherlands Forensic Institute
----------
-----
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30- Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
--------------------------------------------------------------- ------------
--- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.426 / Virus Database: 270.14.83/2526 - Release Date: 11/25/09 19:43:00
--------------------------------------------------------------- --------------- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
Hi Harm, I had promised to get back to on this problem during the week; I did not quite make that, here I am trying to make what might be your definition of 'before the beginning of next week'. But apologies for the delay. You provided a test query on which you saw the 'recursion' error occurring. We know this error can occur on certain very large queries (in the test web there is a query with 100 XPath steps which triggers it). As there are limits to any program, we accept that this bug will happen in very extreme cases. However, the query you provided did not seem very intimidating. Therefore it may be a serious issue. However, your report so far has proven irreproducible for me. I tried it on the latest release with the mclient utility to no avail. I also tried with the 'pf' program standalone, as well as with in-server (using the query wrapped in a 'xquery()' MIL command). Are you sure this query triggers the error? Of course, I might go ahead and create an option for changing the recursion limit in the code. I held back from that because the risk is that instead of an error you might be getting a segmentation violation instead (after all, the fence is there for a reason). I could mitigate that risk by also giving the thread more stack space (does not work very easily on all architectures). Still, understanding the problem better will make the solution better, so getting this to reproduce would help. Thanks in advance, Peter BTW did NFI try the Nov2009 release already? It is the one with the pf:cache pragmas that will help performance quite a bit. I tested this with all 126 versions of MonetDB4 currently in Stefan's archive: $ for i in `locate bin/pf | egrep -e 'pf$'`; do echo $i; $i /tmp/t 2>&1| fgrep ecurs; done /export/scratch0/manegold/BenQ/2008-05-15.F8/sda1/home/manegold/monetdb-curr ent/program-i686/bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda1/usr/bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda1/var/www/cgi-bin/MonetDB/bi n/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda2/Win/home/manegold/Developm ent/Current/.GNU.32.32.d.--enable-debug.PREFIX./bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda2/Win/home/manegold/Developm ent/Update/.GNU.32.32.d.--enable-debug.PREFIX./bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda3/SuSE/FC5/home/manegold/mon etdb-current/program-i686/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperBall/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperBall-SP1/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperBall-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperBall-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperBall-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperBall-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperBall-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperBall/prefix.db g.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperBall/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperBall/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperBall-SP1/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperBall-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperBall-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperBall-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperBall-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperBall-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperBall-SP2/prefi x.opt.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperBall-SP2/prefi x.opt.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperBall-SP2/prefi x.opt.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperBall/prefix.db g.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperBall/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperBall/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperBall/prefix.op t.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperBall/prefix.op t.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperBall/prefix.op t.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperBall-SP1/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperBall-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperBall-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperBall-SP1/prefi x.opt.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperBall-SP1/prefi x.opt.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperBall-SP1/prefi x.opt.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperBall-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperBall-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperBall-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperBall-SP2/prefi x.opt.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperBall-SP2/prefi x.opt.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperBall-SP2/prefi x.opt.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperBall/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperBall/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperBall-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperBall-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperBall-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperBall-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperBall-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2009-SuperBall/prefix.db g.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2009-SuperBall/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2009-SuperBall/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix./bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enable-bits~32/bi n/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enable-bits~32_-- enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enable-oid32/bin/ pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enable-oid32_--en able-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enable-strict_--d isable-debug_--enable-optimize_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enable-strict_--d isable-debug_--enable-optimize_--disable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enable-strict_--d isable-debug_--enable-optimize_--disable-assert_CC~icc/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enable-strict_--d isable-debug_--enable-optimize_--disable-assert_CC~icc_--enable-probxml/bin/ pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enable-strict_--d isable-debug_--enable-optimize_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enable-strict_--d isable-debug_--enable-optimize_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enable-strict_--d isable-optimize_--disable-debug_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enable-strict_--d isable-optimize_--disable-debug_--disable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enable-strict_--d isable-optimize_--enable-debug_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enable-strict_--d isable-optimize_--enable-debug_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix._--enable-probxml/b in/pf /export/scratch0/manegold/Monet/Testing/Current/prefix./bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable-bits~32/bin/ pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable-bits~32.pf_M 5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable-bits~32_--en able-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable-oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable-oid32.pf_M5X Q/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable-oid32_--enab le-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable-strict_--dis able-debug_--enable-optimize_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable-strict_--dis able-debug_--enable-optimize_--disable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable-strict_--dis able-debug_--enable-optimize_--disable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable-strict_--dis able-debug_--enable-optimize_--disable-assert_CC~icc/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable-strict_--dis able-debug_--enable-optimize_--disable-assert_CC~icc.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable-strict_--dis able-debug_--enable-optimize_--disable-assert_CC~icc_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable-strict_--dis able-debug_--enable-optimize_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable-strict_--dis able-debug_--enable-optimize_--enable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable-strict_--dis able-debug_--enable-optimize_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable-strict_--dis able-optimize_--disable-debug_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable-strict_--dis able-optimize_--disable-debug_--disable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable-strict_--dis able-optimize_--disable-debug_--disable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable-strict_--dis able-optimize_--enable-debug_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable-strict_--dis able-optimize_--enable-debug_--enable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable-strict_--dis able-optimize_--enable-debug_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix..pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix._--enable-probxml/bin /pf /export/scratch0/manegold/Monet/Testing/Stable/prefix./bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable-bits~32/bin/p f /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable-bits~32.pf_NF I/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable-bits~32_--ena ble-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable-oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable-oid32.pf_NFI/ bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable-oid32_--enabl e-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable-strict_--disa ble-debug_--enable-optimize_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable-strict_--disa ble-debug_--enable-optimize_--disable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable-strict_--disa ble-debug_--enable-optimize_--disable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable-strict_--disa ble-debug_--enable-optimize_--disable-assert_CC~icc/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable-strict_--disa ble-debug_--enable-optimize_--disable-assert_CC~icc.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable-strict_--disa ble-debug_--enable-optimize_--disable-assert_CC~icc_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable-strict_--disa ble-debug_--enable-optimize_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable-strict_--disa ble-debug_--enable-optimize_--enable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable-strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-bits~32/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable-strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-bits~32_--enable-probxm l/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable-strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable-strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-oid32_--enable-probxml/ bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable-strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable-strict_--disa ble-optimize_--disable-debug_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable-strict_--disa ble-optimize_--disable-debug_--disable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable-strict_--disa ble-optimize_--disable-debug_--disable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable-strict_--disa ble-optimize_--enable-debug_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable-strict_--disa ble-optimize_--enable-debug_--enable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable-strict_--disa ble-optimize_--enable-debug_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix..pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix._--enable-probxml/bin/ pf /export/scratch0/manegold/MonetDB_4-12/.GNU.64.64.d..PREFIX./PATHFINDER/bin/ pf /export/scratch0/manegold/MonetDB_Release_Build/prefix/bin/pf /export/scratch0/manegold/tmp/BenQ_FC6/home/manegold/monetdb-current/program -i686/bin/pf /export/scratch0/manegold/tmp/BenQ_FC6/var/www/cgi-bin/MonetDB/bin/pf /usr/bin/pf -----Original Message----- From: Harm van Beek(DT) [mailto:harm@holmes.nl] Sent: vrijdag 27 november 2009 14:49 To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion Hi Peter, We indeed execute a large query (which is constructed by progra mcode, so hard to change). You can find the query at the bottom of my reply. I understand that you introduced the limits to not run out of stack space. However, since we are in full control of the system on which the query is executed, we might be able to adapt the limits to our environment. Can you give us directions on where to set this? Thanks again, Harm -- query: element result { for $name in distinct-values( ( let $var1 := let $var2 := let $var3 := let $var4 := let $var5 := let $var6 := let $var7 := let $var8 := let $var9 := let $var10 := let $var11 := let $var12 := doc('document.xml')//properties/.. return $var12[ (self::registry) ] return $var11/ancestor-or-self::file[1]/properties/.. return $var10[ properties/(name)[ contains( lower-case(.), 'system' ) ] ] return $var9/child::registry/properties/.. return $var8/child::key/properties/.. return $var7/child::key/properties/.. return $var6/child::key/properties/.. return $var5[ properties/(name)[ . = 'Enum' ] ] return $var4/descendant-or-self::key/properties/.. return $var3[ properties/(name)[ contains( lower-case(.), 'usb' ) ] ] return $var2/descendant-or-self::value/properties/.. return $var1[ properties/(name)[ . = 'LocationInformation' ] ] )/properties/*/name() ) return element name { $name } }
-----Original Message----- From: Peter Boncz [mailto:P.Boncz@cwi.nl] Sent: Friday, November 27, 2009 10:26 AM To: 'Communication channel for MonetDB users' Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
Hi Harm,
The 'too deep recursion' error is typically triggered by very large queries that send the pathfinder compiler into a very deep (and time consuming) search. The error is given to prevent a crash due to lack of stack space. Recently, I do recall that the limits were tightened as there are some operating systems in our nightly test farm that give their processes little stack space, and could still cause pathfinder to crash due to lack of stack space.
So, it depends. The limit when to break off search is arbitrary and based currently on the lowest common denominator among the systems we used. We could potentially increase it (and possibly fix the settings on our testing machines to allow more stack space). We could even introduce a user-controller limit.
However, this phenomenon typically happens only on strange queries (e.fg doing tens or hundreds XPath steps after each other).
Could you share what query produced such an error for you?
Best,
Peter Boncz
-----Original Message----- From: Martin Kersten [mailto:Martin.Kersten@cwi.nl] Sent: Thursday, November 26, 2009 4:28 PM To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
During the development of our tool, using Xquery on MonetDB, we ran onto the following error:
ERROR = !fatal error: aborted too deep recursion
From previous posts (I guess) I can conclude that this relates to an Xquery that simply is too big (too many operators) to create a good plan for it. All systems have limits that either cause an emergency stop. In this case,
Harm van Beek(DT) wrote: the operating system has such a limit.
I tried several MonetDB releases (May2009-SP2,
Aug2009-SP2, Nov2009-DB0)
on Red Hat Enterprise Linux Server release 5.4 (Tikanga), all showing the same result.
Does anyone have any directions on how to solve this? Yes. Programming cleverness comes into play. Especially if you use a tool to generate queries, then that tool should be a little more conservative in the kind of queries generated.
Can we reconfigure the compiler to fix this (or recompile it with options)? No. At some point the optimization space becomes just too big. People that can spare the time to find the optimal settings for a larger application space are more then welcome to try it out on the source code made available.
regards, Martin
Thanks in advance!
Harm van Beek Netherlands Forensic Institute
----------
-----
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30- Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
--------------------------------------------------------------- ------------
--- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.426 / Virus Database: 270.14.83/2526 - Release Date: 11/25/09 19:43:00
--------------------------------------------------------------- --------------- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
---------------------------------------------------------------------------- -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.426 / Virus Database: 270.14.83/2526 - Release Date: 11/26/09 19:42:00
Dear Peter, Thanks for your reply! I double-checked; the query sent earlier DOES run into the "too deep recursion" problem. One major difference with respect to earlier development is that we currently are using the RedHat Enterprise Linux (Tikanga) operation system on our servers. We know that this might cause the problem, however, as far as we can see the user limits match (or outnumber) the limits on the Fedora systems. If you can point us where/how to check which limits are used in the calculation used for limiting recursion depth, we might be able to adapt the settings. We installed the November release in our development environment and will soon start testing with it. Kind regards, Harm
-----Original Message----- From: Peter Boncz [mailto:P.Boncz@cwi.nl] Sent: Monday, December 14, 2009 12:25 AM To: Harm van Beek (DT) Cc: Communication channel for MonetDB users Subject: RE: [MonetDB-users] Fatal error: aborted too deep recursion
Hi Harm,
I had promised to get back to on this problem during the week; I did not quite make that, here I am trying to make what might be your definition of 'before the beginning of next week'. But apologies for the delay.
You provided a test query on which you saw the 'recursion' error occurring. We know this error can occur on certain very large queries (in the test web there is a query with 100 XPath steps which triggers it). As there are limits to any program, we accept that this bug will happen in very extreme cases. However, the query you provided did not seem very intimidating. Therefore it may be a serious issue.
However, your report so far has proven irreproducible for me.
I tried it on the latest release with the mclient utility to no avail. I also tried with the 'pf' program standalone, as well as with in-server (using the query wrapped in a 'xquery()' MIL command).
Are you sure this query triggers the error? Of course, I might go ahead and create an option for changing the recursion limit in the code. I held back from that because the risk is that instead of an error you might be getting a segmentation violation instead (after all, the fence is there for a reason). I could mitigate that risk by also giving the thread more stack space (does not work very easily on all architectures). Still, understanding the problem better will make the solution better, so getting this to reproduce would help.
Thanks in advance,
Peter
BTW did NFI try the Nov2009 release already? It is the one with the pf:cache pragmas that will help performance quite a bit.
I tested this with all 126 versions of MonetDB4 currently in Stefan's archive:
$ for i in `locate bin/pf | egrep -e 'pf$'`; do echo $i; $i /tmp/t 2>&1| fgrep ecurs; done /export/scratch0/manegold/BenQ/2008-05-15.F8/sda1/home/manegold /monetdb-curr ent/program-i686/bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda1/usr/bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda1/var/www/cgi-b in/MonetDB/bi n/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda2/Win/home/mane gold/Developm ent/Current/.GNU.32.32.d.--enable-debug.PREFIX./bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda2/Win/home/mane gold/Developm ent/Update/.GNU.32.32.d.--enable-debug.PREFIX./bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda3/SuSE/FC5/home /manegold/mon etdb-current/program-i686/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP1/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all/prefix.db g.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP1/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.opt.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.opt.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.opt.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.db g.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.op t.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.op t.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.op t.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.opt.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.opt.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.opt.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.opt.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.opt.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.opt.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2009-SuperB all/prefix.db g.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2009-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix./bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-bits~32/bi n/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-bits~32_-- enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-oid32/bin/ pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-oid32_--en able-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--disable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--disable-assert_CC~icc/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--disable-assert_CC~icc_--enable -probxml/bin/ pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-optimize_--disable-debug_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-optimize_--disable-debug_--disable-assert_--enable-probx ml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-optimize_--enable-debug_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-optimize_--enable-debug_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix._--ena ble-probxml/b in/pf /export/scratch0/manegold/Monet/Testing/Current/prefix./bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -bits~32/bin/ pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -bits~32.pf_M 5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -bits~32_--en able-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -oid32.pf_M5X Q/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -oid32_--enab le-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert_CC~icc/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert_CC~icc.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert_CC~icc_--enable-p robxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--enable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--disable-debug_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--disable-debug_--disable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--disable-debug_--disable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--enable-debug_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--enable-debug_--enable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--enable-debug_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix..pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix._--enabl e-probxml/bin /pf /export/scratch0/manegold/Monet/Testing/Stable/prefix./bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- bits~32/bin/p f /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- bits~32.pf_NF I/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- bits~32_--ena ble-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- oid32.pf_NFI/ bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- oid32_--enabl e-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_CC~icc/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_CC~icc.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_CC~icc_--enable-pr obxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-bits~32/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-bits~32_-- enable-probxm l/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-oid32_--en able-probxml/ bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--disable-debug_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--disable-debug_--disable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--disable-debug_--disable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--enable-debug_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--enable-debug_--enable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--enable-debug_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix..pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix._--enable -probxml/bin/ pf /export/scratch0/manegold/MonetDB_4-12/.GNU.64.64.d..PREFIX./PA THFINDER/bin/ pf /export/scratch0/manegold/MonetDB_Release_Build/prefix/bin/pf /export/scratch0/manegold/tmp/BenQ_FC6/home/manegold/monetdb-cu rrent/program -i686/bin/pf /export/scratch0/manegold/tmp/BenQ_FC6/var/www/cgi-bin/MonetDB/bin/pf /usr/bin/pf
-----Original Message----- From: Harm van Beek(DT) [mailto:harm@holmes.nl] Sent: vrijdag 27 november 2009 14:49 To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
Hi Peter,
We indeed execute a large query (which is constructed by progra mcode, so hard to change). You can find the query at the bottom of my reply.
I understand that you introduced the limits to not run out of stack space. However, since we are in full control of the system on which the query is executed, we might be able to adapt the limits to our environment. Can you give us directions on where to set this?
Thanks again,
Harm
-- query:
element result { for $name in distinct-values( ( let $var1 := let $var2 := let $var3 := let $var4 := let $var5 := let $var6 := let $var7 := let $var8 := let $var9 := let $var10 := let $var11 := let $var12 := doc('document.xml')//properties/.. return $var12[ (self::registry) ] return $var11/ancestor-or-self::file[1]/properties/.. return $var10[ properties/(name)[ contains( lower-case(.), 'system' ) ] ] return $var9/child::registry/properties/.. return $var8/child::key/properties/.. return $var7/child::key/properties/.. return $var6/child::key/properties/.. return $var5[ properties/(name)[ . = 'Enum' ] ] return $var4/descendant-or-self::key/properties/.. return $var3[ properties/(name)[ contains( lower-case(.), 'usb' ) ] ] return $var2/descendant-or-self::value/properties/.. return $var1[ properties/(name)[ . = 'LocationInformation' ] ] )/properties/*/name() ) return element name { $name } }
-----Original Message----- From: Peter Boncz [mailto:P.Boncz@cwi.nl] Sent: Friday, November 27, 2009 10:26 AM To: 'Communication channel for MonetDB users' Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
Hi Harm,
The 'too deep recursion' error is typically triggered by very large queries that send the pathfinder compiler into a very deep (and time consuming) search. The error is given to prevent a crash due to lack of stack space. Recently, I do recall that the limits were tightened as there are some operating systems in our nightly test farm that give their processes little stack space, and could still cause pathfinder to crash due to lack of stack space.
So, it depends. The limit when to break off search is arbitrary and based currently on the lowest common denominator among the systems we used. We could potentially increase it (and possibly fix the settings on our testing machines to allow more stack space). We could even introduce a user-controller limit.
However, this phenomenon typically happens only on strange queries (e.fg doing tens or hundreds XPath steps after each other).
Could you share what query produced such an error for you?
Best,
Peter Boncz
-----Original Message----- From: Martin Kersten [mailto:Martin.Kersten@cwi.nl] Sent: Thursday, November 26, 2009 4:28 PM To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
During the development of our tool, using Xquery on MonetDB, we ran onto the following error:
ERROR = !fatal error: aborted too deep recursion
From previous posts (I guess) I can conclude that this relates to an Xquery that simply is too big (too many operators) to create a good plan for it. All systems have limits that either cause an emergency stop. In this case,
Harm van Beek(DT) wrote: the operating system has such a limit.
I tried several MonetDB releases (May2009-SP2,
Aug2009-SP2, Nov2009-DB0)
on Red Hat Enterprise Linux Server release 5.4 (Tikanga), all showing the same result.
Does anyone have any directions on how to solve this? Yes. Programming cleverness comes into play. Especially if you use a tool to generate queries, then that tool should be a little more conservative in the kind of queries generated.
Can we reconfigure the compiler to fix this (or recompile it with options)? No. At some point the optimization space becomes just too big. People that can spare the time to find the optimal settings for a larger application space are more then welcome to try it out on the source code made available.
regards, Martin
Thanks in advance!
Harm van Beek Netherlands Forensic Institute
----------
-----
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30- Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
--------------------------------------------------------------- ------------
--- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.426 / Virus Database: 270.14.83/2526 - Release Date: 11/25/09 19:43:00
--------------------------------------------------------------- --------------- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
--------------------------------------------------------------- ------------- -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.426 / Virus Database: 270.14.83/2526 - Release Date: 11/26/09 19:42:00
Beste Harm, you are running on Red Hat Enterprise Linux Server release 5.4, right? Is your OS and hardware 64-bit or 32-bit? Did you install pre-built binaries (RPMs) or did you compile MonetDB yourself? If the former, which RPMs did you get from where? (Official release or nightly build?) If the latter, did you compile from CVS sources or from a (released or nightly built?) super source ball, which configure setting did you use, and which compiler (incl. version) did you use? Like Peter, I haven't yet managed to reproduce your problem with the given query. The aove information might help us to test in a setup that is as close to yours as possible ... Stefan On Mon, Dec 14, 2009 at 09:09:51AM +0100, Harm van Beek (DT) wrote:
Dear Peter,
Thanks for your reply!
I double-checked; the query sent earlier DOES run into the "too deep recursion" problem. One major difference with respect to earlier development is that we currently are using the RedHat Enterprise Linux (Tikanga) operation system on our servers. We know that this might cause the problem, however, as far as we can see the user limits match (or outnumber) the limits on the Fedora systems.
If you can point us where/how to check which limits are used in the calculation used for limiting recursion depth, we might be able to adapt the settings.
We installed the November release in our development environment and will soon start testing with it.
Kind regards,
Harm
-----Original Message----- From: Peter Boncz [mailto:P.Boncz@cwi.nl] Sent: Monday, December 14, 2009 12:25 AM To: Harm van Beek (DT) Cc: Communication channel for MonetDB users Subject: RE: [MonetDB-users] Fatal error: aborted too deep recursion
Hi Harm,
I had promised to get back to on this problem during the week; I did not quite make that, here I am trying to make what might be your definition of 'before the beginning of next week'. But apologies for the delay.
You provided a test query on which you saw the 'recursion' error occurring. We know this error can occur on certain very large queries (in the test web there is a query with 100 XPath steps which triggers it). As there are limits to any program, we accept that this bug will happen in very extreme cases. However, the query you provided did not seem very intimidating. Therefore it may be a serious issue.
However, your report so far has proven irreproducible for me.
I tried it on the latest release with the mclient utility to no avail. I also tried with the 'pf' program standalone, as well as with in-server (using the query wrapped in a 'xquery()' MIL command).
Are you sure this query triggers the error? Of course, I might go ahead and create an option for changing the recursion limit in the code. I held back from that because the risk is that instead of an error you might be getting a segmentation violation instead (after all, the fence is there for a reason). I could mitigate that risk by also giving the thread more stack space (does not work very easily on all architectures). Still, understanding the problem better will make the solution better, so getting this to reproduce would help.
Thanks in advance,
Peter
BTW did NFI try the Nov2009 release already? It is the one with the pf:cache pragmas that will help performance quite a bit.
I tested this with all 126 versions of MonetDB4 currently in Stefan's archive:
$ for i in `locate bin/pf | egrep -e 'pf$'`; do echo $i; $i /tmp/t 2>&1| fgrep ecurs; done /export/scratch0/manegold/BenQ/2008-05-15.F8/sda1/home/manegold /monetdb-curr ent/program-i686/bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda1/usr/bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda1/var/www/cgi-b in/MonetDB/bi n/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda2/Win/home/mane gold/Developm ent/Current/.GNU.32.32.d.--enable-debug.PREFIX./bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda2/Win/home/mane gold/Developm ent/Update/.GNU.32.32.d.--enable-debug.PREFIX./bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda3/SuSE/FC5/home /manegold/mon etdb-current/program-i686/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP1/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all/prefix.db g.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP1/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.opt.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.opt.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.opt.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.db g.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.op t.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.op t.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.op t.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.opt.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.opt.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.opt.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.opt.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.opt.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.opt.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2009-SuperB all/prefix.db g.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2009-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix./bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-bits~32/bi n/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-bits~32_-- enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-oid32/bin/ pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-oid32_--en able-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--disable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--disable-assert_CC~icc/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--disable-assert_CC~icc_--enable -probxml/bin/ pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-optimize_--disable-debug_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-optimize_--disable-debug_--disable-assert_--enable-probx ml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-optimize_--enable-debug_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-optimize_--enable-debug_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix._--ena ble-probxml/b in/pf /export/scratch0/manegold/Monet/Testing/Current/prefix./bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -bits~32/bin/ pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -bits~32.pf_M 5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -bits~32_--en able-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -oid32.pf_M5X Q/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -oid32_--enab le-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert_CC~icc/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert_CC~icc.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert_CC~icc_--enable-p robxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--enable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--disable-debug_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--disable-debug_--disable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--disable-debug_--disable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--enable-debug_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--enable-debug_--enable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--enable-debug_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix..pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix._--enabl e-probxml/bin /pf /export/scratch0/manegold/Monet/Testing/Stable/prefix./bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- bits~32/bin/p f /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- bits~32.pf_NF I/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- bits~32_--ena ble-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- oid32.pf_NFI/ bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- oid32_--enabl e-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_CC~icc/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_CC~icc.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_CC~icc_--enable-pr obxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-bits~32/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-bits~32_-- enable-probxm l/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-oid32_--en able-probxml/ bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--disable-debug_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--disable-debug_--disable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--disable-debug_--disable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--enable-debug_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--enable-debug_--enable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--enable-debug_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix..pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix._--enable -probxml/bin/ pf /export/scratch0/manegold/MonetDB_4-12/.GNU.64.64.d..PREFIX./PA THFINDER/bin/ pf /export/scratch0/manegold/MonetDB_Release_Build/prefix/bin/pf /export/scratch0/manegold/tmp/BenQ_FC6/home/manegold/monetdb-cu rrent/program -i686/bin/pf /export/scratch0/manegold/tmp/BenQ_FC6/var/www/cgi-bin/MonetDB/bin/pf /usr/bin/pf
-----Original Message----- From: Harm van Beek(DT) [mailto:harm@holmes.nl] Sent: vrijdag 27 november 2009 14:49 To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
Hi Peter,
We indeed execute a large query (which is constructed by progra mcode, so hard to change). You can find the query at the bottom of my reply.
I understand that you introduced the limits to not run out of stack space. However, since we are in full control of the system on which the query is executed, we might be able to adapt the limits to our environment. Can you give us directions on where to set this?
Thanks again,
Harm
-- query:
element result { for $name in distinct-values( ( let $var1 := let $var2 := let $var3 := let $var4 := let $var5 := let $var6 := let $var7 := let $var8 := let $var9 := let $var10 := let $var11 := let $var12 := doc('document.xml')//properties/.. return $var12[ (self::registry) ] return $var11/ancestor-or-self::file[1]/properties/.. return $var10[ properties/(name)[ contains( lower-case(.), 'system' ) ] ] return $var9/child::registry/properties/.. return $var8/child::key/properties/.. return $var7/child::key/properties/.. return $var6/child::key/properties/.. return $var5[ properties/(name)[ . = 'Enum' ] ] return $var4/descendant-or-self::key/properties/.. return $var3[ properties/(name)[ contains( lower-case(.), 'usb' ) ] ] return $var2/descendant-or-self::value/properties/.. return $var1[ properties/(name)[ . = 'LocationInformation' ] ] )/properties/*/name() ) return element name { $name } }
-----Original Message----- From: Peter Boncz [mailto:P.Boncz@cwi.nl] Sent: Friday, November 27, 2009 10:26 AM To: 'Communication channel for MonetDB users' Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
Hi Harm,
The 'too deep recursion' error is typically triggered by very large queries that send the pathfinder compiler into a very deep (and time consuming) search. The error is given to prevent a crash due to lack of stack space. Recently, I do recall that the limits were tightened as there are some operating systems in our nightly test farm that give their processes little stack space, and could still cause pathfinder to crash due to lack of stack space.
So, it depends. The limit when to break off search is arbitrary and based currently on the lowest common denominator among the systems we used. We could potentially increase it (and possibly fix the settings on our testing machines to allow more stack space). We could even introduce a user-controller limit.
However, this phenomenon typically happens only on strange queries (e.fg doing tens or hundreds XPath steps after each other).
Could you share what query produced such an error for you?
Best,
Peter Boncz
-----Original Message----- From: Martin Kersten [mailto:Martin.Kersten@cwi.nl] Sent: Thursday, November 26, 2009 4:28 PM To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
During the development of our tool, using Xquery on MonetDB, we ran onto the following error:
ERROR = !fatal error: aborted too deep recursion
From previous posts (I guess) I can conclude that this relates to an Xquery that simply is too big (too many operators) to create a good plan for it. All systems have limits that either cause an emergency stop. In this case,
Harm van Beek(DT) wrote: the operating system has such a limit.
I tried several MonetDB releases (May2009-SP2,
Aug2009-SP2, Nov2009-DB0)
on Red Hat Enterprise Linux Server release 5.4 (Tikanga), all showing the same result.
Does anyone have any directions on how to solve this? Yes. Programming cleverness comes into play. Especially if you use a tool to generate queries, then that tool should be a little more conservative in the kind of queries generated.
Can we reconfigure the compiler to fix this (or recompile it with options)? No. At some point the optimization space becomes just too big. People that can spare the time to find the optimal settings for a larger application space are more then welcome to try it out on the source code made available.
regards, Martin
Thanks in advance!
Harm van Beek Netherlands Forensic Institute
----------
-----
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30- Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
--------------------------------------------------------------- ------------
--- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.426 / Virus Database: 270.14.83/2526 - Release Date: 11/25/09 19:43:00
--------------------------------------------------------------- --------------- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
--------------------------------------------------------------- ------------- -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.426 / Virus Database: 270.14.83/2526 - Release Date: 11/26/09 19:42:00
------------------------------------------------------------------------------ Return on Information: Google Enterprise Search pays you back Get the facts. http://p.sf.net/sfu/google-dev2dev _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
-- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |
Hi Stefan, See my answers below.
-----Original Message----- From: Stefan Manegold [mailto:Stefan.Manegold@cwi.nl] Sent: Monday, December 14, 2009 12:43 PM To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
Beste Harm,
you are running on Red Hat Enterprise Linux Server release 5.4, right? Correct.
Is your OS and hardware 64-bit or 32-bit?
64 bit.
Did you install pre-built binaries (RPMs) or did you compile MonetDB yourself?
We compiled MonetDB from the superball.
If the former, which RPMs did you get from where? (Official release or nightly build?) If the latter, did you compile from CVS sources or from a (released or nightly built?) super source ball, which configure setting did you use, and which compiler (incl. version) did you use?
We compiled the Super Source Tarball, http://monetdb.cwi.nl/downloads/sources/Nov2009/MonetDB-Nov2009-SuperBal l.tar.bz2. We ran monetdb-install.sh with --prefix and --enable-xquery options only. We used gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-46) for compilation.
Like Peter, I haven't yet managed to reproduce your problem with the given query. The aove information might help us to test in a setup that is as close to yours as possible ...
We also cannot reproduce the problem on our Fedora machine. Hopefully, the answers above help you. If you need more info, please let me know! Thanks, Harm
Stefan
Dear Peter,
Thanks for your reply!
I double-checked; the query sent earlier DOES run into the "too deep recursion" problem. One major difference with respect to earlier development is that we currently are using the RedHat Enterprise Linux (Tikanga) operation system on our servers. We know that this might cause the problem, however, as far as we can see the user limits match (or outnumber) the limits on the Fedora systems.
If you can point us where/how to check which limits are used in the calculation used for limiting recursion depth, we might be able to adapt the settings.
We installed the November release in our development environment and will soon start testing with it.
Kind regards,
Harm
-----Original Message----- From: Peter Boncz [mailto:P.Boncz@cwi.nl] Sent: Monday, December 14, 2009 12:25 AM To: Harm van Beek (DT) Cc: Communication channel for MonetDB users Subject: RE: [MonetDB-users] Fatal error: aborted too deep recursion
Hi Harm,
I had promised to get back to on this problem during the week; I did not quite make that, here I am trying to make what might be your definition of 'before the beginning of next week'. But apologies for the delay.
You provided a test query on which you saw the 'recursion' error occurring. We know this error can occur on certain very large queries (in the test web there is a query with 100 XPath steps which triggers it). As there are limits to any program, we accept that this bug will happen in very extreme cases. However, the query you provided did not seem very intimidating. Therefore it may be a serious issue.
However, your report so far has proven irreproducible for me.
I tried it on the latest release with the mclient utility to no avail. I also tried with the 'pf' program standalone, as well as with in-server (using the query wrapped in a 'xquery()' MIL command).
Are you sure this query triggers the error? Of course, I might go ahead and create an option for changing the recursion limit in the code. I held back from that because the risk is that instead of an error you might be getting a segmentation violation instead (after all, the fence is
reason). I could mitigate that risk by also giving the thread more stack space (does not work very easily on all architectures). Still, understanding the problem better will make the solution better, so getting this to reproduce would help.
Thanks in advance,
Peter
BTW did NFI try the Nov2009 release already? It is the one with the pf:cache pragmas that will help performance quite a bit.
I tested this with all 126 versions of MonetDB4 currently in Stefan's archive:
$ for i in `locate bin/pf | egrep -e 'pf$'`; do echo $i; $i /tmp/t 2>&1| fgrep ecurs; done /export/scratch0/manegold/BenQ/2008-05-15.F8/sda1/home/manegold /monetdb-curr ent/program-i686/bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda1/usr/bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda1/var/www/cgi-b in/MonetDB/bi n/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda2/Win/home/mane gold/Developm ent/Current/.GNU.32.32.d.--enable-debug.PREFIX./bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda2/Win/home/mane gold/Developm ent/Update/.GNU.32.32.d.--enable-debug.PREFIX./bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda3/SuSE/FC5/home /manegold/mon etdb-current/program-i686/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP1/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all/prefix.db g.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP1/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.opt.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.opt.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.opt.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.db g.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.op t.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.op t.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.op t.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.opt.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.opt.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.opt.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.opt.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.opt.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.opt.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2009-SuperB all/prefix.db g.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2009-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix./bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-bits~32/bi n/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-bits~32_-- enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-oid32/bin/ pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-oid32_--en able-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d
isable-debug_--enable-optimize_--disable-assert_--enable-probx ml/bin/pf
/export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--disable-assert_CC~icc/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--disable-assert_CC~icc_--enable -probxml/bin/ pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d
isable-debug_--enable-optimize_--enable-assert_--enable-probxml/bin/pf
/export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-optimize_--disable-debug_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-optimize_--disable-debug_--disable-assert_--enable-probx ml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-optimize_--enable-debug_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d
isable-optimize_--enable-debug_--enable-assert_--enable-probxml/bin/pf
/export/scratch0/manegold/Monet/Testing/Candidate/prefix._--ena ble-probxml/b in/pf /export/scratch0/manegold/Monet/Testing/Current/prefix./bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -bits~32/bin/ pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -bits~32.pf_M 5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -bits~32_--en able-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -oid32.pf_M5X Q/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -oid32_--enab le-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis
able-debug_--enable-optimize_--disable-assert_--enable-probxml/bin/pf
/export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert_CC~icc/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert_CC~icc.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert_CC~icc_--enable-p robxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--enable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--disable-debug_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--disable-debug_--disable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis
able-optimize_--disable-debug_--disable-assert_--enable-probxml/bin/pf
/export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--enable-debug_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--enable-debug_--enable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--enable-debug_--enable-assert_--enable-probxml/bin/pf
/export/scratch0/manegold/Monet/Testing/Current/prefix..pf_M5XQ/bin/pf
/export/scratch0/manegold/Monet/Testing/Current/prefix._--enabl e-probxml/bin /pf /export/scratch0/manegold/Monet/Testing/Stable/prefix./bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- bits~32/bin/p f /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- bits~32.pf_NF I/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- bits~32_--ena ble-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- oid32.pf_NFI/ bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- oid32_--enabl e-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_CC~icc/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_CC~icc.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_CC~icc_--enable-pr obxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-bits~32/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-bits~32_-- enable-probxm l/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-oid32_--en able-probxml/ bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--disable-debug_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--disable-debug_--disable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa
ble-optimize_--disable-debug_--disable-assert_--enable-probxml/bin/pf
/export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--enable-debug_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--enable-debug_--enable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--enable-debug_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix..pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix._--enable -probxml/bin/ pf /export/scratch0/manegold/MonetDB_4-12/.GNU.64.64.d..PREFIX./PA THFINDER/bin/ pf /export/scratch0/manegold/MonetDB_Release_Build/prefix/bin/pf /export/scratch0/manegold/tmp/BenQ_FC6/home/manegold/monetdb-cu rrent/program -i686/bin/pf
/export/scratch0/manegold/tmp/BenQ_FC6/var/www/cgi-bin/MonetDB/bin/pf
/usr/bin/pf
-----Original Message----- From: Harm van Beek(DT) [mailto:harm@holmes.nl] Sent: vrijdag 27 november 2009 14:49 To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
Hi Peter,
We indeed execute a large query (which is constructed by
On Mon, Dec 14, 2009 at 09:09:51AM +0100, Harm van Beek (DT) wrote: there for a progra mcode,
so hard to change). You can find the query at the bottom of my reply.
I understand that you introduced the limits to not run out of stack space. However, since we are in full control of the system on which the query is executed, we might be able to adapt the limits to our environment. Can you give us directions on where to set this?
Thanks again,
Harm
-- query:
element result { for $name in distinct-values( ( let $var1 := let $var2 := let $var3 := let $var4 := let $var5 := let $var6 := let $var7 := let $var8 := let $var9 := let $var10 := let $var11 := let $var12 := doc('document.xml')//properties/.. return $var12[ (self::registry) ] return $var11/ancestor-or-self::file[1]/properties/.. return $var10[ properties/(name)[ contains( lower-case(.), 'system' ) ] ] return $var9/child::registry/properties/.. return $var8/child::key/properties/.. return $var7/child::key/properties/.. return $var6/child::key/properties/.. return $var5[ properties/(name)[ . = 'Enum' ] ] return $var4/descendant-or-self::key/properties/.. return $var3[ properties/(name)[ contains( lower-case(.), 'usb' ) ] ] return $var2/descendant-or-self::value/properties/.. return $var1[ properties/(name)[ . = 'LocationInformation' ] ] )/properties/*/name() ) return element name { $name } }
-----Original Message----- From: Peter Boncz [mailto:P.Boncz@cwi.nl] Sent: Friday, November 27, 2009 10:26 AM To: 'Communication channel for MonetDB users' Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
Hi Harm,
The 'too deep recursion' error is typically triggered by very large queries that send the pathfinder compiler into a very deep (and time consuming) search. The error is given to prevent a crash due to lack of stack space. Recently, I do recall that the limits were tightened as there are some operating systems in our nightly test farm that give their processes little stack space, and could still cause pathfinder to crash due to lack of stack space.
So, it depends. The limit when to break off search is arbitrary and based currently on the lowest common denominator among the systems we used. We could potentially increase it (and possibly fix the settings on our testing machines to allow more stack space). We could even introduce a user-controller limit.
However, this phenomenon typically happens only on strange queries (e.fg doing tens or hundreds XPath steps after each other).
Could you share what query produced such an error for you?
Best,
Peter Boncz
-----Original Message----- From: Martin Kersten [mailto:Martin.Kersten@cwi.nl] Sent: Thursday, November 26, 2009 4:28 PM To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
During the development of our tool, using Xquery on MonetDB, we ran onto the following error:
ERROR = !fatal error: aborted too deep recursion
>From previous posts (I guess) I can conclude that this relates to an Xquery that simply is too big (too many operators) to create a good plan for it. All systems have limits that either cause an emergency stop. In this case,
Harm van Beek(DT) wrote: the operating system has such a limit.
I tried several MonetDB releases (May2009-SP2,
Aug2009-SP2, Nov2009-DB0)
on Red Hat Enterprise Linux Server release 5.4 (Tikanga), all showing the same result.
Does anyone have any directions on how to solve this? Yes. Programming cleverness comes into play. Especially if you use a tool to generate queries, then that tool should be a little more conservative in the kind of queries generated.
Can we reconfigure the compiler to fix this (or recompile it with options)? No. At some point the optimization space becomes just too big. People that can spare the time to find the optimal settings for a larger application space are more then welcome to try it out on the source code made available.
regards, Martin
Thanks in advance!
Harm van Beek Netherlands Forensic Institute
----------
-----
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30- Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
--------------------------------------------------------------- ------------
--- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.426 / Virus Database: 270.14.83/2526 - Release Date: 11/25/09 19:43:00
--------------------------------------------------------------- --------------- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
--------------------------------------------------------------- ------------- -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.426 / Virus Database: 270.14.83/2526 - Release Date: 11/26/09 19:42:00
--------------------------------------------------------------- ---------------
Return on Information: Google Enterprise Search pays you back Get the facts. http://p.sf.net/sfu/google-dev2dev _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
-- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |
--------------------------------------------------------------- --------------- Return on Information: Google Enterprise Search pays you back Get the facts. http://p.sf.net/sfu/google-dev2dev _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
Harm, different platforms appear to have different demands for stack space with recursive function calls. For instance, we experienced the Intel compiler, in particular on Windows, requires much more stack space than. e.g., the GNU compiler on Fedora. Hence, I guess, that also on your Red Hat Enterprise Linux Server machine with the gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-46), the stack demands are higher than on your and our Fedora machines ... We do no longer have (access to) any Red Hat Enterprise Linux Server machine; hence, we cannot test and investigate the problem on that platform ourselves. However, I just added your query (using an empty document) as test to our testing system. Hence, as of tomorrow morning, we can check under http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/... http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/... how your query performs on our various testing platforms (cf., http://monetdb.cwi.nl/Development/TestWeb/Platforms/index.html; note, the "RedHat EL WS 4" listed has recently died; we haven't yet updated the web site accordingly). Hopefully, this will give us some more hints. Regards, Stefan On Mon, Dec 14, 2009 at 01:12:19PM +0100, Harm van Beek (DT) wrote:
Hi Stefan,
See my answers below.
-----Original Message----- From: Stefan Manegold [mailto:Stefan.Manegold@cwi.nl] Sent: Monday, December 14, 2009 12:43 PM To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
Beste Harm,
you are running on Red Hat Enterprise Linux Server release 5.4, right? Correct.
Is your OS and hardware 64-bit or 32-bit?
64 bit.
Did you install pre-built binaries (RPMs) or did you compile MonetDB yourself?
We compiled MonetDB from the superball.
If the former, which RPMs did you get from where? (Official release or nightly build?) If the latter, did you compile from CVS sources or from a (released or nightly built?) super source ball, which configure setting did you use, and which compiler (incl. version) did you use?
We compiled the Super Source Tarball, http://monetdb.cwi.nl/downloads/sources/Nov2009/MonetDB-Nov2009-SuperBal l.tar.bz2.
We ran monetdb-install.sh with --prefix and --enable-xquery options only. We used gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-46) for compilation.
Like Peter, I haven't yet managed to reproduce your problem with the given query. The aove information might help us to test in a setup that is as close to yours as possible ...
We also cannot reproduce the problem on our Fedora machine. Hopefully, the answers above help you.
If you need more info, please let me know!
Thanks,
Harm
Stefan
Dear Peter,
Thanks for your reply!
I double-checked; the query sent earlier DOES run into the "too deep recursion" problem. One major difference with respect to earlier development is that we currently are using the RedHat Enterprise Linux (Tikanga) operation system on our servers. We know that this might cause the problem, however, as far as we can see the user limits match (or outnumber) the limits on the Fedora systems.
If you can point us where/how to check which limits are used in the calculation used for limiting recursion depth, we might be able to adapt the settings.
We installed the November release in our development environment and will soon start testing with it.
Kind regards,
Harm
-----Original Message----- From: Peter Boncz [mailto:P.Boncz@cwi.nl] Sent: Monday, December 14, 2009 12:25 AM To: Harm van Beek (DT) Cc: Communication channel for MonetDB users Subject: RE: [MonetDB-users] Fatal error: aborted too deep recursion
Hi Harm,
I had promised to get back to on this problem during the week; I did not quite make that, here I am trying to make what might be your definition of 'before the beginning of next week'. But apologies for the delay.
You provided a test query on which you saw the 'recursion' error occurring. We know this error can occur on certain very large queries (in the test web there is a query with 100 XPath steps which triggers it). As there are limits to any program, we accept that this bug will happen in very extreme cases. However, the query you provided did not seem very intimidating. Therefore it may be a serious issue.
However, your report so far has proven irreproducible for me.
I tried it on the latest release with the mclient utility to no avail. I also tried with the 'pf' program standalone, as well as with in-server (using the query wrapped in a 'xquery()' MIL command).
Are you sure this query triggers the error? Of course, I might go ahead and create an option for changing the recursion limit in the code. I held back from that because the risk is that instead of an error you might be getting a segmentation violation instead (after all, the fence is
reason). I could mitigate that risk by also giving the thread more stack space (does not work very easily on all architectures). Still, understanding the problem better will make the solution better, so getting this to reproduce would help.
Thanks in advance,
Peter
BTW did NFI try the Nov2009 release already? It is the one with the pf:cache pragmas that will help performance quite a bit.
I tested this with all 126 versions of MonetDB4 currently in Stefan's archive:
$ for i in `locate bin/pf | egrep -e 'pf$'`; do echo $i; $i /tmp/t 2>&1| fgrep ecurs; done /export/scratch0/manegold/BenQ/2008-05-15.F8/sda1/home/manegold /monetdb-curr ent/program-i686/bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda1/usr/bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda1/var/www/cgi-b in/MonetDB/bi n/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda2/Win/home/mane gold/Developm ent/Current/.GNU.32.32.d.--enable-debug.PREFIX./bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda2/Win/home/mane gold/Developm ent/Update/.GNU.32.32.d.--enable-debug.PREFIX./bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda3/SuSE/FC5/home /manegold/mon etdb-current/program-i686/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP1/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all/prefix.db g.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP1/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.opt.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.opt.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.opt.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.db g.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.op t.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.op t.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.op t.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.opt.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.opt.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.opt.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.opt.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.opt.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.opt.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2009-SuperB all/prefix.db g.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2009-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix./bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-bits~32/bi n/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-bits~32_-- enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-oid32/bin/ pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-oid32_--en able-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d
isable-debug_--enable-optimize_--disable-assert_--enable-probx ml/bin/pf
/export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--disable-assert_CC~icc/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--disable-assert_CC~icc_--enable -probxml/bin/ pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d
isable-debug_--enable-optimize_--enable-assert_--enable-probxml/bin/pf
/export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-optimize_--disable-debug_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-optimize_--disable-debug_--disable-assert_--enable-probx ml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-optimize_--enable-debug_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d
isable-optimize_--enable-debug_--enable-assert_--enable-probxml/bin/pf
/export/scratch0/manegold/Monet/Testing/Candidate/prefix._--ena ble-probxml/b in/pf /export/scratch0/manegold/Monet/Testing/Current/prefix./bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -bits~32/bin/ pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -bits~32.pf_M 5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -bits~32_--en able-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -oid32.pf_M5X Q/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -oid32_--enab le-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis
able-debug_--enable-optimize_--disable-assert_--enable-probxml/bin/pf
/export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert_CC~icc/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert_CC~icc.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert_CC~icc_--enable-p robxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--enable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--disable-debug_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--disable-debug_--disable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis
able-optimize_--disable-debug_--disable-assert_--enable-probxml/bin/pf
/export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--enable-debug_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--enable-debug_--enable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--enable-debug_--enable-assert_--enable-probxml/bin/pf
/export/scratch0/manegold/Monet/Testing/Current/prefix..pf_M5XQ/bin/pf
/export/scratch0/manegold/Monet/Testing/Current/prefix._--enabl e-probxml/bin /pf /export/scratch0/manegold/Monet/Testing/Stable/prefix./bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- bits~32/bin/p f /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- bits~32.pf_NF I/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- bits~32_--ena ble-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- oid32.pf_NFI/ bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- oid32_--enabl e-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_CC~icc/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_CC~icc.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_CC~icc_--enable-pr obxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-bits~32/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-bits~32_-- enable-probxm l/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-oid32_--en able-probxml/ bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--disable-debug_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--disable-debug_--disable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa
ble-optimize_--disable-debug_--disable-assert_--enable-probxml/bin/pf
/export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--enable-debug_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--enable-debug_--enable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--enable-debug_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix..pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix._--enable -probxml/bin/ pf /export/scratch0/manegold/MonetDB_4-12/.GNU.64.64.d..PREFIX./PA THFINDER/bin/ pf /export/scratch0/manegold/MonetDB_Release_Build/prefix/bin/pf /export/scratch0/manegold/tmp/BenQ_FC6/home/manegold/monetdb-cu rrent/program -i686/bin/pf
/export/scratch0/manegold/tmp/BenQ_FC6/var/www/cgi-bin/MonetDB/bin/pf
/usr/bin/pf
-----Original Message----- From: Harm van Beek(DT) [mailto:harm@holmes.nl] Sent: vrijdag 27 november 2009 14:49 To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
Hi Peter,
We indeed execute a large query (which is constructed by
On Mon, Dec 14, 2009 at 09:09:51AM +0100, Harm van Beek (DT) wrote: there for a progra mcode,
so hard to change). You can find the query at the bottom of my reply.
I understand that you introduced the limits to not run out of stack space. However, since we are in full control of the system on which the query is executed, we might be able to adapt the limits to our environment. Can you give us directions on where to set this?
Thanks again,
Harm
-- query:
element result { for $name in distinct-values( ( let $var1 := let $var2 := let $var3 := let $var4 := let $var5 := let $var6 := let $var7 := let $var8 := let $var9 := let $var10 := let $var11 := let $var12 := doc('document.xml')//properties/.. return $var12[ (self::registry) ] return $var11/ancestor-or-self::file[1]/properties/.. return $var10[ properties/(name)[ contains( lower-case(.), 'system' ) ] ] return $var9/child::registry/properties/.. return $var8/child::key/properties/.. return $var7/child::key/properties/.. return $var6/child::key/properties/.. return $var5[ properties/(name)[ . = 'Enum' ] ] return $var4/descendant-or-self::key/properties/.. return $var3[ properties/(name)[ contains( lower-case(.), 'usb' ) ] ] return $var2/descendant-or-self::value/properties/.. return $var1[ properties/(name)[ . = 'LocationInformation' ] ] )/properties/*/name() ) return element name { $name } }
-----Original Message----- From: Peter Boncz [mailto:P.Boncz@cwi.nl] Sent: Friday, November 27, 2009 10:26 AM To: 'Communication channel for MonetDB users' Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
Hi Harm,
The 'too deep recursion' error is typically triggered by very large queries that send the pathfinder compiler into a very deep (and time consuming) search. The error is given to prevent a crash due to lack of stack space. Recently, I do recall that the limits were tightened as there are some operating systems in our nightly test farm that give their processes little stack space, and could still cause pathfinder to crash due to lack of stack space.
So, it depends. The limit when to break off search is arbitrary and based currently on the lowest common denominator among the systems we used. We could potentially increase it (and possibly fix the settings on our testing machines to allow more stack space). We could even introduce a user-controller limit.
However, this phenomenon typically happens only on strange queries (e.fg doing tens or hundreds XPath steps after each other).
Could you share what query produced such an error for you?
Best,
Peter Boncz
-----Original Message----- From: Martin Kersten [mailto:Martin.Kersten@cwi.nl] Sent: Thursday, November 26, 2009 4:28 PM To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
Harm van Beek(DT) wrote: > During the development of our tool, using Xquery on MonetDB, we ran onto > the following error: > > ERROR = !fatal error: aborted too deep recursion > >>From previous posts (I guess) I can conclude that this relates to an > Xquery that simply is too big (too many operators) to create a good plan > for it. All systems have limits that either cause an emergency stop. In this case, the operating system has such a limit. > > I tried several MonetDB releases (May2009-SP2, Aug2009-SP2, Nov2009-DB0) > on Red Hat Enterprise Linux Server release 5.4 (Tikanga), all showing > the same result. > > Does anyone have any directions on how to solve this? Yes. Programming cleverness comes into play. Especially if you use a tool to generate queries, then that tool should be a little more conservative in the kind of queries generated.
> Can we reconfigure the compiler to fix this (or recompile it with > options)? No. At some point the optimization space becomes just too big. People that can spare the time to find the optimal settings for a larger application space are more then welcome to try it out on the source code made available.
regards, Martin > > Thanks in advance! > > Harm van Beek > Netherlands Forensic Institute > >
----------
----- > Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30- Day > trial. Simplify your report design, integration and deployment - and focus on > what you do best, core application coding. Discover what's new with > Crystal Reports now. http://p.sf.net/sfu/bobj-july > _______________________________________________ > MonetDB-users mailing list > MonetDB-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-users
--------------------------------------------------------------- ------------
--- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.426 / Virus Database: 270.14.83/2526 - Release Date: 11/25/09 19:43:00
--------------------------------------------------------------- --------------- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
--------------------------------------------------------------- ------------- -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.426 / Virus Database: 270.14.83/2526 - Release Date: 11/26/09 19:42:00
--------------------------------------------------------------- ---------------
Return on Information: Google Enterprise Search pays you back Get the facts. http://p.sf.net/sfu/google-dev2dev _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
-- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |
--------------------------------------------------------------- --------------- Return on Information: Google Enterprise Search pays you back Get the facts. http://p.sf.net/sfu/google-dev2dev _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
------------------------------------------------------------------------------ Return on Information: Google Enterprise Search pays you back Get the facts. http://p.sf.net/sfu/google-dev2dev _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
-- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |
Hi Harm, I have been able to reproduce this problem, and it is a compiler bug. This is clearly gcc 4.1.2 related, with optimizing flags. Our gentoo system also has this compiler function. Though the reduce() function in the milgen module of the pathfinder declares just a few variables, the compiler produced code ups the stack pointer by a hefty 16KB per call. This means that stack space is gone after 100 calls or so. In my other main activity (vectorwise) we have just also experienced problems with this compiler. It just generates wrong code as well. So the problems are not restricted to stack space overuse. My conclusion is that despite the "reliable" RHEL label, gcc 4.1.2 is a release that is not enterprise proof. At least that is my conclusion. The best solution is upgrading your compiler. Another solution is using a build produced on a different system, or use a binary distributed by CWI. Hope this helps, Peter -----Original Message----- From: Harm van Beek (DT) [mailto:harm@holmes.nl] Sent: maandag 14 december 2009 13:12 To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion Hi Stefan, See my answers below.
-----Original Message----- From: Stefan Manegold [mailto:Stefan.Manegold@cwi.nl] Sent: Monday, December 14, 2009 12:43 PM To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
Beste Harm,
you are running on Red Hat Enterprise Linux Server release 5.4, right? Correct.
Is your OS and hardware 64-bit or 32-bit?
64 bit.
Did you install pre-built binaries (RPMs) or did you compile MonetDB yourself?
We compiled MonetDB from the superball.
If the former, which RPMs did you get from where? (Official release or nightly build?) If the latter, did you compile from CVS sources or from a (released or nightly built?) super source ball, which configure setting did you use, and which compiler (incl. version) did you use?
We compiled the Super Source Tarball, http://monetdb.cwi.nl/downloads/sources/Nov2009/MonetDB-Nov2009-SuperBal l.tar.bz2. We ran monetdb-install.sh with --prefix and --enable-xquery options only. We used gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-46) for compilation.
Like Peter, I haven't yet managed to reproduce your problem with the given query. The aove information might help us to test in a setup that is as close to yours as possible ...
We also cannot reproduce the problem on our Fedora machine. Hopefully, the answers above help you. If you need more info, please let me know! Thanks, Harm
Stefan
Dear Peter,
Thanks for your reply!
I double-checked; the query sent earlier DOES run into the "too deep recursion" problem. One major difference with respect to earlier development is that we currently are using the RedHat Enterprise Linux (Tikanga) operation system on our servers. We know that this might cause the problem, however, as far as we can see the user limits match (or outnumber) the limits on the Fedora systems.
If you can point us where/how to check which limits are used in the calculation used for limiting recursion depth, we might be able to adapt the settings.
We installed the November release in our development environment and will soon start testing with it.
Kind regards,
Harm
-----Original Message----- From: Peter Boncz [mailto:P.Boncz@cwi.nl] Sent: Monday, December 14, 2009 12:25 AM To: Harm van Beek (DT) Cc: Communication channel for MonetDB users Subject: RE: [MonetDB-users] Fatal error: aborted too deep recursion
Hi Harm,
I had promised to get back to on this problem during the week; I did not quite make that, here I am trying to make what might be your definition of 'before the beginning of next week'. But apologies for the delay.
You provided a test query on which you saw the 'recursion' error occurring. We know this error can occur on certain very large queries (in the test web there is a query with 100 XPath steps which triggers it). As there are limits to any program, we accept that this bug will happen in very extreme cases. However, the query you provided did not seem very intimidating. Therefore it may be a serious issue.
However, your report so far has proven irreproducible for me.
I tried it on the latest release with the mclient utility to no avail. I also tried with the 'pf' program standalone, as well as with in-server (using the query wrapped in a 'xquery()' MIL command).
Are you sure this query triggers the error? Of course, I might go ahead and create an option for changing the recursion limit in the code. I held back from that because the risk is that instead of an error you might be getting a segmentation violation instead (after all, the fence is
reason). I could mitigate that risk by also giving the thread more stack space (does not work very easily on all architectures). Still, understanding the problem better will make the solution better, so getting this to reproduce would help.
Thanks in advance,
Peter
BTW did NFI try the Nov2009 release already? It is the one with the pf:cache pragmas that will help performance quite a bit.
I tested this with all 126 versions of MonetDB4 currently in Stefan's archive:
$ for i in `locate bin/pf | egrep -e 'pf$'`; do echo $i; $i /tmp/t 2>&1| fgrep ecurs; done /export/scratch0/manegold/BenQ/2008-05-15.F8/sda1/home/manegold /monetdb-curr ent/program-i686/bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda1/usr/bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda1/var/www/cgi-b in/MonetDB/bi n/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda2/Win/home/mane gold/Developm ent/Current/.GNU.32.32.d.--enable-debug.PREFIX./bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda2/Win/home/mane gold/Developm ent/Update/.GNU.32.32.d.--enable-debug.PREFIX./bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda3/SuSE/FC5/home /manegold/mon etdb-current/program-i686/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP1/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all/prefix.db g.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP1/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.opt.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.opt.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.opt.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.db g.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.op t.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.op t.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.op t.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.opt.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.opt.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.opt.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.opt.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.opt.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.opt.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2009-SuperB all/prefix.db g.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2009-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix./bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-bits~32/bi n/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-bits~32_-- enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-oid32/bin/ pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-oid32_--en able-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d
isable-debug_--enable-optimize_--disable-assert_--enable-probx ml/bin/pf
/export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--disable-assert_CC~icc/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--disable-assert_CC~icc_--enable -probxml/bin/ pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d
isable-debug_--enable-optimize_--enable-assert_--enable-probxml/bin/pf
/export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-optimize_--disable-debug_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-optimize_--disable-debug_--disable-assert_--enable-probx ml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-optimize_--enable-debug_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d
isable-optimize_--enable-debug_--enable-assert_--enable-probxml/bin/pf
/export/scratch0/manegold/Monet/Testing/Candidate/prefix._--ena ble-probxml/b in/pf /export/scratch0/manegold/Monet/Testing/Current/prefix./bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -bits~32/bin/ pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -bits~32.pf_M 5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -bits~32_--en able-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -oid32.pf_M5X Q/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -oid32_--enab le-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis
able-debug_--enable-optimize_--disable-assert_--enable-probxml/bin/pf
/export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert_CC~icc/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert_CC~icc.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert_CC~icc_--enable-p robxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--enable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--disable-debug_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--disable-debug_--disable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis
able-optimize_--disable-debug_--disable-assert_--enable-probxml/bin/pf
/export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--enable-debug_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--enable-debug_--enable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--enable-debug_--enable-assert_--enable-probxml/bin/pf
/export/scratch0/manegold/Monet/Testing/Current/prefix..pf_M5XQ/bin/pf
/export/scratch0/manegold/Monet/Testing/Current/prefix._--enabl e-probxml/bin /pf /export/scratch0/manegold/Monet/Testing/Stable/prefix./bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- bits~32/bin/p f /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- bits~32.pf_NF I/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- bits~32_--ena ble-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- oid32.pf_NFI/ bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- oid32_--enabl e-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_CC~icc/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_CC~icc.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_CC~icc_--enable-pr obxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-bits~32/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-bits~32_-- enable-probxm l/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-oid32_--en able-probxml/ bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--disable-debug_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--disable-debug_--disable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa
ble-optimize_--disable-debug_--disable-assert_--enable-probxml/bin/pf
/export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--enable-debug_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--enable-debug_--enable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--enable-debug_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix..pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix._--enable -probxml/bin/ pf /export/scratch0/manegold/MonetDB_4-12/.GNU.64.64.d..PREFIX./PA THFINDER/bin/ pf /export/scratch0/manegold/MonetDB_Release_Build/prefix/bin/pf /export/scratch0/manegold/tmp/BenQ_FC6/home/manegold/monetdb-cu rrent/program -i686/bin/pf
/export/scratch0/manegold/tmp/BenQ_FC6/var/www/cgi-bin/MonetDB/bin/pf
/usr/bin/pf
-----Original Message----- From: Harm van Beek(DT) [mailto:harm@holmes.nl] Sent: vrijdag 27 november 2009 14:49 To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
Hi Peter,
We indeed execute a large query (which is constructed by
On Mon, Dec 14, 2009 at 09:09:51AM +0100, Harm van Beek (DT) wrote: there for a progra mcode,
so hard to change). You can find the query at the bottom of my reply.
I understand that you introduced the limits to not run out of stack space. However, since we are in full control of the system on which the query is executed, we might be able to adapt the limits to our environment. Can you give us directions on where to set this?
Thanks again,
Harm
-- query:
element result { for $name in distinct-values( ( let $var1 := let $var2 := let $var3 := let $var4 := let $var5 := let $var6 := let $var7 := let $var8 := let $var9 := let $var10 := let $var11 := let $var12 := doc('document.xml')//properties/.. return $var12[ (self::registry) ] return $var11/ancestor-or-self::file[1]/properties/.. return $var10[ properties/(name)[ contains( lower-case(.), 'system' ) ] ] return $var9/child::registry/properties/.. return $var8/child::key/properties/.. return $var7/child::key/properties/.. return $var6/child::key/properties/.. return $var5[ properties/(name)[ . = 'Enum' ] ] return $var4/descendant-or-self::key/properties/.. return $var3[ properties/(name)[ contains( lower-case(.), 'usb' ) ] ] return $var2/descendant-or-self::value/properties/.. return $var1[ properties/(name)[ . = 'LocationInformation' ] ] )/properties/*/name() ) return element name { $name } }
-----Original Message----- From: Peter Boncz [mailto:P.Boncz@cwi.nl] Sent: Friday, November 27, 2009 10:26 AM To: 'Communication channel for MonetDB users' Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
Hi Harm,
The 'too deep recursion' error is typically triggered by very large queries that send the pathfinder compiler into a very deep (and time consuming) search. The error is given to prevent a crash due to lack of stack space. Recently, I do recall that the limits were tightened as there are some operating systems in our nightly test farm that give their processes little stack space, and could still cause pathfinder to crash due to lack of stack space.
So, it depends. The limit when to break off search is arbitrary and based currently on the lowest common denominator among the systems we used. We could potentially increase it (and possibly fix the settings on our testing machines to allow more stack space). We could even introduce a user-controller limit.
However, this phenomenon typically happens only on strange queries (e.fg doing tens or hundreds XPath steps after each other).
Could you share what query produced such an error for you?
Best,
Peter Boncz
-----Original Message----- From: Martin Kersten [mailto:Martin.Kersten@cwi.nl] Sent: Thursday, November 26, 2009 4:28 PM To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
During the development of our tool, using Xquery on MonetDB, we ran onto the following error:
ERROR = !fatal error: aborted too deep recursion
>From previous posts (I guess) I can conclude that this relates to an Xquery that simply is too big (too many operators) to create a good plan for it. All systems have limits that either cause an emergency stop. In this case,
Harm van Beek(DT) wrote: the operating system has such a limit.
I tried several MonetDB releases (May2009-SP2,
Aug2009-SP2, Nov2009-DB0)
on Red Hat Enterprise Linux Server release 5.4 (Tikanga), all showing the same result.
Does anyone have any directions on how to solve this? Yes. Programming cleverness comes into play. Especially if you use a tool to generate queries, then that tool should be a little more conservative in the kind of queries generated.
Can we reconfigure the compiler to fix this (or recompile it with options)? No. At some point the optimization space becomes just too big. People that can spare the time to find the optimal settings for a larger application space are more then welcome to try it out on the source code made available.
regards, Martin
Thanks in advance!
Harm van Beek Netherlands Forensic Institute
----------
-----
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30- Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
--------------------------------------------------------------- ------------
--- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.426 / Virus Database: 270.14.83/2526 - Release Date: 11/25/09 19:43:00
--------------------------------------------------------------- --------------- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
--------------------------------------------------------------- ------------- -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.426 / Virus Database: 270.14.83/2526 - Release Date: 11/26/09 19:42:00
--------------------------------------------------------------- ---------------
Return on Information: Google Enterprise Search pays you back Get the facts. http://p.sf.net/sfu/google-dev2dev _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
-- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |
--------------------------------------------------------------- --------------- Return on Information: Google Enterprise Search pays you back Get the facts. http://p.sf.net/sfu/google-dev2dev _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
---------------------------------------------------------------------------- -- Return on Information: Google Enterprise Search pays you back Get the facts. http://p.sf.net/sfu/google-dev2dev _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.427 / Virus Database: 270.14.106/2563 - Release Date: 12/13/09 19:47:00
Hi Peter, Thanks for your reply and time spent to find it out! Too bad to head that the RHEL distribution has a bug in its gcc compiler. We will internally discuss how to proceed with this. I'll let you know which way we plan to go. Thanks again! Kind regards, Harm
-----Original Message----- From: Peter Boncz [mailto:P.Boncz@cwi.nl] Sent: Wednesday, December 23, 2009 6:47 PM To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
Hi Harm,
I have been able to reproduce this problem, and it is a compiler bug.
This is clearly gcc 4.1.2 related, with optimizing flags. Our gentoo system also has this compiler function. Though the reduce() function in the milgen module of the pathfinder declares just a few variables, the compiler produced code ups the stack pointer by a hefty 16KB per call. This means that stack space is gone after 100 calls or so.
In my other main activity (vectorwise) we have just also experienced problems with this compiler. It just generates wrong code as well. So the problems are not restricted to stack space overuse.
My conclusion is that despite the "reliable" RHEL label, gcc 4.1.2 is a release that is not enterprise proof. At least that is my conclusion.
The best solution is upgrading your compiler. Another solution is using a build produced on a different system, or use a binary distributed by CWI.
Hope this helps,
Peter
-----Original Message----- From: Harm van Beek (DT) [mailto:harm@holmes.nl] Sent: maandag 14 december 2009 13:12 To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
Hi Stefan,
See my answers below.
-----Original Message----- From: Stefan Manegold [mailto:Stefan.Manegold@cwi.nl] Sent: Monday, December 14, 2009 12:43 PM To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
Beste Harm,
you are running on Red Hat Enterprise Linux Server release 5.4, right? Correct.
Is your OS and hardware 64-bit or 32-bit?
64 bit.
Did you install pre-built binaries (RPMs) or did you compile MonetDB yourself?
We compiled MonetDB from the superball.
If the former, which RPMs did you get from where? (Official release or nightly build?) If the latter, did you compile from CVS sources or from a (released or nightly built?) super source ball, which configure setting did you use, and which compiler (incl. version) did you use?
We compiled the Super Source Tarball, http://monetdb.cwi.nl/downloads/sources/Nov2009/MonetDB-Nov2009 -SuperBal l.tar.bz2.
We ran monetdb-install.sh with --prefix and --enable-xquery options only. We used gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-46) for compilation.
Like Peter, I haven't yet managed to reproduce your problem with the given query. The aove information might help us to test in a setup
that is as
close to yours as possible ... We also cannot reproduce the problem on our Fedora machine. Hopefully, the answers above help you.
If you need more info, please let me know!
Thanks,
Harm
Stefan
On Mon, Dec 14, 2009 at 09:09:51AM +0100, Harm van Beek (DT) wrote:
Dear Peter,
Thanks for your reply!
I double-checked; the query sent earlier DOES run into the "too deep recursion" problem. One major difference with respect to earlier development is that we currently are using the RedHat Enterprise Linux (Tikanga) operation system on our servers. We know that this might cause the problem, however, as far as we can see the user limits match (or outnumber) the limits on the Fedora systems.
If you can point us where/how to check which limits are used in the calculation used for limiting recursion depth, we might be able to adapt the settings.
We installed the November release in our development environment and will soon start testing with it.
Kind regards,
Harm
-----Original Message----- From: Peter Boncz [mailto:P.Boncz@cwi.nl] Sent: Monday, December 14, 2009 12:25 AM To: Harm van Beek (DT) Cc: Communication channel for MonetDB users Subject: RE: [MonetDB-users] Fatal error: aborted too deep
recursion
Hi Harm,
I had promised to get back to on this problem during the week; I did not quite make that, here I am trying to make what might be your definition of 'before the beginning of next week'. But apologies for the delay.
You provided a test query on which you saw the 'recursion' error occurring. We know this error can occur on certain very large queries (in the test web there is a query with 100 XPath steps which triggers it).
limits to any program, we accept that this bug will happen in very extreme cases. However, the query you provided did not seem very intimidating. Therefore it may be a serious issue.
However, your report so far has proven irreproducible for me.
I tried it on the latest release with the mclient utility to no avail. I also tried with the 'pf' program standalone, as well as with in-server (using the query wrapped in a 'xquery()' MIL command).
Are you sure this query triggers the error? Of course, I might go ahead and create an option for changing the recursion limit in the code. I held back from that because the risk is that instead of an error you might be getting a segmentation violation instead (after all, the fence is
As there are there for a
reason). I could mitigate that risk by also giving the thread more stack space (does not work very easily on all architectures). Still, understanding the problem better will make the solution better, so getting this to reproduce would help.
Thanks in advance,
Peter
BTW did NFI try the Nov2009 release already? It is the one with the pf:cache pragmas that will help performance quite a bit.
I tested this with all 126 versions of MonetDB4 currently in Stefan's archive:
$ for i in `locate bin/pf | egrep -e 'pf$'`; do echo $i; $i /tmp/t 2>&1| fgrep ecurs; done /export/scratch0/manegold/BenQ/2008-05-15.F8/sda1/home/manegold /monetdb-curr ent/program-i686/bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda1/usr/bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda1/var/www/cgi-b in/MonetDB/bi n/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda2/Win/home/mane gold/Developm ent/Current/.GNU.32.32.d.--enable-debug.PREFIX./bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda2/Win/home/mane gold/Developm ent/Update/.GNU.32.32.d.--enable-debug.PREFIX./bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda3/SuSE/FC5/home /manegold/mon etdb-current/program-i686/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP1/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all/prefix.db g.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP1/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.opt.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.opt.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.opt.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.db g.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.op t.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.op t.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.op t.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.opt.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.opt.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.opt.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.opt.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.opt.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.opt.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2009-SuperB all/prefix.db g.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2009-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix./bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-bits~32/bi n/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-bits~32_-- enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-oid32/bin/ pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-oid32_--en able-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d
isable-debug_--enable-optimize_--disable-assert_--enable-probx ml/bin/pf
/export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--disable-assert_CC~icc/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--disable-assert_CC~icc_--enable -probxml/bin/ pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d
isable-debug_--enable-optimize_--enable-assert_--enable-probx ml/bin/pf
/export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-optimize_--disable-debug_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-optimize_--disable-debug_--disable-assert_--enable-probx ml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-optimize_--enable-debug_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d
isable-optimize_--enable-debug_--enable-assert_--enable-probx ml/bin/pf
/export/scratch0/manegold/Monet/Testing/Candidate/prefix._--ena ble-probxml/b in/pf /export/scratch0/manegold/Monet/Testing/Current/prefix./bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -bits~32/bin/ pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -bits~32.pf_M 5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -bits~32_--en able-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -oid32.pf_M5X Q/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -oid32_--enab le-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis
able-debug_--enable-optimize_--disable-assert_--enable-probxml/bin/pf
/export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert_CC~icc/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert_CC~icc.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert_CC~icc_--enable-p robxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--enable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis
able-debug_--enable-optimize_--enable-assert_--enable-probxml/bin/pf
/export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--disable-debug_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--disable-debug_--disable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis
able-optimize_--disable-debug_--disable-assert_--enable-probx ml/bin/pf
/export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--enable-debug_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--enable-debug_--enable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis
able-optimize_--enable-debug_--enable-assert_--enable-probxml/bin/pf
/export/scratch0/manegold/Monet/Testing/Current/prefix..pf_M5
XQ/bin/pf
/export/scratch0/manegold/Monet/Testing/Current/prefix._--enabl e-probxml/bin /pf /export/scratch0/manegold/Monet/Testing/Stable/prefix./bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- bits~32/bin/p f /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- bits~32.pf_NF I/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- bits~32_--ena ble-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- oid32.pf_NFI/ bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- oid32_--enabl e-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa
ble-debug_--enable-optimize_--disable-assert_--enable-probxml/bin/pf
/export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_CC~icc/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_CC~icc.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_CC~icc_--enable-pr obxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-bits~32/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-bits~32_-- enable-probxm l/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-oid32_--en able-probxml/ bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--disable-debug_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--disable-debug_--disable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa
ble-optimize_--disable-debug_--disable-assert_--enable-probxml/bin/pf
/export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--enable-debug_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--enable-debug_--enable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--enable-debug_--enable-assert_--enable-probxml/bin/pf
/export/scratch0/manegold/Monet/Testing/Stable/prefix._--enable -probxml/bin/ pf /export/scratch0/manegold/MonetDB_4-12/.GNU.64.64.d..PREFIX./PA THFINDER/bin/ pf /export/scratch0/manegold/MonetDB_Release_Build/prefix/bin/pf /export/scratch0/manegold/tmp/BenQ_FC6/home/manegold/monetdb-cu rrent/program -i686/bin/pf
/export/scratch0/manegold/tmp/BenQ_FC6/var/www/cgi-bin/MonetDB/bin/pf
/usr/bin/pf
-----Original Message----- From: Harm van Beek(DT) [mailto:harm@holmes.nl] Sent: vrijdag 27 november 2009 14:49 To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
Hi Peter,
We indeed execute a large query (which is constructed by
/export/scratch0/manegold/Monet/Testing/Stable/prefix..pf_NFI/bin/pf progra mcode,
so hard to change). You can find the query at the bottom of my reply.
I understand that you introduced the limits to not run out of stack space. However, since we are in full control of the system on which the query is executed, we might be able to adapt the limits to our environment. Can you give us directions on where to set this?
Thanks again,
Harm
-- query:
element result { for $name in distinct-values( ( let $var1 := let $var2 := let $var3 := let $var4 := let $var5 := let $var6 := let $var7 := let $var8 := let $var9 := let $var10 := let $var11 := let $var12 := doc('document.xml')//properties/.. return $var12[ (self::registry) ] return $var11/ancestor-or-self::file[1]/properties/.. return $var10[ properties/(name)[ contains( lower-case(.), 'system' ) ] ] return $var9/child::registry/properties/.. return $var8/child::key/properties/.. return $var7/child::key/properties/.. return $var6/child::key/properties/.. return $var5[ properties/(name)[ . = 'Enum' ] ] return $var4/descendant-or-self::key/properties/.. return $var3[ properties/(name)[ contains( lower-case(.), 'usb' ) ] ] return $var2/descendant-or-self::value/properties/.. return $var1[ properties/(name)[ . = 'LocationInformation' ] ] )/properties/*/name() ) return element name { $name } }
-----Original Message----- From: Peter Boncz [mailto:P.Boncz@cwi.nl] Sent: Friday, November 27, 2009 10:26 AM To: 'Communication channel for MonetDB users' Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
Hi Harm,
The 'too deep recursion' error is typically triggered by very large queries that send the pathfinder compiler into a very deep (and time consuming) search. The error is given to prevent a crash due to lack of stack space. Recently, I do recall that the limits were tightened as there are some operating systems in our nightly test farm that give their processes little stack space, and could still cause pathfinder to crash due to lack of stack space.
So, it depends. The limit when to break off search is arbitrary and based currently on the lowest common denominator among the systems we used. We could potentially increase it (and possibly fix the settings on our testing machines to allow more stack space). We could even introduce a user-controller limit.
However, this phenomenon typically happens only on strange queries (e.fg doing tens or hundreds XPath steps after each other).
Could you share what query produced such an error for you?
Best,
Peter Boncz
-----Original Message----- From: Martin Kersten [mailto:Martin.Kersten@cwi.nl] Sent: Thursday, November 26, 2009 4:28 PM To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
Harm van Beek(DT) wrote: > During the development of our tool, using Xquery on MonetDB, we ran onto > the following error: > > ERROR = !fatal error: aborted too deep recursion > >>From previous posts (I guess) I can conclude that this relates to an > Xquery that simply is too big (too many operators) to create a good plan > for it. All systems have limits that either cause an emergency stop. In this case, the operating system has such a limit. > > I tried several MonetDB releases (May2009-SP2, Aug2009-SP2, Nov2009-DB0) > on Red Hat Enterprise Linux Server release 5.4 (Tikanga), all showing > the same result. > > Does anyone have any directions on how to solve this? Yes. Programming cleverness comes into play. Especially if you use a tool to generate queries, then that tool should be a little more conservative in the kind of queries generated.
> Can we reconfigure the compiler to fix this (or recompile it with > options)? No. At some point the optimization space becomes just too big. People that can spare the time to find the optimal settings for a larger application space are more then welcome to try it out on the source code made available.
regards, Martin > > Thanks in advance! > > Harm van Beek > Netherlands Forensic Institute > >
----------
----- > Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30- Day > trial. Simplify your report design, integration and deployment - and focus on > what you do best, core application coding. Discover what's new with > Crystal Reports now. http://p.sf.net/sfu/bobj-july > _______________________________________________ > MonetDB-users mailing list > MonetDB-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-users
--------------------------------------------------------------- ------------
--- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.426 / Virus Database: 270.14.83/2526 - Release Date: 11/25/09 19:43:00
--------------------------------------------------------------- --------------- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
--------------------------------------------------------------- ------------- -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.426 / Virus Database: 270.14.83/2526 - Release Date: 11/26/09 19:42:00
--------------------------------------------------------------- ---------------
Return on Information: Google Enterprise Search pays you back Get the facts. http://p.sf.net/sfu/google-dev2dev _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
-- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |
--------------------------------------------------------------- --------------- Return on Information: Google Enterprise Search pays you back Get the facts. http://p.sf.net/sfu/google-dev2dev _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
--------------------------------------------------------------- ------------- -- Return on Information: Google Enterprise Search pays you back Get the facts. http://p.sf.net/sfu/google-dev2dev _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.427 / Virus Database: 270.14.106/2563 - Release Date: 12/13/09 19:47:00
--------------------------------------------------------------- --------------- This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
On 24-12-2009 08:25:26 +0100, Harm van Beek (DT) wrote:
Thanks for your reply and time spent to find it out! Too bad to head that the RHEL distribution has a bug in its gcc compiler. We will internally discuss how to proceed with this. I'll let you know which way we plan to go.
It is usually not too hard to compile a GCC compiler (provided your host platform is relatively sane). So you might just be able to compile GCC 4.2.4 and do your compilations with that. PS.: pegathree now uses GCC 4.4 again, I had switched it to 4.1.2 because our codebase wasn't yet resistant against 4.4 when I switched it on at that time.
participants (6)
-
Fabian Groffen -
Harm van Beek (DT) -
Harm van Beek(DT)
-
Martin Kersten -
Peter Boncz -
Stefan Manegold