Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime shredder.mx, Feb2009, 1.153.2.3, 1.153.2.4
Jan, unfortunately, this checkin breaks more than 30 pathfinder tests: benchmarks/XBench/TC/MD/shred_docs.milS benchmarks/XBench/TC/MD/load_docs.xq benchmarks/XBench/TC/MD/count_nodes.xq benchmarks/XBench/TC/MD/q01.xq ... benchmarks/XBench/TC/MD/q19.xq benchmarks/XMach-1/load_docs.xq benchmarks/XMach-1/count_nodes.xq benchmarks/XMach-1/q1.xq ... benchmarks/XMach-1/q8.xq tests/BugTracker/immune_for_updates.SF-1766259.xq tests/BugTracker/immune_for_updates.SF-1981852.xq Stefan On Fri, Feb 20, 2009 at 09:06:47AM +0000, Jan Flokstra wrote:
Update of /cvsroot/monetdb/pathfinder/runtime In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv26141
Modified Files: Tag: Feb2009 shredder.mx Log Message: - fix [ monetdb-Bugs-2586088 ] XQ: large text nodes
U shredder.mx Index: shredder.mx =================================================================== RCS file: /cvsroot/monetdb/pathfinder/runtime/shredder.mx,v retrieving revision 1.153.2.3 retrieving revision 1.153.2.4 diff -u -d -r1.153.2.3 -r1.153.2.4 --- shredder.mx 6 Feb 2009 22:52:30 -0000 1.153.2.3 +++ shredder.mx 20 Feb 2009 09:06:45 -0000 1.153.2.4 @@ -362,6 +362,7 @@ int sp; /* the stackpointer */ /* buffer administration */ int content; /* content ptr in char buff */ + int content_max; /* max size of content buffer */ char *content_buf; /* character buffer */ /* */ size_t fileSize; /* size of the input file */ @@ -1105,14 +1106,13 @@ int n) { shredCtxStruct *shredCtx = (shredCtxStruct*) xmlCtx; - int l = MIN(PFSHRED_STRLEN_MAX -(int) shredCtx->content, n); - - memcpy(&(shredCtx->content_buf[shredCtx->content]), cs, l); - shredCtx->content += l;
- if (l < n) - stream_printf(GDKout, "!WARNING: shred_characters truncated text node > %d characters(starts with `%.16s...')\n", - (int) PFSHRED_STRLEN_MAX, cs); + if ( (shredCtx->content + n + 1) < shredCtx->content_max) { + shredCtx->content_max = shredCtx->content + n + 1; + shredCtx->content_buf = GDKrealloc(shredCtx->content_buf, shredCtx->content_max); + } + memcpy(&(shredCtx->content_buf[shredCtx->content]), cs, n); + shredCtx->content += n; }
/** @@ -1831,7 +1831,8 @@ /* allocate some buffers used during parse */ shredCtx->val = (char*) GDKmalloc(PFSHRED_BUFLEN+1); shredCtx->lifo = (node_t *) GDKmalloc((XML_DEPTH_MAX+1) * sizeof(node_t)); - shredCtx->content_buf = (char*) GDKmalloc(PFSHRED_STRLEN_MAX); + shredCtx->content_max = PFSHRED_STRLEN_MAX; + shredCtx->content_buf = (char*) GDKmalloc(shredCtx->content_max); if (shredCtx->val == NULL || shredCtx->content_buf == NULL || shredCtx->lifo == NULL) { return shredder_free(shredCtx,-shredCtx->incremental); }
------------------------------------------------------------------------------ Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise -Strategies to boost innovation and cut costs with open source participation -Receive a $600 discount off the registration fee with the source code: SFAD http://p.sf.net/sfu/XcvMzF8H _______________________________________________ Monetdb-pf-checkins mailing list Monetdb-pf-checkins@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins
-- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |
Sh*t. Does this mean I have to rebuild the new candidate release? I am building a new candidate release (nearly finished), but it contains Jan's fix. I can of course revert the change and rebuild. If I do that, Jan will have more time to fix the bug, the fix then won't make it into this release but will have to wait until the bug-fix release. Stefan Manegold wrote:
Jan,
unfortunately, this checkin breaks more than 30 pathfinder tests:
benchmarks/XBench/TC/MD/shred_docs.milS benchmarks/XBench/TC/MD/load_docs.xq benchmarks/XBench/TC/MD/count_nodes.xq benchmarks/XBench/TC/MD/q01.xq ... benchmarks/XBench/TC/MD/q19.xq benchmarks/XMach-1/load_docs.xq benchmarks/XMach-1/count_nodes.xq benchmarks/XMach-1/q1.xq ... benchmarks/XMach-1/q8.xq tests/BugTracker/immune_for_updates.SF-1766259.xq tests/BugTracker/immune_for_updates.SF-1981852.xq
Stefan
On Fri, Feb 20, 2009 at 09:06:47AM +0000, Jan Flokstra wrote:
Update of /cvsroot/monetdb/pathfinder/runtime In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv26141
Modified Files: Tag: Feb2009 shredder.mx Log Message: - fix [ monetdb-Bugs-2586088 ] XQ: large text nodes
U shredder.mx Index: shredder.mx =================================================================== RCS file: /cvsroot/monetdb/pathfinder/runtime/shredder.mx,v retrieving revision 1.153.2.3 retrieving revision 1.153.2.4 diff -u -d -r1.153.2.3 -r1.153.2.4 --- shredder.mx 6 Feb 2009 22:52:30 -0000 1.153.2.3 +++ shredder.mx 20 Feb 2009 09:06:45 -0000 1.153.2.4 @@ -362,6 +362,7 @@ int sp; /* the stackpointer */ /* buffer administration */ int content; /* content ptr in char buff */ + int content_max; /* max size of content buffer */ char *content_buf; /* character buffer */ /* */ size_t fileSize; /* size of the input file */ @@ -1105,14 +1106,13 @@ int n) { shredCtxStruct *shredCtx = (shredCtxStruct*) xmlCtx; - int l = MIN(PFSHRED_STRLEN_MAX -(int) shredCtx->content, n); - - memcpy(&(shredCtx->content_buf[shredCtx->content]), cs, l); - shredCtx->content += l;
- if (l < n) - stream_printf(GDKout, "!WARNING: shred_characters truncated text node > %d characters(starts with `%.16s...')\n", - (int) PFSHRED_STRLEN_MAX, cs); + if ( (shredCtx->content + n + 1) < shredCtx->content_max) { + shredCtx->content_max = shredCtx->content + n + 1; + shredCtx->content_buf = GDKrealloc(shredCtx->content_buf, shredCtx->content_max); + } + memcpy(&(shredCtx->content_buf[shredCtx->content]), cs, n); + shredCtx->content += n; }
/** @@ -1831,7 +1831,8 @@ /* allocate some buffers used during parse */ shredCtx->val = (char*) GDKmalloc(PFSHRED_BUFLEN+1); shredCtx->lifo = (node_t *) GDKmalloc((XML_DEPTH_MAX+1) * sizeof(node_t)); - shredCtx->content_buf = (char*) GDKmalloc(PFSHRED_STRLEN_MAX); + shredCtx->content_max = PFSHRED_STRLEN_MAX; + shredCtx->content_buf = (char*) GDKmalloc(shredCtx->content_max); if (shredCtx->val == NULL || shredCtx->content_buf == NULL || shredCtx->lifo == NULL) { return shredder_free(shredCtx,-shredCtx->incremental); }
------------------------------------------------------------------------------ Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise -Strategies to boost innovation and cut costs with open source participation -Receive a $600 discount off the registration fee with the source code: SFAD http://p.sf.net/sfu/XcvMzF8H _______________________________________________ Monetdb-pf-checkins mailing list Monetdb-pf-checkins@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins
-- Sjoerd Mullender
participants (2)
-
Sjoerd Mullender -
Stefan Manegold