On Sun, Jan 31, 2010 at 04:43:29PM +0000, Martin Kersten wrote:
Update of /cvsroot/monetdb/MonetDB/src/gdk In directory sfp-cvsdas-1.v30.ch3.sourceforge.com:/tmp/cvs-serv28035
Modified Files: Tag: Feb2010 gdk_storage.mx Log Message: A pity that the first code was not properly checked, because it did not even tried to force the preload by memory touching properly.
As per request (though admittedly belated ...): Wouldn't it be sufficient (and faster and less memory consuming) to preload only that part of the heaps that is indeed used (i.e., holds useful/relevant data) instead of preloading the whole (possibly consciously over-allocated) heaps? I.e., replacing ...heap->size by ...heap->free (as it was in the original code, only that the original code incorrecly assumed ...heap->free to be an absolute address instead of a relative byte offset ...) Stefan
Index: gdk_storage.mx =================================================================== RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_storage.mx,v retrieving revision 1.149.2.5 retrieving revision 1.149.2.6 diff -u -d -r1.149.2.5 -r1.149.2.6 --- gdk_storage.mx 31 Jan 2010 15:50:19 -0000 1.149.2.5 +++ gdk_storage.mx 31 Jan 2010 16:43:27 -0000 1.149.2.6 @@ -703,28 +703,32 @@ @c size_t BATpreload_(BAT *b) { - size_t dummy = 0, *i; + size_t dummy = 0, *i, *limit; size_t step = MT_pagesize()/sizeof(size_t); size_t pages = (size_t) (0.8 * MT_npages());
- if ( b->T->vheap && b->T->vheap->base ){ - IODEBUG THRprintf(GDKout,"#BATpreload(%s): T->vheap\n", BATgetId(b)); - for ( i = (size_t *) b->T->vheap->base; i < (size_t *) b->T->vheap->free && pages > 0; i+= step, pages--) + if ( b->T->vheap && b->T->vheap->base && pages){ + IODEBUG THRprintf(GDKout,"#BATpreloaded(%s): T->vheap\n", BATgetId(b)); + limit = (size_t *) (b->T->vheap->base + b->T->vheap->size); + for ( i = (size_t *) b->T->vheap->base; i < limit && pages > 0; i+= step, pages--) dummy += *i; } - if ( b->H->vheap && b->H->vheap->base ){ - IODEBUG THRprintf(GDKout,"#BATpreload(%s): H->vheap\n", BATgetId(b)); - for ( i = (size_t *) b->H->vheap->base; i < (size_t *) b->H->vheap->free && pages > 0 ; i+= step, pages--) + if ( b->T->heap.base && pages ){ + IODEBUG THRprintf(GDKout,"#BATpreload(%s): T->heap\n", BATgetId(b)); + limit = (size_t *) (b->T->heap.base + b->T->heap.size); + for ( i = (size_t *) b->T->heap.base; i < limit && pages > 0; i+= step, pages--) dummy += *i; } - if ( b->hheap && b->hheap->base ){ - IODEBUG THRprintf(GDKout,"#BATpreload(%s): hheap\n", BATgetId(b)); - for ( i = (size_t *) b->hheap->base; i < (size_t *)b->hheap->free && pages > 0 ; i+= step, pages--) + if ( b->H->heap.base && pages ){ + IODEBUG THRprintf(GDKout,"#BATpreload(%s): H->heap\n", BATgetId(b)); + limit = (size_t *) (b->H->heap.base + b->H->heap.size); + for ( i = (size_t *) b->H->heap.base; i < limit && pages > 0 ; i+= step, pages--) dummy += *i; } - if ( b->theap && b->theap->base ){ - IODEBUG THRprintf(GDKout,"#BATpreload(%s): theap\n", BATgetId(b)); - for ( i = (size_t *) b->theap->base; i < (size_t *) b->theap->free && pages > 0; i+= step, pages--) + if ( b->H->vheap && b->H->vheap->base && pages ){ + IODEBUG THRprintf(GDKout,"#BATpreload(%s): H->vheap\n", BATgetId(b)); + limit = (size_t *) (b->H->vheap->base + b->H->vheap->size); + for ( i = (size_t *) b->H->vheap->base; i < limit && pages > 0 ; i+= step, pages--) dummy += *i; } return dummy;
------------------------------------------------------------------------------ The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com _______________________________________________ Monetdb-checkins mailing list Monetdb-checkins@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
-- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |