I don't think BATs created in VIEWcreate_ should end up in GDKupgradevarheap. Those BATs are views, and I would think views are read-only, and read-only BATs should not end up here.
Yes, intuitively it seemed a bit unusual when I thought about it, but then again I am not much used to the MonetDB internals yet.
I had found GDKupgradevarheap being supplied with a capacity less than heap.size >> shift (it was capacity == 256, heap.size == 512 and shift == 0), and was wondering where it came from. So I added assert statements to all locations modifying capacity, shift or the heap allocation, which errored out in VIEWcreate_. It happened about the same time where I would have expected the GDKupgradevarheap issue to occur, so I found it most likely to be related.
Of course the problematic BAT might still come from another source, so if you say views won't reach GDKupgradevarheap, I'll probably go for another run without the assertions in VIEWcreate_. On the other hand, will it suffice to put assert(!isVIEW(b)) statements in front of the GDKupgradevarheap calls in the code to verify?
If you get to this situation again, and you happen to catch it in the debugger (inside GDKupgradevarheap), perhaps you can look at the c->parentid value. I expect it to be 0, meaning the column is not shared with another BAT (i.e. not a view). I'll try an assert to that effect and see what that gives in our tests.
My current setup makes it a bit impractical to attach a debugger, but I'm investigating further using asserts. I had an "assert(max_capacity == capacity)" to find out when it happens again, and now I have an "assert(c->parentid == 0 || capacity == max_capacity)" right in front of it, so I can inform you if the sizes are indeed mismatching with c->parentid being non-zero at the same time. Right now, my search for clues let me stumble on yet another issue for which I'm not sure if it's related: [1]. At least it always corrupts my data before I get to the point of investigating the heap upgrading issue. Best regards, Isidor Zeuner [1] http://sourceforge.net/tracker/?func=detail&aid=2938065&group_id=56967&atid=482468