Hi,

I am reading Monetdb code to understand the functionality of BAT Buffer Pool (BBP). I understood that it doesn’t allocate allocate memory for actual BAT but BATs are first instantiated on the HEAP memory using BATcreatedesc, BATnewstorage where BATcreatedesc also registers the entry of new BAT into BBP. BBP only allocates HEAP memory for maintaining the bookkeeping information for the BATs and keeping the directory structure information. I understood from the code that BBP keeps looking for the HOT/COLD state of BATs and performs unloading if memory is growing or descriptors are increasing using BBPTrim. Does this mean File descriptor here?

Questions:

1) I want to know if MonetDB allocates a large heap space internally then reuse that space for allocating memory chunks to BATs on demand or every time MonetDB ask operating system to extend heap size when new BAT is created?

2) I want to understand how BBP manages the HEAP memory. When a BAT is no longer needed, then what BBP does with that memory. Does it return that memory to OS and shrinks the heap size used by MonetDB or it holds that space for future allocations?

3) A typical TPCH query generates 100’s of intermediate BATs. Does MonetDB allocates memory for those BATs during query execution and returns it back to operating system when query finishes?

4) When we start MonetDB then how much initial heap memory is allocated by the MonetDB system. Does BBP allocates this memory on the HEAP? If not, then which part of the code knows which BATs to load at the start time and how much HEAP memory needs to be allocated for those BATs.

I found the list of all BATs in Monetdb through bbp.get() mal call. I also understand that SQL catalog maintains the mapping of table to column names and column names to BAT_in_BBP mapping. But I couldn’t able to find which BAT I should access through mal code to get the sql catalog mappings. I want to use MAL to see which table is composed of which column is linked to which BAT in BBP.

Thanks for the help.

Kind Regards, Ahmad