See http://www.monetdb.org/Documentation/Cookbooks/SQLrecipies/storage-model for a gross overview. On 1/18/13 5:37 PM, Hassan, Ahmad wrote:
Hi Martin,
Thanks for pointing out the relevant sections. I read some of your lecture slides on MonetDB (http://homepages.cwi.nl/~manegold/teaching/adt/lectures/lecture2.pdf) and it seems like BAT (Binary Association Tables) data structures are the relevant code regions for me to analyse. In the beginning, I am interested in analysing the main memory access pattern of BAT data structures for a database benchmark schema (i.e. TPC-H) and any temporary BAT structures that gets created during the workload execution.
So in order to focus only on this part, would you please point me to relevant codebase where the BAT structures for certain schema are initialised and declared?
After looking at the GDK kernel source, I can see that it has gdk_bat.c implementation and other database operator implementation. As my focus is on BAT structures memory access pattern, so it will really helpful if I can get the information about where BAT data structures for certain database schema and workload are declared and initialised. For example if I run TPC-H workload on monetDB, which part of the code initialises BAT structures for TPC-H schema and where temporary BAT structures (if any) are creating during query execution.
Notes on Building MonetDB under LLVM Framework:
At the moment I directly modified 'Makefile' of monetdbd and mserver5 source sub tree. Compiling and Instrumenting under LLVM framework involves the following steps:
1) Compile C/C++ files into LLVM bitcode files 2) Link individual LLVM bitcode files into one aggregated bitcode file 3) Apply LLVM based instrumentation on bitcode file 4) Perform linking with shared libraries and generate executable
So following the above sequence, I added the following rule in the monetdbd Makefile:
CC=clang CFLAGS = -g -O2 -emit-llvm -c $(X_CFLAGS) //-emit-llvm -c for generating bitcode instead of object files monetdbd_LLVMMERGE = llvm-link -o merge-$@.bc monetdbd_LLVMOPT = opt -o opt-$@.bc -load mypass.so -mypass < merge-$@.bc //Only required if adding instrumentation
monetdbd$(EXEEXT): $(monetdbd_OBJECTS) $(monetdbd_DEPENDENCIES) $(EXTRA_monetdbd_DEPENDENCIES) @rm -f monetdbd$(EXEEXT) @rm -f opt-$@.bc merge-$@.bc $(monetdbd_LLVMMERGE) $(monetdbd_OBJECTS) $(monetdbd_LLVMOPT) //Only required if adding instrumentation $(monetdbd_LINK) opt-$@.bc $(monetdbd_LDADD) $(LIBS)
At the moment, it is very hacky way of building monetdb under llvm but this approach can eventually go to 'configure' and 'Makefile.in' to add support for llvm based compilation.
Thanks.
Kind Regards, Ahmad
-----Original Message----- From: users-list-bounces+ahmad.hassan=sap.com@monetdb.org [mailto:users-list-bounces+ahmad.hassan=sap.com@monetdb.org] On Behalf Of Martin Kersten Sent: 18 January 2013 13:42 To: users-list@monetdb.org Subject: Re: MonetDB under LLVM
Hi Ahmad,
Such an analysis calls for a deep study of the code base, which you already have at your disposal. For datastructures related to all table operations you can start by studying the .../gdk library to raise from MonetDB naive to MonetDB informed user. Thereafter, the MAL interpreter becomes your target for studies. Once done, you might browse around in the SQL code base for further details on catalog datastructures.
Success, regards, Martin ps. As ever, the source code is your most detailed documentation.
On 1/18/13 1:09 PM, Hassan, Ahmad wrote:
Hi Team,
I have modified the monetdb build system to generate executable using LLVM Bitcode and added instrumentation rather than generating exe directly from Object files. At the moment the changes are limited to following executables:
monetdbd: tools/merovingian/daemon mserver5: tools/mserver
I have a question please. I want to analyse the storage manager of monetdb that implements the data structures for holding Table's data and Indexes. More precisely, I want to analyse all the data structures involved during Query processing explicitly i.e indexes, data structures holding tables data in main memory. May I ask which sub-tree of Monet source tree deals with that. Being naïve in monetdb internals, I assumed that monetdbd and mserver5 would be relevant because they are server side part of the code.
If I get any documentation on that, then it will be very useful.
Thanks.
Kind Regards, Ahmad
-----Original Message----- From: users-list-bounces+ahmad.hassan=sap.com@monetdb.org [mailto:users-list-bounces+ahmad.hassan=sap.com@monetdb.org] On Behalf Of Fabian Groffen Sent: 10 January 2013 17:35 To: users-list@monetdb.org Subject: Re: MonetDB under LLVM
On 10-01-2013 17:23:27 +0000, Hassan, Ahmad wrote:
Yes Fabian told me this earlier. I tried that option but it only uses 'clang' to compile the source code. It doesn't produce LLVM bitcode files. It produces standard object files.
A quick search with your favourite search engine would have told you that you'll need at least some intrusive changes to the build-system, and a different way of building.
Since you seem not to have reached that conclusion, I doubt whether you are the one to actually find out what you need to get it working. It is not of a concern to MonetDB, at the moment.
_______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list _______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list