Extending core data types is not going to happen, I am not using the word never though:) For the cases you describe you will have to do wrappers around the types that monetdb's kernel (aka gdk) provides you with. To give you an example, imagine gdk being your c compiler and monetdb5 modules being your c programs. You would not ask the c compiler to support a new data type, rather you would create a struct in your c program that wraps that new type you need. Of course there are performance penalties, but it is a trade-off: if we made all types part of gdk and then again you would have performance issues. Now there are many ways to do things, and many times if it is done correctly you would see small to zero performance problems. For example if you need a 16 byte type, maybe you need to define 2 columns of 8 bytes each, and create monetdb5 procedures that understand this semantics by using the primitives provided by the gdk. As for your uint_32, maybe allowing a regular sorting of signed 32 bit ints and then flipping (in the monetdb5 wrapper procedure) the negatives at the end of the column would not be the optimal solution but close to it, since a flip of this kind would not cost too much, especially if the memory allocation is down correctly from the start. Just some quick thoughts, maybe I am missing something. lefteris On Fri, Mar 8, 2013 at 4:53 PM, Mike Gomez wrote:

Hi lefteris, Thanks for your advice.

i get your point. i have confirmed that OIDs arithmetic is unsigned type. but that again opens up a can of worms.

x32 bit is passing out and x64 bit is in. but considering each uint32 will now take 64 bit, that's double the space.It might not count much for 1 or 10 million rows, but when the row count increases, the system will hit the memory wall 50% earlier, (considering only this change).

also, your advice mentions years of stuff to get GDK atoms in place SO i will fall back to external atoms (monetdb5/modules/atoms). but suppose i want a datatype of say 12 or 16 bytes or even more, then how we do we get around it. Limited understanding of mine tells, it not into possible.so how do i put around that limitation or is it not possible !!

also,i have a question about external atoms. say i cast lng into few bit fields, and later on i want to sort the table according to bit fields. since inherent type is lng 9 as you have mentioned, arithmetic is done on lng), so again this is impossible! right?

is there work going to extend core data types ?

more pointers ??

Thanks. Mike

...

Hi Mike,

your second solution through the kernel is definitely not the right way to go. Data types in the kernel are first class citizens and used to optimize the code in many different ways. Adding a type there is inefficient, next to impossible if you do not have years of experience in monet, and 99.9% of the cases I can think of unnecessary.

The correct way to do it is through monetdb5/modules/atoms

But you have to expose the data type to the sql frontend too (I assume that is what you are using).

Your uint must be mapped to a datatype that monetdb kernel knows and has the same behaviour. That is oid's for this case (even if oid's may be larger than uints).

MonetDB does not use function pointers for comparisons in the kernel, so you can not just add another comparison function and expect everything to work out of the box. Rather you map your new type in monetdb5 to a type in the kernel that subsumes it.

Hope this helps.

cheers,

lefteris

On Fri, Mar 8, 2013 at 2:24 PM, Mike Gomez wrote:

...

Hi, i am working on monetdb so as it can be used efficiently by more users(myself included).However, the issue i seem to face ( as maybe others) is the data types.

for many weeks now,I have been going through the source code, understanding and learning it.Data types in monetdb can be extended in 2 types, external and core GDK.

external method problem ( maybe bug in monetdb) ========================

I have created new data type, say unsigned int 32 bit, using extensions (such as inet,color and so on).However, the bug is monetdb system doesn't use my comparison operation, instead uses the default inbuilt operators.

eg. in the uint mal file( MonetDB/monetdb5/modules/atoms/uint.mal), i have declared

-------------- atom uint:int; .. ..............skipped source code

however, when i sort the table, the system still places negative values ahead, means instead of using my declared comparison algos, it's using internal algos.

my question is how do i override that ( or is it even possible to do it).

core GDK method =============== After failing that, i thought lets add the core data type uint in the gdk_atoms.c. i modified the parsor(& everything else) and the system recognizes my data type.

however,when i run simple insert query such as below, i get insert into test1 values('2'); TypeException:user.s1_1[3]:'calc.uint' undefined in: _5:any := calc.uint(_4:str)

however, i have got completely stuck in the MAL operators and language. so, how do i get the algebra, operations and everything right, so it works?

Any pointers will help. let me know if something more is required.

i am working on this out of my interest,so eventually if i get it to work somehow, the code goes back to project.

Thanks for help. mike

_______________________________________________ developers-list mailing list developers-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/developers-list