We are transferring a lot of data as the result of some of our queries (20-30 columns of integers and floats) for about 50,000 records, aggregating these from multiple MonetDB instances (on different machines).

For this pattern of query, the largest amount of time spent on the query (from the network onwards, ignoring Mserver5 lookup time) is on the data conversion.

When running “perf top” on our application, we notice we are spending a significant amount of time (the second most samples per perf top item at 14%) converting the data returned by libmapi from “const char *” to int64_t, int32_t, double, etc.  We know the type of the data, and we have very few string columns in the database.

Is there an option to MonetDB that allows it to pass binary data rather than textual data back and forth on the wire?  Is there an API that will give the data back in the native type rather than in the on-the-wire string type?  Both client and server are always running on 64-bit X86 boxes running Linux (meaning, little endian is guaranteed)?  We use serialized protobufs whenever we send data on the wire when communicating between our various programs, and the data stream is significantly smaller than sending integers back and forth in ascii.  If not, would switching to ODBC  / JDBC give us better performance in this area?

Thanks,

Dave