Hello Stefan, Thanks for your quick and detailed response. My interest in MonetDB is commercial, to be honest, I'm looking for a solution that helps me with ETL operations on NOAA datasets and probably from other agencies, that will be used in a commercial product we're making. "We" are a recently created startup (just 2 partners) and we're looking for a data storage solution that fits our needs, as cheap as possible. Me, as the CTO, I'm responsible for evaluating the state-of-the-art in array databases out there. So far I've been looking at MonetDB, SciDB, Rasdaman and InifiniDB. Another option is go for a Hadoop cluster and use a NoSQL approach (Mongo, HBase, Cassandra, Redis, etc), but I'd like to try a more "scientific" data storage solution for this. This leads me to another question, actually...can MonetDB displayed as a cluster? That said, it would be nice if MonetDB supports this kind of scientific formats (not just for NOAA) in the future, and if I can help in some way I'd love to. Kind regards, Alejandro On Fri, Feb 7, 2014 at 9:56 AM, Stefan Manegold wrote:

Hi Alejandro,

first of all thank you very much for your interest in MonetDB!

The latest release (Feb2013-SP6) as well as the upcoming Jan2014 release of MonetDB do unfortunately not support the formats you mentioned, yet.

We have some initial code to load NetCDF files into SciQL [1] arrays but for now that is neither documented nor released, let alone "production ready".

MonetDB does currently not provide any analytical tools other than standard SQL. However, there is an integration of MonetDB with R [2] that might provide you with what you need.

The idea of the MonetDB Data Vault framework [3,4] is that no preparation is required, but the DBMS itself "knows" how to load specific file formats (provided it has been "tought" once how to do so).

Having said that, NOAA sounds very interesting!

The development of specific features in MonetDB is mainly driven by our own research focus, but also by concrete requests and needs of commercial users.

In case your background and intended use/deployment of MonetDB is in a research or non-commercial environment, it might be an idea to look for funding opportunities if research interests overlap.

In case your background and intended use/deployment of MonetDB is in a commercial context, you might want to consider getting into contact with MonetDB Solutions [5].

Please let us know, whether any of the above is suitable / interesting for you, or whether you have different requirements.

Best, Stefan

[1] http://www.sciql.org/ [2] http://monetr.r-forge.r-project.org/ [3] https://repository.cwi.nl/noauth/search/searchrepository.php?isneditor=all&id=21397 [4] https://repository.cwi.nl/noauth/search/searchrepository.php?isneditor=all&id=21469 [5] http://www.monetdbsolutions.com/

On Thu, Feb 06, 2014 at 12:58:37PM +0100, Alejandro Villamarin wrote:

...

Hi everyone,

I've got some questions regarding MonetDB and scientifc data...more precisely, I'd like to know if MonetDB is suitable for ETL on datasets coming from NOAA in the following formats: GRIB2, NetCDF and HDF5, which all are multdimensional value grid files (to put it simple). I've already installed MonetDB and played with the tutorial, but so far I looks like a traditional RDBMS to me (yet)...any insights in this direction? Should I deep dive into the reference Manual?

Another question would be...could I directly insert the data or do I have to do any parsing before? Does MonetDB provide analytical tools to do, for instance, correlation on that type of datasets?

Thanks in advance, Alejandro

...

_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list

-- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list