Hello Stefan,

Thanks for your quick and detailed response. My interest in MonetDB is commercial, to be honest, I'm looking for a solution that helps me with ETL operations on NOAA datasets and probably from other agencies, that will be used in a commercial product we're making. "We" are a recently created startup (just 2 partners) and we're looking for a data storage solution that fits our needs, as cheap as possible.

Me, as the CTO, I'm responsible for evaluating the state-of-the-art in array databases out there. So far I've been looking at MonetDB, SciDB, Rasdaman and InifiniDB. Another option is go for a Hadoop cluster and use a NoSQL approach (Mongo, HBase, Cassandra, Redis, etc), but I'd like to try a more "scientific" data storage solution for this. This leads me to another question, actually...can MonetDB displayed as a cluster?

That said, it would be nice if MonetDB supports this kind of scientific formats (not just for NOAA) in the future, and if I can help in some way I'd love to.

Kind regards,
Alejandro


On Fri, Feb 7, 2014 at 9:56 AM, Stefan Manegold <Stefan.Manegold@cwi.nl> wrote:
Hi Alejandro,

first of all thank you very much for your interest in MonetDB!

The latest release (Feb2013-SP6) as well as the upcoming Jan2014 release of
MonetDB do unfortunately not support the formats you mentioned, yet.

We have some initial code to load NetCDF files into SciQL [1] arrays but for
now that is neither documented nor released, let alone "production ready".

MonetDB does currently not provide any analytical tools other than standard
SQL.  However, there is an integration of MonetDB with R [2] that might
provide you with what you need.

The idea of the MonetDB Data Vault framework [3,4] is that no preparation is
required, but the DBMS itself "knows" how to load specific file formats
(provided it has been "tought" once how to do so).

Having said that, NOAA sounds very interesting!

The development of specific features in MonetDB is mainly driven by our own
research focus, but also by concrete requests and needs of commercial users.

In case your background and intended use/deployment of MonetDB is in a
research or non-commercial environment, it might be an idea to look for
funding opportunities if research interests overlap.

In case your background and intended use/deployment of MonetDB is in a
commercial context, you might want to consider getting into contact with
MonetDB Solutions [5].

Please let us know, whether any of the above is suitable / interesting for
you, or whether you have different requirements.

Best,
Stefan


[1] http://www.sciql.org/
[2] http://monetr.r-forge.r-project.org/
[3] https://repository.cwi.nl/noauth/search/searchrepository.php?isneditor=all&id=21397
[4] https://repository.cwi.nl/noauth/search/searchrepository.php?isneditor=all&id=21469
[5] http://www.monetdbsolutions.com/


On Thu, Feb 06, 2014 at 12:58:37PM +0100, Alejandro Villamarin wrote:
> Hi everyone,
>
> I've got some questions regarding MonetDB and scientifc data...more
> precisely, I'd like to know if MonetDB is suitable for ETL on datasets
> coming from NOAA in the following formats: GRIB2, NetCDF and HDF5, which
> all are multdimensional value grid files (to put it simple). I've already
> installed MonetDB and played with the tutorial, but so far I looks like a
> traditional RDBMS to me (yet)...any insights in this direction? Should I
> deep dive into the reference Manual?
>
> Another question would be...could I directly insert the data or do I have
> to do any parsing before? Does MonetDB provide analytical tools to do, for
> instance, correlation on that type of datasets?
>
> Thanks in advance,
> Alejandro

> _______________________________________________
> users-list mailing list
> users-list@monetdb.org
> https://www.monetdb.org/mailman/listinfo/users-list


--
| Stefan.Manegold@CWI.nl | DB Architectures   (DA) |
|  www.CWI.nl/~manegold  | Science Park 123 (L321) |
|   +31 (0)20 592-4212   | 1098 XG Amsterdam  (NL) |
_______________________________________________
users-list mailing list
users-list@monetdb.org
https://www.monetdb.org/mailman/listinfo/users-list