Hello, I too would be interested in TPC-H comparisons of MonetDB versus other open source databases like MySQL and PostgreSQL, in addition to commercial databases like Oracle. If anyone has done such comparisons, please let me know. Martin also mentioned that the plans are not currently optimized but that it is targeted for V5. Are any of these optimizations already in the V5 CVS code (correlated subqueries?)? Thanks in advance.

-----Original Message----- From: monetdb-users-bounces@lists.sourceforge.net [mailto:monetdb-users-bounces@lists.sourceforge.net] On Behalf Of Martin Kersten Sent: Saturday, September 16, 2006 7:39 AM To: monetdb-users@lists.sourceforge.net Subject: Re: [MonetDB-users] Comparing TPC-H benchmark results?

If I a remember correctly the scale factor is 0.01 in the testweb. Beware that some queries in TPC-H require a lot of intermediate space and the plans produced for MIL are not optimized. (It is the target for the MonetDB V5 release)

Actually, we would be interested in experiences in the upcoming TPC-DS, which is more modern and not well explored by any system.

Fabian Groffen wrote:

...

Hi,

No need to resubmit your question. It was received, but I'm sorry that we've been too busy to answer this question.

On 15-09-2006 21:25:15 +0200, Wouter Scherphof wrote:

...

Hi,

The TPC-H benchmark (http://www.tpc.org/tpch/default.asp) is part of the MonetDB SQL source and the benchmark is run during weekends, as I can see on the TestWeb

(http://monetdb.cwi.nl/testing/projects/monetdb/Current/sql/.mTests03

...

/times.html). How should I interpret the listed results? According to TPC, results should be stated as a "QphH@Size" value, where "QphH" is the number of queries per hour for the H benchmark, and "Size" is a "scaling factor". On the TestWeb, the results are stated as seconds of execution time.

The testweb tpc-h is very very very small, to keep both testweb small in diskspace, and to keep the test time reasonable on our slower testing machines.

...

Is it correct to compute the QphH value as follows: Queries 01-22 took 1.985 seconds, so QphH is (3600 / 1.985) * 22 = 39.899 ?

Well, the testweb framework does way more than just executing the test, so the performance time of the test might not be accurate. Also, you probably want to test with a higher workload, to get a more accurate result.

...

What scaling factor is used in the benchmark runs; is this 1? Is it possible to let the TestWeb run with scaling factor 100? Because that's the smallest Size for which results are published on the TPC web site.

It's very small. I don't recall how small, but very very small.

...

Did anyone yet try to determine the Price of the test platforms, following the TPC Pricing Spec? TPC-H results normally include a Price/Performance metric in the form of $/QphH@Size, and I guess MonetDB could score particularly well on this metric.

We never really did this. But, feel free to try and setup such test, we are very interested in the outcomes, of course.

I hope this answers your questions.

Regards

----------------------------------------------------------------------

...

--- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo

http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057& dat=121642 _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users --

moredata@fastmail.net -- http://www.fastmail.fm - IMAP accessible web-mail