On Sat, Dec 1, 2012 at 4:30 PM,
<users-list-request@monetdb.org> wrote:
Send users-list mailing list submissions to
users-list@monetdb.org
To subscribe or unsubscribe via the World Wide Web, visit
http://mail.monetdb.org/mailman/listinfo/users-list
or, via email, send a message with subject or body 'help' to
users-list-request@monetdb.org
You can reach the person managing the list at
users-list-owner@monetdb.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of users-list digest..."
Today's Topics:
1. Same SQL Statement with different throughput time. (swapnil joshi)
2. Re: Same SQL Statement with different throughput time.
(Charalampos Nikolaou)
3. Re: Same SQL Statement with different throughput time. (Lefteris)
4. Re: Same SQL Statement with different throughput time.
(Stefan Manegold)
----------------------------------------------------------------------
Message: 1
Date: Fri, 30 Nov 2012 16:59:57 +0530
From: swapnil joshi <webmaster.swapnil@gmail.com>
To: Stefan.Manegold@cwi.nl, users-list@monetdb.org
Subject: Same SQL Statement with different throughput time.
Message-ID:
<CAFBccRCGURekT6sCL3SSvOGMWCBwaJv+kFD_bNnyVWEkmC8f3Q@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
Hi Stefan,
If monetdb doesn't maintain any caching then when i am creating test1
database
which contain demo table which contain 100 million records.
When first time i am executing "SELECT SUM(V1) FROM DEMO;"
monetdb gives me result in (11.4s)
when immediate i am trying to execute same query "SELECT SUM(V1) FROM DEMO;"
It gives me result in (31.065ms)
How / why this happen?
I have also checked my OS memory cache and I have erased OS Memory cache.
Now, I am trying same query "SELECT SUM(V1) FROM DEMO;"
now also he take same time in milliseconds approximate(30.723ms)
why he first time take lot of time and every next time he take negligible
time.
I hope you will clear my problem related of throughput time.
--
Regards,
Swapnil K. Joshi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.monetdb.org/pipermail/users-list/attachments/20121130/49651049/attachment.html>
------------------------------
Message: 2
Date: Fri, 30 Nov 2012 13:42:27 +0200
From: Charalampos Nikolaou <charnik@di.uoa.gr>
To: Communication channel for MonetDB users <users-list@monetdb.org>
Subject: Re: Same SQL Statement with different throughput time.
Message-ID:
<CAGVKX=fX5zwht8h3rs=oPXu5GidifcB-tT+=ovWiREqhepxfhg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
I think the reason is that for the first query the time is spent
mainly for loading your table in main memory.
Then, the second query has only the overhead of computing the sum over
column V1.
To see this, you can run the query, restart monetdbd, and then run the
query again. You should get similar performance to that of the first
run.
Babis
On Fri, Nov 30, 2012 at 1:29 PM, swapnil joshi
<webmaster.swapnil@gmail.com> wrote:
> Hi Stefan,
> If monetdb doesn't maintain any caching then when i am creating test1
> database
> which contain demo table which contain 100 million records.
>
> When first time i am executing "SELECT SUM(V1) FROM DEMO;"
> monetdb gives me result in (11.4s)
> when immediate i am trying to execute same query "SELECT SUM(V1) FROM DEMO;"
> It gives me result in (31.065ms)
>
> How / why this happen?
>
> I have also checked my OS memory cache and I have erased OS Memory cache.
> Now, I am trying same query "SELECT SUM(V1) FROM DEMO;"
> now also he take same time in milliseconds approximate(30.723ms)
>
> why he first time take lot of time and every next time he take negligible
> time.
>
> I hope you will clear my problem related of throughput time.
>
> --
> Regards,
> Swapnil K. Joshi
>
>
> _______________________________________________
> users-list mailing list
> users-list@monetdb.org
> http://mail.monetdb.org/mailman/listinfo/users-list
>
------------------------------
Message: 3
Date: Fri, 30 Nov 2012 12:58:20 +0100
From: Lefteris <lsidir@gmail.com>
To: Communication channel for MonetDB users <users-list@monetdb.org>
Subject: Re: Same SQL Statement with different throughput time.
Message-ID:
<CAO=W8YYnvgfNE40LjzvYMce2uTJ21G+GxYV38eAQnjqWULsjLQ@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
just restarting monetdb wont do it (OS will still keep in memory the data),
you will also need to clean the memory, for example alloc all memory space,
write 1's and then free again the memory.
Swapnil, which version of monetdb are you using?
IMO, 11 seconds vs. 30 milliseconds is too much of a difference to be just
I/O. But maybe I am wrong, maybe your dbfarm is on a network?
On Fri, Nov 30, 2012 at 12:42 PM, Charalampos Nikolaou <charnik@di.uoa.gr>wrote:
> I think the reason is that for the first query the time is spent
> mainly for loading your table in main memory.
> Then, the second query has only the overhead of computing the sum over
> column V1.
>
> To see this, you can run the query, restart monetdbd, and then run the
> query again. You should get similar performance to that of the first
> run.
>
> Babis
>
> On Fri, Nov 30, 2012 at 1:29 PM, swapnil joshi
> <webmaster.swapnil@gmail.com> wrote:
> > Hi Stefan,
> > If monetdb doesn't maintain any caching then when i am creating
> test1
> > database
> > which contain demo table which contain 100 million records.
> >
> > When first time i am executing "SELECT SUM(V1) FROM DEMO;"
> > monetdb gives me result in (11.4s)
> > when immediate i am trying to execute same query "SELECT SUM(V1) FROM
> DEMO;"
> > It gives me result in (31.065ms)
> >
> > How / why this happen?
> >
> > I have also checked my OS memory cache and I have erased OS Memory cache.
> > Now, I am trying same query "SELECT SUM(V1) FROM DEMO;"
> > now also he take same time in milliseconds approximate(30.723ms)
> >
> > why he first time take lot of time and every next time he take negligible
> > time.
> >
> > I hope you will clear my problem related of throughput time.
> >
> > --
> > Regards,
> > Swapnil K. Joshi
> >
> >
> > _______________________________________________
> > users-list mailing list
> > users-list@monetdb.org
> > http://mail.monetdb.org/mailman/listinfo/users-list
> >
> _______________________________________________
> users-list mailing list
> users-list@monetdb.org
> http://mail.monetdb.org/mailman/listinfo/users-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.monetdb.org/pipermail/users-list/attachments/20121130/ccf3f583/attachment.html>
------------------------------
Message: 4
Date: Fri, 30 Nov 2012 13:25:12 +0100 (CET)
From: Stefan Manegold <Stefan.Manegold@cwi.nl>
To: "=?utf-8?B?Q29tbXVuaWNhdGlvbiBjaGFubmVsIGZvciBNb25ldERCIHVzZXJz?="
<users-list@monetdb.org>
Subject: Re: Same SQL Statement with different throughput time.
Message-ID: <FDD9B5D3-5D99-4792-A2FF-71D12F90E8EA@cwi.nl>
Content-Type: text/plain; charset="utf-8"
I guess it's a good idea to run your query in both cases, slow & fast prefixed with TRACE to see where time is spent.
BTW, are we talking about genuine MonetDB (as Letters asked: which version) and hence original MonetDB SUM()-implementation, or has the code been modified or extended; f the latter: where & how?
S.
--
| Stefan.Manegold @ CWI.nl | DB Architectures (INS1) |
| http://CWI.nl/~manegold/ | Science Park 123 (L321) |
| Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |
----- Reply message -----
From: "Lefteris" <lsidir@gmail.com>
To: "Communication channel for MonetDB users" <users-list@monetdb.org>
Subject: Same SQL Statement with different throughput time.
Date: Fri, Nov 30, 2012 12:58
just restarting monetdb wont do it (OS will still keep in memory the data),
you will also need to clean the memory, for example alloc all memory space,
write 1's and then free again the memory.
Swapnil, which version of monetdb are you using?
IMO, 11 seconds vs. 30 milliseconds is too much of a difference to be just
I/O. But maybe I am wrong, maybe your dbfarm is on a network?
On Fri, Nov 30, 2012 at 12:42 PM, Charalampos Nikolaou <charnik@di.uoa.gr>wrote:
> I think the reason is that for the first query the time is spent
> mainly for loading your table in main memory.
> Then, the second query has only the overhead of computing the sum over
> column V1.
>
> To see this, you can run the query, restart monetdbd, and then run the
> query again. You should get similar performance to that of the first
> run.
>
> Babis
>
> On Fri, Nov 30, 2012 at 1:29 PM, swapnil joshi
> <webmaster.swapnil@gmail.com> wrote:
> > Hi Stefan,
> > If monetdb doesn't maintain any caching then when i am creating
> test1
> > database
> > which contain demo table which contain 100 million records.
> >
> > When first time i am executing "SELECT SUM(V1) FROM DEMO;"
> > monetdb gives me result in (11.4s)
> > when immediate i am trying to execute same query "SELECT SUM(V1) FROM
> DEMO;"
> > It gives me result in (31.065ms)
> >
> > How / why this happen?
> >
> > I have also checked my OS memory cache and I have erased OS Memory cache.
> > Now, I am trying same query "SELECT SUM(V1) FROM DEMO;"
> > now also he take same time in milliseconds approximate(30.723ms)
> >
> > why he first time take lot of time and every next time he take negligible
> > time.
> >
> > I hope you will clear my problem related of throughput time.
> >
> > --
> > Regards,
> > Swapnil K. Joshi
> >
> >
> > _______________________________________________
> > users-list mailing list
> > users-list@monetdb.org
> > http://mail.monetdb.org/mailman/listinfo/users-list
> >
> _______________________________________________
> users-list mailing list
> users-list@monetdb.org
> http://mail.monetdb.org/mailman/listinfo/users-list
>
------------------------------
_______________________________________________
users-list mailing list
users-list@monetdb.org
http://mail.monetdb.org/mailman/listinfo/users-list
End of users-list Digest, Vol 4, Issue 1
****************************************