[MonetDB-users] Fwd: Re: DB Corruption

This is what one of my coworkers reports about the problem. both columns are varchar(24). We have confirmed that there are no null values. We have a theory… Approximately every 6 hours, monetdb stops responding to queries and starts thrashing the disk. This behavior has been confirmed without any inserts. Because of this behavior, we routinely stop and start the service. After a restart, the service returns to a normal state. This appears to be the core problem. Our guess is that we restarted the service in the middle of a massive import. We will make sure this doesn't happen again, however, we still cannot find any sort of resolution as to why monetdb routinely starts thrashing the disk (on the order of writing 150MB/s) without any inserts, only select queries. Sorry to change the topic, but we're moving on the assumption we corrupted our db by restarting the service in the middle of a massive "INSERT INTO FROM" query. Can you think of any reason why monetdb would need to write to the disk in such an aggressive manner while ONLY serving select queries? This appears to be the core problem. Thanks On Mar 26, 2012, at 9:23 AM, Joseph Brower wrote:

What would you like me to tell them?

-------- Original Message -------- Subject: Re: [MonetDB-users] DB Corruption Date: Sun, 25 Mar 2012 09:15:24 +0200 From: Martin Kersten Reply-To: Martin.Kersten@cwi.nl, Communication channel for MonetDB users To: Communication channel for MonetDB users

What is the column type? Is the situation repeatable using a fresh database?

regards, Martin

On 3/25/12 6:34 AM, Joseph Brower wrote:

...

We just checked. There are no nulls in the database at all. Additionally, there is only one column that has a different count.

Thanks,

Joseph Brower

On 03/24/2012 09:58 AM, Stefan Manegold wrote:

...

Joseph,

since you did not specify your queries in more detail, I assume they are as simple as (1) select count(*) from; and (2) select count(single_column) from;

In that case, you could try the following sanity check:

(3) select count(*) from where single_column is null;

and check whether that returns the difference between your counts.

If that indeed returns the difference between your global count(*) and count(single_column), this would confirm the existence of NULL values, and you'd need to invest which of the values in your CVS file have been interpreted as NULL and why.

In case the result of queries (2) and (3) do not add up to the number returned by query (1), you might want to check the output(s) of

TRACE select count(*) from; TRACE select count(single_column) from; TRACE select count(*) from where single_column is null;

(or have us check them, provided/once we find the time).

Stefan

On Sat, Mar 24, 2012 at 09:37:38AM -0600, Joseph Brower wrote:

...

There shouldn't be any nulls. The records were all added by a "Copy Into" command. The files that we imported had no nulls in it at all.

On 03/24/2012 01:57 AM, Sjoerd Mullender wrote:

...

On 2012-03-24 00:14, Joseph Brower wrote:

...

We were troubleshooting some odd results coming back from our database, and found that it's missing some entries from a column. This appears to be causing some sort of offset. Does anyone have any ideas on what can be done to resolve it or what may have caused it? Is there any information we could send that would be helpful in troubleshooting this or for a bug report?

If we do a count(*) and then a count('single_column') we are getting two different results (when they should be indentical) count (*) counts all rows, whereas count(single_column) only count non-null values. Are there any nulls in the single_column?

------------------------------------------------------------------------------ This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure

_______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users

---

This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users

------------------------------------------------------------------------------ This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users

------------------------------------------------------------------------------ This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users