[MonetDB-users] Slow import from a SQL (20 tps)
I'm evaluating MonetDB and importing an SQL file containing INSERT statements, so that I can populate the following table : CREATE TABLE session ( cmd character varying(25), date bigint, session_id character varying(64), score integer, src_ip character varying(25), dst_ip character varying(25), tx_bytes integer, rx_bytes integer, retx_bytes integer, scrambling character varying(16), url character varying(64), country character varying(32), orig character varying(3), status integer ) No index was created.
From what I can see, I'm somewhere between 20 and 40 insertions per second (Core i5, 4GB RAM, Windows7, MonetDb MonetDB5-SQL-Installer-i386-20111021.msi).
The Windows task manager shows a disk write throughput of 300 kB/s which looks like very small, processor usage around 5%. I imported the file with mclient -u monetdb < myfile.sql I expected to import at 200 rows/second minimum. Am I doing something wrong ? What can be done to improve the loading speed ? Thank you SC
You should consider turning off auto-commit or use the COPY INTO statement . On 1/18/12 2:17 PM, Sékine Coulibaly wrote:
I’m evaluating MonetDB and importing an SQL file containing INSERT statements, so that I can populate the following table :
CREATE TABLE session
(
cmd character varying(25),
date bigint,
session_id character varying(64),
score integer,
src_ip character varying(25),
dst_ip character varying(25),
tx_bytes integer,
rx_bytes integer,
retx_bytes integer,
scrambling character varying(16),
url character varying(64),
country character varying(32),
orig character varying(3),
status integer
)
No index was created.
From what I can see, I’m somewhere between 20 and 40 insertions per second (Core i5, 4GB RAM, Windows7, MonetDb MonetDB5-SQL-Installer-i386-20111021.msi).
The Windows task manager shows a disk write throughput of 300 kB/s which looks like very small, processor usage around 5%.
I imported the file with mclient –u monetdb < myfile.sql
I expected to import at 200 rows/second minimum. Am I doing something wrong ?
What can be done to improve the loading speed ?
Thank you
SC
------------------------------------------------------------------------------ Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d
_______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
alternative use: mclient -u monetdb myfile.sql On 1/18/12 5:00 PM, Martin Kersten wrote:
You should consider turning off auto-commit or use the COPY INTO statement .
On 1/18/12 2:17 PM, Sékine Coulibaly wrote:
I’m evaluating MonetDB and importing an SQL file containing INSERT statements, so that I can populate the following table :
CREATE TABLE session
(
cmd character varying(25),
date bigint,
session_id character varying(64),
score integer,
src_ip character varying(25),
dst_ip character varying(25),
tx_bytes integer,
rx_bytes integer,
retx_bytes integer,
scrambling character varying(16),
url character varying(64),
country character varying(32),
orig character varying(3),
status integer
)
No index was created.
From what I can see, I’m somewhere between 20 and 40 insertions per second (Core i5, 4GB RAM, Windows7, MonetDb MonetDB5-SQL-Installer-i386-20111021.msi).
The Windows task manager shows a disk write throughput of 300 kB/s which looks like very small, processor usage around 5%.
I imported the file with mclient –u monetdb< myfile.sql
I expected to import at 200 rows/second minimum. Am I doing something wrong ?
What can be done to improve the loading speed ?
Thank you
SC
------------------------------------------------------------------------------ Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d
_______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
------------------------------------------------------------------------------ Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
participants (2)
-
Martin Kersten -
Sékine Coulibaly