HI All, While calculating monetdb size I am observing below : When I am inserting records to Monet DB using JDBC total memory and VM usage is going to 10 G + but when I see file system size of the database directory it is showing something like 1 GB . Is some compression coming to picture here while storing data in file system? Any pointers will be helpful. Regards, Ashish _______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list
Also it will be interesting to know if large memory usage is due to intermediate results then when can be expect them to be cleaned? ON statement and connection close?? Regards, Ashish From: Ashish Kumar Singh Sent: Saturday, November 10, 2012 3:09 PM To: users-list@monetdb.org Cc: Ashish Kumar Singh Subject: Database Memory Usage HI All, While calculating monetdb size I am observing below : When I am inserting records to Monet DB using JDBC total memory and VM usage is going to 10 G + but when I see file system size of the database directory it is showing something like 1 GB . Is some compression coming to picture here while storing data in file system? Any pointers will be helpful. Regards, Ashish _______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list
Hi, I ran some test on a simple flat schema and below are results hinting at some bug! Vmem and RES are collected from top command. It seems when you close connection at that time only monetdb releases server side resources which is quite undesirable. All queries are insert queries sent using JDBC. Insertion Model Number of record Vmem (MB) RES System Monet DB Health Single Connection Multiple Statement 45603 3596m 2.8g Crash Single Connection Single Statement 69699 2508m 1.9g Crash Single Connection Single Statement Disposed and recreated after every 100 record 396980 713 M 66 M Good Health Single Connection Single Statement Disposed and recreated after every 100 record 800000 713 M 66 M Good Health Single Connection Single Statement Disposed and recreated after every 100 record 396980 773 M 67 M Good Health Single Connection Single Statement Disposed and recreated after every 100 record 8000000 1603 M 97 M Good Health Schema: CREATE TABLE rc_event( roe_type bigint, dc_servername varchar(50), roe_start_datetime timestamp , resources_used_vmem double, resource_list_vmem double, roe_source_id varchar(120), walltime_splitt bigint, ncpu_walltime bigint, custom_resource_1 bigint, custom_resource_2 bigint, custom_resource_3 bigint, custom_resource_4 bigint, custom_resource_5 bigint); I ran these test on CentOS 5.8 2 core 4 GB ram . Regards, Ashish From: Ashish Kumar Singh [mailto:ashishkumar.singh@altair.com] Sent: Saturday, November 10, 2012 5:28 PM To: users-list@monetdb.org Cc: ashishk@altair.com Subject: RE: Database Memory Usage Also it will be interesting to know if large memory usage is due to intermediate results then when can be expect them to be cleaned? ON statement and connection close?? Regards, Ashish From: Ashish Kumar Singh Sent: Saturday, November 10, 2012 3:09 PM To: users-list@monetdb.org Cc: Ashish Kumar Singh Subject: Database Memory Usage HI All, While calculating monetdb size I am observing below : When I am inserting records to Monet DB using JDBC total memory and VM usage is going to 10 G + but when I see file system size of the database directory it is showing something like 1 GB . Is some compression coming to picture here while storing data in file system? Any pointers will be helpful. Regards, Ashish _______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list
Hi, Any pointers on this? Regards, Ashish From: Ashish Kumar Singh [mailto:ashishkumar.singh@altair.com] Sent: Sunday, November 11, 2012 2:51 PM To: users-list@monetdb.org Cc: ashishk@altair.com Subject: RE: Database Memory Usage Hi, I ran some test on a simple flat schema and below are results hinting at some bug! Vmem and RES are collected from top command. It seems when you close connection at that time only monetdb releases server side resources which is quite undesirable. All queries are insert queries sent using JDBC. Insertion Model Number of record Vmem (MB) RES System Monet DB Health Single Connection Multiple Statement 45603 3596m 2.8g Crash Single Connection Single Statement 69699 2508m 1.9g Crash Single Connection Single Statement Disposed and recreated after every 100 record 396980 713 M 66 M Good Health Single Connection Single Statement Disposed and recreated after every 100 record 800000 713 M 66 M Good Health Single Connection Single Statement Disposed and recreated after every 100 record 396980 773 M 67 M Good Health Single Connection Single Statement Disposed and recreated after every 100 record 8000000 1603 M 97 M Good Health Schema: CREATE TABLE rc_event( roe_type bigint, dc_servername varchar(50), roe_start_datetime timestamp , resources_used_vmem double, resource_list_vmem double, roe_source_id varchar(120), walltime_splitt bigint, ncpu_walltime bigint, custom_resource_1 bigint, custom_resource_2 bigint, custom_resource_3 bigint, custom_resource_4 bigint, custom_resource_5 bigint); I ran these test on CentOS 5.8 2 core 4 GB ram . Regards, Ashish From: Ashish Kumar Singh [mailto:ashishkumar.singh@altair.com] Sent: Saturday, November 10, 2012 5:28 PM To: users-list@monetdb.org Cc: ashishk@altair.com Subject: RE: Database Memory Usage Also it will be interesting to know if large memory usage is due to intermediate results then when can be expect them to be cleaned? ON statement and connection close?? Regards, Ashish From: Ashish Kumar Singh Sent: Saturday, November 10, 2012 3:09 PM To: users-list@monetdb.org Cc: Ashish Kumar Singh Subject: Database Memory Usage HI All, While calculating monetdb size I am observing below : When I am inserting records to Monet DB using JDBC total memory and VM usage is going to 10 G + but when I see file system size of the database directory it is showing something like 1 GB . Is some compression coming to picture here while storing data in file system? Any pointers will be helpful. Regards, Ashish _______________________________________________ users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list
participants (1)
-
Ashish Kumar Singh