Hi Looking at the log it appears that your system might be confronted with memory fragmentation. Because GDKmmap fails for such small sizes. This indicates that probably another seemingly innocent malloc()also failed, for which there missed defensive code to take care of it. This may lead to a pointer dereference causing a segfault. A gdb trace may isolate the area. In the development branch a number of these tiny holes have been fixed and will appear in the next release. The easiest workaround for you would be to restart the server after a series of batch updates. On 19/08/16 03:47, 王彤 wrote:

Hi,

I try to synchronize data from mysql to monetdb. Total rows is about 15 million. I insert the rows in batch. Each batch has about 10000 rows. I use 'copy into from stdin' as in http://dev.monetdb.org/hg/MonetDB/file/tip/java/example/SQLcopyinto.java . The virtual memory continuously grows. When it grows to about 80G, the server is killed by signal SIGSEGV. The database size on disk is about 30GB.

Below is the server infomation and error log. Any help would be greatly appreciated!

OS version: centos-release-7-0.1406.el7.centos.2.5.x86_64 mserver5 --version MonetDB 5 server v11.23.7 "Jun2016-SP1" (64-bit, 64-bit oids, 128-bit integers) Copyright (c) 1993-July 2008 CWI Copyright (c) August 2008-2016 MonetDB B.V., all rights reserved Visit http://www.monetdb.org/ for further information Found 7.4GiB available memory, 2 available cpu cores Libraries: libpcre: 8.32 2012-11-30 (compiled with 8.32) openssl: OpenSSL 1.0.1e 11 Feb 2013 (compiled with OpenSSL 1.0.1e-fips 11 Feb 2013) libxml2: 2.9.1 (compiled with 2.9.1) Compiled by: root@jstu87d9w6zj4u (x86_64-unknown-linux-gnu) Compilation: gcc -g -O2 Linking : /usr/bin/ld -m elf_x86_64

in /data/dbfarm/merovingian.log 2016-08-18 17:29:04 ERR shomopcrm[10246]: #GDKmmap(38862848) fails, try to free up space [memory in use=98280928,virtual memory in use=3779503584] 2016-08-18 17:29:04 ERR shomopcrm[10246]: #GDKmmap(38862848) result [mem=98280928,vm=3779503584] 2016-08-18 17:29:04 MSG merovingian[10194]: proxying client 10.173.87.201:46953 for database 'shomopcrm' to mapi:monetdb:///data/dbfarm/shomopcrm/.mapi.sock?database=shomopcrm 2016-08-18 17:29:04 MSG merovingian[10194]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying 2016-08-18 17:29:04 MSG merovingian[10194]: proxying client 10.153.193.249:55915 for database 'shomopcrm' to mapi:monetdb:///data/dbfarm/shomopcrm/.mapi.sock?database=shomopcrm 2016-08-18 17:29:04 MSG merovingian[10194]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying 2016-08-18 17:29:04 ERR shomopcrm[10246]: #GDKmmap(38862848) fails, try to free up space [memory in use=185008352,virtual memory in use=3840344288] 2016-08-18 17:29:04 ERR shomopcrm[10246]: #GDKmmap(38862848) result [mem=185008352,vm=3840344288] 2016-08-18 17:29:04 MSG merovingian[10194]: proxying client 10.173.87.201:46963 for database 'shomopcrm' to mapi:monetdb:///data/dbfarm/shomopcrm/.mapi.sock?database=shomopcrm 2016-08-18 17:29:04 MSG merovingian[10194]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying 2016-08-18 17:29:04 ERR shomopcrm[10246]: #GDKmmap(51838976) fails, try to free up space [memory in use=99455712,virtual memory in use=3766391520] 2016-08-18 17:29:04 ERR shomopcrm[10246]: #GDKmmap(51838976) result [mem=99455712,vm=3766391520] 2016-08-18 17:29:04 MSG merovingian[10194]: database 'shomopcrm' (10246) was killed by signal SIGSEGV 2016-08-18 17:29:04 MSG merovingian[10194]: starting database 'shomopcrm', up min/avg/max: 17s/15h/5d, crash average: 1.00 0.90 0.40 (23-11=12) 2016-08-18 17:29:05 MSG shomopcrm[984]: arguments: /usr/local/bin/mserver5 --dbpath=/data/dbfarm/shomopcrm --set merovingian_uri=mapi:monetdb://jstu87d9w6zj4u:30001/shomopcrm --set mapi_open=false --set mapi_port=0 --set mapi_usock=/data/dbfarm/shomopcrm/.mapi.sock --set monet_vault_key=/data/dbfarm/shomopcrm/.vaultkey --set gdk_nr_threads=2 --set max_clients=64 --set sql_optimizer=default_pipe --set monet_daemon=yes 2016-08-18 17:29:05 MSG shomopcrm[984]: # MonetDB 5 server v11.23.7 "Jun2016-SP1" 2016-08-18 17:29:05 MSG shomopcrm[984]: # Serving database 'shomopcrm', using 2 threads 2016-08-18 17:29:05 MSG shomopcrm[984]: # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs and 128bit integers dynamically linked 2016-08-18 17:29:05 MSG shomopcrm[984]: # Found 7.390 GiB available main-memory. 2016-08-18 17:29:05 MSG shomopcrm[984]: # Copyright (c) 1993-July 2008 CWI. 2016-08-18 17:29:05 MSG shomopcrm[984]: # Copyright (c) August 2008-2016 MonetDB B.V., all rights reserved 2016-08-18 17:29:05 MSG shomopcrm[984]: # Visit http://www.monetdb.org/ for further information 2016-08-18 17:29:05 MSG shomopcrm[984]: # Listening for UNIX domain connection requests on mapi:monetdb:///data/dbfarm/shomopcrm/.mapi.sock 2016-08-18 17:29:05 MSG shomopcrm[984]: # MonetDB/SQL module loaded

in /var/log/messages, there is an error: kernel: mserver5[981]: segfault at 18 ip 00007fe10f845a89 sp 00007fcf38e79760 error 4 in lib_sql.so[7fe10f817000+175000]

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- wang.tong@shomop.com

_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list