[MonetDB-users] Monetdb5 threading question.
When I start the default demo database (or a new created one) process on Windows XP, the following messaged appears in the command window : # Serving databse 'demo', using 1 thread Does it mean the whole database process is single-threaded? The following section in the monetdb5.config file shows we can config multiple threads # Mserver 5 recognizes a series of command line names instead:# --threads ( 1 | PARMASK)# --memory (MEMMASK )# --properties (CHECKMASK | PROPMASK | BATMASK )# --io (IOMASK | PERFMASK )# --transactions ( TMMASK | DELTAMASK | TEMMASK)# --modules (DLMASK | LOADMASK)# --algorithms (ALGOMASK | ESTIMASK)# --xproperties (XPROPMASK )# --performance (JOINPROPMASK | DEADBEEFMASK) What does 'parmsk' mean, and how to config monetdb to use multiple threads? _________________________________________________________________ More than messages–check out the rest of the Windows Live™. http://www.microsoft.com/windows/windowslive/
小波 顾 wrote:
When I start the default demo database (or a new created one) process on Windows XP, the following messaged appears in the command window : # Serving databse 'demo', using 1 thread
Does it mean the whole database process is single-threaded? no, it has detected a single core machine. We need to make the message more explicit. The following section in the monetdb5.config file shows we can config multiple threads
These are debugging flags only
# Mserver 5 recognizes a series of command line names instead: # --threads ( 1 | PARMASK) # --memory (MEMMASK ) # --properties (CHECKMASK | PROPMASK | BATMASK ) # --io (IOMASK | PERFMASK ) # --transactions ( TMMASK | DELTAMASK | TEMMASK) # --modules (DLMASK | LOADMASK) # --algorithms (ALGOMASK | ESTIMASK) # --xproperties (XPROPMASK ) # --performance (JOINPROPMASK | DEADBEEFMASK)
What does 'parmsk' mean, and how to config monetdb to use multiple threads?
------------------------------------------------------------------------ check out the rest of the Windows Live™. More than mail–Windows Live™ goes way beyond your inbox. More than messages http://www.microsoft.com/windows/windowslive/ ------------------------------------------------------------------------
------------------------------------------------------------------------------
------------------------------------------------------------------------
_______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
But the machine I am using have two cores.> Date: Fri, 2 Jan 2009 09:37:42 +0100> From: Martin.Kersten@cwi.nl> To: monetdb-users@lists.sourceforge.net> Subject: Re: [MonetDB-users] Monetdb5 threading question.> > 小波 顾 wrote:> > When I start the default demo database (or a new created one) process > > on Windows XP, the following messaged appears in the command window :> > # Serving databse 'demo', using 1 thread> > > > Does it mean the whole database process is single-threaded?> no, it has detected a single core machine. We need to make the message > more explicit.> > The following section in the monetdb5.config file shows we can config > > multiple threads> > > These are debugging flags only> > # Mserver 5 recognizes a series of command line names instead:> > # --threads ( 1 | PARMASK)> > # --memory (MEMMASK )> > # --properties (CHECKMASK | PROPMASK | BATMASK )> > # --io (IOMASK | PERFMASK )> > # --transactions ( TMMASK | DELTAMASK | TEMMASK)> > # --modules (DLMASK | LOADMASK)> > # --algorithms (ALGOMASK | ESTIMASK)> > # --xproperties (XPROPMASK )> > # --performance (JOINPROPMASK | DEADBEEFMASK)> > > > What does 'parmsk' mean, and how to config monetdb to use multiple > > threads?> >> > > >> > ------------------------------------------------------------------------> > check out the rest of the Windows Live™. More than mail–Windows Live™ > > goes way beyond your inbox. More than messages > > http://www.microsoft.com/windows/windowslive/> > ------------------------------------------------------------------------> >> > ------------------------------------------------------------------------------> > > > ------------------------------------------------------------------------> >> > _______________________________________________> > MonetDB-users mailing list> > MonetDB-users@lists.sourceforge.net> > https://lists.sourceforge.net/lists/listinfo/monetdb-users> > > > > ------------------------------------------------------------------------------> _______________________________________________> MonetDB-users mailing list> MonetDB-users@lists.sourceforge.net> https://lists.sourceforge.net/lists/listinfo/monetdb-users _________________________________________________________________ Show them the way! Add maps and directions to your party invites. http://www.microsoft.com/windows/windowslive/events.aspx
On Fri, Jan 02, 2009 at 04:28:31PM +0800, 小波 顾 wrote:
When I start the default demo database (or a new created one) process on Windows XP, the following messaged appears in the command window : # Serving databse 'demo', using 1 thread
Does it mean the whole database process is single-threaded? The following section in the monetdb5.config file shows we can config multiple threads
# Mserver 5 recognizes a series of command line names instead:# --threads ( 1 | PARMASK)# --memory (MEMMASK )# --properties (CHECKMASK | PROPMASK | BATMASK )# --io (IOMASK | PERFMASK )# --transactions ( TMMASK | DELTAMASK | TEMMASK)# --modules (DLMASK | LOADMASK)# --algorithms (ALGOMASK | ESTIMASK)# --xproperties (XPROPMASK )# --performance (JOINPROPMASK | DEADBEEFMASK)
What does 'parmsk' mean, and how to config monetdb to use multiple threads?
If you read the whole monetdb5.conf file, you'll find ======== [...] # You can enable debug for specific parts. By default debug is # switched off. gdk_debug=0 # possible values: (can be a combination of) # 1 = thread-specific debug output # 2 = CHECKMASK = property enforcing on new BATs # 4 = MEMMASK = memory allocation # 8 = PROPMASK = property checking on all values: tells about # wrongly set properties # 16 = IOMASK = major IO activity # 32 = BATMASK = BAT handling # 64 = PARSEMASK = parser debugging # 128 = PARMASK = Thread management ^^^ ^^^^^^^ ^^^^^^^^^^^^^^^^^ # 256 = TRGMASK = Event management # 512 = TMMASK = Transaction management # 1024 = TEMMASK = Locks and Triggers # 2048 = DLMASK = Dynamic loading # 4096 = PERFMASK = BBP Performance (?) # 8192 = DELTAMASK = Delta debugging (?) # 16384 = LOADMASK = Module loading # 32768 = YACCMASK = Yacc specific error messages # 65536 = obsolete = tcpip (??) # 131072 = obsolete = monet_multiplex: setaggr debugging (??) # 262144 = obsolete = \ # 524288 = obsolete = ddbench (???) # 1048576 = obsolete = / # 2097152 = ALGOMASK = show join/select algorithm chosen # 4194304 = ESTIMASK = show result size estimations (for join,select) # 8388608 = XPROPMASK = extended property checking: tells also about # not set properties #16777216 = JOINPROPMASK = disable property checking with join & outerjoin # (e.g., for performance measurements) #33554432 = DEADBEEFMASK = disable "cleaning" of freed memory in GDKfree() # (e.g., for performance measurements) # # Mserver 5 recognizes a series of command line names instead: # --threads ( 1 | PARMASK) # --memory (MEMMASK ) # --properties (CHECKMASK | PROPMASK | BATMASK ) # --io (IOMASK | PERFMASK ) # --transactions ( TMMASK | DELTAMASK | TEMMASK) # --modules (DLMASK | LOADMASK) # --algorithms (ALGOMASK | ESTIMASK) # --xproperties (XPROPMASK ) # --performance (JOINPROPMASK | DEADBEEFMASK) [...] ======== Hence, the '--thread' option is "just" to get debug output about multi threading. MonetDB runs multi-threaded, but by default only uses one thread per client/query (multiple clients are served concurrently); to get parallel processing of a single query (makes sense only on multi-core SMP machines) you can set the number of threads on the mserver5 command line via `mserver5 --set gdk_nr_threads=<nr-threads> ...` or in the monetdb5.conf file via 'gdk_nr_threads=<nr-threads>' Stefan
_________________________________________________________________ More than messages–check out the rest of the Windows Live™. http://www.microsoft.com/windows/windowslive/ ------------------------------------------------------------------------------
_______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
-- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |
Yes , after I add gdk_nr_threads=2 to the config file, the mserver.exe says 2 threads are used, but it seems there are some problems: When I use SQuirrel SQL Client connect to my new database, the mserver.exe process' CPU usage is very high(98%) whenever a statement is sent to the server. the following sample stataments even runs for a second. create table t1 ( a int not null, b varchar(1024) ); create table t2 ( a int not null, c varchar(1024)); insert into t1 values(1, 'b1'); insert into t1 values(2, 'b2'); insert into t1 values(3, 'b3'); insert into t1 values(1, 'c1'); insert into t1 values(2, 'c2'); select t1.* ,t2.c from t1 left outer join t2 on t1.a = t2.a There is also an error message like " memory heap courput" which I can't re produce.
Date: Fri, 2 Jan 2009 09:44:32 +0100> From: Stefan.Manegold@cwi.nl> To: monetdb-users@lists.sourceforge.net> Subject: Re: [MonetDB-users] Monetdb5 threading question.> > On Fri, Jan 02, 2009 at 04:28:31PM +0800, 小波 顾 wrote:> > > > When I start the default demo database (or a new created one) process on Windows XP, the following messaged appears in the command window :> > # Serving databse 'demo', using 1 thread> > > > Does it mean the whole database process is single-threaded?> > The following section in the monetdb5.config file shows we can config multiple threads> > > > # Mserver 5 recognizes a series of command line names instead:# --threads ( 1 | PARMASK)# --memory (MEMMASK )# --properties (CHECKMASK | PROPMASK | BATMASK )# --io (IOMASK | PERFMASK )# --transactions ( TMMASK | DELTAMASK | TEMMASK)# --modules (DLMASK | LOADMASK)# --algorithms (ALGOMASK | ESTIMASK)# --xproperties (XPROPMASK )# --performance (JOINPROPMASK | DEADBEEFMASK)> > > > What does 'parmsk' mean, and how to config monetdb to use multiple threads?> > If you read the whole monetdb5.conf file, you'll find> > ========> [...]> # You can enable debug for specific parts. By default debug is> # switched off.> gdk_debug=0> # possible values: (can be a combination of)> # 1 = thread-specific debug output> # 2 = CHECKMASK = property enforcing on new BATs> # 4 = MEMMASK = memory allocation> # 8 = PROPMASK = property checking on all values: tells about> # wrongly set properties> # 16 = IOMASK = major IO activity> # 32 = BATMASK = BAT handling> # 64 = PARSEMASK = parser debugging > # 128 = PARMASK = Thread management> ^^^ ^^^^^^^ ^^^^^^^^^^^^^^^^^> # 256 = TRGMASK = Event management> # 512 = TMMASK = Transaction management> # 1024 = TEMMASK = Locks and Triggers> # 2048 = DLMASK = Dynamic loading> # 4096 = PERFMASK = BBP Performance (?)> # 8192 = DELTAMASK = Delta debugging (?)> # 16384 = LOADMASK = Module loading> # 32768 = YACCMASK = Yacc specific error messages> # 65536 = obsolete = tcpip (??)> # 131072 = obsolete = monet_multiplex: setaggr debugging (??)> # 262144 = obsolete = \> # 524288 = obsolete = ddbench (???)> # 1048576 = obsolete = /> # 2097152 = ALGOMASK = show join/select algorithm chosen> # 4194304 = ESTIMASK = show result size estimations (for join,select)> # 8388608 = XPROPMASK = extended property checking: tells also about> # not set properties> #16777216 = JOINPROPMASK = disable property checking with join & outerjoin> # (e.g., for performance measurements)> #33554432 = DEADBEEFMASK = disable "cleaning" of freed memory in GDKfree()> # (e.g., for performance measurements)> # > # Mserver 5 recognizes a series of command line names instead: > # --threads ( 1 | PARMASK)> # --memory (MEMMASK )> # --properties (CHECKMASK | PROPMASK | BATMASK )> # --io (IOMASK | PERFMASK )> # --transactions ( TMMASK | DELTAMASK | TEMMASK)> # --modules (DLMASK | LOADMASK)> # --algorithms (ALGOMASK | ESTIMASK)> # --xproperties (XPROPMASK ) > # --performance (JOINPROPMASK | DEADBEEFMASK)> [...]> ========> > Hence, the '--thread' option is "just" to get debug output about multi threading.> > MonetDB runs multi-threaded, but by default only uses one thread per> client/query (multiple clients are served concurrently); to get parallel> processing of a single query (makes sense only on multi-core SMP machines)> you can set the number of threads on the mserver5 command line via> > `mserver5 --set gdk_nr_threads=<nr-threads> ...`> > or in the monetdb5.conf file via> > 'gdk_nr_threads=<nr-threads>'> > > Stefan> > > _________________________________________________________________> > More than messages–check out the rest of the Windows Live™.> > http://www.microsoft.com/windows/windowslive/> > ------------------------------------------------------------------------------> > > _______________________________________________> > MonetDB-users mailing list> > MonetDB-users@lists.sourceforge.net> > https://lists.sourceforge.net/lists/listinfo/monetdb-users> > > -- > | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl |> | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ |> | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 |> | The Netherlands | Fax : +31 (20) 592-4312 |> > ------------------------------------------------------------------------------> _______________________________________________> MonetDB-users mailing list> MonetDB-users@lists.sourceforge.net> https://lists.sourceforge.net/lists/listinfo/monetdb-users
Invite your mail contacts to join your friends list with Windows Live Spaces. It's easy! http://spaces.live.com/spacesapi.aspx?wx_action=create&wx_url=/friends.aspx&mkt=en-us
participants (3)
-
Martin Kersten -
Stefan Manegold -
小波 顾