[Monetdb-developers] MonetDB on UltraSPARC T1
Hello, It took me some time, and others might be interested in it: Below my experiences with (successfully) compiling MonetDB on an UltraSPARC T1 running Solaris 10. Appearantly the UltraSPARC T1 with its many slow cores, is not the ultimate system for MonetDB (which likes fast CPU's the most), but someone has to try it... I configured with "--enable-bits=64 --enable-oid32 --enable-shared --disable-static". (using gcc) By patching iso/math_c99.h according to http://gcc.gnu.org/ml/gcc-patches/2005-05/msg02123.html as Fabian suggested in MonetDB-bug: 1304078, the first hurdle was taken. The warnings below could be worked around by deleting lines 512 t/m 516 from src/gdk/gdk_utils.mx gcc -m64 -DHAVE_CONFIG_H -I. -I/usr/users/wouter/scratch/MonetDB-4.12/src/gdk -I../.. -I/usr/users/wouter/scratch/MonetDB-4.12/src/gdk -I../common -I/usr/users/wouter/scratch/MonetDB-4.12/src/gdk/../common -DLIBGDK -O2 -std=c99 -D__EXTENSIONS__ -Wall -W -fno-strict-aliasing -O2 -fomit-frame-pointer -finline-functions -DHWCOUNTERS -DHW_SunOS -DHW_sun4v -Werror-implicit-function-declaration -Werror -Wno-unused-function -Wno-unused-label -Wno-uninitialized -D_REENTRANT -c gdk_utils.c -fPIC -DPIC -o .libs/libbat_la-gdk_utils.o /usr/users/wouter/scratch/MonetDB-4.12/src/gdk/gdk_utils.mx: In function `GDKmem_heapcheck': /usr/users/wouter/scratch/MonetDB-4.12/src/gdk/gdk_utils.mx:513: warning: comparison of unsigned expression < 0 is always false /usr/users/wouter/scratch/MonetDB-4.12/src/gdk/gdk_utils.mx:513: warning: comparison of unsigned expression < 0 is always false /usr/users/wouter/scratch/MonetDB-4.12/src/gdk/gdk_utils.mx:513: warning: comparison of unsigned expression < 0 is always false gmake[5]: *** [libbat_la-gdk_utils.lo] Error 1 gmake[5]: Leaving directory `/usr/users/wouter/scratch/MonetDB-4.12/SunOS/src/gdk' gmake[4]: *** [all] Error 2 gmake[4]: Leaving directory `/usr/users/wouter/scratch/MonetDB-4.12/SunOS/src/gdk' gmake[3]: *** [all-recursive] Error 1 gmake[3]: Leaving directory `/usr/users/wouter/scratch/MonetDB-4.12/SunOS/src' gmake[2]: *** [all] Error 2 gmake[2]: Leaving directory `/usr/users/wouter/scratch/MonetDB-4.12/SunOS/src' gmake[1]: *** [all-recursive] Error 1 gmake[1]: Leaving directory `/usr/users/wouter/scratch/MonetDB-4.12/SunOS' gmake: *** [all] Error 2 MonetDB then compiled successfully, without warnings. After setting: LD_LIBRARY_PATH=/usr/sfw/lib/64/:/lib/64/ and installing GNU diffutils, make check reported: 1 out of 313 could not be executed, 11 SIGNIFICANTLY different... Among which: two tests that fail due to 64bits vs 32oids in src/gdk:casts (it seems that this test does not take into account the possible difference between bit- and oid-size), 1 java example failed: output was written to err instead of std, two due to timeouts, two failed because of some additional linebrakes in tcp-output, nothing very serious it seems. I have the mTests results available if anyone is interested in those. Pathfinder compiled without problems and make check reported: 15 (out of 1187) could not be executed, 5 SIGNIFICANTLY different). Then the results of some simple queries (on a document with 1.4M elements and 2.3M attributes): -bash-3.00$ time echo "count(doc(\"patrick.xml\")//file)" | MapiClient -lxq 89017 real 0m0.110s user 0m0.021s sys 0m0.034s -bash-3.00$ time echo "count(doc(\"patrick.xml\")//file[@name])" | MapiClient -lxq 89017 real 0m8.255s user 0m0.021s sys 0m0.034s -bash-3.00$ The result of the last query is (in my eyes) dramatic (more than 4 times as slow as a Xeon 2.8). The next step will be to compile with the native compiler, which I haven't done yet, and neither do I know whether I will succeed. Greetings, Wouter
Hi Wouter, thank you very much for sorting this out and for sharing your experiences with us! I immediately have some questions: (1) Which version of MonetDB & pathfinder did you use? (especially the exact revision of src/gdk/gdk_utils.mx would be interesting to know for all those who also need to delete lines 512 t/m 516 --- or try to fix them properly ;-)) (2) Which version of gcc do you uses? (On our Solaris 10/x86 machines, compilation works fine with gcc 3.4.5, but failes (don't recall the details right now) with gcc 3.4.3 ...) (3) Could you please send the timings of your sample queries that MapiClients reports with "-T"? (4) Would the "patrick.xml" document be available? (5) What are the detailed specifications of the hardware you used (next to #cores & clockspeed especially the amount of main memory). (6) Would the machine you used be accessable for us for nightly testing? (Sorry, couldn't resist asking ;-)) Thanks again, Stefan On Fri, Sep 08, 2006 at 03:13:18PM +0200, Wouter Alink (DT) wrote:
Hello,
It took me some time, and others might be interested in it:
Below my experiences with (successfully) compiling MonetDB on an UltraSPARC T1 running Solaris 10. Appearantly the UltraSPARC T1 with its many slow cores, is not the ultimate system for MonetDB (which likes fast CPU's the most), but someone has to try it...
I configured with "--enable-bits=64 --enable-oid32 --enable-shared --disable-static". (using gcc)
By patching iso/math_c99.h according to http://gcc.gnu.org/ml/gcc-patches/2005-05/msg02123.html as Fabian suggested in MonetDB-bug: 1304078, the first hurdle was taken.
The warnings below could be worked around by deleting lines 512 t/m 516 from src/gdk/gdk_utils.mx
gcc -m64 -DHAVE_CONFIG_H -I. -I/usr/users/wouter/scratch/MonetDB-4.12/src/gdk -I../.. -I/usr/users/wouter/scratch/MonetDB-4.12/src/gdk -I../common -I/usr/users/wouter/scratch/MonetDB-4.12/src/gdk/../common -DLIBGDK -O2 -std=c99 -D__EXTENSIONS__ -Wall -W -fno-strict-aliasing -O2 -fomit-frame-pointer -finline-functions -DHWCOUNTERS -DHW_SunOS -DHW_sun4v -Werror-implicit-function-declaration -Werror -Wno-unused-function -Wno-unused-label -Wno-uninitialized -D_REENTRANT -c gdk_utils.c -fPIC -DPIC -o .libs/libbat_la-gdk_utils.o /usr/users/wouter/scratch/MonetDB-4.12/src/gdk/gdk_utils.mx: In function `GDKmem_heapcheck': /usr/users/wouter/scratch/MonetDB-4.12/src/gdk/gdk_utils.mx:513: warning: comparison of unsigned expression < 0 is always false /usr/users/wouter/scratch/MonetDB-4.12/src/gdk/gdk_utils.mx:513: warning: comparison of unsigned expression < 0 is always false /usr/users/wouter/scratch/MonetDB-4.12/src/gdk/gdk_utils.mx:513: warning: comparison of unsigned expression < 0 is always false gmake[5]: *** [libbat_la-gdk_utils.lo] Error 1 gmake[5]: Leaving directory `/usr/users/wouter/scratch/MonetDB-4.12/SunOS/src/gdk' gmake[4]: *** [all] Error 2 gmake[4]: Leaving directory `/usr/users/wouter/scratch/MonetDB-4.12/SunOS/src/gdk' gmake[3]: *** [all-recursive] Error 1 gmake[3]: Leaving directory `/usr/users/wouter/scratch/MonetDB-4.12/SunOS/src' gmake[2]: *** [all] Error 2 gmake[2]: Leaving directory `/usr/users/wouter/scratch/MonetDB-4.12/SunOS/src' gmake[1]: *** [all-recursive] Error 1 gmake[1]: Leaving directory `/usr/users/wouter/scratch/MonetDB-4.12/SunOS' gmake: *** [all] Error 2
MonetDB then compiled successfully, without warnings.
After setting: LD_LIBRARY_PATH=/usr/sfw/lib/64/:/lib/64/ and installing GNU diffutils, make check reported: 1 out of 313 could not be executed, 11 SIGNIFICANTLY different... Among which: two tests that fail due to 64bits vs 32oids in src/gdk:casts (it seems that this test does not take into account the possible difference between bit- and oid-size), 1 java example failed: output was written to err instead of std, two due to timeouts, two failed because of some additional linebrakes in tcp-output, nothing very serious it seems. I have the mTests results available if anyone is interested in those.
Pathfinder compiled without problems and make check reported: 15 (out of 1187) could not be executed, 5 SIGNIFICANTLY different).
Then the results of some simple queries (on a document with 1.4M elements and 2.3M attributes):
-bash-3.00$ time echo "count(doc(\"patrick.xml\")//file)" | MapiClient -lxq 89017
real 0m0.110s user 0m0.021s sys 0m0.034s -bash-3.00$ time echo "count(doc(\"patrick.xml\")//file[@name])" | MapiClient -lxq 89017
real 0m8.255s user 0m0.021s sys 0m0.034s -bash-3.00$
The result of the last query is (in my eyes) dramatic (more than 4 times as slow as a Xeon 2.8). The next step will be to compile with the native compiler, which I haven't done yet, and neither do I know whether I will succeed.
Greetings, Wouter
------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
-- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |
I forgot to mention: it was MonetDB 4.12.1 from yesterdays CVS -bas-3.00$ gcc --version gcc (GCC) 3.4.3 (csl-sol210-3_4-branch+sol_rpath) The system is a Sun Firebird T2000, 8core (32 threads), 32GB (try-before-buy, so I don't know about nightly testing) Patrick.xml will not be available, but in an earlier bug-report I submitted a small script which could produce similar documents. I have to look it up. -T: 20ms in Trans, 8202ms in Query, 1ms in print. I think you also want to know whether something was swapped to disk: I tried MonetDB with --debug=16, and no IO was performed during the queries. And while I'm trying to compile with the native compiler, it correctly?? (at least after having a quick look at the code) complained about: gdk_scanselect.mx:346: never reached gdk_relop.mx:2032: never reached gdk_relop.mx:2185: never reached gdk_relop.mx:1226: never reached (forgot the exact line-number, but a similar case) gdk_bbp.mx:1450: function should return void. and when linking the src/modules/calibrator it complained about "CFLAGS = -O0" (commenting out this line in Makefile.ag did the trick) Haven't made it to a full compile yet... Grtz, Wouter
-----Original Message----- From: Stefan Manegold [mailto:Stefan.Manegold@cwi.nl] Sent: Friday, September 08, 2006 3:32 PM To: Wouter Alink (DT) Cc: monetdb-developers@lists.sourceforge.net Subject: Re: [Monetdb-developers] MonetDB on UltraSPARC T1
Hi Wouter,
thank you very much for sorting this out and for sharing your experiences with us!
I immediately have some questions:
(1) Which version of MonetDB & pathfinder did you use? (especially the exact revision of src/gdk/gdk_utils.mx would be interesting to know for all those who also need to delete lines 512 t/m 516 --- or try to fix them properly ;-))
(2) Which version of gcc do you uses? (On our Solaris 10/x86 machines, compilation works fine with gcc 3.4.5, but failes (don't recall the details right now) with gcc 3.4.3 ...)
(3) Could you please send the timings of your sample queries that MapiClients reports with "-T"?
(4) Would the "patrick.xml" document be available?
(5) What are the detailed specifications of the hardware you used (next to #cores & clockspeed especially the amount of main memory).
(6) Would the machine you used be accessable for us for nightly testing? (Sorry, couldn't resist asking ;-))
Thanks again,
Stefan
On Fri, Sep 08, 2006 at 03:13:18PM +0200, Wouter Alink (DT) wrote:
Hello,
It took me some time, and others might be interested in it:
Below my experiences with (successfully) compiling MonetDB on an UltraSPARC T1 running Solaris 10. Appearantly the UltraSPARC T1 with its many slow cores, is not the ultimate system for MonetDB (which likes fast CPU's the most), but someone has to try it...
I configured with "--enable-bits=64 --enable-oid32 --enable-shared --disable-static". (using gcc)
By patching iso/math_c99.h according to http://gcc.gnu.org/ml/gcc-patches/2005-05/msg02123.html as Fabian suggested in MonetDB-bug: 1304078, the first hurdle was taken.
The warnings below could be worked around by deleting lines 512 t/m 516 from src/gdk/gdk_utils.mx
gcc -m64 -DHAVE_CONFIG_H -I. -I/usr/users/wouter/scratch/MonetDB-4.12/src/gdk -I../.. -I/usr/users/wouter/scratch/MonetDB-4.12/src/gdk -I../common -I/usr/users/wouter/scratch/MonetDB-4.12/src/gdk/../common -DLIBGDK -O2 -std=c99 -D__EXTENSIONS__ -Wall -W -fno-strict-aliasing -O2 -fomit-frame-pointer -finline-functions -DHWCOUNTERS -DHW_SunOS -DHW_sun4v -Werror-implicit-function-declaration -Werror -Wno-unused-function -Wno-unused-label -Wno-uninitialized -D_REENTRANT -c gdk_utils.c -fPIC -DPIC -o .libs/libbat_la-gdk_utils.o /usr/users/wouter/scratch/MonetDB-4.12/src/gdk/gdk_utils.mx: In function `GDKmem_heapcheck': /usr/users/wouter/scratch/MonetDB-4.12/src/gdk/gdk_utils.mx:513: warning: comparison of unsigned expression < 0 is always false /usr/users/wouter/scratch/MonetDB-4.12/src/gdk/gdk_utils.mx:513: warning: comparison of unsigned expression < 0 is always false /usr/users/wouter/scratch/MonetDB-4.12/src/gdk/gdk_utils.mx:513: warning: comparison of unsigned expression < 0 is always false gmake[5]: *** [libbat_la-gdk_utils.lo] Error 1 gmake[5]: Leaving directory `/usr/users/wouter/scratch/MonetDB-4.12/SunOS/src/gdk' gmake[4]: *** [all] Error 2 gmake[4]: Leaving directory `/usr/users/wouter/scratch/MonetDB-4.12/SunOS/src/gdk' gmake[3]: *** [all-recursive] Error 1 gmake[3]: Leaving directory `/usr/users/wouter/scratch/MonetDB-4.12/SunOS/src' gmake[2]: *** [all] Error 2 gmake[2]: Leaving directory `/usr/users/wouter/scratch/MonetDB-4.12/SunOS/src' gmake[1]: *** [all-recursive] Error 1 gmake[1]: Leaving directory `/usr/users/wouter/scratch/MonetDB-4.12/SunOS' gmake: *** [all] Error 2
MonetDB then compiled successfully, without warnings.
After setting: LD_LIBRARY_PATH=/usr/sfw/lib/64/:/lib/64/ and installing GNU diffutils, make check reported: 1 out of 313 could not be executed, 11 SIGNIFICANTLY different... Among which: two tests that fail due to 64bits vs 32oids in src/gdk:casts (it seems that this test does not take into account the possible difference between bit- and oid-size), 1 java example failed: output was written to err instead of std, two due to timeouts, two failed because of some additional linebrakes in tcp-output, nothing very serious it seems. I have the mTests results available if anyone is interested in those.
Pathfinder compiled without problems and make check reported: 15 (out of 1187) could not be executed, 5 SIGNIFICANTLY different).
Then the results of some simple queries (on a document with 1.4M elements and 2.3M attributes):
-bash-3.00$ time echo "count(doc(\"patrick.xml\")//file)" | MapiClient -lxq 89017
real 0m0.110s user 0m0.021s sys 0m0.034s -bash-3.00$ time echo "count(doc(\"patrick.xml\")//file[@name])" | MapiClient -lxq 89017
real 0m8.255s user 0m0.021s sys 0m0.034s -bash-3.00$
The result of the last query is (in my eyes) dramatic (more than 4 times as slow as a Xeon 2.8). The next step will be to compile with the native compiler, which I haven't done yet, and neither do I know whether I will succeed.
Greetings, Wouter
----------------------------------------------------------------------
--- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=1216
42 _______________________________________________ Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
-- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | CWI, P.O.Box | 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |
participants (2)
-
Stefan Manegold -
Wouter Alink (DT)