[Monetdb-developers] Default of "--enable-strict/--disable-strict" in release branch
Dear all, I do see the point that we do not necessarily distribute released source tarballs with "--enable-strict" as default, as this might "unnecessarily" hinder compilation with compilers and/or environments that "we don't know yet". However, nightly testing does explicitly use --enable-strict both both the development trunk ("Current") and the release branch ("Stable") to ensure "correctness" of the code. This leads to the "inconvenient" situation that, during bug fixing, developers that do not explicitly use "--enable-strict" compile with the default "--disable-strict" and consequently might miss small (but "nasty") problems that then cause nightly testing to fail. To solve this "dilemma", I came up with the idea to make the default setting "more flexible" and "automatically detect" whether a developer is compiling from CVS or a user is compiling from source tarball(s). In the former case, the default is "--enable-strict", it the latter it is "--disable-strict". Here's my patch: ======== --- buildtools/conf/MonetDB.m4 24 May 2008 17:13:41 -0000 1.75.2.1 +++ buildtools/conf/MonetDB.m4 6 Jun 2008 06:14:32 -0000 @@ -17,7 +17,15 @@ dnl Defaults that differ between development trunk and release branch: AC_DEFUN([AM_MONETDB_DEFAULTS], [ -dft_strict=no +if test -f "$srcdir"/vertoo.data; then + dnl Developers compiling from CVS: + dnl default is --enable-strict + dft_strict=yes +else + dnl Users compiling from source tarball(s): + dnl default is --disable-strict + dft_strict=no +fi dft_assert=no dft_optimi=yes dft_warning=no ======== In case there are no (strong) objections, I'd like to check this in later today. Open question: shall we only do this on the release branch and keep "--enable-strict" as default in all cases in the development trunk, or shall we apply the same strategy o the development trunk, too? Yours, Stefan On Fri, Jun 06, 2008 at 06:45:45AM +0200, Testweb Compilation wrote:
opt_icol.c:376: warning: enumeration value 'la_dummy' not handled in switch
374 if (L(p)->kind == la_twig && 375 !PFprop_icol (p->prop, L(p)->sem.iter_item.item)) 376 switch (LL(p)->kind) { 377 378 case la_docnode:
/home/monet/testing/monet.GNU.32.32.d.2271/pathfinder/compiler/algebra/opt/opt_icol.c:376: warning: enumeration value 'la_string_join' not handled in switch /home/monet/testing/monet.GNU.32.32.d.2271/pathfinder/compiler/algebra/opt/opt_icol.c:376: warning: enumeration value 'la_dummy' not handled in switch make[8]: *** [libopt_la-opt_icol.lo] Error 1
http://www.ins.cwi.nl/~monet/permastore/2008-06-06/Stable/make-pathfinder-GN...
CVS committers: boncz@cwi.nl jan.rittinger@in.tum.de lsidir@cwi.nl manegold@cwi.nl sjoerd@cwi.nl zhang@cwi.nl
An identical error message was detected on the following systems: GNU.32.32.d-Debian4.0 GNU.32.32.d-Fedora6 GNU.32.32.d-Fedora8 GNU.64.32.d-Fedora8 GNU.64.64.d-Fedora8 GNU.64.64.s-Fedora8
-- At this time you cannot configure Testing mail, sorry You are receiving this mail because: - you checked in some code on pathfinder (Stable) - you are a fixed recipient
-- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |
On Fri, Jun 06, 2008 at 08:28:10AM +0200, Stefan Manegold wrote:
Dear all,
I do see the point that we do not necessarily distribute released source tarballs with "--enable-strict" as default, as this might "unnecessarily" hinder compilation with compilers and/or environments that "we don't know yet".
However, nightly testing does explicitly use --enable-strict both both the development trunk ("Current") and the release branch ("Stable") to ensure "correctness" of the code.
This leads to the "inconvenient" situation that, during bug fixing, developers that do not explicitly use "--enable-strict" compile with the default "--disable-strict" and consequently might miss small (but "nasty") problems that then cause nightly testing to fail.
To solve this "dilemma", I came up with the idea to make the default setting "more flexible" and "automatically detect" whether a developer is compiling from CVS or a user is compiling from source tarball(s). In the former case, the default is "--enable-strict", it the latter it is "--disable-strict".
Here's my patch: ======== --- buildtools/conf/MonetDB.m4 24 May 2008 17:13:41 -0000 1.75.2.1 +++ buildtools/conf/MonetDB.m4 6 Jun 2008 06:14:32 -0000 @@ -17,7 +17,15 @@ dnl Defaults that differ between development trunk and release branch: AC_DEFUN([AM_MONETDB_DEFAULTS], [ -dft_strict=no +if test -f "$srcdir"/vertoo.data; then + dnl Developers compiling from CVS: + dnl default is --enable-strict + dft_strict=yes +else + dnl Users compiling from source tarball(s): + dnl default is --disable-strict + dft_strict=no +fi dft_assert=no dft_optimi=yes dft_warning=no ========
In case there are no (strong) objections, I'd like to check this in later today.
Open question: shall we only do this on the release branch and keep "--enable-strict" as default in all cases in the development trunk, or shall we apply the same strategy o the development trunk, too?
Same strategy sounds good for the development trunk too. There the users of the tar.balls also may have other platforms. Niels
Yours, Stefan
On Fri, Jun 06, 2008 at 06:45:45AM +0200, Testweb Compilation wrote:
opt_icol.c:376: warning: enumeration value 'la_dummy' not handled in switch
374 if (L(p)->kind == la_twig && 375 !PFprop_icol (p->prop, L(p)->sem.iter_item.item)) 376 switch (LL(p)->kind) { 377 378 case la_docnode:
/home/monet/testing/monet.GNU.32.32.d.2271/pathfinder/compiler/algebra/opt/opt_icol.c:376: warning: enumeration value 'la_string_join' not handled in switch /home/monet/testing/monet.GNU.32.32.d.2271/pathfinder/compiler/algebra/opt/opt_icol.c:376: warning: enumeration value 'la_dummy' not handled in switch make[8]: *** [libopt_la-opt_icol.lo] Error 1
http://www.ins.cwi.nl/~monet/permastore/2008-06-06/Stable/make-pathfinder-GN...
CVS committers: boncz@cwi.nl jan.rittinger@in.tum.de lsidir@cwi.nl manegold@cwi.nl sjoerd@cwi.nl zhang@cwi.nl
An identical error message was detected on the following systems: GNU.32.32.d-Debian4.0 GNU.32.32.d-Fedora6 GNU.32.32.d-Fedora8 GNU.64.32.d-Fedora8 GNU.64.64.d-Fedora8 GNU.64.64.s-Fedora8
-- At this time you cannot configure Testing mail, sorry You are receiving this mail because: - you checked in some code on pathfinder (Stable) - you are a fixed recipient
-- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |
------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php _______________________________________________ Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
-- Niels Nes, Centre for Mathematics and Computer Science (CWI) Kruislaan 413, 1098 SJ Amsterdam, The Netherlands room C0.02, phone ++31 20 592-4098, fax ++31 20 592-4312 url: http://www.cwi.nl/~niels e-mail: Niels.Nes@cwi.nl
On 06-06-2008 08:49:45 +0200, Niels Nes wrote:
On Fri, Jun 06, 2008 at 08:28:10AM +0200, Stefan Manegold wrote:
To solve this "dilemma", I came up with the idea to make the default setting "more flexible" and "automatically detect" whether a developer is compiling from CVS or a user is compiling from source tarball(s). In the former case, the default is "--enable-strict", it the latter it is "--disable-strict".
Here's my patch: ======== --- buildtools/conf/MonetDB.m4 24 May 2008 17:13:41 -0000 1.75.2.1 +++ buildtools/conf/MonetDB.m4 6 Jun 2008 06:14:32 -0000 @@ -17,7 +17,15 @@ dnl Defaults that differ between development trunk and release branch: AC_DEFUN([AM_MONETDB_DEFAULTS], [ -dft_strict=no +if test -f "$srcdir"/vertoo.data; then + dnl Developers compiling from CVS: + dnl default is --enable-strict + dft_strict=yes +else + dnl Users compiling from source tarball(s): + dnl default is --disable-strict + dft_strict=no +fi
Would it be possible to do this check based on the existence of a .CVS directory?
In case there are no (strong) objections, I'd like to check this in later today.
Open question: shall we only do this on the release branch and keep "--enable-strict" as default in all cases in the development trunk, or shall we apply the same strategy o the development trunk, too?
Same strategy sounds good for the development trunk too. There the users of the tar.balls also may have other platforms.
It would make sense to me if the nightly tarballs, which include the superballs all compile with strict, optimize and assert being *disabled* by default, for a sane code distribution. Please note that for some compiler issues to catch, one has to use -O2 (--enable optimize?), however doing that makes the binary hard to debug, which results in a natural inconvenience here...
On Fri, Jun 06, 2008 at 08:59:35AM +0200, Fabian Groffen wrote:
On 06-06-2008 08:49:45 +0200, Niels Nes wrote:
On Fri, Jun 06, 2008 at 08:28:10AM +0200, Stefan Manegold wrote: [...]
Would it be possible to do this check based on the existence of a .CVS directory?
Would be. However, then it only detects cvs checkouts, but not cvs exports. I just used the same test that we already use to detect whether we need code generation tools (flex, bison, etc.), or not. I'll discuss with Sjoerd. [...]
Open question: shall we only do this on the release branch and keep "--enable-strict" as default in all cases in the development trunk, or shall we apply the same strategy o the development trunk, too?
Same strategy sounds good for the development trunk too. There the users of the tar.balls also may have other platforms.
Good. Less diff between Stable & Current make thinks even simpler.
It would make sense to me if the nightly tarballs, which include the superballs all compile with strict, optimize and assert being *disabled* by default, for a sane code distribution.
Please note that for some compiler issues to catch, one has to use -O2 (--enable optimize?), however doing that makes the binary hard to debug, which results in a natural inconvenience here...
Even without --enable-optimize the configure default (not our choice, but that of the makers of configure!) (at least for gcc, not the most rare compiler) is "-g -O2", hence, the default does not omit "-O2". Moreover, there is the issue of performance experiments/comparisons in case the default is not --enable-optimize . Let try to make a final decision "life" during MADAM on Monday. Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |
On Fri, Jun 06, 2008 at 09:12:39AM +0200, Stefan Manegold wrote:
On Fri, Jun 06, 2008 at 08:59:35AM +0200, Fabian Groffen wrote:
On 06-06-2008 08:49:45 +0200, Niels Nes wrote:
On Fri, Jun 06, 2008 at 08:28:10AM +0200, Stefan Manegold wrote: [...] It would make sense to me if the nightly tarballs, which include the superballs all compile with strict, optimize and assert being *disabled* by default, for a sane code distribution.
Please note that for some compiler issues to catch, one has to use -O2 (--enable optimize?), however doing that makes the binary hard to debug, which results in a natural inconvenience here...
Even without --enable-optimize the configure default (not our choice, but that of the makers of configure!) (at least for gcc, not the most rare compiler) is "-g -O2", hence, the default does not omit "-O2".
Moreover, there is the issue of performance experiments/comparisons in case the default is not --enable-optimize . Let try to make a final decision "life" during MADAM on Monday.
Of course, until then, suggestions, ideas, concers, etc. as input for the desicion are more than welcome to be shared and discussed on this list! Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |
On 06-06-2008 09:12:39 +0200, Stefan Manegold wrote:
It would make sense to me if the nightly tarballs, which include the superballs all compile with strict, optimize and assert being *disabled* by default, for a sane code distribution.
Please note that for some compiler issues to catch, one has to use -O2 (--enable optimize?), however doing that makes the binary hard to debug, which results in a natural inconvenience here...
Even without --enable-optimize the configure default (not our choice, but that of the makers of configure!) (at least for gcc, not the most rare compiler) is "-g -O2", hence, the default does not omit "-O2".
"-g -O2" is an autoconf default, used if you don't set CFLAGS, which is pretty much sane for a normal installation. If you want more, because you understand more (risk assessment, etc.) you can tweak your CFLAGS to suit your needs, e.g. "-march=native -pipe -O3" and way beyond that. That's how the autotools world works.
Moreover, there is the issue of performance experiments/comparisons in case the default is not --enable-optimize .
There is a difference between "user distribution" and "micro-detail-performance-experiment-hackery usage". The latter should be setup by someone who knows what he/she is actually doing.
Let try to make a final decision "life" during MADAM on Monday.
On Fri, Jun 06, 2008 at 09:24:24AM +0200, Fabian Groffen wrote:
On 06-06-2008 09:12:39 +0200, Stefan Manegold wrote:
It would make sense to me if the nightly tarballs, which include the superballs all compile with strict, optimize and assert being *disabled* by default, for a sane code distribution.
Please note that for some compiler issues to catch, one has to use -O2 (--enable optimize?), however doing that makes the binary hard to debug, which results in a natural inconvenience here...
Even without --enable-optimize the configure default (not our choice, but that of the makers of configure!) (at least for gcc, not the most rare compiler) is "-g -O2", hence, the default does not omit "-O2".
"-g -O2" is an autoconf default, used if you don't set CFLAGS, which is pretty much sane for a normal installation. If you want more, because you understand more (risk assessment, etc.) you can tweak your CFLAGS to suit your needs, e.g. "-march=native -pipe -O3" and way beyond that. That's how the autotools world works.
I know and agree. In addition, for know compilers and platforms, we provide an "agressive" setting that is tested and found working not only fast(?) but also correctly(!) via the --enable-optimize switch such that uses do not have to spend their time on finding these settings themselves, but still benefit from their gains in performance.
Moreover, there is the issue of performance experiments/comparisons in case the default is not --enable-optimize .
There is a difference between "user distribution" and "micro-detail-performance-experiment-hackery usage". The latter should be setup by someone who knows what he/she is actually doing.
Well, my view and intension has been and still is a bit more detailed, consisting of even three scenarios: 1) "developers debugging 'distribution'" This is the (unpackaged?) CVS code base without --enable-optimize, but with --enable-strict and --enable-assert as default. Maybe, we [cs]ould even consider making --endable-debug the default to get rid of the "-O2" with gcc to improve gdb-debuggability. 2) "users production distribution" Ready-to-use for users with all (save) perfoamce benefits, but no (default) need to debug the code line by line (let's face it: which *user* does understand out code in details?). This is (a) the source tarball(s) with defaults --disable-strict --disable-assert --disable-debug --enable-optimize (I don't consider --enable-optimize a "hack" but a well-tested setting for know compiles and scenarios, and hence want users to benefit from it) and (b) the binary packages compiled with the same defaults. "Power users" that need to debug the code should know how to use configure with explicit --disable-optimize --enable-debug --enable-assert. 3) "Performance geeks" Take any of the above defaults and use the available knobs (configre switches, CFLAGS settings, etc.) to tune to your desire/needs. We might want to look into our configure files, though, to check whether the way how we modify the CFALGS (overwrite, append, prepend) is convenient and/or correct or could be imporoved for both purposes. Suggestions and help are welcome. Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |
On 06-06-2008 10:06:19 +0200, Stefan Manegold wrote:
2) "users production distribution" Ready-to-use for users with all (save) perfoamce benefits, but no (default) need to debug the code line by line (let's face it: which *user* does understand out code in details?). This is (a) the source tarball(s) with defaults --disable-strict --disable-assert --disable-debug --enable-optimize (I don't consider --enable-optimize a "hack" but a well-tested setting for know compiles and scenarios, and hence want users to benefit from it) and (b) the binary packages compiled with the same defaults.
This is where we disagree. a) we should standardise, and not be another project that annoyingly thinks they know better than the very well thought of defaults b) with agressive optimisations, such as --enable-optimize, it is IMO quite wrong to distribute the resulting binary; you never know what CPU or situation someone is in If you want to know how your favourite distribution compiles packages, check `vim --version` to see that they use very mild optimisation settings (-mtune=generic -O2 -pipe).
We might want to look into our configure files, though, to check whether the way how we modify the CFALGS (overwrite, append, prepend) is convenient and/or correct or could be imporoved for both purposes. Suggestions and help are welcome.
I use --disable-everything and set CFLAGS and X_CFLAGS, and that works correctly enough.
Users should first and foremost find it easy to install and effective to build their applications. Power users, e.g. with very large database, complex queries, should be pointed towards a supportive documentation page to squeeze out more performance and the risk assesment. They are also the users we would like to know better. MySQL and Postgresql are also good examples based on low-default settings, ofcourse with all the consequences. Fabian Groffen wrote:
On 06-06-2008 10:06:19 +0200, Stefan Manegold wrote:
2) "users production distribution" Ready-to-use for users with all (save) perfoamce benefits, but no (default) need to debug the code line by line (let's face it: which *user* does understand out code in details?). This is (a) the source tarball(s) with defaults --disable-strict --disable-assert --disable-debug --enable-optimize (I don't consider --enable-optimize a "hack" but a well-tested setting for know compiles and scenarios, and hence want users to benefit from it) and (b) the binary packages compiled with the same defaults.
This is where we disagree. a) we should standardise, and not be another project that annoyingly thinks they know better than the very well thought of defaults b) with agressive optimisations, such as --enable-optimize, it is IMO quite wrong to distribute the resulting binary; you never know what CPU or situation someone is in
If you want to know how your favourite distribution compiles packages, check `vim --version` to see that they use very mild optimisation settings (-mtune=generic -O2 -pipe).
We might want to look into our configure files, though, to check whether the way how we modify the CFALGS (overwrite, append, prepend) is convenient and/or correct or could be imporoved for both purposes. Suggestions and help are welcome.
I use --disable-everything and set CFLAGS and X_CFLAGS, and that works correctly enough.
------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php _______________________________________________ Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
participants (4)
-
Fabian Groffen -
Martin Kersten -
Niels Nes -
Stefan Manegold