On Mon, Nov 21, 2005 at 11:45:35AM +0100, Sjoerd Mullender wrote:
I guess it depends on the submitter. For some it is (almost) trivial to create a test script, and in any case, it helps the developper enormously if a small test can be made that shows the problem. In fact I would go so far as to say that for all real bugs (i.e. not compilation issues and typos and such) a small test is essential. I do think it is the developper's responsibility that the test gets added to the test web (that doesn't mean the developper has to do it, it means the developper must make sure it is done).
That exactly what I intended to say --- apparently, I need to work on the "efficiency" of my words ... ;-) Users and developers should not be "eminies" blaming each other, rather they should work together, and support each other as good as possible to solve bug with minimal overhead. The user know how to trigger the bug, hence (s)he should provide the developer with as much information as possible to resproduce the bug and thus check whether her/his fixed indeed solve the problem --- and a test query/script is the most efficient way to describe and reproduce a bug. Likewise, the developer is responsible for ensuring that his/her fix solves the problem *permanently*; again, running the users test script/query interatively via Mtest AND automatically every night in our test system is the most efficient/convenient/effective way to do so. Since we cannot assume that every user is familiar with our testing facilities, it is the developer's responsibility that test scripts indeed get added to CVS (and hence the testing system and test web) before a bug is closed. Both user and developer share the reposibility of verifying that the provided solution indeed solved the problem permanently.
- Please provide a detailed description how to repeat the bug you report, or even a concise test script to do so. Preferably, the test script should also be added to the standard test-suite (in the CVS repository; see "Adding test scripts" below for more details!).
I strongly suggest to drop the CVS thing. Many users don't have it, and it's confusing and more work for devs, see below.
I'm sort of with Fabian here. Maybe the "preferably" should be changed in "if possible".
fine with me.
- Please feel free to close your bug report, in case you notice it does not occur any more (your test script in the standard test-suite is your free-of-charge monitor; simply check our "TestWeb" at http://monetdb.cwi.nl/Development/TestWeb/ ;-)). See "Closing bug reports" below for more details!
This is weird. So a bug can suddenly vanish? Yes it can, but then the exact cause of the bug was not found, so it can be gone just by coincidence, which is *not* good. Submitters should only close their bug if they think they made a mistake and that their bug is invalid.
Bugs do sometimes disappear because of fixes that were made for some other reason. It is appropriate that the report gets closed in that case. The resolution should then be "Out of Date".
Exactly. ;-)
- Once a developer closes your bug report, please double-check whether the solution indeed works for you (simply check the status of your test script in our "TestWeb" at http://monetdb.cwi.nl/Development/TestWeb/ ;-)). If is does not work for you, feel free to re-open the bug report, or file a new one. In either case, please give a detailed description of the remaining or new problem!
Why not let the developer *always* add the test script before closing the bug. Checking that a bug is already there (hoping it complies to the naming conventions as pointed out below) is just some extra hassle. Besides that, IMHO it's just more well organised if the dev adds the script once (s)he fixes the problem and as such the correct output can be determined as well as the test possibly made somewhat more extensive to cover more possible problems. The dev that fixes the problem should ideally have the best knowledge on why something went wrong.
As I mentioned above, it is the developer's responsibility. That doesn't mean the developer must do it, and if the submitter can create and check in a test script, I would encourage that.
This (c|sh)ould have been my words ;-) Anyway, thank you all very much for your comments so far. I'll try to include them (as well as the comments that are still to come) in the MonetDB-Bug-Etiquette. Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |