Re: Add a new optimizer pipeline.

10 Aug 2016


      The garbagecollector should be the last optimizer in the pipe.
Reconsider it putting it in the 'right' place
On 10/08/16 12:08, Xu,Wenjian wrote:
...
Hi Martin,
On Wed, Aug 10, 2016 at 5:19 PM, Martin Kersten mailto:martin.kersten@cwi.nl> wrote:
On 10/08/16 10:49, Xu,Wenjian wrote:
Hi all,
MonetDB version: 11.23.7
        Ubuntu version: 14.04
I created a new optimizer pipeline by adding an additional optimization step after 'default_pipe', i.e.:
        '*my_customized_pipe*'='optimizer.inline();optimizer.candidates();optimizer.remap();optimizer.costModel();optimizer.coercions();optimizer.evaluate();optimizer.pushselect();optimizer.aliases();optimizer.mitosis();optimizer.mergetable();optimizer.deadcode();optimizer.aliases();optimizer.constants();optimizer.commonTerms();optimizer.projectionpath();optimizer.deadcode();optimizer.reorder();optimizer.reduce();optimizer.matpack();optimizer.dataflow();optimizer.querylog();optimizer.multiplex();optimizer.generator();optimizer.profiler();optimizer.garbageCollector();*optimizer.customizedstep()*'
What I have done is:
        1) Create an empty optimizer step:
            -- 1.1  add /opt_customizedstep.c/h/ in package /monetdb/optimizer/*/, leave function body /OPTcustomizedstepImplementation() /empty/, /i.e. with one statement */return 0*/
        /    -- /1.2 register the step by modifying /opt_wrapper.c /and/optimizer.mal/
        2) Add the definition of the new pipeline in /opt_pipes.c/
        3) Modify the configuration file to compile the added source files.
After re-build the system, I can enable the new optimizer pipeline by *set optimizer='my_customized_pipe'*. However, I have two issues:
i) using EXPLAIN statement, I found the MAL plan different from the one generated by 'default_pipe' (which should be the same since I did nothing in the added step?). Specifically, the plan seems to be not affected by /optimizer.reorder() /step, i.e.,
        bat.new() and bat.append() instructions appear in the end of the plan.
1) make sure you restarted the server.
    2) plans are cached, so if you attempt the following
    select count(*) from tables
    set optimizer='my_customized_pipe'
    select count(*) from tables
it still uses the default plan
Furthermore, the optimizer history is not saved unless you
    call DEBUG select....
I am sure that I have restarted the server. Also, I have excluded the factor of plan caching by *set optimizer='my_customized_pipe' *immediately after I entered the sql interface. But my problem is still there...
ii) then I entered mdb mode by DEBUG statement. But it seems that some command did not work as before. For example, *optimizer* command does not display actions performed by each optimizer step.
For this issue, I highly suspected that I have missed some steps to make a new pipeline work correctly...
Thank you very much for your kind reply.
Best regards,
Wenjian
_______________________________________________
        users-list mailing list
        users-list@monetdb.org mailto:users-list@monetdb.org
        https://www.monetdb.org/mailman/listinfo/users-list https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________
    users-list mailing list
    users-list@monetdb.org mailto:users-list@monetdb.org
    https://www.monetdb.org/mailman/listinfo/users-list https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________
users-list mailing list
users-list@monetdb.org
https://www.monetdb.org/mailman/listinfo/users-list