Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer opt_derivepath.mx, Feb2009, NONE, 1.1.2.1 Makefile.ag, Feb2009, 1.66, 1.66.2.1 opt_prelude.mx, Feb2009, 1.55.2.3, 1.55.2.4

5 Mar 2009

On Thu, Mar 05, 2009 at 10:26:17AM +0100, Martin Kersten wrote:
...
Sjoerd Mullender wrote:
...
How is this not a new feature?
Oops, yes, unless Stefan wants this in for McK
As I said, this was/is just an idea, has low(est) priority (for me),
effects/benefits are (at least to me) yet unknown, is not crucial for McK

--- hence, yes, it is a new feature that should be only in the development
trunk --- I can later take care of revoming it from the release branch ...

Stefan
...
...
Martin Kersten wrote:
...
Update of /cvsroot/monetdb/MonetDB5/src/optimizer
In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv2998/optimizer
Modified Files:
      Tag: Feb2009
  Makefile.ag opt_prelude.mx 
Added Files:
      Tag: Feb2009
  opt_derivepath.mx 
Log Message:
Added the framework for the commissioned derivepath optimizer.
U opt_prelude.mx
Index: opt_prelude.mx
===================================================================
RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/opt_prelude.mx,v
retrieving revision 1.55.2.3
retrieving revision 1.55.2.4
diff -u -d -r1.55.2.3 -r1.55.2.4
--- opt_prelude.mx	1 Mar 2009 19:15:30 -0000	1.55.2.3
+++ opt_prelude.mx	5 Mar 2009 00:17:12 -0000	1.55.2.4
@@ -57,6 +57,7 @@
 opt_export  str deleteRef;
 opt_export  str depositRef;
 opt_export  str deriveRef;
+opt_export  str derivePathRef;
 opt_export  str differenceRef;
 opt_export  str divRef;
 opt_export  str disconnectRef;
@@ -216,6 +217,7 @@
 str deleteRef;
 str depositRef;
 str deriveRef;
+str derivePathRef;
 str differenceRef;
 str divRef;
 str disconnectRef;
@@ -371,6 +373,7 @@
      deleteRef = putName("delete",6);
      depositRef = putName("deposit",7);
      deriveRef = putName("derive",6);
+		derivePathRef = putName("derivePath",10);
      differenceRef= putName("difference",10);
      divRef = putName("/",1);
      disconnectRef= putName("disconnect",10);
U Makefile.ag
Index: Makefile.ag
===================================================================
RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/Makefile.ag,v
retrieving revision 1.66
retrieving revision 1.66.2.1
diff -u -d -r1.66 -r1.66.2.1
--- Makefile.ag	18 Jan 2009 11:44:14 -0000	1.66
+++ Makefile.ag	5 Mar 2009 00:17:12 -0000	1.66.2.1
@@ -32,7 +32,7 @@
      opt_singleton.mx opt_costModel.mx opt_reduce.mx opt_macro.mx \
      opt_accumulators.mx opt_qep.mx opt_mergetable.mx \
      opt_remoteQueries.mx opt_joinselect.mx opt_partitions.mx \
-		opt_evaluate.mx opt_inline.mx opt_pushranges.mx \
+		opt_evaluate.mx opt_inline.mx opt_pushranges.mx opt_derivepath.mx \
      opt_accessmode.mx opt_joinpath.mx opt_heuristics.mx opt_remap.mx \
      opt_statistics.mx opt_trace.mx  opt_recycler.mx opt_dataflow.mx \
      opt_mitosis.mx opt_octopus.mx opt_replicator.mx opt_history.mx
@@ -53,7 +53,7 @@
      opt_singleton.mx opt_costModel.mx opt_reduce.mx opt_macro.mx \
      opt_accumulators.mx opt_qep.mx opt_mergetable.mx \
      opt_remoteQueries.mx opt_joinselect.mx opt_partitions.mx \
-		opt_evaluate.mx opt_inline.mx opt_pushranges.mx \
+		opt_evaluate.mx opt_inline.mx opt_pushranges.mx opt_derivepath.mx \
      opt_accessmode.mx opt_joinpath.mx opt_heuristics.mx opt_remap.mx \
      opt_statistics.mx opt_trace.mx  opt_recycler.mx opt_dataflow.mx \
      opt_mitosis.mx opt_octopus.mx opt_replicator.mx opt_history.mx
@@ -72,7 +72,7 @@
      opt_singleton.mx opt_costModel.mx opt_reduce.mx opt_macro.mx \
      opt_accumulators.mx opt_qep.mx opt_mergetable.mx \
      opt_remoteQueries.mx opt_joinselect.mx opt_partitions.mx \
-		opt_evaluate.mx opt_inline.mx opt_pushranges.mx \
+		opt_evaluate.mx opt_inline.mx opt_pushranges.mx opt_derivepath.mx \
      opt_accessmode.mx opt_joinpath.mx opt_heuristics.mx opt_remap.mx \
      opt_statistics.mx opt_trace.mx  opt_recycler.mx opt_dataflow.mx \
      opt_mitosis.mx opt_octopus.mx opt_replicator.mx opt_history.mx
--- NEW FILE: opt_derivepath.mx ---
@' The contents of this file are subject to the MonetDB Public License
@' Version 1.1 (the "License"); you may not use this file except in
@' compliance with the License. You may obtain a copy of the License at
@' http://monetdb.cwi.nl/Legal/MonetDBLicense-1.1.html
@'
@' Software distributed under the License is distributed on an "AS IS"
@' basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the
@' License for the specific language governing rights and limitations
@' under the License.
@'
@' The Original Code is the MonetDB Database System.
@'
@' The Initial Developer of the Original Code is CWI.
@' Portions created by CWI are Copyright (C) 1997-July 2008 CWI.
@' Copyright August 2008-2009 MonetDB B.V.
@' All Rights Reserved.
@f opt_derivepath
@- Group derive paths
The routine @sc{optimizer.derivepaths()}
walks through the program looking for grouping operations
and cascades them into a single multiple group paths.
To illustrate, consider
@verbatim
  (ext1,grp1) := group.new(b);
  (ext2,grp2) := group.derive(ext1,grp1, c);
  (ext3,grp3) := group.derive(ext2,grp2, d);
@end verbatim
The result becomes.
@verbatim
  (ext3,grp3) := group.derivepath(b,c,d);
@end verbatim
@{
@mal
pattern optimizer.derivePath():str
address OPTderivePath;
pattern optimizer.derivePath(mod:str, fcn:str):str
address OPTderivePath
comment "Join path constructor";
pattern group.derivePath(l:bat[:any,:any]...):bat[:any,:any]
address ALGderivePath
comment "internal routine to handle join paths.
  The type analysis is rather tricky.";
@h
#ifndef _OPT_DERIVEPATH_
#define _OPT_DERIVEPATH_
#include "opt_prelude.h"
#include "opt_support.h"
#include "mal_interpreter.h"
/* #define DEBUG_OPT_DERIVEPATH  */
@c
#include "mal_config.h"
#include "opt_derivepath.h"
static int
OPTderivePathImplementation(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr p)
{
  int i, actions=0;
  int *pc;
  str derivePathRef = putName("derivePath",10);
  InstrPtr q;
  InstrPtr *old;
  int limit,slimit;
(void) cntxt;
  (void) stk;
  if (varGetProp(mb, getArg(mb->stmt[0], 0), inlineProp) != NULL)
      return 0;
old= mb->stmt;
  limit= mb->stop;
  slimit= mb->ssize;
  newMalBlkStmt(mb,mb->ssize);
/* beware, new variables and instructions are introduced */
  pc= (int*) GDKmalloc(sizeof(int)* mb->vtop * 2); /* to find last assignment */
  memset((char*) pc, 0, sizeof(int)* mb->vtop *2);
for (i = 0; ifdout,"new derivePath instruction\n");
      		printInstruction(cntxt->fdout,mb, 0, q, LIST_MAL_ALL);
#endif
      	}
      } 
      pushInstruction(mb,p);
  }
  for(; ifdout,"#opt_derivepath: %d statements glued\n",actions);
  return actions;
}
@include optimizerWrapper.mx
@h
@:exportOptimizer(derivePath)@
#endif
opt_export str ALGderivePath(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr pci);
@c
#include "opt_statistics.h"
@:wrapOptimizer(derivePath,OPT_CHECK_ALL)@
@-
The derive path optimizer takes a derivation sequence and
attempts to minimize the intermediate result.
The choice depends on a good estimate of intermediate
results using properties.
For the time being, we use a simplistic model, based
on the assumption that most joins are foreign key joins anyway.
@c
str
ALGderivePath(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr pci)
{
  (void) cntxt;
  (void) mb;
  (void) stk;
  (void) pci;
  throw(MAL,"group.derivePath","nyi");
}
@}
------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Monetdb-checkins mailing list
Monetdb-checkins@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
------------------------------------------------------------------------
------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
------------------------------------------------------------------------
_______________________________________________
Monetdb-checkins mailing list
Monetdb-checkins@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Monetdb-checkins mailing list
Monetdb-checkins@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
-- 
| Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl |
| CWI,  P.O.Box 94079 | http://www.cwi.nl/~manegold/  |
| 1090 GB Amsterdam   | Tel.: +31 (20) 592-4212       |
| The Netherlands     | Fax : +31 (20) 592-4312       |

    