Re: [Monetdb-developers] MonetDB with Perl in windows
Romulo Goncalves wrote:
Hello,
My name is Romulo (INS-1) and I am trying to create a windows installer with Perl. My colleagues told me that you are the Perl expert in MonetDB...
To compile PERL I did:
install perl in C:\perl_5\ and unzip the sources in same directory as MonetDB SQL MONETDB5 .... (The same steps as we should do for php}
In the MonetDB\conf\rules.msc I added:
PERL_SRCDIR = $(TOPDIR)\..\..\perl-5.8.8 PERL_INSTDIR = C:\perl-5 # for these, also see configure.py PERL_EXTENSIONDIR = lib\perl5
Note: I am compiling for M5 and sql from cvs branch.
For the compilation I set the flag HAVE_PERL=1.
Everything compiled.
Now I need to know which files I have to add to the installer. I was looking for a perl lib in cliens\NT\lib\ and It does not have such lib.
Could you help me a bit please?
First and foremost, let me admit that I did not find time to implement the new protocol (8) yet - thus the perl modules may not work for M5 (at least the pure perl (MapiPP) client). Anyway, this has nothing to do with the installation procedure. In the spirit of Perl’s motto, "There’s More Than One Way To Do It" (TMTOWTDI), there are many perl modules in clients/src/perl: - Mapi.pm is a simple pure perl client, maintained by CWI and works (AFAIK) with the current protocol - MapiLib (in Cimpl) is a wrapper around libMapi and you need SWIG to build it - The modules in MonetDB-CLI* are maintained by me: - MonetDB::CLI::MapiPP is pure perl - MonetDB::CLI::MapiLib is pure perl but uses MapiLib - MonetDB::CLI::MapiXS is a XS (that is: C) wrapper around libMapi - MonetDB::CLI is a simple loader for one of the MonetDB::CLI::* modules above - DBD::monetdb (in DBD) is a DBI client on top of MonetDB::CLI In general, you install a perl module in the following way: perl Makefile.PL make make test make install In this way, perl saves you from all the drudgery. Installing by hand is easy for pure perl modules (though not recommended): simple copy all *.pm into your perl lib directory. Example (MonetDB::CLI::MapiPP): MonetDB-CLI-MapiPP/MapiPP.pm -> /perl/site/lib/MonetDB/CLI/MapiPP.pm (assuming /perl/site/lib is in the lib search path - try 'perl -V' and look for @INC and PERL5LIB. Later, if you try to load MonetDB::CLI::MapiPP, perl replaces '::' with '/', adds .pm and looks in every path from @INC.) However, installing an extension module (XS or SWIG) is difficult! Perl builds additional files (e.g. .dll) which have to be installed into different places. Best you call the standard 'make install' from the installer or at least watch what 'make install' would do. Regards, Steffen
Steffen Goeldner wrote:
Romulo Goncalves wrote:
Hello,
My name is Romulo (INS-1) and I am trying to create a windows installer with Perl. My colleagues told me that you are the Perl expert in MonetDB...
To compile PERL I did:
install perl in C:\perl_5\ and unzip the sources in same directory as MonetDB SQL MONETDB5 .... (The same steps as we should do for php}
In the MonetDB\conf\rules.msc I added:
PERL_SRCDIR = $(TOPDIR)\..\..\perl-5.8.8 PERL_INSTDIR = C:\perl-5 # for these, also see configure.py PERL_EXTENSIONDIR = lib\perl5
Note: I am compiling for M5 and sql from cvs branch.
For the compilation I set the flag HAVE_PERL=1.
Everything compiled.
Now I need to know which files I have to add to the installer. I was looking for a perl lib in cliens\NT\lib\ and It does not have such lib.
Could you help me a bit please?
First and foremost, let me admit that I did not find time to implement the new protocol (8) yet - thus the perl modules may not work for M5 (at least the pure perl (MapiPP) client).
Anyway, this has nothing to do with the installation procedure.
In the spirit of Perl’s motto, "There’s More Than One Way To Do It" (TMTOWTDI), there are many perl modules in clients/src/perl:
- Mapi.pm is a simple pure perl client, maintained by CWI and works (AFAIK) with the current protocol
- MapiLib (in Cimpl) is a wrapper around libMapi and you need SWIG to build it
- The modules in MonetDB-CLI* are maintained by me:
- MonetDB::CLI::MapiPP is pure perl - MonetDB::CLI::MapiLib is pure perl but uses MapiLib - MonetDB::CLI::MapiXS is a XS (that is: C) wrapper around libMapi - MonetDB::CLI is a simple loader for one of the MonetDB::CLI::* modules above
- DBD::monetdb (in DBD) is a DBI client on top of MonetDB::CLI
In general, you install a perl module in the following way:
perl Makefile.PL make make test make install To do this I have to use cygwin which takes too long and it cannot be used for windows 64 bits. Do you know the steps for native compiler (nmake)?
Regards, Romulo
In this way, perl saves you from all the drudgery.
Installing by hand is easy for pure perl modules (though not recommended): simple copy all *.pm into your perl lib directory. Example (MonetDB::CLI::MapiPP):
MonetDB-CLI-MapiPP/MapiPP.pm -> /perl/site/lib/MonetDB/CLI/MapiPP.pm
(assuming /perl/site/lib is in the lib search path - try 'perl -V' and look for @INC and PERL5LIB. Later, if you try to load MonetDB::CLI::MapiPP, perl replaces '::' with '/', adds .pm and looks in every path from @INC.)
However, installing an extension module (XS or SWIG) is difficult! Perl builds additional files (e.g. .dll) which have to be installed into different places. Best you call the standard 'make install' from the installer or at least watch what 'make install' would do.
Regards, Steffen
Romulo Goncalves wrote:
Steffen Goeldner wrote:
In general, you install a perl module in the following way:
perl Makefile.PL make make test make install
To do this I have to use cygwin which takes too long and it cannot be used for windows 64 bits. Do you know the steps for native compiler (nmake)?
Nmake should do. For extension modules (i.e. C), it is recommended that you use the same compiler as for the perl build (indeed, other combinations are almost impossible). AFAIK, ActiveState builds with VC - thus VC/nmake is o.k. (Personally, I use the ancient Borland C++ 5.5 with dmake.) BTW: You find 'your' make with perl -MConfig -e "print $Config{make}" Regards, Steffen
Steffen Goeldner wrote:
Romulo Goncalves wrote:
Steffen Goeldner wrote:
In general, you install a perl module in the following way:
perl Makefile.PL make make test make install
To do this I have to use cygwin which takes too long and it cannot be used for windows 64 bits. Do you know the steps for native compiler (nmake)?
Nmake should do. For extension modules (i.e. C), it is recommended that you use the same compiler as for the perl build (indeed, other combinations are almost impossible). AFAIK, ActiveState builds with VC - thus VC/nmake is o.k. (Personally, I use the ancient Borland C++ 5.5 with dmake.)
BTW: You find 'your' make with
perl -MConfig -e "print $Config{make}" I am using visual studio 2003 with service pack 1.
To compile I did: nmake NEED_MX=1 HAVE_JAVA=1 HAVE_PYTHON=1 HAVE_PYTHON_SWIG=1 HAVE_PHP=1 HAVE_PERL=1 HAVE_PERL_DEVEL=1 HAVE_PERL_SWIG=1 HAVE_PEAR=1 NO_MONETDB4=1 HAVE_MONETDB5=1 And I got the following error: if exist libMonetODBC.dll.manifest mt -nologo -manifest libMonetODBC.dll.manifest -outputresource:libMonetODBC.dll;2 if not exist "samples" mkdir "samples" copy ".\..\..\..\src\odbc\samples\Makefile.msc" "samples\Makefile" 1 file(s) copied. cd "samples" && nmake /nologo "prefix=C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\clients\NT" all if not exist "python" mkdir "python" copy ".\..\..\..\src\odbc\python\Makefile.msc" "python\Makefile" 1 file(s) copied. cd "python" && nmake /nologo "prefix=C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\clients\NT" all if not exist "perl" mkdir "perl" copy ".\..\..\src\perl\Makefile.msc" "perl\Makefile" 1 file(s) copied. cd "perl" && nmake /nologo "prefix=C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\clients\NT" all NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-dir' NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-dir' NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-dir' if not exist "MonetDB-CLI-MapiPP" mkdir "MonetDB-CLI-MapiPP" NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-Makefile' NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-Makefile' NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-Makefile' copy ".\..\..\..\src\perl\MonetDB-CLI-MapiPP\Makefile.msc" "MonetDB-CLI-MapiPP\Makefile" 1 file(s) copied. NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-all' NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-all' NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-all' cd "MonetDB-CLI-MapiPP" && nmake /nologo "prefix=C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\clients\NT" all if not exist "Cimpl" mkdir "Cimpl" copy ".\..\..\..\src\perl\Cimpl\Makefile.msc" "Cimpl\Makefile" 1 file(s) copied. cd "Cimpl" && nmake /nologo "prefix=C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\clients\NT" all cl -GF -W3 -wd4273 -wd4102 -MD -nologo -Zi -Ox -I. -I.\..\..\.. "-IC:\Pthreads\include" -I.\..\..\..\..\src\perl\Cimp "-I..\..\mapilib" -I"C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\MonetDB\NT\include" -I"C:\cygwin\home\goncalve\MonetDB\Mone DB_SQL\MonetDB\NT\include\common" -I"C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\MonetDB\NT\include\gdk" -LD -FeMapiLib.dll /link ..\..\mapilib\libMapi.lib cl : Command line error D2003 : missing source filename NMAKE : fatal error U1077: 'cl' : return code '0x2' Stop. NMAKE : fatal error U1077: 'cd' : return code '0x2' Stop. NMAKE : fatal error U1077: 'cd' : return code '0x2' Stop. NMAKE : fatal error U1077: 'cd' : return code '0x2' Stop. NMAKE : fatal error U1077: '"E:\Visual Studio .NET 2003\VC7\BIN\nmake.exe"' : return code '0x2' Stop. C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\clients\NT>cd C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\clients\NT C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\clients\NT> Did I forget something or it is an error from your part? Regards, Romulo
Regards, Steffen
Romulo Goncalves wrote:
I am using visual studio 2003 with service pack 1.
To compile I did: nmake NEED_MX=1 HAVE_JAVA=1 HAVE_PYTHON=1 HAVE_PYTHON_SWIG=1 HAVE_PHP=1 HAVE_PERL=1 HAVE_PERL_DEVEL=1 HAVE_PERL_SWIG=1 HAVE_PEAR=1 NO_MONETDB4=1 HAVE_MONETDB5=1
And I got the following error: if exist libMonetODBC.dll.manifest mt -nologo -manifest libMonetODBC.dll.manifest -outputresource:libMonetODBC.dll;2 if not exist "samples" mkdir "samples" copy ".\..\..\..\src\odbc\samples\Makefile.msc" "samples\Makefile" 1 file(s) copied. cd "samples" && nmake /nologo "prefix=C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\clients\NT" all if not exist "python" mkdir "python" copy ".\..\..\..\src\odbc\python\Makefile.msc" "python\Makefile" 1 file(s) copied. cd "python" && nmake /nologo "prefix=C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\clients\NT" all if not exist "perl" mkdir "perl" copy ".\..\..\src\perl\Makefile.msc" "perl\Makefile" 1 file(s) copied. cd "perl" && nmake /nologo "prefix=C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\clients\NT" all NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-dir' NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-dir' NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-dir' if not exist "MonetDB-CLI-MapiPP" mkdir "MonetDB-CLI-MapiPP" NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-Makefile' NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-Makefile' NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-Makefile' copy ".\..\..\..\src\perl\MonetDB-CLI-MapiPP\Makefile.msc" "MonetDB-CLI-MapiPP\Makefile" 1 file(s) copied. NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-all' NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-all' NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-all' cd "MonetDB-CLI-MapiPP" && nmake /nologo "prefix=C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\clients\NT" all if not exist "Cimpl" mkdir "Cimpl" copy ".\..\..\..\src\perl\Cimpl\Makefile.msc" "Cimpl\Makefile" 1 file(s) copied. cd "Cimpl" && nmake /nologo "prefix=C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\clients\NT" all cl -GF -W3 -wd4273 -wd4102 -MD -nologo -Zi -Ox -I. -I.\..\..\.. "-IC:\Pthreads\include" -I.\..\..\..\..\src\perl\Cimp "-I..\..\mapilib" -I"C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\MonetDB\NT\include" -I"C:\cygwin\home\goncalve\MonetDB\Mone DB_SQL\MonetDB\NT\include\common" -I"C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\MonetDB\NT\include\gdk" -LD -FeMapiLib.dll /link ..\..\mapilib\libMapi.lib cl : Command line error D2003 : missing source filename NMAKE : fatal error U1077: 'cl' : return code '0x2' Stop. NMAKE : fatal error U1077: 'cd' : return code '0x2' Stop. NMAKE : fatal error U1077: 'cd' : return code '0x2' Stop. NMAKE : fatal error U1077: 'cd' : return code '0x2' Stop. NMAKE : fatal error U1077: '"E:\Visual Studio .NET 2003\VC7\BIN\nmake.exe"' : return code '0x2' Stop.
C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\clients\NT>cd C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\clients\NT C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\clients\NT>
Did I forget something or it is an error from your part?
Aah, I think I understand. What I described: perl Makefile.PL nmake nmake test nmake install is completely unrelated to the MonetDB build. You can try this e.g. in clients/src/perl/MonetDB-CLI-MapiPP - I'm sure it works. 'perl Makefile.PL' will generate a 'Makefile', which is used by nmake. But again, this is all in perl space and has nothing to do with the MonetDB build. Now to the MonetDB build. It uses a similar approach as 'perl Makefile.PL'. Here, 'python autogen.py' generates 'Makefile.msc', which is used by nmake. However, I never tried this to build the perl modules. AFAICS, it work on UNIX (somehow) for Mapi and MapiLib, but (IMHO) it's a difficult and risky way. I think it would be easier to teach autogen how to generate a Makefile which calls 'perl Makefile.PL && make'. Unfortunately, I'm not familar with autogen - but hey, it's a python script; who if not the kind people at CWI can solve this? Two small problems come into my mind: 1. the generated Makefiles may clash on some platforms, if no file extension is used. 2. For Cimpl, SWIG needs to be in the path and a rule for .i files known Regards, Steffen
Romulo Goncalves wrote:
I am using visual studio 2003 with service pack 1.
To compile I did: nmake NEED_MX=1 HAVE_JAVA=1 HAVE_PYTHON=1 HAVE_PYTHON_SWIG=1 HAVE_PHP=1 HAVE_PERL=1 HAVE_PERL_DEVEL=1 HAVE_PERL_SWIG=1 HAVE_PEAR=1 NO_MONETDB4=1 HAVE_MONETDB5=1
And I got the following error: if exist libMonetODBC.dll.manifest mt -nologo -manifest libMonetODBC.dll.manifest -outputresource:libMonetODBC.dll;2 if not exist "samples" mkdir "samples" copy ".\..\..\..\src\odbc\samples\Makefile.msc" "samples\Makefile" 1 file(s) copied. cd "samples" && nmake /nologo "prefix=C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\clients\NT" all if not exist "python" mkdir "python" copy ".\..\..\..\src\odbc\python\Makefile.msc" "python\Makefile" 1 file(s) copied. cd "python" && nmake /nologo "prefix=C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\clients\NT" all if not exist "perl" mkdir "perl" copy ".\..\..\src\perl\Makefile.msc" "perl\Makefile" 1 file(s) copied. cd "perl" && nmake /nologo "prefix=C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\clients\NT" all NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-dir' NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-dir' NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-dir' if not exist "MonetDB-CLI-MapiPP" mkdir "MonetDB-CLI-MapiPP" NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-Makefile' NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-Makefile' NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-Makefile' copy ".\..\..\..\src\perl\MonetDB-CLI-MapiPP\Makefile.msc" "MonetDB-CLI-MapiPP\Makefile" 1 file(s) copied. NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-all' NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-all' NMAKE : warning U4004: too many rules for target 'HAVE_PERL_DEVEL-all' cd "MonetDB-CLI-MapiPP" && nmake /nologo "prefix=C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\clients\NT" all if not exist "Cimpl" mkdir "Cimpl" copy ".\..\..\..\src\perl\Cimpl\Makefile.msc" "Cimpl\Makefile" 1 file(s) copied. cd "Cimpl" && nmake /nologo "prefix=C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\clients\NT" all cl -GF -W3 -wd4273 -wd4102 -MD -nologo -Zi -Ox -I. -I.\..\..\.. "-IC:\Pthreads\include" -I.\..\..\..\..\src\perl\Cimp "-I..\..\mapilib" -I"C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\MonetDB\NT\include" -I"C:\cygwin\home\goncalve\MonetDB\Mone DB_SQL\MonetDB\NT\include\common" -I"C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\MonetDB\NT\include\gdk" -LD -FeMapiLib.dll /link ..\..\mapilib\libMapi.lib cl : Command line error D2003 : missing source filename NMAKE : fatal error U1077: 'cl' : return code '0x2' Stop. NMAKE : fatal error U1077: 'cd' : return code '0x2' Stop. NMAKE : fatal error U1077: 'cd' : return code '0x2' Stop. NMAKE : fatal error U1077: 'cd' : return code '0x2' Stop. NMAKE : fatal error U1077: '"E:\Visual Studio .NET 2003\VC7\BIN\nmake.exe"' : return code '0x2' Stop.
C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\clients\NT>cd C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\clients\NT C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\clients\NT>
Did I forget something or it is an error from your part?
Aah, I think I understand. What I described:
perl Makefile.PL nmake nmake test nmake install
is completely unrelated to the MonetDB build. You can try this e.g. in clients/src/perl/MonetDB-CLI-MapiPP - I'm sure it works. 'perl Makefile.PL' will generate a 'Makefile', which is used by nmake. But again, this is all in perl space and has nothing to do with the MonetDB build.
Now to the MonetDB build. It uses a similar approach as 'perl Makefile.PL'. Here, 'python autogen.py' generates 'Makefile.msc', which is used by nmake. However, I never tried this to build the perl modules. AFAICS, it work on UNIX (somehow) for Mapi and MapiLib, but (IMHO) it's a difficult and risky way. I think it would be easier to teach autogen how to generate a Makefile which calls 'perl Makefile.PL && make'. Unfortunately, I'm not familar with autogen - but hey, it's a python script; who if not the kind people at CWI can solve this? Yap the autogen has to be modified. The compilation crashed after I created the Makefiles. I did in your directories:
Steffen Goeldner wrote: perl Makefile.PL nmake nmake install The installation is installing the libs in the directory where my Perl is installed. However, I could not find perl_monetdb.dll Then I removed Makefile.msc in the top directory to tell autogen to recreate all the Makefiles (I know that it does know how to do it for perl) and it crashed in the cleanup. cd ".\.." && autogen.py Traceback (most recent call last): File "C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\buildtools\autogen\autogen.py", line 159, in <module> os.path.walk(topdir, cleanup, None) File "C:\Python25\lib\ntpath.py", line 334, in walk walk(name, func, arg) File "C:\Python25\lib\ntpath.py", line 334, in walk walk(name, func, arg) File "C:\Python25\lib\ntpath.py", line 334, in walk walk(name, func, arg) File "C:\Python25\lib\ntpath.py", line 334, in walk walk(name, func, arg) File "C:\Python25\lib\ntpath.py", line 334, in walk walk(name, func, arg) File "C:\Python25\lib\ntpath.py", line 334, in walk walk(name, func, arg) File "C:\Python25\lib\ntpath.py", line 334, in walk walk(name, func, arg) File "C:\Python25\lib\ntpath.py", line 328, in walk func(arg, top, names) File "C:\cygwin\home\goncalve\MonetDB\MonetDB_SQL\buildtools\autogen\autogen.py", line 157, in cleanup os.unlink(os.path.join(dirname, name)) WindowsError: [Error 13] Access is denied: 'C:\\cygwin\\home\\goncalve\\MonetDB\\MonetDB_SQL\\clients\\src\\perl\\DBD\\blib\\l ib\\DBD\\monetdb\\.incs.ag' NMAKE : fatal error U1077: 'cd' : return code '0x1' Stop. I will talk with the ones responsible for autogen to add what we need. Thanks for the help. ;) Regards, Romulo
Two small problems come into my mind:
1. the generated Makefiles may clash on some platforms, if no file extension is used. 2. For Cimpl, SWIG needs to be in the path and a rule for .i files known
Regards, Steffen
participants (2)
-
Romulo Goncalves -
Steffen Goeldner