- developers-list - monetdb.org

[Monetdb-developers] web-page: Broken link....
by Romulo Goncalves 22 Oct '09

22 Oct '09

Hello, In this page: http://monetdb.cwi.nl/projects/monetdb/SQL/Documentation/Start-and-Stop.htm… I click on See for a more complete session "VOC demo." and I end up in the same page, I think the right link should be: http://monetdb.cwi.nl/projects/monetdb/SQL/QuickTour/VOC/index.html Regards, Romulo

2 2

[Monetdb-developers] web-page: mclient options
by Romulo Goncalves 22 Oct '09

22 Oct '09

In monetdb web-page I read the VOC tutorial and I see the the following: shell> mclient -lsql -uvoc -Pvoc sql> \< voc_dump.sql Then I went to: http://monetdb.cwi.nl/projects/monetdb/SQL/Documentation/MAPI-Client.html#M… And for client options I see: -P passwd | --passwd=passwd password I just tried in my cvs head version and the option -P does not work. Reading some check ins in clients module, it seems we have to use .monetdb file to avoid typing the password. However, the syntax of the file was not specified. I create the file .monetdb with following content: monetdb monetdb And I get the following message: [goncalve@sirius scripts]$ ./mclient current debug sql .monetdb:1: syntax error: monetdb .monetdb:2: syntax error: monetdb password: I think we need some extra information about this new feature ;) Regards, Romulo

3 3

Re: [Monetdb-developers] [Monetdb-checkins] clients/src/mapiclient dump.c, Aug2009, 1.45.2.1, 1.45.2.2
by Milena Ivanova 22 Oct '09

22 Oct '09

Fabian wrote: > Update of /cvsroot/monetdb/clients/src/mapiclient > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv31397 > > Modified Files: > Tag: Aug2009 > dump.c > Log Message: > Stefan dK ran into a 'table is not unique' message, which IMO can only indicate something is seriously fubared > > Would you please write in English? http://www.merriam-webster.com/dictionary/ showed empty result on 'fubared' and International Maritime Organization for the shortening IMO, Thanks, Milena > Index: dump.c > =================================================================== > RCS file: /cvsroot/monetdb/clients/src/mapiclient/dump.c,v > retrieving revision 1.45.2.1 > retrieving revision 1.45.2.2 > diff -u -d -r1.45.2.1 -r1.45.2.2 > --- dump.c 22 Sep 2009 08:26:36 -0000 1.45.2.1 > +++ dump.c 22 Oct 2009 08:37:26 -0000 1.45.2.2 > @@ -342,9 +342,10 @@ > > if (cnt != 1) { > if (cnt == 0) > - fprintf(stderr, "Table %s.%s does not exist.\n", schema, tname); > + fprintf(stderr, "table %s.%s does not exist\n", schema, tname); > else > - fprintf(stderr, "Table %s.%s not unique.\n", schema, tname); > + fprintf(stderr, "table %s.%s is not unique, corrupt catalog?\n", > + schema, tname); > goto bailout; > } > > > > ------------------------------------------------------------------------------ > Come build with us! The BlackBerry(R) Developer Conference in SF, CA > is the only developer event you need to attend this year. Jumpstart your > developing skills, take BlackBerry mobile applications to market and stay > ahead of the curve. Join us from November 9 - 12, 2009. Register now! > http://p.sf.net/sfu/devconference > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins >

2 2

Re: [Monetdb-developers] [Monetdb-checkins] clients/src/python/monetdb mapi2.py, , NONE, 1.1 mapi.py, , 1.17, 1.18
by Sjoerd Mullender 22 Oct '09

22 Oct '09

Sounds like something that should have gone to the Candidate (Nov2009) branch? Gijs Molenaar wrote: > Update of /cvsroot/monetdb/clients/src/python/monetdb > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv12494 > > Modified Files: > mapi.py > Added Files: > mapi2.py > Log Message: > > Both python2 and python3 can do import monetdb.mapi now. Change is backwards > compatible. > > Previously you needed to do import monetdb.mapi3 in python3. > > > > --- NEW FILE: mapi2.py --- > # The contents of this file are subject to the MonetDB Public License > # Version 1.1 (the "License"); you may not use this file except in > # compliance with the License. You may obtain a copy of the License at > # http://monetdb.cwi.nl/Legal/MonetDBLicense-1.1.html > # > # Software distributed under the License is distributed on an "AS IS" > # basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the > # License for the specific language governing rights and limitations > # under the License. > # > # The Original Code is the MonetDB Database System. > # > # The Initial Developer of the Original Code is CWI. > # Portions created by CWI are Copyright (C) 1997-July 2008 CWI. > # Copyright August 2008-2009 MonetDB B.V. > # All Rights Reserved. > > """ > this is the python 2.* implementation of the mapi API. > > If you use python 3.* you should use mapi3.py > """ > > import socket > import logging > import struct > > from monetdb.monetdb_exceptions import * > > > logger = logging.getLogger("monetdb") > > MAX_PACKAGE_LENGTH = (1024*8)-2 > > MSG_PROMPT = "" > MSG_INFO = "#" > MSG_ERROR = "!" > MSG_Q = "&" > MSG_QTABLE = "&1" > MSG_QUPDATE = "&2" > MSG_QSCHEMA = "&3" > MSG_QTRANS = "&4" > MSG_QPREPARE = "&5" > MSG_QBLOCK = "&6" > MSG_HEADER = "%" > MSG_TUPLE = "[" > MSG_REDIRECT = "^" > > STATE_INIT = 0 > STATE_READY = 1 > > > class Server: > def __init__(self): > self.state = STATE_INIT > self._result = None > > def connect(self, hostname, port, username, password, database, language): > """ connect to a MonetDB database using the mapi protocol""" > > self.hostname = hostname > self.port = port > self.username = username > self.password = password > self.database = database > self.language = language > > self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) > > try: > self.socket.connect((hostname, port)) > except socket.error, error: > (error_code, error_str) = error > raise OperationalError(error_str) > > self.__login() > > > def __login(self, iteration=0): > """ Reads challenge from line, generate response and check if > everything is okay """ > > challenge = self.__getblock() > response = self.__challenge_response(challenge) > self.__putblock(response) > prompt = self.__getblock().strip() > > if len(prompt) == 0: > # Empty response, server is happy > pass > elif prompt.startswith(MSG_INFO): > logger.info("II %s" % prompt[1:]) > > elif prompt.startswith(MSG_ERROR): > logger.error(prompt[1:]) > raise DatabaseError(prompt[1:]) > > elif prompt.startswith(MSG_REDIRECT): > # a redirect can contain multiple redirects, for now we only use > # the first > response = prompt.split()[0][1:].split(':') > if response[1] == "merovingian": > logger.debug("II: merovingian proxy, restarting " + > "authenticatiton") > if iteration <= 10: > self.__login(iteration=iteration+1) > else: > raise OperationalError("maximal number of redirects " + > "reached (10)") > > elif response[1] == "monetdb": > self.hostname = response[2][2:] > self.port, self.database = response[3].split('/') > self.port = int(self.port) > logger.info("II: merovingian redirect to monetdb://%s:%s/%s" % > (self.hostname, self.port, self.database)) > self.socket.close() > self.connect(self.hostname, self.port, self.username, > self.password, self.database, self.language) > > else: > logger.error('!' + prompt[0]) > raise ProgrammingError("unknown redirect: %s" % prompt) > > else: > logger.error('!' + prompt[0]) > raise ProgrammingError("unknown state: %s" % prompt) > > self.state = STATE_READY > return True > > > def disconnect(self): > """ disconnect from the monetdb server """ > self.state = STATE_INIT > self.socket.close() > > > def cmd(self, operation): > """ put a mapi command on the line""" > logger.debug("II: executing command %s" % operation) > > if self.state != STATE_READY: > raise(ProgrammingError, "Not connected") > > self.__putblock(operation) > response = self.__getblock() > if not len(response): > return > if response[0] in [MSG_Q, MSG_HEADER, MSG_TUPLE]: > return response > elif response[0] == MSG_ERROR: > raise OperationalError(response[1:]) > else: > raise ProgrammingError("unknown state: %s" % response) > > > def __challenge_response(self, challenge): > """ generate a response to a mapi login challenge """ > challenges = challenge.split(':') > salt, identity, protocol, hashes, endian = challenges[:5] > > password = self.password > > if protocol == '9': > algo = challenges[5] > if algo == 'SHA512': > import hashlib > password = hashlib.sha512(password).hexdigest() > elif algo == 'SHA384': > import hashlib > password = hashlib.sha384(password).hexdigest() > elif algo == 'SHA256': > import hashlib > password = hashlib.sha256(password).hexdigest() > elif algo == 'SHA224': > import hashlib > password = hashlib.sha224(password).hexdigest() > elif algo == 'SHA1': > import hashlib > password = hashlib.sha1(password).hexdigest() > elif algo == 'MD5': > import hashlib > password = hashlib.md5(password).hexdigest() > else: > raise NotSupportedError("The %s hash algorithm is not " + > "supported" % algo) > elif protocol != "8": > raise NotSupportedError("We only speak protocol v8 and v9") > > h = hashes.split(",") > if "SHA1" in h: > import hashlib > s = hashlib.sha1() > s.update(password.encode()) > s.update(salt.encode()) > pwhash = "{SHA1}" + s.hexdigest() > elif "MD5" in h: > import hashlib > m = hashlib.md5() > m.update(password.encode()) > m.update(salt.encode()) > pwhash = "{MD5}" + m.hexdigest() > elif "crypt" in h: > import crypt > pwhash = "{crypt}" + crypt.crypt((password+salt)[:8], salt[-2:]) > else: > pwhash = "{plain}" + password + salt > > return ":".join(["BIG", self.username, pwhash, self.language, > self.database]) + ":" > > > def __getblock(self): > """ read one mapi encoded block """ > result = [] > last = 0 > while not last: > flag = self.__getbytes(2) > if len(flag) != 2: > raise OperationalError("server returned %s bytes, I need 2" % > len(flag)) > > # unpack (little endian short) > unpacked = struct.unpack('<H', flag)[0] > length = unpacked >> 1 > last = unpacked & 1 > logger.debug("II: reading %i bytes" % length) > if length > 0: > result.append(self.__getbytes(length)) > > result_str = "".join(result) > logger.debug("RX: %s" % result_str) > return result_str > > > def __getbytes(self, bytes): > """Read an amount of bytes from the socket""" > try: > return self.socket.recv(bytes) > except socket.error, error: > raise OperationalError(error[1]) > > > def __putblock(self, block): > """ wrap the line in mapi format and put it into the socket """ > pos = 0 > last = 0 > while not last: > data = block[pos:pos+MAX_PACKAGE_LENGTH] > length = len(data) > if length < MAX_PACKAGE_LENGTH: > last = 1 > flag = struct.pack( '<H', ( length << 1 ) + last ) > try: > logger.debug("II: sending %i bytes, last: %s" % (length, last)) > logger.debug("TX: %s" % data) > self.socket.send(flag) > self.socket.send(data) > except socket.error, error: > raise OperationalError(error[1]) > pos += length > > > Index: mapi.py > =================================================================== > RCS file: /cvsroot/monetdb/clients/src/python/monetdb/mapi.py,v > retrieving revision 1.17 > retrieving revision 1.18 > diff -u -d -r1.17 -r1.18 > --- mapi.py 7 Jul 2009 11:15:45 -0000 1.17 > +++ mapi.py 22 Oct 2009 10:25:35 -0000 1.18 > @@ -1,263 +1,10 @@ > -# The contents of this file are subject to the MonetDB Public License > -# Version 1.1 (the "License"); you may not use this file except in > -# compliance with the License. You may obtain a copy of the License at > -# http://monetdb.cwi.nl/Legal/MonetDBLicense-1.1.html > -# > -# Software distributed under the License is distributed on an "AS IS" > -# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the > -# License for the specific language governing rights and limitations > -# under the License. > -# > -# The Original Code is the MonetDB Database System. > -# > -# The Initial Developer of the Original Code is CWI. > -# Portions created by CWI are Copyright (C) 1997-July 2008 CWI. > -# Copyright August 2008-2009 MonetDB B.V. > -# All Rights Reserved. > - > -""" > -this is the python 2.* implementation of the mapi API. > - > -If you use python 3.* you should use mapi3.py > -""" > - > -import socket > -import logging > -import struct > - > -from monetdb.monetdb_exceptions import * > - > - > -logger = logging.getLogger("monetdb") > - > -MAX_PACKAGE_LENGTH = (1024*8)-2 > - > -MSG_PROMPT = "" > -MSG_INFO = "#" > -MSG_ERROR = "!" > -MSG_Q = "&" > -MSG_QTABLE = "&1" > -MSG_QUPDATE = "&2" > -MSG_QSCHEMA = "&3" > -MSG_QTRANS = "&4" > -MSG_QPREPARE = "&5" > -MSG_QBLOCK = "&6" > -MSG_HEADER = "%" > -MSG_TUPLE = "[" > -MSG_REDIRECT = "^" > - > -STATE_INIT = 0 > -STATE_READY = 1 > - > - > -class Server: > - def __init__(self): > - self.state = STATE_INIT > - self._result = None > - > - def connect(self, hostname, port, username, password, database, language): > - """ connect to a MonetDB database using the mapi protocol""" > - > - self.hostname = hostname > - self.port = port > - self.username = username > - self.password = password > - self.database = database > - self.language = language > - > - self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) > - > - try: > - self.socket.connect((hostname, port)) > - except socket.error, error: > - (error_code, error_str) = error > - raise OperationalError(error_str) > - > - self.__login() > - > - > - def __login(self, iteration=0): > - """ Reads challenge from line, generate response and check if > - everything is okay """ > - > - challenge = self.__getblock() > - response = self.__challenge_response(challenge) > - self.__putblock(response) > - prompt = self.__getblock().strip() > - > - if len(prompt) == 0: > - # Empty response, server is happy > - pass > - elif prompt.startswith(MSG_INFO): > - logger.info("II %s" % prompt[1:]) > - > - elif prompt.startswith(MSG_ERROR): > - logger.error(prompt[1:]) > - raise DatabaseError(prompt[1:]) > - > - elif prompt.startswith(MSG_REDIRECT): > - # a redirect can contain multiple redirects, for now we only use > - # the first > - response = prompt.split()[0][1:].split(':') > - if response[1] == "merovingian": > - logger.debug("II: merovingian proxy, restarting " + > - "authenticatiton") > - if iteration <= 10: > - self.__login(iteration=iteration+1) > - else: > - raise OperationalError("maximal number of redirects " + > - "reached (10)") > - > - elif response[1] == "monetdb": > - self.hostname = response[2][2:] > - self.port, self.database = response[3].split('/') > - self.port = int(self.port) > - logger.info("II: merovingian redirect to monetdb://%s:%s/%s" % > - (self.hostname, self.port, self.database)) > - self.socket.close() > - self.connect(self.hostname, self.port, self.username, > - self.password, self.database, self.language) > - > - else: > - logger.error('!' + prompt[0]) > - raise ProgrammingError("unknown redirect: %s" % prompt) > - > - else: > - logger.error('!' + prompt[0]) > - raise ProgrammingError("unknown state: %s" % prompt) > - > - self.state = STATE_READY > - return True > - > - > - def disconnect(self): > - """ disconnect from the monetdb server """ > - self.state = STATE_INIT > - self.socket.close() > - > - > - def cmd(self, operation): > - """ put a mapi command on the line""" > - logger.debug("II: executing command %s" % operation) > - > - if self.state != STATE_READY: > - raise(ProgrammingError, "Not connected") > - > - self.__putblock(operation) > - response = self.__getblock() > - if not len(response): > - return > - if response[0] in [MSG_Q, MSG_HEADER, MSG_TUPLE]: > - return response > - elif response[0] == MSG_ERROR: > - raise OperationalError(response[1:]) > - else: > - raise ProgrammingError("unknown state: %s" % response) > - > - > - def __challenge_response(self, challenge): > - """ generate a response to a mapi login challenge """ > - challenges = challenge.split(':') > - salt, identity, protocol, hashes, endian = challenges[:5] > - > - password = self.password > - > - if protocol == '9': > - algo = challenges[5] > - if algo == 'SHA512': > - import hashlib > - password = hashlib.sha512(password).hexdigest() > - elif algo == 'SHA384': > - import hashlib > - password = hashlib.sha384(password).hexdigest() > - elif algo == 'SHA256': > - import hashlib > - password = hashlib.sha256(password).hexdigest() > - elif algo == 'SHA224': > - import hashlib > - password = hashlib.sha224(password).hexdigest() > - elif algo == 'SHA1': > - import hashlib > - password = hashlib.sha1(password).hexdigest() > - elif algo == 'MD5': > - import hashlib > - password = hashlib.md5(password).hexdigest() > - else: > - raise NotSupportedError("The %s hash algorithm is not " + > - "supported" % algo) > - elif protocol != "8": > - raise NotSupportedError("We only speak protocol v8 and v9") > - > - h = hashes.split(",") > - if "SHA1" in h: > - import hashlib > - s = hashlib.sha1() > - s.update(password.encode()) > - s.update(salt.encode()) > - pwhash = "{SHA1}" + s.hexdigest() > - elif "MD5" in h: > - import hashlib > - m = hashlib.md5() > - m.update(password.encode()) > - m.update(salt.encode()) > - pwhash = "{MD5}" + m.hexdigest() > - elif "crypt" in h: > - import crypt > - pwhash = "{crypt}" + crypt.crypt((password+salt)[:8], salt[-2:]) > - else: > - pwhash = "{plain}" + password + salt > - > - return ":".join(["BIG", self.username, pwhash, self.language, > - self.database]) + ":" > - > - > - def __getblock(self): > - """ read one mapi encoded block """ > - result = [] > - last = 0 > - while not last: > - flag = self.__getbytes(2) > - if len(flag) != 2: > - raise OperationalError("server returned %s bytes, I need 2" % > - len(flag)) > - > - # unpack (little endian short) > - unpacked = struct.unpack('<H', flag)[0] > - length = unpacked >> 1 > - last = unpacked & 1 > - logger.debug("II: reading %i bytes" % length) > - if length > 0: > - result.append(self.__getbytes(length)) > - > - result_str = "".join(result) > - logger.debug("RX: %s" % result_str) > - return result_str > - > - > - def __getbytes(self, bytes): > - """Read an amount of bytes from the socket""" > - try: > - return self.socket.recv(bytes) > - except socket.error, error: > - raise OperationalError(error[1]) > > +import sys > > - def __putblock(self, block): > - """ wrap the line in mapi format and put it into the socket """ > - pos = 0 > - last = 0 > - while not last: > - data = block[pos:pos+MAX_PACKAGE_LENGTH] > - length = len(data) > - if length < MAX_PACKAGE_LENGTH: > - last = 1 > - flag = struct.pack( '<H', ( length << 1 ) + last ) > - try: > - logger.debug("II: sending %i bytes, last: %s" % (length, last)) > - logger.debug("TX: %s" % data) > - self.socket.send(flag) > - self.socket.send(data) > - except socket.error, error: > - raise OperationalError(error[1]) > - pos += length > +# a ugly hack to support python 2 and 3 at the same time > +(major, minor, micro, level, serial) = sys.version_info > +if (major == 3): > + from monetdb.mapi3 import * > +else: > + from monetdb.mapi2 import * > > > > ------------------------------------------------------------------------------ > Come build with us! The BlackBerry(R) Developer Conference in SF, CA > is the only developer event you need to attend this year. Jumpstart your > developing skills, take BlackBerry mobile applications to market and stay > ahead of the curve. Join us from November 9 - 12, 2009. Register now! > http://p.sf.net/sfu/devconference > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/mal tablet_sql.mx, , 1.9, 1.10
by Sjoerd Mullender 06 Oct '09

06 Oct '09

Shouldn't this have gone to the Nov2009 branch? Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/mal > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv8289 > > Modified Files: > tablet_sql.mx > Log Message: > Handle the case that an integer column is fed a string value that > appears to match the null string. > > > U tablet_sql.mx > Index: tablet_sql.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/mal/tablet_sql.mx,v > retrieving revision 1.9 > retrieving revision 1.10 > diff -u -d -r1.9 -r1.10 > --- tablet_sql.mx 2 Oct 2009 14:31:39 -0000 1.9 > +++ tablet_sql.mx 6 Oct 2009 13:33:45 -0000 1.10 > @@ -88,12 +88,21 @@ > #endif > adt = fmt->nildata; > fmt->c[0]->T->nonil = 0; > - } else if ( quote && *s == quote ) { > - s++; /* find the last quote */ > + } else > + if ( quote && *s == quote ) { > + /* strip the quotes when present */ > + s++; > for ( t = e = s; *t ; t++) > if ( *t == quote) e = t; > *e = 0; > adt = fmt->frstr(fmt, fmt->adt, s, e, 0); > + /* The user might have specified a null string escape > + e.g. NULL as '', which should be tested */ > + if (adt == NULL && s == e && fmt->nullstr && > + strncasecmp(s, fmt->nullstr, fmt->null_length+1) == 0){ > + adt = fmt->nildata; > + fmt->c[0]->T->nonil = 0; > + } > } else { > for( e=s; *e; e++) > ; > > > ------------------------------------------------------------------------------ > Come build with us! The BlackBerry® Developer Conference in SF, CA > is the only developer event you need to attend this year. Jumpstart your > developing skills, take BlackBerry mobile applications to market and stay > ahead of the curve. Join us from November 9-12, 2009. Register now! > http://p.sf.net/sfu/devconf > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/backends/monet5 sql_result.mx, , 1.127, 1.128
by Sjoerd Mullender 06 Oct '09

06 Oct '09

Shouldn't this have gone to the Nov2009 branch? Martin Kersten wrote: > Update of /cvsroot/monetdb/sql/src/backends/monet5 > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv7452 > > Modified Files: > sql_result.mx > Log Message: > The loader now properly recognizes decimals read as integer values. > > > U sql_result.mx > Index: sql_result.mx > =================================================================== > RCS file: /cvsroot/monetdb/sql/src/backends/monet5/sql_result.mx,v > retrieving revision 1.127 > retrieving revision 1.128 > diff -u -d -r1.127 -r1.128 > --- sql_result.mx 23 Sep 2009 07:36:21 -0000 1.127 > +++ sql_result.mx 6 Oct 2009 13:30:13 -0000 1.128 > @@ -381,8 +381,17 @@ > } > > len = (*BATatoms[type].atomFromStr)(s, &c->len, (ptr) &c->data); > - if (len == 0 || len != e-s) > + if (len == 0 || len != e-s){ > + /* decimals can be converted to integers when *.000 */ > + if ( s[len++] == '.') > + switch( type ){ > + case TYPE_bte: case TYPE_int: case TYPE_lng: case TYPE_sht: > + while (s[len]=='0') len++; > + if ( s[len] == 0) > + return c->data; > + } > return NULL; > + } > return c->data; > } > > > > ------------------------------------------------------------------------------ > Come build with us! The BlackBerry® Developer Conference in SF, CA > is the only developer event you need to attend this year. Jumpstart your > developing skills, take BlackBerry mobile applications to market and stay > ahead of the curve. Join us from November 9-12, 2009. Register now! > http://p.sf.net/sfu/devconf > _______________________________________________ > Monetdb-sql-checkins mailing list > Monetdb-sql-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-sql-checkins -- Sjoerd Mullender

1 0

Re: [Monetdb-developers] [Monetdb-sql-checkins] sql/src/backends/monet5/merovingian ChangeLog.Aug2009, Aug2009, NONE, 1.1.2.1 ChangeLog, Aug2009, 1.18.2.2, NONE
by Sjoerd Mullender 30 Sep '09

30 Sep '09

Fabian wrote: > Update of /cvsroot/monetdb/sql/src/backends/monet5/merovingian > In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv13466 > > Added Files: > Tag: Aug2009 > ChangeLog.Aug2009 > Removed Files: > Tag: Aug2009 > ChangeLog > Log Message: > switch to branch specific ChangeLog (please don't remove ChangeLog in HEAD) I took care of the not removing part. -- Sjoerd Mullender

1 0

[Monetdb-developers] Important Announcement: New release branch
by Sjoerd Mullender 29 Sep '09

29 Sep '09

I have created a new branch which is to be used for the upcoming Nov2009 release. This branch is called "Nov2009". There are now two "stable" branches, the Aug2009 and Nov2009 branch. The Aug2009 branch will be used for the Aug2009-SP2 release which is scheduled for October. The Nov2009 branch will be used for the Nov2009 feature release in November (and the Nov2009-SP1 and Nov2009-SP2 releases in December and January). The reason for making the Nov2009 branch now, and not waiting until after the Aug2009-SP2 release is that this way there is more time to stabilize the code for the release. There are a number of new features that are planned for the November release. Some of these features are not completed yet, and all of these features have to be properly tested and fixed if they're broken. The not-yet-completed features *must* be completed before the end of October, or else they *will* be removed from the release. New features for the November 2009 release are: - new SQL import from CSV files - new code for large joins - monetdb/merovingian distribution - database replication - native PHP client - others? Features that will definitely not be part of the November 2009 release include: - Stratos's slicing and histogram stuff - more? These will be removed from the Nov2009 branch but of course kept in the development trunk. All developers *must* now decide where each change they want to make to the code belongs. If a bug occurs in the Aug2009 branch, the bug *must* be fixed in that branch. If a bug does not occur in the Aug2009 branch, but it does occur in the Nov2009 branch, it *must* be fixed in the Nov2009 branch (this includes fixes to complete the above-mentioned features). All other changes *must* go to the development trunk. Propagation will be done from the Aug2009 branch to the Nov2009 branch, and from the Nov2009 branch to the development trunk. -- Sjoerd Mullender

3 2

[Monetdb-developers] Announcement: New Aug2009-SP1 bugfix release of MonetDB suite
by Sjoerd Mullender 28 Sep '09

28 Sep '09

The MonetDB team at CWI/MonetDB BV is pleased to announce the Aug2009-SP1 bug fix release of the MonetDB suite of programs. This is the first bug fix release for the Aug2009 release. More information on this release can be found at <http://monetdb.cwi.nl/projects/monetdb//Development/Releases/Aug2009/#Aug20…>. The sources and binaries are available at <http://monetdb.cwi.nl/projects/monetdb/Download/>. -- Sjoerd Mullender

1 0

[Monetdb-developers] Remote Merovingian control
by Fabian Groffen 25 Sep '09

25 Sep '09

This message only applies to the current development branch of MonetDB, and is informational only, mainly addressed to others working with the current development branch. monetdb[1] goes abroad After quite some hacking to move code back and forth between components, we now got the first version of merovingian[2] and monetdb[1] that can do remote control. I will give some small examples here so everyone that is interested can get up to speed fast, and/or get an idea on the possibilities. Basically, all functionality that used to be implemented in monetdb, is now implemented in merovingian. That means, for instance creation of a database is performed by merovingian. A positive side-effect of this is that the discovery aspect of databases is now real-time, e.g.: % monetdb release dumptest taken database out of maintenance mode: dumptest 2009-09-25 11:54:10 MSG control[8590]: (local): released database 'dumptest' 2009-09-25 11:54:10 MSG discovery[8590]: new database mapi:monetdb://pegasus.ins.cwi.nl:50000/dumptest (ttl=660s) We can see that when database is released again using monetdb release, it is immediately announced in the network, making it available for use. Locking a database has the oposite effect, of course. Next, the "(local)" string indicates that this control command was received over the local UNIX socket, as was the case in previous releases of merovingian. In addition to this local access, monetdb now also can connect over a TCP socket, which needs not to be remote, e.g.: % monetdb -hjarl.ins.cwi.nl -Pxxxxxx status name state uptime health last crash testdbatjenniesmachine stopped 100%, 4m - 2009-09-25 11:56:19 MSG control[11449]: pegasus.ins.cwi.nl:49652: served status list Here we can see that the merovingian at jarl reports it served a status list to pegasus, which is the host I performed the monetdb status at. Note the -Pxxxxxxxx in there, which is the passphrase of the remote merovingian. This passphrase is by default a random string, but can be set to something else by editing the .merovingian_pass file in the dbfarm and restarting merovingian. The way to find other merovingians in the vincinity is by using monetdb discover. Its output now includes how to reach each merovingian, if remote control is enabled for them. Remote control over TCP can be disabled in monetdb5.conf by unsetting mero_controlport. Control entries can be recognised in discover's output by the trailing asterisk, which one can also glob for: % monetdb discover "*\*" location pegasus.ins.cwi.nl:50001* volund.ins.cwi.nl:50001* jarl.ins.cwi.nl:64001* This command just returned all merovingians that have remote control enabled and reports their hostname and port to connect to. The remote control protocol is very simple, and I'm planning to make at least a Java library available to perform these commands, such that monetdb[1] no longer needs to be the only application to perform commands with merovingian. [1] http://homepages.cwi.nl/~fabian/MonetDB/Man%20Pages.html/monetdb.html [2] http://homepages.cwi.nl/~fabian/MonetDB/Man%20Pages.html/merovingian.html

1 0