[Monetdb-developers] Any efforts on using bitmap indices?
Hello, does anybody consider supporting bitmap indices? An indexing software called FastBit was integrated to MonetDB (http://crd.lbl.gov/~kewu/ps/LBNL-61768.pdf) and some benchmarks show that there were serious improvements concerning combined numerical and text queries (offers up to 10 times better performance then MonetDB without FastBit). Gabriel
On Fri, Jul 13, 2007 at 01:37:42PM +0200, Gabriel Welsche wrote:
Hello,
does anybody consider supporting bitmap indices?
An indexing software called FastBit was integrated to MonetDB (http://crd.lbl.gov/~kewu/ps/LBNL-61768.pdf) and some benchmarks show that there were serious improvements concerning combined numerical and text queries (offers up to 10 times better performance then MonetDB without FastBit).
We plan to look at the code from the FastBit group. They are working hard to make their code available in open source. Than the MonetDB team will have a look at their efforts integrating that into MonetDB/SQL. We hopefully get a maintainable patch for MonetDB out of this. Niels
Gabriel
------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ _______________________________________________ Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
-- Niels Nes, Centre for Mathematics and Computer Science (CWI) Kruislaan 413, 1098 SJ Amsterdam, The Netherlands room C0.02, phone ++31 20 592-4098, fax ++31 20 592-4312 url: http://www.cwi.nl/~niels e-mail: Niels.Nes@cwi.nl
Hi, Is there a plan to support full text search(FTS) in MonetDB as well? The pdf (http://crd.lbl.gov/~kewu/ps/LBNL-61768.pdf) talks about integrating FTS into monetdb sql using lucene as tokenizer. Postgresql has tsearch2 but lucene is a clear win when it comes to performance. It will be nice to have the power of MonetDB 'join' with the fts capability. Thanks Bharani Niels Nes wrote:
On Fri, Jul 13, 2007 at 01:37:42PM +0200, Gabriel Welsche wrote:
Hello,
does anybody consider supporting bitmap indices?
An indexing software called FastBit was integrated to MonetDB (http://crd.lbl.gov/~kewu/ps/LBNL-61768.pdf) and some benchmarks show that there were serious improvements concerning combined numerical and text queries (offers up to 10 times better performance then MonetDB without FastBit).
We plan to look at the code from the FastBit group. They are working hard to make their code available in open source. Than the MonetDB team will have a look at their efforts integrating that into MonetDB/SQL. We hopefully get a maintainable patch for MonetDB out of this.
Niels
Gabriel
------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ _______________________________________________ Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
--
Niels Nes, Centre for Mathematics and Computer Science (CWI) Kruislaan 413, 1098 SJ Amsterdam, The Netherlands room C0.02, phone ++31 20 592-4098, fax ++31 20 592-4312 url: http://www.cwi.nl/~niels e-mail: Niels.Nes@cwi.nl
------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ _______________________________________________ Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
-- View this message in context: http://www.nabble.com/Any-efforts-on-using-bitmap-indices--tf4073766.html#a1... Sent from the monetdb-developers mailing list archive at Nabble.com.
Dear Bharani, There is support for full text search in MonetDB/XQuery, see: http://dbappl.cs.utwente.nl/pftijah/ Best, Djoerd. -- Djoerd Hiemstra University of Twente Department of Computer Science PO Box 217, 7500 AE Enschede, The Netherlands URL: www.cs.utwente.nl/~hiemstra Tel: +31 53 4892335 On Mon, 23 Jul 2007, Bharani wrote:
Hi,
Is there a plan to support full text search(FTS) in MonetDB as well? The pdf (http://crd.lbl.gov/~kewu/ps/LBNL-61768.pdf) talks about integrating FTS into monetdb sql using lucene as tokenizer.
Postgresql has tsearch2 but lucene is a clear win when it comes to performance. It will be nice to have the power of MonetDB 'join' with the fts capability.
Thanks Bharani
Niels Nes wrote:
On Fri, Jul 13, 2007 at 01:37:42PM +0200, Gabriel Welsche wrote:
Hello,
does anybody consider supporting bitmap indices?
An indexing software called FastBit was integrated to MonetDB (http://crd.lbl.gov/~kewu/ps/LBNL-61768.pdf) and some benchmarks show that there were serious improvements concerning combined numerical and text queries (offers up to 10 times better performance then MonetDB without FastBit).
We plan to look at the code from the FastBit group. They are working hard to make their code available in open source. Than the MonetDB team will have a look at their efforts integrating that into MonetDB/SQL. We hopefully get a maintainable patch for MonetDB out of this.
Niels
Gabriel
------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ _______________________________________________ Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
--
Niels Nes, Centre for Mathematics and Computer Science (CWI) Kruislaan 413, 1098 SJ Amsterdam, The Netherlands room C0.02, phone ++31 20 592-4098, fax ++31 20 592-4312 url: http://www.cwi.nl/~niels e-mail: Niels.Nes@cwi.nl
------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ _______________________________________________ Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
-- View this message in context: http://www.nabble.com/Any-efforts-on-using-bitmap-indices--tf4073766.html#a1... Sent from the monetdb-developers mailing list archive at Nabble.com.
------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
Thanks for the link, i didnt know MonetDB had fts support. However i am looking for the full text search capability in the SQL. Something like select * from table where column ='+include this text -but with out this' so that we can combine the boolean text query along with the normal joins. FastBit ( accorrding to the doc) has integrated this with MonetDB by overriding the = operator. Thanks Bharani Djoerd Hiemstra-2 wrote:
Dear Bharani,
There is support for full text search in MonetDB/XQuery, see: http://dbappl.cs.utwente.nl/pftijah/
Best, Djoerd.
-- Djoerd Hiemstra University of Twente Department of Computer Science PO Box 217, 7500 AE Enschede, The Netherlands
URL: www.cs.utwente.nl/~hiemstra Tel: +31 53 4892335
On Mon, 23 Jul 2007, Bharani wrote:
Hi,
Is there a plan to support full text search(FTS) in MonetDB as well? The pdf (http://crd.lbl.gov/~kewu/ps/LBNL-61768.pdf) talks about integrating FTS into monetdb sql using lucene as tokenizer.
Postgresql has tsearch2 but lucene is a clear win when it comes to performance. It will be nice to have the power of MonetDB 'join' with the fts capability.
Thanks Bharani
Niels Nes wrote:
On Fri, Jul 13, 2007 at 01:37:42PM +0200, Gabriel Welsche wrote:
Hello,
does anybody consider supporting bitmap indices?
An indexing software called FastBit was integrated to MonetDB (http://crd.lbl.gov/~kewu/ps/LBNL-61768.pdf) and some benchmarks show that there were serious improvements
concerning
combined numerical and text queries (offers up to 10 times better performance then MonetDB without FastBit).
We plan to look at the code from the FastBit group. They are working hard to make their code available in open source. Than the MonetDB team will have a look at their efforts integrating that into MonetDB/SQL. We hopefully get a maintainable patch for MonetDB out of this.
Niels
Gabriel
This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ _______________________________________________ Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
--
Niels Nes, Centre for Mathematics and Computer Science (CWI) Kruislaan 413, 1098 SJ Amsterdam, The Netherlands room C0.02, phone ++31 20 592-4098, fax ++31 20 592-4312 url: http://www.cwi.nl/~niels e-mail: Niels.Nes@cwi.nl
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ _______________________________________________ Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
-- View this message in context: http://www.nabble.com/Any-efforts-on-using-bitmap-indices--tf4073766.html#a1... Sent from the monetdb-developers mailing list archive at Nabble.com.
------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
-- View this message in context: http://www.nabble.com/Any-efforts-on-using-bitmap-indices--tf4073766.html#a1... Sent from the monetdb-developers mailing list archive at Nabble.com.
Hi, We have this high on the wishlist, but had no time to do it yet. The direction we expect to take is to make the underlying text indexing stuff from PF/Tijah accessible from SQL. Notice that we are not so much interested in creating another coupled architecture, but rather plan to do all FT query processing in plain MonetDB code. (We have named this approach a multi-model dbms architecture in the past) So, no integration with Lucene nor Postgres's engine. Many greetings, Arjen | Thanks for the link, i didnt know MonetDB had fts support. However i am | looking for the full text search capability in the SQL. Something like | | select * from table where column ='+include this text -but with out this' | so that we can combine the boolean text query along with the normal joins. | FastBit ( accorrding to the doc) has integrated this with MonetDB by | overriding the = operator. | | Thanks | Bharani
Hi, The only comfort with lucene is the query syntax and the speed becuase of inverted index ( might be faster in BAT but have to wait to compare the performance). I will be happy to use it when it is available. Thanks Bharani Arjen P. de Vries wrote:
Hi,
We have this high on the wishlist, but had no time to do it yet.
The direction we expect to take is to make the underlying text indexing stuff from PF/Tijah accessible from SQL.
Notice that we are not so much interested in creating another coupled architecture, but rather plan to do all FT query processing in plain MonetDB code. (We have named this approach a multi-model dbms architecture in the past) So, no integration with Lucene nor Postgres's engine.
Many greetings,
Arjen
| Thanks for the link, i didnt know MonetDB had fts support. However i am | looking for the full text search capability in the SQL. Something like | | select * from table where column ='+include this text -but with out this' | so that we can combine the boolean text query along with the normal joins. | FastBit ( accorrding to the doc) has integrated this with MonetDB by | overriding the = operator. | | Thanks | Bharani
------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
-- View this message in context: http://www.nabble.com/Any-efforts-on-using-bitmap-indices--tf4073766.html#a1... Sent from the monetdb-developers mailing list archive at Nabble.com.
participants (5)
-
Arjen P. de Vries -
Bharani -
Djoerd Hiemstra -
Gabriel Welsche -
Niels Nes