Dear Wouter,
Thanks for your response.
Well what I am doing is somewhat open ended. I am still in early stages and have not yet defined specific tasks yet. But what I intend to do is to be able to query different parts of a webpages and combine results from different webpages effectively. The least I want to be able to do is to have queries of the following form run efficiently:
(1) Retrieve the whole (specific) section of webpages efficiently (for example retrieve "Summary" section of all webpages).
(2)Retrieve all webpages that have specific words in specific sections (for example the word "Shakespeare" in "Introduction" section)
Again, I have about 1 million webpages. I also have resources for parallelizing things if needed.
Thank you again for your help.
Andy
-------------------------------------------------------------
From: Wouter Alink <wouter.alink@gm...> - 2009-08-08 18:04
Hello (?),
It completely depends on your application whether MonetDB/XQuery is
the right solution. MonetDB/XQuery has been and is being used for very
large XML collections, but this also depends on how you would like to
query the data. Could you perhaps give some more information about the
application you have in mind?
Greetings,
Wouter
2009/8/8 ?listanand@gm... <listanand@gm...>:
> Dear all,
>
> I am new to MonetDB/XQuery and am considering using it for storing and
> processing a large number of webpages (~1million) in XML format. I am hoping
> to know if this is indeed the right tool, and how well it will scale to
> handle tasks of this magnitude.
>
> Thanks in advance
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
> trial. Simplify your report design, integration and deployment - and focus
> on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now. http://p.sf.net/sfu/bobj-july
> _______________________________________________
> MonetDB-users mailing list
> MonetDB-users@li...
> https://lists.sourceforge.net/lists/listinfo/monetdb-users
>
>