Hi there,

I need a string tokenizer in MonetDB.

The problem I have is not with the function itself, but with the fact that this is a 1 to N rows function.

Implementing this for a single string value is easy enough, using a table function that takes a string a returns a table:

create function tokenize(s string)

returns table (token string)

external name tokenize;

select *

from tokenize("one two three");

That's fine.

The issue I'm having is with extending this to a column of strings.

Ideally, given a string column

one two three

four five six

seven eight

I'd like to get an output along these lines (simplistic representation here):

one two three | one

one two three | two

one two three | three

four five six | four

four five six | five

four five six | six

seven eight | seven

seven eight | eight

I can sure code the c function and the mal wrapper to implement this, but I can't see how to map it to SQL, given that table functions don't accept identifiers as parameters.

Any idea? Any possible workaround?

Thanks, Roberto