[Monetdb-developers] [M5] RFC: function shipping semantics
Hi all, I'd like some input on an issue that I can't get a clear vision on for now. Recently I added the remote module in MonetDB 5 which basically has 3 functions which perform actions with a remote (MonetDB 5) server. The functions get, put and exec retrieve, store and execute an object from, to or on a remote host. get and put allow for retrieval and storage of basically any type, scalars, strings, BATs, etc. exec only executes functions available on the remote host. The idea is to have one to "put()" a function to the remote host, and then execute it, retrieving its return value. exec is currently implemented that way, however, get and put of functions is not because it is unclear to me how this should exactly happen. For the case of put() I can think of the following which seems quite intuitive and ok to me: function user.func(v:int):str; r := "something"; return(r); end func; m := remote.put("host", "user.func(:int):str"); t := 5; q := remote.exec("host", m, t); io.print(q); The example puts function func to the remote host and executes it with argument t which is 5. The return string kept in q will contain the string "something". Because the identifier of the function on the remote host is returned and assigned to m, the function can easily be executed, since exec requires the function name as string. Currently that function name is just the function without its signature. The signature (which is not required, even not desired currently) can be deduced from the arguments given, resulting in a normal function resolution and (slightly misleading) error reporting if that fails. The other way around (thus to get a remote function) is still unclear to me how to do it. I could imagine the following that should work: v := remote.get("host", m); w := remote.get("host", "user.func(:int):str"); Problem with v is that m is not a full signature, so a conflict may arise if m is overloaded. This is not necessarily a problem, since m is a unique identifier, so no overloading for it exists. However, it is a requirement that a get after a put should always succeed and result in the same object that was put. The bigger problem here, however, is how to deal with the function being get. Since a function cannot be stored in a variable in MAL, it has to be stored on the local machine, or its contents as a string returned. In the first case the get() function would return its local identifier. In the latter case it would return a (big) string. In both cases, the function cannot be "used" as function in the program. A variable containing a function name cannot be used as in e.g.: k := v(t); where v, a variable, would be used as function, "dereferencing" v's value or something. Of course with a human factor one can do: io.print(v); and use the value of v to construct a call to the locally stored function. In the case of a string, the function has to be made, which I don't know how to do that, but would give explicit control over its name to the program. I don't think the concept of eval is clearing up things, as well as possible at all (if you think about the k := statement where only v should be expanded, not t neither k). Ideas?
That's a biggy for breakfast Fabian Groffen wrote:
Hi all,
I'd like some input on an issue that I can't get a clear vision on for now.
Recently I added the remote module in MonetDB 5 which basically has 3 functions which perform actions with a remote (MonetDB 5) server. The functions get, put and exec retrieve, store and execute an object from, to or on a remote host. get and put allow for retrieval and storage of basically any type, scalars, strings, BATs, etc. exec only executes functions available on the remote host. The idea is to have one to "put()" a function to the remote host, and then execute it, retrieving its return value. exec is currently implemented that way, however, get and put of functions is not because it is unclear to me how this should exactly happen. Simply put, function objects should be cast to strings before being thrown and reparsed at the other side.
For the case of put() I can think of the following which seems quite intuitive and ok to me:
function user.func(v:int):str; r := "something"; return(r); end func;
m := remote.put("host", "user.func(:int):str"); t := 5; q := remote.exec("host", m, t); io.print(q); ms:= inspect.toString("user","func"); remote.parse("host",ms); t:= 5; q:= remote.exec("host","user","func",t);
The example puts function func to the remote host and executes it with argument t which is 5. The return string kept in q will contain the string "something". Because the identifier of the function on the remote host is returned and assigned to m, the function can easily be
executed, since exec requires the function name as string. Currently that function name is just the function without its signature. The signature (which is not required, even not desired currently) can be deduced from the arguments given, resulting in a normal function resolution and (slightly misleading) error reporting if that fails.
The other way around (thus to get a remote function) is still unclear to me how to do it. I could imagine the following that should work:
v := remote.get("host", m); w := remote.get("host", "user.func(:int):str"); ms:= remote.get("host","inspect.toString(\"user\",\"func\");"); lang.parse(ms);
Problem with v is that m is not a full signature, so a conflict may just extend the arguments of toString arise if m is overloaded. This is not necessarily a problem, since m is a unique identifier, so no overloading for it exists. However, it is a requirement that a get after a put should always succeed and result in the same object that was put. The function returned will shield the old version in the symbol table. I consider it a garbage collection if we are looking for exact copies.
The bigger problem here, however, is how to deal with the function being get. Since a function cannot be stored in a variable in MAL, it has to be stored on the local machine, or its contents as a string returned. I don't see the problem. Either you reparse the string directly, which gives
No. The identifier is determined at the sending side. In case of a conflict, you either dump it in a 'private' remote module or rename the function locally first. the remote side control over the module it is placed into, or you overrule the remote side by doing something line lang.parse("myDefaultModule",ms);
In the first case the get() function would return its local identifier. In the latter case it would return a (big) string. In both cases, the function cannot be "used" as function in the program. A variable containing a function name cannot be used as in e.g.:
k := v(t);
where v, a variable, would be used as function, "dereferencing" v's value or something. Of course with a human factor one can do:
In general, in this case there is a role for a MAL-MAL translator that performs the resolution before execution. You simple accumulate the mapping in a table understood by the trafo.
io.print(v);
and use the value of v to construct a call to the locally stored function. In the case of a string, the function has to be made, which I don't know how to do that, but would give explicit control over its name to the program.
I don't think the concept of eval is clearing up things, as well as possible at all (if you think about the k := statement where only v should be expanded, not t neither k).
Ideas?
------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ _______________________________________________ Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
On 12-07-2007 15:40:09 +0200, Martin Kersten wrote:
Simply put, function objects should be cast to strings before being thrown and reparsed at the other side.
Technically speaking that's fine with me. I don't expect anything else.
For the case of put() I can think of the following which seems quite intuitive and ok to me: function user.func(v:int):str; r := "something"; return(r); end func; m := remote.put("host", "user.func(:int):str"); t := 5; q := remote.exec("host", m, t); io.print(q); ms:= inspect.toString("user","func"); remote.parse("host",ms);
This is unacceptable to me. (on this level)
t:= 5; q:= remote.exec("host","user","func",t);
Here I have a little problem that "user.func" is not "user.func" on host "host". The module may not exist, another function may exist that has the same signature, etc. etc.
The example puts function func to the remote host and executes it with argument t which is 5. The return string kept in q will contain the string "something". Because the identifier of the function on the remote host is returned and assigned to m, the function can easily be No. The identifier is determined at the sending side. In case of a conflict, you either dump it in a 'private' remote module or rename the function locally first.
No. That's not "consistent" with how the function works on other objects. And I don't like the fixed/static smell of it either. A function is either an object, or it is not. And if it is not, then it is something else, but what? It's not trivial to me how to treat it.
executed, since exec requires the function name as string. Currently that function name is just the function without its signature. The signature (which is not required, even not desired currently) can be deduced from the arguments given, resulting in a normal function resolution and (slightly misleading) error reporting if that fails. The other way around (thus to get a remote function) is still unclear to me how to do it. I could imagine the following that should work: v := remote.get("host", m); w := remote.get("host", "user.func(:int):str"); ms:= remote.get("host","inspect.toString(\"user\",\"func\");"); lang.parse(ms);
inspect.toString doesn't dump functions at the moment, by the way.
Problem with v is that m is not a full signature, so a conflict may just extend the arguments of toString arise if m is overloaded. This is not necessarily a problem, since m is a unique identifier, so no overloading for it exists. However, it is a requirement that a get after a put should always succeed and result in the same object that was put. The function returned will shield the old version in the symbol table. I consider it a garbage collection if we are looking for exact copies.
I consider that a not so nice solution. In the end you may have a function locally that is named like the function you fetch from the remote site. And you may definitely not want to overwrite it. I don't like to assume/enforce global uniqueness of the namespace.
The bigger problem here, however, is how to deal with the function being get. Since a function cannot be stored in a variable in MAL, it has to be stored on the local machine, or its contents as a string returned. I don't see the problem. Either you reparse the string directly, which gives the remote side control over the module it is placed into, or you overrule the remote side by doing something line lang.parse("myDefaultModule",ms);
All functions that you mention here do not exist. However the actual implementation is not the biggest concern for me now. I try to get the right semantics for dealing with functions clear. Doing a string dump/reparse doesn't trivially allow for plan stacks to be sent over. Question remains whether this is necessary or not.
In the first case the get() function would return its local identifier. In the latter case it would return a (big) string. In both cases, the function cannot be "used" as function in the program. A variable containing a function name cannot be used as in e.g.: k := v(t); where v, a variable, would be used as function, "dereferencing" v's value or something. Of course with a human factor one can do:
In general, in this case there is a role for a MAL-MAL translator that performs the resolution before execution. You simple accumulate the mapping in a table understood by the trafo.
io.print(v); and use the value of v to construct a call to the locally stored function. In the case of a string, the function has to be made, which I don't know how to do that, but would give explicit control over its name to the program. I don't think the concept of eval is clearing up things, as well as possible at all (if you think about the k := statement where only v should be expanded, not t neither k). Ideas?
After a night of thinking, I've come to the conclusion that functions cannot be dealt with using put and get. Since you can't make a reference to a function (i.e. assign it to a variable) it cannot work in the contract of get and put. This leads to the conclusion that separate functions are necessary to deal with functions, i.e. the list() function I proposed earlier. Such function, returning a string representation of the function (body?) could be used to inline, parse or whatever of the function on the local site. In great lines this is analogous to Martin's ideas, though separated from normal get and put, since a function is not an adressable object in MAL. On 12-07-2007 16:01:45 +0200, Fabian wrote:
On 12-07-2007 15:40:09 +0200, Martin Kersten wrote:
Simply put, function objects should be cast to strings before being thrown and reparsed at the other side.
Technically speaking that's fine with me. I don't expect anything else.
For the case of put() I can think of the following which seems quite intuitive and ok to me: function user.func(v:int):str; r := "something"; return(r); end func; m := remote.put("host", "user.func(:int):str"); t := 5; q := remote.exec("host", m, t); io.print(q); ms:= inspect.toString("user","func"); remote.parse("host",ms);
This is unacceptable to me. (on this level)
t:= 5; q:= remote.exec("host","user","func",t);
Here I have a little problem that "user.func" is not "user.func" on host "host". The module may not exist, another function may exist that has the same signature, etc. etc.
The example puts function func to the remote host and executes it with argument t which is 5. The return string kept in q will contain the string "something". Because the identifier of the function on the remote host is returned and assigned to m, the function can easily be No. The identifier is determined at the sending side. In case of a conflict, you either dump it in a 'private' remote module or rename the function locally first.
No. That's not "consistent" with how the function works on other objects. And I don't like the fixed/static smell of it either. A function is either an object, or it is not. And if it is not, then it is something else, but what? It's not trivial to me how to treat it.
executed, since exec requires the function name as string. Currently that function name is just the function without its signature. The signature (which is not required, even not desired currently) can be deduced from the arguments given, resulting in a normal function resolution and (slightly misleading) error reporting if that fails. The other way around (thus to get a remote function) is still unclear to me how to do it. I could imagine the following that should work: v := remote.get("host", m); w := remote.get("host", "user.func(:int):str"); ms:= remote.get("host","inspect.toString(\"user\",\"func\");"); lang.parse(ms);
inspect.toString doesn't dump functions at the moment, by the way.
Problem with v is that m is not a full signature, so a conflict may just extend the arguments of toString arise if m is overloaded. This is not necessarily a problem, since m is a unique identifier, so no overloading for it exists. However, it is a requirement that a get after a put should always succeed and result in the same object that was put. The function returned will shield the old version in the symbol table. I consider it a garbage collection if we are looking for exact copies.
I consider that a not so nice solution. In the end you may have a function locally that is named like the function you fetch from the remote site. And you may definitely not want to overwrite it. I don't like to assume/enforce global uniqueness of the namespace.
The bigger problem here, however, is how to deal with the function being get. Since a function cannot be stored in a variable in MAL, it has to be stored on the local machine, or its contents as a string returned. I don't see the problem. Either you reparse the string directly, which gives the remote side control over the module it is placed into, or you overrule the remote side by doing something line lang.parse("myDefaultModule",ms);
All functions that you mention here do not exist. However the actual implementation is not the biggest concern for me now. I try to get the right semantics for dealing with functions clear. Doing a string dump/reparse doesn't trivially allow for plan stacks to be sent over. Question remains whether this is necessary or not.
In the first case the get() function would return its local identifier. In the latter case it would return a (big) string. In both cases, the function cannot be "used" as function in the program. A variable containing a function name cannot be used as in e.g.: k := v(t); where v, a variable, would be used as function, "dereferencing" v's value or something. Of course with a human factor one can do:
In general, in this case there is a role for a MAL-MAL translator that performs the resolution before execution. You simple accumulate the mapping in a table understood by the trafo.
io.print(v); and use the value of v to construct a call to the locally stored function. In the case of a string, the function has to be made, which I don't know how to do that, but would give explicit control over its name to the program. I don't think the concept of eval is clearing up things, as well as possible at all (if you think about the k := statement where only v should be expanded, not t neither k). Ideas?
participants (3)
-
Fabian -
Fabian Groffen
-
Martin Kersten