Query database inside embedded R
Hello, We are trying to transition from a Postgres database to MonetDB. We had a few R function calls, so we decided to try embedded R. One of our R functions is designed to query the postgres database to get some more data to process. The problem, now, is how to integrate this function in Monet. We are using embeddedR to reduce latency, so using MonetDB.R inside the embedded function (if it is even possible) to query the database would be a total waste. Clearly the ultimate solution would be to make all the queries outside the function, but it would be really really hard to achieve this result given how the function is designed. What I am asking is the is some solution to this, like having some way to query the database inside embedded R avoiding the overhead of MonetDB.R. Thank you, Stefano
Hi Stefano,
On 31 May 2016, at 11:18, Stefano Fioravanzo
wrote: We are trying to transition from a Postgres database to MonetDB. We had a few R function calls, so we decided to try embedded R. One of our R functions is designed to query the postgres database to get some more data to process. The problem, now, is how to integrate this function in Monet. We are using embeddedR to reduce latency, so using MonetDB.R inside the embedded function (if it is even possible) to query the database would be a total waste.
Clearly the ultimate solution would be to make all the queries outside the function, but it would be really really hard to achieve this result given how the function is designed.
What I am asking is the is some solution to this, like having some way to query the database inside embedded R avoiding the overhead of MonetDB.R.
Yeah that’s what I call loopback queries. We have them in embedded Python already and could possibly make them available from embedded R as well. Would you be willing to try a pre-release version of MonetDB to see whether whatever we come up with works for you? Hannes
Could please give me a snipped of python code using a loopback query? Just to understand if we are talking the same thing and so I can try with python first? I have not been able to find any reference to this in the web. Stefano
On 31 May 2016, at 13:14, Hannes Mühleisen
wrote: Hi Stefano,
On 31 May 2016, at 11:18, Stefano Fioravanzo
wrote: We are trying to transition from a Postgres database to MonetDB. We had a few R function calls, so we decided to try embedded R. One of our R functions is designed to query the postgres database to get some more data to process. The problem, now, is how to integrate this function in Monet. We are using embeddedR to reduce latency, so using MonetDB.R inside the embedded function (if it is even possible) to query the database would be a total waste.
Clearly the ultimate solution would be to make all the queries outside the function, but it would be really really hard to achieve this result given how the function is designed.
What I am asking is the is some solution to this, like having some way to query the database inside embedded R avoiding the overhead of MonetDB.R.
Yeah that’s what I call loopback queries. We have them in embedded Python already and could possibly make them available from embedded R as well. Would you be willing to try a pre-release version of MonetDB to see whether whatever we come up with works for you?
Hannes
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Never Mind! I have found an example on the “Voter Classification using MonetDB/Python”. I have tested _conn.executed and it works as expected. Now, as you can understand we cannot use a development version of the database to test the new R feature. If we did chose to try your pre-release version, how much time would you think it would take to reach a stable version? We want to understand if we had better start a R to Python conversion of our functions, or if it is acceptable to wait a little bit for you to include the new feature in a stable version. Let me know, Stefano
On 31 May 2016, at 14:33, Stefano Fioravanzo
wrote: Could please give me a snipped of python code using a loopback query? Just to understand if we are talking the same thing and so I can try with python first? I have not been able to find any reference to this in the web.
Hi Stefano,
On 31 May 2016, at 16:04, Stefano Fioravanzo
wrote: Now, as you can understand we cannot use a development version of the database to test the new R feature. If we did chose to try your pre-release version, how much time would you think it would take to reach a stable version? We want to understand if we had better start a R to Python conversion of our functions, or if it is acceptable to wait a little bit for you to include the new feature in a stable version.
We plan to include the loopback functionality in the next release that is planned for this month. Hannes
Ok, we will stick with embedded python for now, waiting for the next release. Thanks for your help! Stefano
On 01 Jun 2016, at 11:45, Hannes Mühleisen
wrote: Hi Stefano,
On 31 May 2016, at 16:04, Stefano Fioravanzo
wrote: Now, as you can understand we cannot use a development version of the database to test the new R feature. If we did chose to try your pre-release version, how much time would you think it would take to reach a stable version? We want to understand if we had better start a R to Python conversion of our functions, or if it is acceptable to wait a little bit for you to include the new feature in a stable version.
We plan to include the loopback functionality in the next release that is planned for this month.
Hannes
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
participants (2)
-
Hannes Mühleisen
-
Stefano Fioravanzo