Re: R UDFs

Hey Dave, If you create the function with LANGUAGE PYTHON_MAP, the function will be called once for every group with the input columns spliced over the group. If you create the function with LANGUAGE PYTHON, the function will be called only once and the groups will be passed in as parameter to the function. See here for more information: https://www.monetdb.org/wiki/Python_User_Defined_Functions#User-Defined_Aggr... I should note that parallel aggregates have not been tested as much as the rest of Python UDFs, so if you experience any problems with them please tell me so I can fix them (also please add a reproducible example when you do ;)) Regards, Mark ----- Original Message ----- From: "Anderson, David B" To: "users-list" Sent: Thursday, June 23, 2016 11:59:20 PM Subject: RE: R UDFs Mark, One more general question. I need to write a Python aggregate UDF which will compute a weighted quartile given a pair of columns. In my case, I will be performing this across a large number of groupings generated by a GROUP BY clause. Will MonetDB pass the entire columns into the Python UDF, or will slices (corresponding to collections of sub-groups) be passed into the UDF? Trying to figure out how parallelized the computation will be. All of the docs mention that computations like quartile() need the entire column, but they really only need the "entire" column for each grouping. I haven't sat down with a debugger and MonetDB to see exactly what is getting passed in. Thanks, Dave -----Original Message----- From: users-list [mailto:users-list-bounces+david.b.anderson=citi.com@monetdb.org] On Behalf Of Mark Raasveldt Sent: Saturday, June 18, 2016 4:49 AM To: users-list Subject: Re: R UDFs Hey Dave, I have only tested it with Python 2.7, but I see no reason why it won't work with Python 2.6. Regards, Mark ----- Original Message ----- From: "Anderson, David B" To: "users-list" Sent: Friday, June 17, 2016 6:07:25 PM Subject: RE: R UDFs Mark, Can I use Python 2.6, or does it have to be 2.7? Dave -----Original Message----- From: users-list [mailto:users-list-bounces+david.b.anderson=citi.com@monetdb.org] On Behalf Of Mark Raasveldt Sent: Friday, June 17, 2016 9:32 AM To: users-list Subject: Re: R UDFs Hey Dave, In the 'default' branch only Python 2.7 UDFs are supported, Python 3 UDFs are currently in the works so you can choose which version you want to use. If you want to use Python 3 UDFs right now you can use the 'python3udf' branch instead. Regards, Mark ----- Original Message ----- From: "Anderson, David B" To: "users-list" Sent: Thursday, June 16, 2016 6:05:30 PM Subject: RE: R UDFs I just read your Embedded Python/NumPy blog post. Excellent! Is there a preferred/required version of Python? I was looking at the config log and I see it checking for Python2 & Python3. -----Original Message----- From: Anderson, David B [ICG-MKTS] Sent: Thursday, June 16, 2016 11:34 AM To: users-list Subject: RE: R UDFs Mark, Thanks for the info. To confirm, If I write my UDFs in Python, they will be executed in parallel (if required)? Thanks, Dave -----Original Message----- From: users-list [mailto:users-list-bounces+david.b.anderson=citi.com@monetdb.org] On Behalf Of Mark Raasveldt Sent: Thursday, June 16, 2016 8:05 AM To: users-list Subject: Re: R UDFs Hey Dave, There's only a single R interpreter running. When an R function is called a lock is acquired that prevents other R functions from running at the same time. There has been some work to running multiple interpreters in parallel in separate processes for Python UDFs, but this has not been ported to R UDFs yet. Regards, Mark ----- Original Message ----- From: "Anderson, David B" To: "users-list" Sent: Thursday, June 16, 2016 1:30:28 PM Subject: R UDFs All, Quick question, how many R interpreters are running within MonetDB if you are using R UDFs? Is there a single shared interpreter or are there multiple copies running? Thanks, Dave _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list