Geometrical grid for spherical data
Hi All, I have spherical coordinates (right ascension, declination) and I'd like to iterate over areas in this 2-dimensional space. Easiest that comes to mind is just create a grid of squares (say 5 arcsecs wide) and iterate over them. Is it easier to do this in MonetDB with just spherical coordinates (some kind of spherical index), or do I have to convert them to x,y,z coordinates and create the grid in 3D? Thank you for your insight. Jiri Nadvornik Astronomical Institute AV CR Stellar department Czech Republic mailto:nadvornik.ji@gmail.com nadvornik.ji@gmail.com
Hi Jiri, What kind of queries would you like to run on this data? Can you give me some examples? Cheers, Kostsi On 03/16/2015 06:04 PM, Jiří Nádvorník wrote:
Hi All,
I have spherical coordinates (right ascension, declination) and I’d like to iterate over areas in this 2-dimensional space. Easiest that comes to mind is just create a grid of squares (say 5 arcsecs wide) and iterate over them.
Is it easier to do this in MonetDB with just spherical coordinates (some kind of spherical index), or do I have to convert them to x,y,z coordinates and create the grid in 3D?
Thank you for your insight.
Jiri Nadvornik
Astronomical Institute AV CR
Stellar department
Czech Republic
nadvornik.ji@gmail.com mailto:nadvornik.ji@gmail.com
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
-- Kostis E. Kyzirakos, Ph.D. Centrum voor Wiskunde en Informatica DB Architectures (DA) Office L315 Science Park 123 1098 XG Amsterdam (NL) tel: +31 (20) 592-4034 mobile: +31 (0) 6422-95345 e-mail: kostis@cwi.nl
Hi Kostsi, Well my query can have only one parameter - the size of the squares in the grid. Then I want the query just yield groups of points (1 group for 1 square in the grid). I don't even need to backtrack the points to the squares in the grid - I don't need a query "Give me all of points in this particular square" (but sure would be better for debugging). I just need to work with these groups of points separately. How to do this practically is another thing - maybe building some hierarchical index which will assign IDs to the squares in the grid will be necessary. An far more advanced example than I need could be https://code.google.com/p/q3c/ for relational DB. Cheers, JN
-----Original Message----- From: users-list [mailto:users-list- bounces+nadvornik.ji=gmail.com@monetdb.org] On Behalf Of Kostis Kyzirakos Sent: 16. března 2015 18:07 To: users-list@monetdb.org Subject: Re: Geometrical grid for spherical data
Hi Jiri, What kind of queries would you like to run on this data? Can you give me some examples?
Cheers, Kostsi
On 03/16/2015 06:04 PM, Jiří Nádvorník wrote:
Hi All,
I have spherical coordinates (right ascension, declination) and I’d like to iterate over areas in this 2-dimensional space. Easiest that comes to mind is just create a grid of squares (say 5 arcsecs wide) and iterate over them.
Is it easier to do this in MonetDB with just spherical coordinates (some kind of spherical index), or do I have to convert them to x,y,z coordinates and create the grid in 3D?
Thank you for your insight.
Jiri Nadvornik
Astronomical Institute AV CR
Stellar department
Czech Republic
nadvornik.ji@gmail.com mailto:nadvornik.ji@gmail.com
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
-- Kostis E. Kyzirakos, Ph.D. Centrum voor Wiskunde en Informatica DB Architectures (DA) Office L315 Science Park 123 1098 XG Amsterdam (NL) tel: +31 (20) 592-4034 mobile: +31 (0) 6422-95345 e-mail: kostis@cwi.nl _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Hi Jiri, Check the LOFAR paper [1] about this and the way they solve this problem from the radio perspective. In optical it is a bit different, since the source density is much higher, but on the other hand the search areas can be well approximated circularly. You may check how SDSS and SkyServer approach it with the HTM indexing. Healpix would also be very nice since every "tile" is of equal area. However, calculating the id may be computationally intensive (at least for the HTM as far as I know). Implementations (in postgresql) of variants, eg, Q3C, are limited and not complete. Cheers Bart [1] http://adsabs.harvard.edu/abs/2015arXiv150301526S On 03/16/2015 10:26 PM, Jiří Nádvorník wrote:
Hi Kostsi,
Well my query can have only one parameter - the size of the squares in the grid. Then I want the query just yield groups of points (1 group for 1 square in the grid). I don't even need to backtrack the points to the squares in the grid - I don't need a query "Give me all of points in this particular square" (but sure would be better for debugging). I just need to work with these groups of points separately.
How to do this practically is another thing - maybe building some hierarchical index which will assign IDs to the squares in the grid will be necessary.
An far more advanced example than I need could be https://code.google.com/p/q3c/ for relational DB.
Cheers,
JN
-----Original Message----- From: users-list [mailto:users-list- bounces+nadvornik.ji=gmail.com@monetdb.org] On Behalf Of Kostis Kyzirakos Sent: 16. března 2015 18:07 To: users-list@monetdb.org Subject: Re: Geometrical grid for spherical data
Hi Jiri, What kind of queries would you like to run on this data? Can you give me some examples?
Cheers, Kostsi
On 03/16/2015 06:04 PM, Jiří Nádvorník wrote:
Hi All,
I have spherical coordinates (right ascension, declination) and I’d like to iterate over areas in this 2-dimensional space. Easiest that comes to mind is just create a grid of squares (say 5 arcsecs wide) and iterate over them.
Is it easier to do this in MonetDB with just spherical coordinates (some kind of spherical index), or do I have to convert them to x,y,z coordinates and create the grid in 3D?
Thank you for your insight.
Jiri Nadvornik
Astronomical Institute AV CR
Stellar department
Czech Republic
nadvornik.ji@gmail.com mailto:nadvornik.ji@gmail.com
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
-- Kostis E. Kyzirakos, Ph.D. Centrum voor Wiskunde en Informatica DB Architectures (DA) Office L315 Science Park 123 1098 XG Amsterdam (NL) tel: +31 (20) 592-4034 mobile: +31 (0) 6422-95345 e-mail: kostis@cwi.nl _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Hi Bart,
thank you for your suggestions.
As far as I know, the most advanced and fast from these is HealPix. The
problem is that I also need to use overlaps. For example if I had a grid of
squares 10 arcsecs wide, I need each square to give me objects from
neighboring squares in range 1 arcsec (1 arcsec overlap).
I have some experience with the Q3C and I know how to do it there, but as
you mentioned the problem is that the squares are not of the same size.
I don't know how to do this in HealPix.
Cheers,
Jiri
2015-03-17 10:30 GMT+01:00 Bart Scheers
Hi Jiri,
Check the LOFAR paper [1] about this and the way they solve this problem from the radio perspective.
In optical it is a bit different, since the source density is much higher, but on the other hand the search areas can be well approximated circularly. You may check how SDSS and SkyServer approach it with the HTM indexing.
Healpix would also be very nice since every "tile" is of equal area. However, calculating the id may be computationally intensive (at least for the HTM as far as I know). Implementations (in postgresql) of variants, eg, Q3C, are limited and not complete.
Cheers Bart
[1] http://adsabs.harvard.edu/abs/2015arXiv150301526S
On 03/16/2015 10:26 PM, Jiří Nádvorník wrote:
Hi Kostsi,
Well my query can have only one parameter - the size of the squares in the grid. Then I want the query just yield groups of points (1 group for 1 square in the grid). I don't even need to backtrack the points to the squares in the grid - I don't need a query "Give me all of points in this particular square" (but sure would be better for debugging). I just need to work with these groups of points separately.
How to do this practically is another thing - maybe building some hierarchical index which will assign IDs to the squares in the grid will be necessary.
An far more advanced example than I need could be https://code.google.com/p/q3c/ for relational DB.
Cheers,
JN
-----Original Message-----
From: users-list [mailto:users-list- bounces+nadvornik.ji=gmail.com@monetdb.org] On Behalf Of Kostis Kyzirakos Sent: 16. března 2015 18:07 To: users-list@monetdb.org Subject: Re: Geometrical grid for spherical data
Hi Jiri, What kind of queries would you like to run on this data? Can you give me some examples?
Cheers, Kostsi
On 03/16/2015 06:04 PM, Jiří Nádvorník wrote:
Hi All,
I have spherical coordinates (right ascension, declination) and I’d like to iterate over areas in this 2-dimensional space. Easiest that comes to mind is just create a grid of squares (say 5 arcsecs wide) and iterate over them.
Is it easier to do this in MonetDB with just spherical coordinates (some kind of spherical index), or do I have to convert them to x,y,z coordinates and create the grid in 3D?
Thank you for your insight.
Jiri Nadvornik
Astronomical Institute AV CR
Stellar department
Czech Republic
nadvornik.ji@gmail.com mailto:nadvornik.ji@gmail.com
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
-- Kostis E. Kyzirakos, Ph.D. Centrum voor Wiskunde en Informatica DB Architectures (DA) Office L315 Science Park 123 1098 XG Amsterdam (NL) tel: +31 (20) 592-4034 mobile: +31 (0) 6422-95345 e-mail: kostis@cwi.nl _______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________ users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
participants (3)
-
Bart Scheers -
Jiří Nádvorník -
Kostis Kyzirakos