Catalog Access: Querying Sky Regions in User-Specified Coordinate Frame#


Learning Goals#

By the end of this tutorial, you will:

  • Understand how to use astropy regions.SphericalSkyRegion region instances to define a selection region in an arbitrary coordinate frame.

  • Understand how to use the FrameTransformerHelper helper class included with this notebook to conveniently obtain coordinate-transformed query syntax from a SphericalSkyRegion instance, and post-process the results table to only include objects within the selected region (in the original frame) and to add columns containing the original frame spatial coordinates.

Table of Contents#

Introduction#

This notebook demonstrates how to translate selection regions of interest defined in a non-ICRS frame, translate that region on the sky to ICRS (the typical coordinate frame used to specify object coordinates in MAST’s and other archives’ catalogs), perform a spatial query, and then ensure the results catalog is populated with the longitude and latitude information for our non-ICRS frame.

The methods demonstrated in this tutorial are broadly applicable. These methods can be used for any of MAST’s catalog holdings — from PanSTARRS (as shown in this notebook) to Roman. Furthermore, these tools can be used for coordinate transform queries of catalogs at other archives (either directly using regions.SphericalSkyRegions to specify ICRS-frame direct/bounding regions, or using the helper class/query syntax with modifications as needed).

As an example, this tutorial demonstrates how to query the MAST PanSTARRS DR2 catalog for objects within regions defined in the Galactic coordinate frame, leveraging the astropy regions package to transform the selection region from Galactic to ICRS coordinates, peforming a catalog query, and then post-processing to add Galactic coordinates to the results table (via an included helper class).

While some catalogs already provide Galactic coordinates (\(\ell, b\)), this generalized approach supports:

  • Selection regions defined in arbitrary, user-specified coordinate frames (such as the Heliocentric spherical coordinate system for the orbit of the GD1 stream described in Koposov et al. 2010, as provided by the gala package).

  • Potential performance improvements by transforming the query to use ICRS coordinates, even if the coordinate frame longitude and latitude are columns (if the database is not spatially indexed on the coordinate in questions, while ICRS position columns are).

Imports#

This tutorial makes use of the following libraries:

%matplotlib inline

import numpy as np
from astropy.coordinates import Angle
import astropy.units as u
from astroquery.mast import Catalogs
from astropy import table
import pyvo as vo
import matplotlib.pyplot as plt
import time
import datetime

from regions import RangeSphericalSkyRegion

# Helper classes for handling coordinate transformations with queries:
from frame_transformer_helper import FrameTransformerHelper

Performing Spatial Catalogs Selections with Changes of Coordinate Frame#

In this notebook, we will construct a sample of bright (\(i<18\)), blue (\(g-y < 0.8\)) stars located along the Galactic plane, combining color, magnitude, and spatial constraints, using PanSTARRS DR1 catalogs.

We will demonstrate how to perform this query using both (1) astroquery and (2) MAST’s Table Access Protocol (TAP) service using the Astronomical Data Query Language (ADQL).

Identifying blue stars on the Galactic plane#

After consulting the PanSTARRS DR2 catalogs documentation, we decide to use the mean_object table. This table provides the relevant photometric (columns gmeanpsfmag, imeanpsfmag, ymeanpsfmag, optimized for point sources; and imeankronmag, used to distinguish between stars and galaxies) and position (ramean, decmean) information.

We define our sample selection as:

  • Located within \(\pm\)0.1 degree of the Galactic plane (\(|b| \leq 0.1\)deg)

  • Have \(i\) band magnitudes brighter than 18

  • Have \(g-y\) colors less than 0.8 (the bluest, reddest filters for PanSTARRS)

  • Are point sources (not extended objects; point sources typically have imeanpsfmag-imeankronmag <= 0.05, as presented in the PanSTARRS documentation).

We additionally apply quality cuts to ensure flux is detected in the \(g\), \(i\), \(y\) bands (magnitudes > -999, the missing value), and there are at least 3 detection in each of these bands (ng, ni, ny > 3; to avoid data artifacts).

Using astroquery#

To begin, we will implement this catalog search using astroquery.

We will begin by defining a small, test region of a limited \((\ell,b)\) range, as we can include only a subset of these constraints in the database query. The remaining constraints — the color cut \(g-y<0.8\), and removing extended sources — will need to be applied in-memory after obtaining the initial results table.

Defining a subset selection region#

We begin by defining a small subset, covering a limited range of \(\ell=[100,101]\) deg, and \(b=[-0.1,0.1]\) deg, using the RangeSphericalSkyRegion class. Note that we must specify the coordinate frame, as either an astropy BaseCoordinateFrame instance or a string indicating an astropy-registered coordinate frame.

# Subset selection region:
lon_range = np.array([100, 101])*u.deg
lat_range = np.array([-0.1, 0.1])*u.deg

sel_region = RangeSphericalSkyRegion(
    frame="galactic", 
    longitude_range=lon_range,
    latitude_range=lat_range, 
)
sel_region
<RangeSphericalSkyRegion(
frame=galactic,
longitude_range=[100. 101.] deg,
latitude_range=[-0.1  0.1] deg
)>

Using the helper class#

We then create an instance of the FrameTransformerHelper class, which we will use to help prepare our transformed query constraints and ensure the results are translated back to our coordinate frame (for this example, Galactic coordinates).

We specify our selection spherical sky region and our target coordinate frame for our query (ICRS).

transf_helper = FrameTransformerHelper(
    sel_region=sel_region, 
    frame_target="icrs"
)

This transformation helper contains a property sel_region_target that represents the transformation of our region into the ICRS coordinate frame:

transf_helper.sel_region_target
<RangeSphericalSkyRegion(
frame=icrs,
longitude_bounds=<LuneSphericalSkyRegion(center_gc1=<SkyCoord (ICRS): (ra, dec) in deg
    (91.94042232, 20.2900295)>, center_gc2=<SkyCoord (ICRS): (ra, dec) in deg
    (272.45586549, -19.41540611)>)>,
latitude_bounds=<CircleAnnulusSphericalSkyRegion(center=<SkyCoord (ICRS): (ra, dec) in deg
    (192.85947789, 27.12825241)>, inner_radius=89.9 deg, outer_radius=90.1 deg)>
)>

This is a convenience method for other internal helper processing that is equivalent to transforming the original region directly:

sel_region.transform_to("icrs")
<RangeSphericalSkyRegion(
frame=icrs,
longitude_bounds=<LuneSphericalSkyRegion(center_gc1=<SkyCoord (ICRS): (ra, dec) in deg
    (91.94042232, 20.2900295)>, center_gc2=<SkyCoord (ICRS): (ra, dec) in deg
    (272.45586549, -19.41540611)>)>,
latitude_bounds=<CircleAnnulusSphericalSkyRegion(center=<SkyCoord (ICRS): (ra, dec) in deg
    (192.85947789, 27.12825241)>, inner_radius=89.9 deg, outer_radius=90.1 deg)>
)>

Obtaining transformed query spatial constraints#

We now need to use this transformed region to determing our spatial constraints.

Currently, the astroquery.mast Catalogs module only supports cone searches. Thus, we will need to use the bounding circle of this transformed region.

Our helper class can provide specifically formatted spatial constraints using the get_bounds_constraints() method. The arguments specify the search type (bounding circle, bounding lon/lat, polygon), the output format (either “ADQL” or “astroquery.mast”), and the column names for longitude, latitude in the database table (“keys_pos_target”; for the PanSTARRS mean object table, these are raMean, decMean

We now use this method to obtain the bounding circle constraints for use with astroquery.

spatial_constraints = transf_helper.get_bounds_constraints(
    search_type="bound_circle", output_format="astroquery.mast", 
)
spatial_constraints
{'coordinates': <SkyCoord (ICRS): (ra, dec) in deg
     (330.70354523, 55.35053603)>,
 'radius': <Angle 0.5099017 deg>}

Constructing and submitting the query#

We now submit the full query using the Catalogs.query_criteria() method, as we will be using a mix of spatial and non-spatial filters. We then apply the post-processing cuts (as noted above) on the preliminary table, to obtain our final sample in this subset region.

# Submit query:
start = time.time()

results = Catalogs.query_criteria(
    collection="ps1_dr2",
    catalog="mean_object",
    **spatial_constraints, 
    select_cols=["objID", "raMean", "decMean",
                 "gMeanPSFMag", "iMeanPSFMag", "yMeanPSFMag",
                 "iMeanKronMag"],    # List of columns to return
    iMeanPSFMag=["<18", ">-999"],  # i < 18; >-999 (missing mag); i band constraint
    gMeanPSFMag=">-999",   # mag > -999; missing magnitudes are -999
    yMeanPSFMag=">-999",   # mag > -999; missing magnitudes are -999
    iMeanKronMag=">-999",  # mag > -999; missing magnitudes are -999
    ng=">3",   # ng > 3; Number of detections in g
    ni=">3",   # ni > 3; Number of detections in i
    ny=">3",   # ny > 3; Number of detections in y
    limit=100000,  # Maximum number of rows to return
)

end = time.time()
print(f"Elapsed time: {str(datetime.timedelta(seconds=end-start))}")

# Note: the returned table column names are all lower case, 
# which we use below.

# Apply post-process cuts:
# 1. Distinguish stars from extended objects using
#    difference of PSF, Kron magnitudes
results = results[
    (results['imeanpsfmag'] - results['imeankronmag']) <= 0.05
]

# 2. Apply color cut
# Copy original catalog for reference:
results = results[
    (results['gmeanpsfmag'] - results['ymeanpsfmag']) < 0.8
]
len(results)
Elapsed time: 0:00:03.874060
204

Using the helper class to transform and trim the results#

Finally, we will use the helper class to post-process the returned table using the helper’s parse_results() method:

  1. As we used a bounding circle for our query, we need to remove entries outside our specified region.

  2. To easily work with our results, the helper class will also add derived columns containing the transformed coordinates. This requires the database position columns to be specified. Here, we set keys_pos_target=["raMean", "decMean"].

By default, the names for the derived columns will be inferred from the coordinate frame longitude/latitude names. However, custom names can be specified using keys_pos_orig (e.g., keys_pos_orig=["l_derived", "b_derived"]). The default units of the output coordinates will be degrees; this can be customized by setting units_orig to some other astropy angular unit.

results = transf_helper.parse_results(
    results,
    keys_pos_target=["ramean", "decmean"]
)
results
Table length=41
objidrameandecmeangmeanpsfmagimeanpsfmagymeanpsfmagimeankronmaglb
degdegmagmagmagmagdegdeg
int64float64float64float32float32float32float32float64float64
174173304387015951330.438696848430555.1460951105436714.961214.332214.205114.3733100.25666651226535-0.0733239967537609
174153300974792642330.0974604145970455.1266484689919214.447113.769313.650113.8123100.089245652757740.028639223390290715
174243303985338935330.3985237237654455.206901250303914.729814.193713.955514.2316100.27484352646091-0.010927951631761761
174053302446793639330.2446857961026655.0441309982182114.365913.866913.711513.8605100.10664775665033-0.08800943466668062
174073301024487454330.1024182366350655.063959093977213.0516.982916.376517.093100.0536271259632-0.023026302327829262
174123301453439176330.1454026204783655.107089955726514.746814.174614.054814.1708100.09928478725283-0.0035091430797302824
174263306037985069330.6038032364570555.2203645311972814.45113.938213.69613.9642100.37664058463054-0.07038129371668994
174143304395985727330.4396059865632755.1208973247362214.750714.101113.975414.1438100.24195452440424-0.09378753501098032
174173303324476748330.3325495255.1468405910.880710.202311.621713.3527100.20862969354452-0.03627276030493398
...........................
174833312484521721331.2484457655.6925205614.857514.356614.163214.3965100.951306379679850.09178800297733512
174493310413019407331.0412899655.4155953515.390614.893814.678514.9507100.69289120671188-0.06196039125780394
174503310746027019331.0745976119874355.4219403235917414.928914.3914.141714.4219100.71187408742465-0.06806709088980577
174723312420544300331.242062419242855.603025392062115.047214.513314.292814.5444100.895504575916720.021727161602114936
174643310880142655331.0880442255.5349629514.421913.933713.668913.9497100.785027450351310.018424365917864963
174663314701059833331.4701688655.5576023114.255713.787313.602213.8174100.97284241340091-0.09099780591410377
174723311320329719331.13204655.6075060413.84413.491813.371513.5626100.848056574071850.062113436088432
174553309943701839330.9943690655.4592725714.377514.066613.921814.0971100.69740207855197-0.011010456369010165
174743311344916461331.1345011755.6214482513.03815.889315.578315.9631100.85743240521830.07252505125748354

Our final sample in this subset region is small (41 objects).

Visualizing the subset region objects#

The targets selected in this subset Galactic longitude/latitute range are shown below in the ICRS (database) and Galactic (selection) frames:

fig, axes = plt.subplots(ncols=2, figsize=(9, 3))

axes[0].set_title("ICRS")
axes[0].set_xlabel("RA")
axes[0].set_ylabel("Dec")
axes[0].grid(True)
axes[0].scatter(
    results["ramean"], results["decmean"],
    marker=".", s=50, lw=0
)

axes[1].set_title("Galactic")
axes[1].set_xlabel(r"$\ell$")
axes[1].set_ylabel(r"$b$")
axes[1].grid(True)
axes[1].scatter(
    results["l"], results["b"],
    marker=".", s=50, lw=0
)

fig.subplots_adjust(top=0.95, bottom=0.0, wspace=0.3)

plt.show()
../../../_images/2bf98b84fc94965cd1cd4adc769e4db6aeefbd00bd288bc44cc79f081fa54eab.png

As this individual chunk query took about 30 seconds, it would take about 2.5 hours query over all chunks; we leave iterating over all chunks as an exercise to interested readers.

However, MAST’s PanSTARRS TAP service runs on a much more powerful database, making it possible to much more quickly iterate over all subsets to obtain the full sample across the whole Galactic plane, as shown below.

Using TAP#

We now demonstrate how to implement this search using ADQL to query MAST’s PanSTARRS TAP service.

This advanced query patterns possible with ADQL streamline this search, enabling us to avoid in-memory post-filtering (for the color and star/extended object cuts).

However, as currently this TAP service only supports cone searches, we will still need to chunk over multiple longitude ranges.

Querying over the full Galactic plane using TAP#

In this case, we’ll proceed directly iterating over all longitude ranges.

First, we connect to the MAST PanSTARRS DR2 TAP service.

# Use `pyvo` to connect to the MAST PanSTARRS TAP service:
TAP_service = vo.dal.TAPService(
    "https://mast.stsci.edu/vo-tap/api/v0.1/mast_catalogs/"
)

Again, we will access the mean object catalog (mean_object here; see the PanSTARRS DR2 catalogs documentation), which contains all the columns we will need.

(Note: this TAP service is backed by a new, more powerful database. The column names in this database are entirely lowercase, but are otherwise equivalent to the columns of the legacy database as documented above.)

We now construct our query, including the color/magnitude constraints and star/extended object separation. As an overview, our query is constructed as follows.

The table, ps1_dr2.mean_object, is specified using the “FROM” statement. We specify the coluns to be returned (objid, ramean, decmean, gmeanpsfmag, imeanpsfmag, ymeanpsfmag, imeankronmag) using the “SELECT” statement. The query constraints are specified in the “WHERE” statement, with multiple constraints linked using “AND”. This includes:

  • the cone search (as in the “CONTAINS(…)” clause; note that for ADQL, the radius must be expressed in degrees),

  • requiring the \(g\), \(i\), \(y\) mean PSF magnitudes and \(i\) mean Kron magnitude to be present (cutting objects that have any of these values missing),

  • the PanSTARRS-specific cut to select stars (imeanpsfmag - imeankronmag <= 0.05), and

  • the \(g-y\) color and \(i\) magnitude cuts.

As before, for each longitude chunk, we (1) define the subset region, (2) leverage the helper to define the ADQL-specific spatial constraint syntax, (3) integrate this into the full query, and (4) query the database and obtain the result, and (5) cut back to our selection region using the helper.

Because we can perform all non-spatial filtering server-side, we use wider longitude ranges (5 degrees) in this case (though still relatively small to ensure no truncation of returned rows).

Note! We are iterating over 72 queries; this cell will take approximately 90 seconds to finish running.

# Selection regions:
# Same latitude range for all chunks
lat_range = np.array([-0.1, 0.1])*u.deg

# Prepare aggregate table:
results_tap = None

start = time.time()
for lon in range(72):
    # Define subset longitude range:
    lon_range = np.array([lon, lon+1])*5*u.deg

    # Define subset spherical region:
    sel_region = RangeSphericalSkyRegion(
        frame="galactic",
        longitude_range=lon_range,
        latitude_range=lat_range,
    )

    # Define transform helper:
    transf_helper = FrameTransformerHelper(
        sel_region=sel_region,
        frame_target="icrs"
    )

    # Obtain query spatial constraints:
    spatial_constraints = transf_helper.get_bounds_constraints(
        search_type="bound_circle", output_format="ADQL",
        keys_pos_target=["ramean", "decmean"],
    )

    # Specify query
    # Note: ADQL comments are preceded by "--"
    adql_query = f"""
    SELECT objid, ramean, decmean, 
        gmeanpsfmag, imeanpsfmag, ymeanpsfmag, 
        imeankronmag
    FROM ps1_dr2.mean_object
    WHERE {spatial_constraints}                -- search constraints from the helper
    AND imeanpsfmag < 18                      -- magnitude cut
    AND (gmeanpsfmag-ymeanpsfmag) < 0.8       -- color cut
    AND (imeanpsfmag - imeankronmag) <= 0.05  -- distinguish stars from extended objects
    AND imeanpsfmag > -999                    -- mag > -999; missing magnitudes are -999
    AND gmeanpsfmag > -999                    -- mag > -999; missing magnitudes are -999
    AND ymeanpsfmag > -999                    -- mag > -999; missing magnitudes are -999
    AND imeankronmag > -999                   -- mag > -999; missing magnitudes are -999
    """

    # Submit query to the MAST TAP service:
    job = TAP_service.run_async(adql_query)
    
    # Retrieve results:
    results_tap_chunk = job.to_table()

    # Parse results & restrict to original region:
    results_tap_chunk = transf_helper.parse_results(results_tap_chunk)

    # Combine with the full results set:
    if results_tap is None:
        results_tap = results_tap_chunk
    else:
        results_tap = table.vstack([results_tap, results_tap_chunk])

end = time.time()
print(f"Elapsed time: {str(datetime.timedelta(seconds=end-start))}")

# Ensure all entries are unique:
results_tap = table.unique(results_tap, keys="objid")
results_tap
/home/runner/micromamba/envs/ci-env/lib/python3.13/site-packages/pyvo/dal/query.py:403: DALOverflowWarning: Results truncated due to server limits. Consider setting a maxrec value.
  warn("Results truncated due to server limits. Consider "
Elapsed time: 0:01:31.318323
Table length=29014
objidrameandecmeangmeanpsfmagimeanpsfmagymeanpsfmagimeankronmaglb
degdegmagmagmagmagdegdeg
int64float64float64float32float32float32float32float64float64
72001200013223769120.00131469904031-29.99708184179613515.739415.114514.98115.161247.08103767408764-0.04555065009153811
72001200684165401120.06841472723237-29.99569691961646415.266614.623314.490714.6652247.110444618094760.0045912063560663435
72001200956252668120.0956245894987-29.99802591366134415.24214.648814.524714.7048247.124835036161580.023396994510247286
72011199430687271119.94304480361075-29.98586072076894414.048913.437613.346213.4685247.04495553333217-0.0825772280946197
72011200126627280120.01264330895721-29.985792521760514.664514.070413.918814.1092247.0765987945898-0.03126727637247311
72011200843455834120.08433206969568-29.9870528317438215.016414.362114.241914.3994247.11035602164680.02086286194769272
72011201614017443120.16137299352742-29.98573308862824814.628414.243414.162414.2586247.144397133146780.07827055962941741
72011201856516198120.18563810356255-29.98674807670708716.842716.308816.102616.3312247.156342454262130.09559276915387996
72021199690015286119.96899020537775-29.97915718752237516.108215.640115.43315.699247.0510676721462-0.05993589389327513
...........................
18354013553965503913.5539908064628862.95357789609345614.937814.390614.156514.42123.247722992579170.0835337399315858
18354013782067323413.78209201558703362.95206772283416514.465614.192614.115314.2356123.351456927922210.08332636565422787
18354014397162734314.39715778673759262.9554970921881215.45415.031914.857315.0816123.63101560484880.09209872977637203
18354014533273742214.53326004430313662.95555405032380515.328914.867214.6814.914123.692878251573290.09369897260282552
18355012846918948312.84691733529748762.9656108064237314.822114.262614.047314.3019122.926216156930020.09386377833133315
18355013461388131913.46144365082202362.9587946019647214.758714.229914.006314.2486123.205593980824490.08832648911503542
18355013702353910913.70237850724543462.9652989135104314.317813.823913.626113.8501123.31503848927710.09605939851848017
18355013993316943013.99335399565444562.9655602490834416.114715.7215.53615.7852123.447275481357150.09835111784095454
18355014316537653914.31650081386591862.96309692978914515.442114.993914.811415.0204123.594180187302640.09884374404115584

Visualizing TAP results across the full Galactic plane#

We now display our final sample using a full-sky Aitoff projection, first in ICRS (left) and then Galactic (right) coordinates.

(Note that the columns from the TAP results table have units.)

fig, axes = plt.subplots(
    ncols=2,
    figsize=(15, 5),
    subplot_kw=dict(projection="aitoff")
)

axes[0].set_title("ICRS, Aitoff projection", pad=15)
axes[0].set_xlabel("RA")
axes[0].set_ylabel("Dec")
axes[0].grid(True)
axes[0].scatter(
    Angle(results_tap["ramean"]).wrap_at(180 * u.deg).radian, 
    Angle(results_tap["decmean"]).radian, 
    marker=".", s=5, lw=0
)

axes[1].set_title("Galactic, Aitoff projection", pad=15)
axes[1].set_xlabel(r"$\ell$")
axes[1].set_ylabel(r"$b$")
axes[1].grid(True)
axes[1].scatter(
    Angle(results_tap["l"]).wrap_at(180 * u.deg).radian,
    Angle(results_tap["b"]).radian,
    marker=".", s=5, lw=0
)

fig.subplots_adjust(top=0.95, bottom=0.0)
../../../_images/24990d14d3b4b94a8c78a3e250d3eebe94c3e7606b0b4dd03a2d6fa0400360eb.png

We can see the observational limitation of PanSTARRS (observing only Decl > -30 degrees) clearly in the left panel (showing ICRS), while the Galactic coordinates of the right panel very closely follow the Galactic midplane (as expected given our spatial constraint).


Additional Resources#

Astropy Coordinate Transformations#

  • The astropy.coordinates documentation provides full details about transforming between coordinate frames using astropy classes.

  • Custom coordinate frames can be defined for use with the astropy.coordinates functionality. To support transformations between custom frames and other frames such as ICRS, it is also necessary to define a transformation to transform between the custom frame and (at least) one of frames included in astropy.coordinates. See the astropy.coordinates documentation for full details.

Astroquery.mast#

Table Access Protocol (TAP) & the Astronomy Query Data Language (ADQL)#

PanSTARRS 1 DR 2#

Citations#

If you use astropy for published research, please cite the authors. Follow these links for more information about citing astropy:

If you use PanSTARRS data accessed through MAST for published research, please include the following acknowledgements, found at the following links:

About This Notebook#

Authors: Sedona Price
Keywords: Tutorial, coordinate frames, astroquery, TAP
Last updated: August 2026


Top of Page Space Telescope Logo