MAST Table Access Protocol PanSTARRS 1 DR2 Demo#


This tutorial demonstrates how to use astroquery to access PanSTARRS 1 Data Release 2 via a Virtual Observatory standard Table Access Protocol (TAP) service at MAST, and work with the resultant data. It relies on Python 3 and astroquery, as well as some other common scientific packages.


Table of Contents#

  1. TAP Service Introduction

  2. Imports

  3. Connecting to a TAP Service

  4. Use Cases

  5. Additional Resources

  6. About This Notebook


TAP Service Introduction#

Table Access Protocol (TAP) services allow more direct and flexible access to astronomical data than the simpler types of IVOA standard data services. Queries are built with the SQL-like Astronomical Data Query Language (ADQL), and can include geographic / spatial queries as well as filtering on other characteristics of the data. This also allows the user fine-grained control over the returned columns, unlike the fixed set of coumns returned from cone, image, and spectral services.

For this example, we’ll be using the astropy affiliated PyVO client, which is interoperable with other valid TAP services, including those at MAST. PyVO documentation is available at ReadTheDocs: https://pyvo.readthedocs.io

We’ll be using PyVO to call the TAP service at MAST serving PanSTARRS 1 Data Release 2, now with individual detection information. The schema is described within the service, and we’ll show how to inspect it. The schema is also the same as the one available via the CasJobs interface, with an additional view added for the most common positional queries. CasJobs has its own copy of the schema documentation, which can be accessed through its own site: http://mastweb.stsci.edu/ps1casjobs/


Imports#

# Use the pyvo library as our client to the data service.
import pyvo as vo

# For resolving objects with tools from MAST
from astroquery.mast import Mast

# For handling ordinary astropy Tables in responses
from astropy.table import Table

# For displaying and manipulating some types of results
%matplotlib inline
import requests
import astropy
import numpy as np
import pylab
import time
import json
from matplotlib import pyplot as plt

# suppress unimportant unit warnings from many TAP services
import warnings
warnings.filterwarnings("ignore", module="astropy.io.votable.*")
warnings.filterwarnings("ignore", module="pyvo.utils.xml.elements")

Connecting to a TAP Service#

The PyVO library is able to connect to any TAP service, given the “base” URL as noted in metadata registry resources describing the service. This is the URL for the PanSTARRS 1 DR2 TAP service.

TAP_service = vo.dal.TAPService("https://mast.stsci.edu/vo-tap/api/v0.1/ps1dr2/")
TAP_service.describe()
Capability ivo://ivoa.net/std/TAP

Interface vs:ParamHTTP
    https://mast.stsci.edu/vo-tap/api/v0.1/ps1dr2

Language ADQL
Output format application/x-votable+xml
    Also available as votable

Output format text/csv;header=present
    Also available as csv

Maximum size of resultsets
    Default 100000 row
    Maximum 100000 row


Capability ivo://ivoa.net/std/DALI#examples

Interface vr:WebBrowser
    https://mast.stsci.edu/vo-tap/api/v0.1/ps1dr2/examples
WARNING: AstropyDeprecationWarning: The indent function is deprecated and may be removed in a future version.
        Use textwrap.indent() instead. [pyvo.io.vosi.voresource]
WARNING: AstropyDeprecationWarning: The indent function is deprecated and may be removed in a future version.
        Use textwrap.indent() instead. [pyvo.io.vosi.tapregext]
WARNING: AstropyDeprecationWarning: The indent function is deprecated and may be removed in a future version.
        Use textwrap.indent() instead. [pyvo.io.vosi.tapregext]
WARNING: AstropyDeprecationWarning: The indent function is deprecated and may be removed in a future version.
        Use textwrap.indent() instead. [pyvo.io.vosi.tapregext]

List available tables#

There are MANY tables available, so we’ll only print out the first few. Use the commented-out for loop to see them all.

TAP_tables = TAP_service.tables

# only show the first three tables
for i, tablename in enumerate(TAP_tables.keys()):
    if i < 10 and not "tap_schema" in tablename:
        TAP_tables[tablename].describe()
        print("Columns={}".format(sorted([k.name for k in TAP_tables[tablename].columns ])))
        print("----")

## PRINT ALL TABLES
# for tablename in TAP_tables.keys():
#     if not "tap_schema" in tablename:  
#         TAP_tables[tablename].describe()
#         print("Columns={}".format(sorted([k.name for k in TAP_tables[tablename].columns ])))
#         print("----")
dbo.Detection
No description

Columns=['airMass', 'apFillF', 'apFlux', 'apFluxErr', 'apRadius', 'dec', 'decErr', 'detectID', 'dvoRegionID', 'expTime', 'extNSigma', 'filterID', 'imageID', 'infoFlag', 'infoFlag2', 'infoFlag3', 'ippDetectID', 'ippObjID', 'kronFlux', 'kronFluxErr', 'kronRad', 'momentM3C', 'momentM3S', 'momentM4C', 'momentM4S', 'momentR1', 'momentRH', 'momentXX', 'momentXY', 'momentYY', 'objID', 'obsTime', 'pltScale', 'posAngle', 'processingVersion', 'psfChiSq', 'psfCore', 'psfFlux', 'psfFluxErr', 'psfLikelihood', 'psfMajorFWHM', 'psfMinorFWHM', 'psfQf', 'psfQfPerfect', 'psfTheta', 'ra', 'raErr', 'randomDetID', 'sky', 'skyErr', 'surveyID', 'telluricExt', 'uniquePspsP2id', 'xPos', 'xPosErr', 'yPos', 'yPosErr', 'zp']
----
dbo.DetectionFlags
No description

Columns=['"value"', 'description', 'hexadecimal', 'name']
----
dbo.DetectionFlags2
No description

Columns=['"value"', 'description', 'hexadecimal', 'name']
----
dbo.DetectionFlags3
No description

Columns=['"value"', 'description', 'hexadecimal', 'name']
----
dbo.DetectionObjectView
No description

Columns=['airMass', 'apFillF', 'apFlux', 'apFluxErr', 'apRadius', 'b', 'batchID', 'beta', 'cx', 'cy', 'cz', 'dec', 'decErr', 'decMean', 'decMeanErr', 'decStack', 'decStackErr', 'detectID', 'dvoRegionID', 'epochMean', 'expTime', 'extNSigma', 'filterID', 'gFlags', 'gMeanApMag', 'gMeanApMagErr', 'gMeanApMagNpt', 'gMeanApMagStd', 'gMeanKronMag', 'gMeanKronMagErr', 'gMeanKronMagNpt', 'gMeanKronMagStd', 'gMeanPSFMag', 'gMeanPSFMagErr', 'gMeanPSFMagMax', 'gMeanPSFMagMin', 'gMeanPSFMagNpt', 'gMeanPSFMagStd', 'gQfPerfect', 'htmID', 'iFlags', 'iMeanApMag', 'iMeanApMagErr', 'iMeanApMagNpt', 'iMeanApMagStd', 'iMeanKronMag', 'iMeanKronMagErr', 'iMeanKronMagNpt', 'iMeanKronMagStd', 'iMeanPSFMag', 'iMeanPSFMagErr', 'iMeanPSFMagMax', 'iMeanPSFMagMin', 'iMeanPSFMagNpt', 'iMeanPSFMagStd', 'iQfPerfect', 'imageID', 'infoFlag', 'infoFlag2', 'infoFlag3', 'ippObjID', 'kronFlux', 'kronFluxErr', 'kronRad', 'l', 'lambda', 'momentM3C', 'momentM3S', 'momentM4C', 'momentM4S', 'momentR1', 'momentRH', 'momentXX', 'momentXY', 'momentYY', 'nDetections', 'nStackDetections', 'nStackObjectRows', 'ng', 'ni', 'nr', 'ny', 'nz', 'objAltName1', 'objAltName2', 'objAltName3', 'objID', 'objInfoFlag', 'objName', 'obsTime', 'pltScale', 'posAngle', 'posMeanChisq', 'processingVersion', 'projectionID', 'psfChiSq', 'psfCore', 'psfFlux', 'psfFluxErr', 'psfLikelihood', 'psfMajorFWHM', 'psfMinorFWHM', 'psfQf', 'psfQfPerfect', 'psfTheta', 'qualityFlag', 'rFlags', 'rMeanApMag', 'rMeanApMagErr', 'rMeanApMagNpt', 'rMeanApMagStd', 'rMeanKronMag', 'rMeanKronMagErr', 'rMeanKronMagNpt', 'rMeanKronMagStd', 'rMeanPSFMag', 'rMeanPSFMagErr', 'rMeanPSFMagMax', 'rMeanPSFMagMin', 'rMeanPSFMagNpt', 'rMeanPSFMagStd', 'rQfPerfect', 'ra', 'raErr', 'raMean', 'raMeanErr', 'raStack', 'raStackErr', 'randomDetID', 'randomID', 'sky', 'skyCellID', 'skyErr', 'surveyID', 'telluricExt', 'tessID', 'uniquePspsOBid', 'uniquePspsP2id', 'xPos', 'xPosErr', 'yFlags', 'yMeanApMag', 'yMeanApMagErr', 'yMeanApMagNpt', 'yMeanApMagStd', 'yMeanKronMag', 'yMeanKronMagErr', 'yMeanKronMagNpt', 'yMeanKronMagStd', 'yMeanPSFMag', 'yMeanPSFMagErr', 'yMeanPSFMagMax', 'yMeanPSFMagMin', 'yMeanPSFMagNpt', 'yMeanPSFMagStd', 'yPos', 'yPosErr', 'yQfPerfect', 'zFlags', 'zMeanApMag', 'zMeanApMagErr', 'zMeanApMagNpt', 'zMeanApMagStd', 'zMeanKronMag', 'zMeanKronMagErr', 'zMeanKronMagNpt', 'zMeanKronMagStd', 'zMeanPSFMag', 'zMeanPSFMagErr', 'zMeanPSFMagMax', 'zMeanPSFMagMin', 'zMeanPSFMagNpt', 'zMeanPSFMagStd', 'zQfPerfect', 'zoneID', 'zp']
----

Use Cases#

Simple Positional Query#

This searches the mean object catalog for objects within .2 degrees of M87 (RA=187.706, Dec=12.391 in degrees). The view used contains information from the ObjectThin table (which has information on object positions and the number of available measurements) and the MeanObject table (which has information on photometry averaged over the multiple epochs of observation).

Note that the results are restricted to objects with nDetections>1, where nDetections is the total number of times the object was detected on the single-epoch images in any filter at any time. Objects with nDetections=1 tend to be artifacts, so this is a quick way to eliminate most spurious objects from the catalog.

This query runs in TAP’s asynchronous mode, which is a queued batch mode with some overhead and longer timeouts, useful for big catalogs like PanSTARRS. It may not be necessary for all queries to PS1 DR2, but the PyVO client can automatically handle the additional processing required over synchronous mode.

job = TAP_service.run_async("""
SELECT objID, RAMean, DecMean, nDetections, ng, nr, ni, nz, ny, gMeanPSFMag, rMeanPSFMag, iMeanPSFMag, zMeanPSFMag, yMeanPSFMag
FROM dbo.MeanObjectView
WHERE
CONTAINS(POINT('ICRS', RAMean, DecMean),CIRCLE('ICRS',187.706,12.391,.2))=1
AND nDetections > 1
  """)
TAP_results = job.to_table()
TAP_results
Table length=6927
objidrameandecmeanndetectionsngnrninznygmeanpsfmagrmeanpsfmagimeanpsfmagzmeanpsfmagymeanpsfmag
degdegmagmagmagmagmag
int64float64float64int16int16int16int16int16int16float32float32float32float32float32
122861877059169881187.7059100330960212.391133459973965410101481210.29510.73211.07611.34211.557
122861877056308967187.7055994812.39030211200002-999.0-999.0-999.0-999.014.9985
122871877063310741187.7063111873214312.39181079630862302010-999.014.4229-999.014.4249-999.0
122861877058698594187.7058091912.39013311200002-999.0-999.0-999.0-999.014.7305
122861877050678994187.7051254260855212.39043055705990652210017.028715.084316.169-999.0-999.0
122871877062551436187.7062236512.39236505200002-999.0-999.0-999.0-999.015.1102
122861877044629638187.7045159812.39086724200020-999.0-999.0-999.015.5294-999.0
122861877056688054187.7055899612.38960002200020-999.0-999.0-999.014.4064-999.0
122871877075050430187.7074791212.3915839200020-999.0-999.0-999.015.5752-999.0
..........................................
122901875031002013187.5031451810709512.41791544021220537710117221.232620.747320.574220.680320.2113
122661878055210091187.8055073626471512.2162813978174585713141261220.905920.351620.069919.995619.8391
122631877536376320187.7536933612.19655761200020-999.0-999.0-999.021.0599-999.0
123031875562633406187.5562239304035812.527354381090412300300-999.0-999.020.5518-999.0-999.0
122681878343382931187.8343815572258412.235251674424532245964022.091221.704221.306821.0164-999.0
123061878259604285187.8258887212.5531376321010021.9049-999.021.4564-999.0-999.0
122751878844752179187.8844679363601512.29302227841492977162414111221.462320.355520.139420.036819.8008
123001875376856622187.5378508678950312.505173082359276403100-999.021.188721.1419-999.0-999.0
123101876640214622187.6639871776572512.586746063373354300210-999.0-999.021.853121.6726-999.0

Get DR2 light curve for RR Lyrae star KQ UMa#

This time we start with the object name, use the MAST name resolver (which relies on Simbad and NED) to convert the name to RA and Dec, and then query the PS1 DR2 mean object catalog at that position. Then we run a spatial query to TAP using those coordinates.

objname = 'KQ UMa'
coords = Mast.resolve_object(objname)
ra,dec = coords.ra.value,coords.dec.value
radius = 1.0/3600.0 # radius = 1 arcsec

query = """
SELECT objID, RAMean, DecMean, nDetections, ng, nr, ni, nz, ny, gMeanPSFMag, 
    rMeanPSFMag, iMeanPSFMag, zMeanPSFMag, yMeanPSFMag
FROM dbo.MeanObjectView
WHERE
CONTAINS(POINT('ICRS', RAMean, DecMean),CIRCLE('ICRS',{},{},{}))=1
AND nDetections > 1
""".format(ra,dec,radius)
print(query)

job = TAP_service.run_async(query)
TAP_results = job.to_table()
TAP_results
SELECT objID, RAMean, DecMean, nDetections, ng, nr, ni, nz, ny, gMeanPSFMag, 
    rMeanPSFMag, iMeanPSFMag, zMeanPSFMag, yMeanPSFMag
FROM dbo.MeanObjectView
WHERE
CONTAINS(POINT('ICRS', RAMean, DecMean),CIRCLE('ICRS',139.33446271609,68.63508880829,0.0002777777777777778))=1
AND nDetections > 1
Table length=1
objidrameandecmeanndetectionsngnrninznygmeanpsfmagrmeanpsfmagimeanpsfmagzmeanpsfmagymeanpsfmag
degdegmagmagmagmagmag
int64float64float64int16int16int16int16int16int16float32float32float32float32float32
190361393344112894139.3344530533415868.635059161692316681021131415.040214.55314.210914.281414.3041

Get Repeated Detection Information#

Extract all the objects with the same object ID from the Detection table, which contains all the individual measurements for this source. The results are joined to the Filter table to convert the filter numbers to names.

objid = TAP_results['objid'][0]
query = """
SELECT
    objID, detectID, Detection.filterID as filterID, Filter.filterType, obsTime, ra, dec,
    psfFlux, psfFluxErr, psfMajorFWHM, psfMinorFWHM, psfQfPerfect, 
    apFlux, apFluxErr, infoFlag, infoFlag2, infoFlag3
FROM Detection
NATURAL JOIN Filter
WHERE objID={}
ORDER BY filterID, obsTime
""".format(objid)
print(query)

job = TAP_service.run_async(query)
detection_TAP_results = job.to_table()
detection_TAP_results
SELECT
    objID, detectID, Detection.filterID as filterID, Filter.filterType, obsTime, ra, dec,
    psfFlux, psfFluxErr, psfMajorFWHM, psfMinorFWHM, psfQfPerfect, 
    apFlux, apFluxErr, infoFlag, infoFlag2, infoFlag3
FROM Detection
NATURAL JOIN Filter
WHERE objID=190361393344112894
ORDER BY filterID, obsTime
Table length=66
objIDdetectIDfilteridfilterTypeobsTimeradecpsfFluxpsfFluxErrpsfMajorFWHMpsfMinorFWHMpsfQfPerfectapFluxapFluxErrinfoFlaginfoFlag2infoFlag3
yrdegdegpixpix
int64int64uint8objectfloat64float64float64float32float32float32float32float32float32float32int64int32int32
1903613933441128941533477163100000101g55634.477414139.334520768.635035770.008261921.14074e-051.881851.767340.9929160.008617051.14233e-05102760517128124782656
1903613933441128941533489683100000081g55634.4899457139.3344882168.635061460.00773731.10268e-051.810311.605180.9984610.007921721.09406e-05102760517128124782656
1903613933441128942322287915600000171g56423.2881719139.3344944168.635039520.003351987.26745e-061.602041.450480.9985890.003386587.2363e-06102760517128108038208
1903613933441128942555598663700000151g56656.5989211139.3344459268.635045890.003729097.37688e-061.828311.686920.9991910.003721637.52088e-06102760517128124815424
1903613933441128942620400703700000161g56721.4009635139.3344514868.635048070.0035767.22894e-061.605851.503960.9986050.003596037.43463e-061027605171287374912
1903613933441128942620407083700000141g56721.4073411139.3344530468.635044060.003505547.14809e-061.624151.488270.9994870.003596247.43434e-061027605171287374912
1903613933441128942642318642600000221g56743.3189045139.334443168.635044970.003438677.81574e-061.539571.47610.9990490.003612397.85237e-061027605171287374912
1903613933441128942642325162600000241g56743.3254198139.3344451268.635052850.003474037.87789e-061.527511.433250.9993670.003603497.8686e-061027605171287374912
1903613933441128941534413404100000122r55635.4136426139.3344717368.63505340.009787581.10843e-051.678791.56530.9985090.009907631.11231e-05102760517128124815424
...................................................
1903613933441128941832551635200000155y55933.5518128139.3344423268.635056690.009155732.64422e-051.113310.953680.9971150.009145152.48501e-0510276051712874483776
1903613933441128941917267252000000115y56018.2674345139.3344607968.635064030.008296462.59461e-051.073321.015630.9979240.008337592.33303e-051027605171287374912
1903613933441128941917273692000000125y56018.2738664139.3344518968.635061520.008199962.55524e-051.095041.059140.9978190.008207532.3153e-051027605171287374912
1903613933441128942291225691500000145y56392.2258719139.3344274268.635062340.007804222.57689e-051.028780.8931950.9986180.007772792.16455e-051027605171287374912
1903613933441128942291232731500000115y56392.2329084139.3344459568.635057440.008035032.55974e-050.9418010.8887880.998050.008070372.2092e-051027605171287374912
1903613933441128942643226481500000245y56744.2269452139.33445468.635058640.005404461.32146e-051.388291.34770.9989220.005498671.13527e-05102760517128124782656
1903613933441128942656230081500000275y56757.2305455139.3344431968.635063750.006366141.60784e-050.8950410.8033650.8525160.004128591.01303e-0510276058112834880
1903613933441128942871653746300000235y56972.6539243139.3344422668.635060870.006878172.79326e-051.536861.288130.9916470.006999142.16585e-051027605171287374912
1903613933441128942872651166300000285y56973.6513438139.3344499168.635057590.006653772.7704e-051.856191.703970.998640.006631062.13156e-051027605171287342144

Plot the light curves#

The psfFlux values from the Detection table are converted from Janskys to AB magnitudes. Measurements in the 5 different filters are plotted separately.

# convert flux in Jy to magnitudes
t = detection_TAP_results['obsTime']
mag = -2.5*np.log10(detection_TAP_results['psfFlux']) + 8.90
xlim = np.array([t.min(),t.max()])
xlim = xlim + np.array([-1,1])*0.02*(xlim[1]-xlim[0])

pylab.rcParams.update({'font.size': 14})
pylab.figure(1,(10,10))

#detection_TAP_results['filterType'] is a byte string, compare accordingly:
for i, filter in enumerate(['g','r','i','z','y']):
    
    pylab.subplot(511+i)
    w = np.where(detection_TAP_results['filterType'] == filter)  
    pylab.plot(t[w],mag[w],'-o')
    pylab.ylabel(f'{filter} [mag]')
    pylab.xlim(xlim)
    pylab.gca().invert_yaxis()
    if i==0:
        pylab.title(objname)
        
pylab.xlabel('Time [MJD]')
pylab.tight_layout()
../../../_images/5ee2b36dd64b4336baf992c79bb6a9e87fdb11b696e991b0a33a9289e667b139.png

Plot differences from the mean magnitudes in the initial search.

# convert flux in Jy to magnitudes
t = detection_TAP_results['obsTime']
mag = -2.5*np.log10(detection_TAP_results['psfFlux']) + 8.90
xlim = np.array([t.min(),t.max()])
xlim = xlim + np.array([-1,1])*0.02*(xlim[1]-xlim[0])

pylab.rcParams.update({'font.size': 14})
pylab.figure(1,(10,10))

#detection_TAP_results['filterType'] is a byte string, compare accordingly:
for i, filter in enumerate(['g','r','i','z','y']):
    pylab.subplot(511+i)
    w = np.where(detection_TAP_results['filterType']==filter)
    magmean = TAP_results[f'{filter}meanpsfmag'][0]
    pylab.plot(t[w],mag[w] - magmean,'-o')
    pylab.ylabel(f'{filter} [mag - {np.round(float(magmean), 4)}]')
    pylab.xlim(xlim)
    pylab.gca().invert_yaxis()
    if i==0:
        pylab.title(objname)
        
pylab.xlabel('Time [MJD]')
pylab.tight_layout()
../../../_images/48cdb23881ab59d3eb0512647842185071507b850d5339a2873a6fced4fea261.png

Identify bad data#

There is one clearly bad \(z\) magnitude with a very large difference. Select the bad point and look at it in more detail.

Note that indexing a table (or numpy array) with a logical expression selects just the rows where that expression is true.

detection_TAP_results[ (detection_TAP_results['filterType']=='z') & (np.abs(mag-TAP_results['zmeanpsfmag'][0]) > 2) ]
Table length=1
objIDdetectIDfilteridfilterTypeobsTimeradecpsfFluxpsfFluxErrpsfMajorFWHMpsfMinorFWHMpsfQfPerfectapFluxapFluxErrinfoFlaginfoFlag2infoFlag3
yrdegdegpixpix
int64int64uint8objectfloat64float64float64float32float32float32float32float32float32float32int64int32int32
1903613933441128941832526275200002344z55933.5264577139.3348816868.635322730.0003179456.73008e-061.075371.01530.3229860.0002132172.36939e-0610276045312832768

From examining this table, it looks like psfQfPerfect is bad. This flag is the PSF-weighted fraction of unmasked pixels in the image (see the documentation for more details). Values near unity indicate good data that is not significantly affected by bad pixels.

Check all the psfQfPerfect values for the \(z\) filter to see if this value really is unusual. The list of values below are sorted by magnitude. The bad point is the only value with psfQfPerfect < 0.95.

w = np.where(detection_TAP_results['filterType']=='z')
zdtab = detection_TAP_results[w]
zdtab['mag'] = mag[w]
zdtab['dmag'] = zdtab['mag'] - TAP_results['zmeanpsfmag'][0]
ii = np.argsort(-np.abs(zdtab['dmag']))
zdtab = zdtab[ii]
zdtab['objID','obsTime','mag','dmag','psfQfPerfect']
Table length=13
objIDobsTimemagdmagpsfQfPerfect
yr
int64float64float32float32float32
19036139334411289455933.526457717.644123.36271950.322986
19036139334411289456289.615934613.890659-0.39074040.997811
19036139334411289456289.624111213.916805-0.364594460.988369
19036139334411289456351.416848313.998972-0.28242780.999257
19036139334411289455281.252828514.5379140.256514550.99754
19036139334411289456351.42407614.032501-0.24889850.999187
19036139334411289455527.650891914.5121170.230717660.997265
19036139334411289456648.567601914.05643-0.224969860.997982
19036139334411289455527.638146914.465140.183740620.99738
19036139334411289455281.262515114.4285540.147153850.955584
19036139334411289455933.53451514.3076390.0262393950.997489
19036139334411289456019.296878214.278636-0.00276374820.997654
19036139334411289456019.303801414.282890.0014905930.997338

Repeat the plot with bad psfQfPerfect values excluded#

Do the plot again but exclude low psfQfPerfect values.

# convert flux in Jy to magnitudes
t = detection_TAP_results['obsTime']
mag = -2.5*np.log10(detection_TAP_results['psfFlux']) + 8.90
magmean = 0.0*mag
for i, filter in enumerate(['g','r','i','z','y']):
    magmean[detection_TAP_results['filterType']==filter] = TAP_results[f'{filter}meanpsfmag'][0]

dmag = mag - magmean
dmag1 = dmag[detection_TAP_results['psfQfPerfect']>0.9]
# fix the x and y axis ranges
xlim = np.array([t.min(),t.max()])
xlim = xlim + np.array([-1,1])*0.02*(xlim[1]-xlim[0])
# flip axis direction for magnitude
ylim = np.array([dmag1.max(),dmag1.min()])
ylim = ylim + np.array([-1,1])*0.02*(ylim[1]-ylim[0])

pylab.rcParams.update({'font.size': 14})
pylab.figure(1,(10,10))

for i, filter in enumerate(['g','r','i','z','y']):
    pylab.subplot(511+i)
    w = np.where((detection_TAP_results['filterType']==filter) & (detection_TAP_results['psfQfPerfect']>0.9))[0]
    pylab.plot(t[w],dmag[w],'-o')
    pylab.ylabel('{} [mag - {:.2f}]'.format(filter,magmean[w[0]]))
    pylab.xlim(xlim)
    pylab.ylim(ylim)
    if i==0:
        pylab.title(objname)
        
pylab.xlabel('Time [MJD]')
pylab.tight_layout()
../../../_images/11df4e5b9ee2c6789fb09a9d6e830911db8cb6da53f309bc65ae98e85cd13a58.png

Plot versus the periodic phase instead of epoch#

Plot versus phase using known RR Lyr period from Simbad (table J/AJ/132/1202/table4).

period = 0.48636 #days, from Simbad
# convert flux in Jy to magnitudes
t = (detection_TAP_results['obsTime'] % period) / period
mag = -2.5*np.log10(detection_TAP_results['psfFlux']) + 8.90
magmean = 0.0*mag
for i, filter in enumerate(['g','r','i','z','y']):
    magmean[detection_TAP_results['filterType']==filter] = TAP_results[f'{filter}meanpsfmag'][0]
    
dmag = mag - magmean
dmag1 = dmag[detection_TAP_results['psfQfPerfect']>0.9]
# fix the x and y axis ranges
xlim = np.array([t.min(),t.max()])
xlim = xlim + np.array([-1,1])*0.02*(xlim[1]-xlim[0])
# flip axis direction for magnitude
ylim = np.array([dmag1.max(),dmag1.min()])
ylim = ylim + np.array([-1,1])*0.02*(ylim[1]-ylim[0])

pylab.rcParams.update({'font.size': 14})
pylab.figure(1,(10,10))

for i, filter in enumerate(['g','r','i','z','y']):
    pylab.subplot(511+i)
    w = np.where((detection_TAP_results['filterType']==filter) & (detection_TAP_results['psfQfPerfect']>0.9))[0]
    w = w[np.argsort(t[w])]
    pylab.plot(t[w],dmag[w],'-o')
    pylab.ylabel('{} [mag - {:.2f}]'.format(filter,magmean[w[0]]))
    pylab.xlim(xlim)
    pylab.ylim(ylim)
    if i==0:
        pylab.title(objname)
        
pylab.xlabel('Phase')
pylab.tight_layout()
../../../_images/107abf32ee5c018fb187623c2cc840a0a405dbfb089861e9c486bcb8c78aae11.png

Repeat search using eclipsing binary KIC 2161623#

From Villanova Kepler Eclipsing Binaries

objname = 'KIC 2161623'
coords = Mast.resolve_object(objname)
ra,dec = coords.ra.value,coords.dec.value
radius = 1.0/3600.0 # radius = 1 arcsec

query = """
SELECT objID, RAMean, DecMean, nDetections, ng, nr, ni, nz, ny, gMeanPSFMag, rMeanPSFMag, iMeanPSFMag, zMeanPSFMag, yMeanPSFMag
FROM dbo.MeanObjectView
WHERE
CONTAINS(POINT('ICRS', RAMean, DecMean),CIRCLE('ICRS',{},{},{}))=1
AND nDetections > 1
""".format(ra,dec,radius)
print(query)

job = TAP_service.run_async(query)
TAP_results = job.to_table()
TAP_results
SELECT objID, RAMean, DecMean, nDetections, ng, nr, ni, nz, ny, gMeanPSFMag, rMeanPSFMag, iMeanPSFMag, zMeanPSFMag, yMeanPSFMag
FROM dbo.MeanObjectView
WHERE
CONTAINS(POINT('ICRS', RAMean, DecMean),CIRCLE('ICRS',291.744461,37.59102,0.0002777777777777778))=1
AND nDetections > 1
Table length=1
objidrameandecmeanndetectionsngnrninznygmeanpsfmagrmeanpsfmagimeanpsfmagzmeanpsfmagymeanpsfmag
degdegmagmagmagmagmag
int64float64float64int16int16int16int16int16int16float32float32float32float32float32
153102917444859851291.7444628363461437.5909988815496967101612151414.599814.282114.158714.200414.0672

Get Repeated Detection Information#

This time include the psfQfPerfect limit directly in the database query.

objid = TAP_results['objid'][0]

query = """
SELECT
    objID, detectID, Detection.filterID as filterID, Filter.filterType, obsTime, ra, dec,
    psfFlux, psfFluxErr, psfMajorFWHM, psfMinorFWHM, psfQfPerfect, 
    apFlux, apFluxErr, infoFlag, infoFlag2, infoFlag3
FROM Detection
NATURAL JOIN Filter
WHERE objID={}
AND psfQfPerfect >= 0.9
ORDER BY filterID, obsTime
""".format(objid)
print(query)

job = TAP_service.run_async(query)
detection_TAP_results = job.to_table()

# add magnitude and difference from mean
detection_TAP_results['magmean'] = 0.0
for i, filter in enumerate([b'g',b'r',b'i',b'z',b'y']):
    detection_TAP_results['magmean'][detection_TAP_results['filterType']==filter] = TAP_results[filter.decode('ascii')+'meanpsfmag'][0]
detection_TAP_results['mag'] = -2.5*np.log10(detection_TAP_results['psfFlux']) + 8.90
detection_TAP_results['dmag'] = detection_TAP_results['mag']-detection_TAP_results['magmean']

detection_TAP_results
SELECT
    objID, detectID, Detection.filterID as filterID, Filter.filterType, obsTime, ra, dec,
    psfFlux, psfFluxErr, psfMajorFWHM, psfMinorFWHM, psfQfPerfect, 
    apFlux, apFluxErr, infoFlag, infoFlag2, infoFlag3
FROM Detection
NATURAL JOIN Filter
WHERE objID=153102917444859851
AND psfQfPerfect >= 0.9
ORDER BY filterID, obsTime
Table length=45
objIDdetectIDfilteridfilterTypeobsTimeradecpsfFluxpsfFluxErrpsfMajorFWHMpsfMinorFWHMpsfQfPerfectapFluxapFluxErrinfoFlaginfoFlag2infoFlag3magmeanmagdmag
yrdegdegpixpix
int64int64uint8objectfloat64float64float64float32float32float32float32float32float32float32int64int32int32float64float32float64
153102917444859851901504437100000881g55002.5047803291.7444639137.59100060.005010086.65687e-061.987441.812110.9994210.005174137.31225e-061027605171281248154240.014.65038914.650388717651367
153102917444859851901519597100000871g55002.5199475291.7444661137.591001560.004873346.48844e-061.984871.707930.9992340.005115427.27714e-061027605171281248154240.014.68043314.68043327331543
1531029174448598511315349965600001291g55416.3502179291.7444608237.590996920.005256228.41646e-061.624031.578930.9985660.00529928.62578e-0610276051712873749120.014.59831614.598316192626953
1531029174448598511315361806500001381g55416.362065291.7444593737.590997330.005330288.62169e-061.320811.271860.9987570.005400828.91577e-061027605171281248154240.014.58312514.583125114440918
1531029174448598512045289305600000761g56146.2895546291.7444591437.590980730.005223998.43794e-061.333691.292110.9997580.005325068.88465e-0610276051712873749120.014.60499414.60499382019043
1531029174448598512045300415600000851g56146.3006673291.7444648637.590976490.005235398.44281e-061.428831.226830.9646460.005324198.85594e-0610276051712873749120.014.60262714.60262680053711
1531029174448598512410322651200000941g56511.322909291.7444565737.590997520.005286418.3427e-061.151231.107890.9987220.005374848.8135e-0610276051712873749120.014.59209814.592098236083984
1531029174448598512410333211200000811g56511.3334693291.7444594537.590996430.005278898.31263e-061.078980.9873230.9952790.00533468.77771e-0610276051712873749120.014.59364314.593643188476562
153102917444859851912525437100001022r55013.5256677291.7444598537.590994540.007002438.7848e-061.901871.686890.9989620.007235139.82915e-0610276051712873749120.014.28687814.286877632141113
............................................................
1531029174448598511604600874100001365y55705.6010605291.7444637237.590996670.008389142.4518e-050.8922940.7857140.9974980.008507052.38093e-0510276051712873749120.014.09070614.090705871582031
1531029174448598511604612744100001345y55705.6129264291.7444647937.590994840.008463762.45964e-050.8435670.7193540.9821990.008459892.36938e-0510276051712873749120.014.08109214.08109188079834
1531029174448598511717343720100001275y55818.3439014291.7444641937.590999730.008995372.60801e-051.256050.9636240.9975090.008931252.37426e-05102760517128577065600.014.01495214.014951705932617
1531029174448598511717355230100001295y55818.3554064291.7444605837.591002080.008938852.61524e-051.26840.9412590.9979290.008983012.38145e-05102760517128409293440.014.02179514.021795272827148
1531029174448598511944620756400001605y56045.6209324291.744462737.590999140.009013122.46438e-051.411571.293440.9983750.009005782.28883e-05102760517128577065600.014.01281214.012811660766602
1531029174448598511944627526400001585y56045.6277083291.7444598937.590997180.008816422.44413e-051.403771.198980.9995120.008887092.26963e-0510276051712873749120.014.03676914.036768913269043
1531029174448598512133193243000001595y56234.1934155291.7444637337.59100090.008511512.42822e-051.045040.8496060.994970.008423522.41261e-0510276051712873749120.014.07498414.074983596801758
1531029174448598512133198803000001635y56234.1989869291.7444691537.591000790.008522162.36733e-051.038710.8439690.9949770.008332252.42534e-05102760517128744837760.014.07362614.073625564575195
1531029174448598512453355593600001735y56554.3560604291.7444648837.590998740.008707051.611e-051.429061.389350.9596460.008868841.39697e-05102760517128409293440.014.05032214.050321578979492
t = detection_TAP_results['obsTime']
dmag = detection_TAP_results['dmag']
xlim = np.array([t.min(),t.max()])
xlim = xlim + np.array([-1,1])*0.02*(xlim[1]-xlim[0])
ylim = np.array([dmag.max(),dmag.min()])
ylim = ylim + np.array([-1,1])*0.02*(ylim[1]-ylim[0])

pylab.rcParams.update({'font.size': 14})
pylab.figure(1,(10,10))

for i, filter in enumerate(['g','r','i','z','y']):
    pylab.subplot(511+i)
    w = np.where(detection_TAP_results['filterType']==filter)[0]
    pylab.plot(t[w],dmag[w],'-o')
    magmean = detection_TAP_results['magmean'][w[0]]
    pylab.ylabel('{} [mag - {:.2f}]'.format(filter,magmean))
    pylab.xlim(xlim)
    pylab.ylim(ylim)
    if i==0:
        pylab.title(objname)
        
pylab.xlabel('Time [MJD]')
pylab.tight_layout()
../../../_images/60821b919cf8ba2d6757b14e7639ad0f000413a09bde55bef42271df893640e8.png

Plot versus phase using known period#

Eclipsing binaries basically vary by same amount in all filters since it is a geometrical effect, so combine the data into a single light curve. Wrap using known period and plot versus phase.

period = 2.2834698
bjd0 = 54999.599837
t = ((detection_TAP_results['obsTime']-bjd0) % period) / period
dmag = detection_TAP_results['dmag']
w = np.argsort(t)
t = t[w]
dmag = dmag[w]
xlim = np.array([t.min(),t.max()])
xlim = xlim + np.array([-1,1])*0.02*(xlim[1]-xlim[0])
ylim = np.array([dmag.max(),dmag.min()])
ylim = ylim + np.array([-1,1])*0.02*(ylim[1]-ylim[0])

pylab.rcParams.update({'font.size': 14})
pylab.figure(1,(10,6))
pylab.plot(t,dmag,'-o')
pylab.xlim(xlim)
pylab.ylim(ylim)
pylab.xlabel('Phase')
pylab.ylabel('Delta magnitude from mean [mag]')
pylab.title(objname)
pylab.tight_layout()
../../../_images/958e8b45fab1710114ef4fd105174477213d73bbf4fb497f13cb09aafdf0f34a.png

Repeat search for another eclipsing binary KIC 8153568#

objname = 'KIC 8153568'
coords = Mast.resolve_object(objname)
ra,dec = coords.ra.value,coords.dec.value
radius = 1.0/3600.0 # radius = 1 arcsec

query = """
SELECT objID, RAMean, DecMean, nDetections, ng, nr, ni, nz, ny, gMeanPSFMag, rMeanPSFMag, iMeanPSFMag, zMeanPSFMag, yMeanPSFMag
FROM dbo.MeanObjectView
WHERE
CONTAINS(POINT('ICRS', RAMean, DecMean),CIRCLE('ICRS',{},{},{}))=1
AND nDetections > 1
""".format(ra,dec,radius)
print(query)

job = TAP_service.run_async(query)
TAP_results = job.to_table()
TAP_results
SELECT objID, RAMean, DecMean, nDetections, ng, nr, ni, nz, ny, gMeanPSFMag, rMeanPSFMag, iMeanPSFMag, zMeanPSFMag, yMeanPSFMag
FROM dbo.MeanObjectView
WHERE
CONTAINS(POINT('ICRS', RAMean, DecMean),CIRCLE('ICRS',286.90445,44.00551,0.0002777777777777778))=1
AND nDetections > 1
Table length=1
objidrameandecmeanndetectionsngnrninznygmeanpsfmagrmeanpsfmagimeanpsfmagzmeanpsfmagymeanpsfmag
degdegmagmagmagmagmag
int64float64float64int16int16int16int16int16int16float32float32float32float32float32
160802869044447231286.9044500512221744.0054794547150988161531101615.182514.989914.890715.199914.8484
objid = TAP_results['objid'][0]
query = """
SELECT
    objID, detectID, Detection.filterID as filterID, Filter.filterType, obsTime, ra, dec,
    psfFlux, psfFluxErr, psfMajorFWHM, psfMinorFWHM, psfQfPerfect, 
    apFlux, apFluxErr, infoFlag, infoFlag2, infoFlag3
FROM Detection
NATURAL JOIN Filter
WHERE objID={}
AND psfQfPerfect >= 0.9
ORDER BY filterID, obsTime
""".format(objid)
print(query)

job = TAP_service.run_async(query)
detection_TAP_results = job.to_table()

# add magnitude and difference from mean
detection_TAP_results['magmean'] = 0.0
for i, filter in enumerate([b'g',b'r',b'i',b'z',b'y']):
    detection_TAP_results['magmean'][detection_TAP_results['filterType']==filter] = TAP_results[filter.decode('ascii')+'meanpsfmag'][0]
detection_TAP_results['mag'] = -2.5*np.log10(detection_TAP_results['psfFlux']) + 8.90
detection_TAP_results['dmag'] = detection_TAP_results['mag']-detection_TAP_results['magmean']

detection_TAP_results
SELECT
    objID, detectID, Detection.filterID as filterID, Filter.filterType, obsTime, ra, dec,
    psfFlux, psfFluxErr, psfMajorFWHM, psfMinorFWHM, psfQfPerfect, 
    apFlux, apFluxErr, infoFlag, infoFlag2, infoFlag3
FROM Detection
NATURAL JOIN Filter
WHERE objID=160802869044447231
AND psfQfPerfect >= 0.9
ORDER BY filterID, obsTime
Table length=60
objIDdetectIDfilteridfilterTypeobsTimeradecpsfFluxpsfFluxErrpsfMajorFWHMpsfMinorFWHMpsfQfPerfectapFluxapFluxErrinfoFlaginfoFlag2infoFlag3magmeanmagdmag
yrdegdegpixpix
int64int64uint8objectfloat64float64float64float32float32float32float32float32float32float32int64int32int32float64float32float64
160802869044447231913364294300001131g55014.3646518286.9044337444.005475750.0031095.98065e-061.781.238110.9983350.003124475.69291e-0610276051712873749120.015.168447515.168447494506836
160802869044447231913374804300001161g55014.3751566286.9044389444.005475020.003130566.00144e-061.669391.327730.9981970.003156135.71358e-061027605171281248154240.015.16094515.160944938659668
1608028690444472311260575651200000921g55361.57592286.9044531744.005474510.003095736.72312e-061.74981.624220.9988260.003100456.69387e-0610276051712873749120.015.17309215.173091888427734
1608028690444472311260587411300001111g55361.587678286.9044515744.005480650.003028156.57752e-061.628391.525790.9983870.00310626.76425e-0610276051712873749120.015.19705615.19705581665039
1608028690444472311287441263100002841g55388.4415151286.904450244.005478910.0008064623.51444e-061.101320.9925010.9976310.0008104163.45294e-061027605171281248154240.016.6335416.6335391998291
1608028690444472311287453413100002491g55388.4536712286.9044504344.005479360.0009817623.82737e-061.10851.074410.9974910.0009855843.78859e-061027605171281248154240.016.41998316.41998291015625
1608028690444472311645448417200000971g55746.4486739286.9044531544.005476460.003066556.32653e-061.328511.166930.998310.003047336.63894e-06102760517128409293440.015.18337415.183374404907227
1608028690444472311647444085300001111g55748.4443394286.9044520744.005478350.003040576.5461e-061.371531.231930.9985440.003078286.69725e-0610276051712873749120.015.19261215.192611694335938
1608028690444472311647455265300001131g55748.4555151286.9044527544.00547970.003047286.57104e-061.275311.264240.9985190.003085646.71944e-0610276051712873749120.015.19021815.190217971801758
............................................................
1608028690444472311360270091200001455y55461.2702758286.9044550344.005478270.004297691.96242e-051.486521.389680.9986860.004233951.58905e-0510276051712873749120.014.81691214.816911697387695
1608028690444472311360282041200001485y55461.2822322286.9044599444.005482290.004174612.0278e-051.765061.7130.9987050.004197361.56164e-0510276051712873749120.014.84845914.848459243774414
1608028690444472311599599934000001475y55700.6001108286.9044469244.005480860.004009831.95314e-051.040480.8986510.9980290.003961771.71112e-05102760517128409293440.014.89218414.892184257507324
1608028690444472311599611654000001425y55700.611833286.9044446344.005480940.004096671.92559e-050.8760260.7410520.9988060.004039491.75641e-0510276051712873749120.014.86892214.868922233581543
1608028690444472311604596264000001595y55705.5964454286.904448944.00548110.003313431.64053e-051.01180.848480.9974650.003279931.48499e-051027605171281248154240.015.09930515.099305152893066
1608028690444472311604607954000001715y55705.6081345286.9044468444.005479640.003211941.64645e-051.081210.9909370.9978730.00318011.44718e-051027605171281248154240.015.13308115.133081436157227
1608028690444472311710235866400001365y55811.2360427286.904452544.005483570.00421441.82169e-050.7231510.6624750.99780.004252461.73593e-0510276051712873421440.014.8381614.838159561157227
1608028690444472311710247766400001515y55811.2479472286.9044527644.005482270.004250271.862e-050.7636580.6921140.9977030.004313361.74107e-0510276051712873421440.014.828958514.828958511352539
1608028690444472312453350024700001535y56554.3504901286.9044508544.005477840.004270931.24367e-051.463481.403940.9726070.004264339.76066e-0610276051712873749120.014.82369314.82369327545166
t = detection_TAP_results['obsTime']
dmag = detection_TAP_results['dmag']
xlim = np.array([t.min(),t.max()])
xlim = xlim + np.array([-1,1])*0.02*(xlim[1]-xlim[0])
ylim = np.array([dmag.max(),dmag.min()])
ylim = ylim + np.array([-1,1])*0.02*(ylim[1]-ylim[0])

pylab.rcParams.update({'font.size': 14})
pylab.figure(1,(10,10))

for i, filter in enumerate(['g','r','i','z','y']):
    pylab.subplot(511+i)
    w = np.where(detection_TAP_results['filterType']==filter)[0]
    pylab.plot(t[w],dmag[w],'-o')
    magmean = detection_TAP_results['magmean'][w[0]]
    pylab.ylabel('{} [mag - {:.2f}]'.format(filter,magmean))
    pylab.xlim(xlim)
    pylab.ylim(ylim)
    if i==0:
        pylab.title(objname)
        
pylab.xlabel('Time [MJD]')
pylab.tight_layout()
../../../_images/382d6dec21551986a12911396e98b6390b49dc2a6810d6a574080b2ede8aaca6.png

Eclipsing binaries basically vary by same amount in all filters since it is a geometrical effect, so combine the data into a single light curve.

Wrap using known period and plot versus phase. Plot two periods of the light curve this time.

This nice light curve appears to show a secondary eclipse.

period = 3.6071431
bjd0 = 54999.289794
t = ((detection_TAP_results['obsTime']-bjd0) % period) / period
dmag = detection_TAP_results['dmag']
w = np.argsort(t)
# extend to two periods
nw = len(w)
w = np.append(w,w)
t = t[w]
# add one to second period
t[-nw:] += 1
dmag = dmag[w]
xlim = [0,2.0]
ylim = np.array([dmag.max(),dmag.min()])
ylim = ylim + np.array([-1,1])*0.02*(ylim[1]-ylim[0])

pylab.rcParams.update({'font.size': 14})
pylab.figure(1,(12,6))
pylab.plot(t,dmag,'-o')
pylab.xlim(xlim)
pylab.ylim(ylim)
pylab.xlabel('Phase')
pylab.ylabel('Delta magnitude from mean [mag]')
pylab.title(objname)
pylab.tight_layout()
../../../_images/b6a0aa0c1a8bc0e2fa1b5677653d4c2ea991faf636a2f7cd5a557684e5ad8880.png

Additional Resources#

Table Access Protocol#

  • IVOA standard for RESTful web service access to tabular data

  • http://www.ivoa.net/documents/TAP/

PanSTARRS 1 DR 2#

  • Catalog for PanSTARRS with additional Detection information

  • https://outerspace.stsci.edu/display/PANSTARRS/

Astronomical Query Data Language (2.0)#

  • IVOA standard for querying astronomical data in tabular format, with geometric search support

  • http://www.ivoa.net/documents/latest/ADQL.html

PyVO#

  • an affiliated package for astropy

  • find and retrieve astronomical data available from archives that support standard IVOA virtual observatory service protocols.

  • https://pyvo.readthedocs.io/en/latest/index.html


About this Notebook#

Authors: Rick White & Theresa Dower, STScI Archive Scientist & Software Engineer
Last Updated: Feb 2024

For support, please contact the Archive HelpDesk at archive@stsci.edu.


Space Telescope Logo

Return to top of page