Find Existing and Planned JWST Observations

Find Existing and Planned JWST Observations#

Introduction#

As with HST, JWST observations that duplicate existing, planned, or approved observations will not be allowed unless investigators provide a scientific justification in their proposal and that request is approved. Consult the JWST Duplicate Observations Policy for details. Broadly speaking, observations you are considering might duplicate observations in the current or prior cycles if your target is of the same astrophysical source, or there is significant spatial overlap of fields, and the following apply:

Similar imaging passband, or overlapping spectral range
Similar (spectral) resolution
Similar exposure depth

Observations with a different scientific instrument than one you are considering might still duplicate if the observing configuration and capabilities are similar (e.g., NIRCam and NIRISS imaging, or NIRCam and NIRSpec spectroscopy).

This notebook illustrates how to use the python package astroquery.mast to search the Mikulski Archive for Space Telescopes (MAST) for potential duplicate observations.

Special Disclaimer#

The capabilities described here will help identify potential duplications between your intended JWST observations and those that have been approved, planned, or that have already executed.

The complete footprint of approved (but not executed) mosaicked or parallel observations is only approximate. That is, while the primary location is reported for an observation, the exact orientation is not known until the observation is scheduled for execution. Moreover metadata in MAST about planned/approved observations may not suffice to determine whether your intended observation is a genuine duplication, particularly for slit or MOS spectroscopy. You are responsible for evaluating the details of the planned observations by using the accepted program's APT file (and/or the Aladin display in APT, as appropriate) to determine if the potential duplications are genuine.

Utility Routines

Queries by Target/Position

Search by Target Name
Single Moving Target

Checking a List of Targets

Loading Targets from CSV

Evaluating Potential Duplications

Trappist-1
V* XX Cha
M 57

Appendix: Caveats
Additional Resources

Strategy#

The following strategy, applied to each target or field, will identify potentially duplicative observations relatively efficiently.

Search for common targets or fields. If no existing or planned observation coincides with your intended target, you are done.
If there is a spatial overlap, determine if the instrument(s) and observing configurations you plan to use are the same or similar to the existing observations. If there is no commonality, you are done.
If there is spatial overlap and common instruments/configuration, determine in detail the overlap in passband/spectral coverage, and exposure depth.
If there is a likely duplication for an intended target, do one of the following:
- Include in your proposal a scientific case for the duplicating observation(s)
- Alter your intended observation(s) in a way that does not duplicate
- Choose a different target

Setup#

Begin by importing some essential python packages: general utilities in astropy, and query services in astroquery.mast.

# Give the notebook cells more of the available width
from IPython.display import display, HTML
display(HTML("<style>.container { width:99% !important; }</style>"))

import astropy
from astropy import table
from astropy.table import Table, unique
from astropy import units as u
from astropy.coordinates import Angle
from astroquery.mast import Mast
from astroquery.mast import Observations
import numpy as np

Utility Routines#

The following utility routine will create URLs to the parent programs of matching observations.

APT_LINK = '(http://www.stsci.edu/cgi-bin/get-proposal-info?id={}&observatory=JWST)'

def get_program_URL(program_id):
    '''
    Generate the URL for program status information, given a program ID. 
    '''
    #return APT_LINK.format(program_id)
    return APT_LINK.format(program_id)

The following utility routine will construct ranges in RA and Dec to facilitate queries in an area surrounding a given place in the sky.

def coord_ranges(ra, dec, extent_ra, extent_dec):
    '''
    Parameters
    ----------
    ra:         right ascension coordinate (deg)
    dec:        declination coordinate (deg)
    extent_ra:  spatial extent in RA (deg)
    extent_dec: spatial extent in Dec (deg)
    '''
    # correct width in ra for declination
    half_width = np.abs(extent_ra / 2. / np.cos(np.deg2rad(dec)))
    ra_range = [ra-half_width, ra+half_width]

    half_width = extent_dec / 2.
    dec_range = [dec-half_width, dec+half_width]
    return (ra_range, dec_range)

Queries by Target/Postion#

All of the queries below search for JWST observations, using a search radius larger than fields of view (FoV) of JWST apertures, to allow for the possibility that the FoV may be rotated when approved-but-unexecuted observations are actually scheduled. Restrict the search to JWST by including the parameter: obs_collection = "JWST".

Searching for source names is not recommended when checking for potential duplications with fixed targets because the matches are not reliable, consistent, or complete. The exception would be for matching Object names of well known solar system bodies.

The best approach is to use an independent source for the coordinates of your desired target. Failing that, the Mast.resolve_object() method will return coordinates in the ICRS reference frame, which you can use to verify whether a target name resolves to the coordinates you intend. See the Appendix for details.

Search by Target Name#

This example shows how to query for a single target with a standard name, HD 104237 which is a T-Tauri star. We first resolve the name and verify that the coordinates are correct.

# Resolve the coordinates if necessary
coords = Mast.resolve_object('HD 104237')
coords

<SkyCoord (ICRS): (ra, dec) in deg
    (180.02119525, -78.19293492)>

In this case the resolved coordinates agree with those in (see Simbad), so it is safe to execute a simple cone search (a search within a specified radius of a place on the sky) to find JWST existing or planned observations. The astroquery.mast package provides the method Observations.query_criteria() to specify the parameters for the search; provide them as key=value pairs. The full set of query parameters for this method may be found on CAOM Field Descriptions.

# If the coordinates are ok
obs = Observations.query_criteria(
        obs_collection='JWST'
        ,objectname='HD 104237'
        ,radius='20s'
        )
print('Number of matching observations: {}'.format(len(obs)))

Number of matching observations: 0

WARNING: NoResultsWarning: Query returned no results. [astroquery.mast.discovery_portal]

The number of matches is zero, so there are no potential duplications with this target.

Single Moving Target#

Moving targets, by definition, do not lend themselves to searches by position. This kind of search is limited to a modest set of solar system bodies with recognized names. Note the use of a wildcard character (*) in case the target name includes other text.

obs = Observations.query_criteria(
        target_name="Io*",
        obs_collection="JWST"
        )
    
print('Number of matching observations: {}'.format(len(obs)))

Number of matching observations: 28

There are some JWST observations of Io, including some that are planned but have not executed as of the beginning of Cy-2 (i.e., calib_level = -1). The details are in the returned table of results, the most essential of which can be viewed below.

out_cols = ['target_name','instrument_name','filters','calib_level','t_exptime','proposal_id']
obs[out_cols].show_in_notebook()

Table length=28

idx	target_name	instrument_name	filters	calib_level	t_exptime	proposal_id
0	IO	MIRI/IFU	--	-1	222.003	4078
1	IO	MIRI/IFU	--	-1	222.003	4078
2	IO	MIRI/IFU	--	-1	222.003	4078
3	IO	MIRI/IFU	--	-1	222.003	4078
4	IO	MIRI/IFU	--	-1	222.003	4078
5	IO	MIRI/IFU	--	-1	222.003	4078
6	IO	MIRI/IFU	--	-1	222.003	4078
7	IO	MIRI/IFU	--	-1	222.003	4078
8	IO	MIRI/IFU	--	-1	222.003	4078
9	IO	MIRI/IFU	--	-1	222.003	4078
10	IO	MIRI/IFU	--	-1	222.003	4078
11	IO	MIRI/IFU	--	-1	222.003	4078
12	IO	MIRI/IFU	--	-1	222.003	4078
13	IO	MIRI/IFU	--	-1	222.003	4078
14	IO	NIRISS/AMI	F430M;NRM	3	1697.4	1373
15	IO	MIRI/IFU	CH4	3	790.8870000000001	1373
16	IO	MIRI/IFU	CH3	3	790.8870000000001	1373
17	IO	MIRI/IFU	CH2	3	790.8870000000001	1373
18	IO	MIRI/IFU	CH1	3	790.8870000000001	1373
19	IO	NIRSPEC/IFU	F170LP;G235H	3	687.152	1373
20	IO	NIRSPEC/IFU	F290LP;G395H	3	343.576	1373
21	IO	NIRSPEC/IFU	F290LP;G395H	3	343.576	1373
22	IO	NIRSPEC/IFU	F170LP;G235H	3	687.152	1373
23	IO	NIRSPEC/IFU	F100LP;G140H	3	5135.288	1373
24	IO	NIRSPEC/IFU	F100LP;G140H	-1	583.556	1373
25	IO	NIRSPEC/IFU	F100LP;G140H	-1	583.556	1373
26	IO	NIRSPEC/IFU	F100LP;G140H	-1	583.556	1373
27	IO	NIRSPEC/IFU	F100LP;G140H	-1	583.556	1373

You may need to examine the specifics of the programs that obtained the observations to know whether your intended observations would duplicate. Here are the unique Program Titles and URLs for the program status pages, which may offer clues:

obs['title'] = [x[:80] for x in obs['obs_title']]
unique(obs['proposal_id','title']).pprint(max_width=-1)
obs['proposal_URL'] = [get_program_URL(x) for x in obs['proposal_id']]
unique(obs['proposal_id','proposal_URL']).pprint(max_width=-1)

proposal_id                                      title                                      
----------- --------------------------------------------------------------------------------
       1373 ERS observations of the Jovian System as a demonstration of JWST capabilities fo
       4078 Mass-loss from Ios volcanic atmosphere: A unique synergy with the Juno Io fly-by
proposal_id                                proposal_URL                              
----------- -------------------------------------------------------------------------
       1373 (http://www.stsci.edu/cgi-bin/get-proposal-info?id=1373&observatory=JWST)
       4078 (http://www.stsci.edu/cgi-bin/get-proposal-info?id=4078&observatory=JWST)

There are images, spectra and IFU observations of this target that could conflict with your intended proposal. Proposed images in passbands that overlap the IFU data would have to be examined closely for duplication.

Checking a List of Targets#

It is often useful to search for individual targets with the MAST Portal because the results are easily visualized. But it is more efficient to search over a large number of targets using astroquery.mast.

Loading Targets from CSV#

For efficiency, make a CSV (comma-separated variable) list of your targets, one per row. The list can alternatively be read from a local file: just substitute the file name for targ_table in the first argument of the Table.read() method. The list can contain many fields, but at a minimum must contain the target name and the equitorial coordinates.

The first row of the file will be interpreted as a column name in the table. This is important.

targ_table = '''
target_name, RA, DEC, Range, Description
Trappist-1, 23:06:29.3684948589, -05:02:29.037301866, 20s, Exoplanet host star
V* XX Cha, 11:11:39.559, -76:20:15.04, 20s, Variable star with debris disk
M 31, 00:42:44.330, +41:16:07.50, 24.0m, Andromeda Galaxy
M 57, 18:53:35.0967659112, +33:01:44.883287544, 4m, Planetary nebula
'''
targets = Table.read(targ_table, format='ascii.csv')

Next, make new columns to hold the coordinates and ranges in units of degrees.

targets['ra_deg'] = [Angle(x+' hours').degree for x in targets['RA']]
targets['dec_deg'] = [Angle(x+' degree').degree for x in targets['DEC']]
targets['range_deg'] = [Angle(x).degree for x in targets['Range']]
targets['N_obs'] = 0  # field to hold the count of matched observations 
targets

Table length=4

target_name	RA	DEC	Range	Description	ra_deg	dec_deg	range_deg	N_obs
str10	str19	str19	str5	str30	float64	float64	float64	int64
Trappist-1	23:06:29.3684948589	-05:02:29.037301866	20s	Exoplanet host star	346.62236872857875	-5.041399250518333	0.005555555555555556	0
V* XX Cha	11:11:39.559	-76:20:15.04	20s	Variable star with debris disk	167.91482916666666	-76.33751111111111	0.005555555555555556	0
M 31	00:42:44.330	+41:16:07.50	24.0m	Andromeda Galaxy	10.684708333333331	41.26875	0.4	0
M 57	18:53:35.0967659112	+33:01:44.883287544	4m	Planetary nebula	283.39623652462996	33.029134246539996	0.06666666666666667	0

It can save some effort to determine the number of JWST Observations that match each of your targets using the query_criteria_count() method. Note that the coordinate parameters must be specified as lower and upper bounds of a range (i.e., a python list), so use the coord_ranges() utility function.

It is good practice to first check the number of matching observations before fetching the results themselves, in case the number of results is very large. This is more important when querying MAST missions with a very large number of observations, such as HST. This type of search is typically fairly fast.

# get the counts of each target
for t in targets:
    (ra_range, dec_range) = coord_ranges(t['ra_deg'],t['dec_deg'], t['range_deg'], t['range_deg'])
    t['N_obs'] = Observations.query_criteria_count(
            obs_collection='JWST'
            ,s_ra=ra_range
            ,s_dec=dec_range
            )

targets['target_name','N_obs'].show_in_notebook()

Table length=4

idx	target_name	N_obs
0	Trappist-1	47
1	V* XX Cha	13
2	M 31	0
3	M 57	147

For targets where no JWST Observations exist, there is no potential for duplication.

Evaluating Potential Duplications#

To examine the matching Observations for the other targets in more detail, use the query_criteria() method.

targets['obs'] = None
for t in targets:
    if t['N_obs'] > 0:
        (ra_range, dec_range) = coord_ranges(t['ra_deg'],t['dec_deg'], t['range_deg'], t['range_deg'])
        t['obs'] = Observations.query_criteria(
            obs_collection="JWST"
            ,s_ra=ra_range
            ,s_dec=dec_range
            )

Targets with matching JWST Observations are examined in the following sub-sections.

Trappist-1#

Trappist-1 is a well known exo-planet host star. If the intended observations are timeseries spectroscopy, the search would naturally be limited to a small area of sky. There are quite a few JWST observations of this target, so it is important to examine the relevant details.

out_cols = ['target_name','instrument_name','filters','t_exptime','proposal_id']
obs = targets[0]['obs']
unique(obs[out_cols]).show_in_notebook()

Table length=3

idx	target_name	instrument_name	filters	t_exptime	proposal_id
0	TRAPPIST-1	MIRI/IMAGE	F1500W	72540.749	3077
1	TRAPPIST-1	MIRI/IMAGE	F1500W	135230.285	3077
2	TRAPPIST-1B	MIRI/IMAGE	F1280W	13983.427	1279

You will need to examine the specifics of the programs that obtained the observations to know for certain which of the 7 known exoplanets was being observed, and therefore whether your intended observations would duplicate. Here are the unique Program Titles, which may offer clues:

obs['title'] = [x[:80] for x in obs['obs_title']]
unique(obs['proposal_id','title']).pprint(max_width=-1)

proposal_id                  title                 
----------- ---------------------------------------
       1279      Thermal emission from Trappist-1 b
       3077 TRAPPIST-1 Planets: Atmospheres Or Not?

The Program Status Pages offer the complete specifications for each existing program:

obs['proposal_URL'] = [get_program_URL(x) for x in obs['proposal_id']]
unique(obs['proposal_id','proposal_URL']).pprint(max_width=-1)

proposal_id                                proposal_URL                              
----------- -------------------------------------------------------------------------
       1279 (http://www.stsci.edu/cgi-bin/get-proposal-info?id=1279&observatory=JWST)
       3077 (http://www.stsci.edu/cgi-bin/get-proposal-info?id=3077&observatory=JWST)

There are both NIRISS and NIRSpec spectroscopic observations of this target that could conflict with your intended proposal.

V* XX Cha#

This pre-main-sequence star has a debris disk, and might be worth a coronagraphic observation. The search area is small (20 arcsec) to exclude nearby targets.

out_cols = ['target_name','instrument_name','filters','t_exptime','calib_level','proposal_id']
obs = targets[1]['obs']
unique(obs[out_cols]).show_in_notebook()

Table length=7

idx	target_name	instrument_name	filters	t_exptime	calib_level	proposal_id
0	V-XX-CHA-2	MIRI/IFU	CH1	3696.348	3	1282
1	V-XX-CHA-2	MIRI/IFU	CH2	3696.348	3	1282
2	V-XX-CHA-2	MIRI/IFU	CH3	3696.348	3	1282
3	V-XX-CHA-2	MIRI/IFU	CH4	3696.348	3	1282
4	V-XX-CHA-2	MIRI/IMAGE	F1280W	3696.348	3	1282
5	V-XX-CHA-2	NIRSPEC/IFU	F290LP;G395H	32.21	-1	1282
6	V-XX-CHA-2	NIRSPEC/IFU	F290LP;G395H	161.052	-1	1282

Note that some of the observations for this target had not been observed (at the beginning of Cy-2).

While JWST spectra from MIRI and NIRSpec exist or are planned, there is no potential for duplication with a coronagraphic observation.

M 57#

The Ring Nebula (NGC 6720) is well observed, particularly with HST and, recently, with JWST. We searched a 4-arcmin region around the source to include potential spectral Observations in the nebular periphery. The following table shows the unique combinations of instrument configurations.

out_cols = ['target_name','instrument_name','filters','t_exptime','proposal_id']
obs = targets[3]['obs']
unique(obs[out_cols]).show_in_notebook()

Table length=43

idx	target_name	instrument_name	filters	t_exptime	proposal_id
0	NGC6720	MIRI/IMAGE	F1000W	444.0079999999999	1558
1	NGC6720	MIRI/IMAGE	F1130W	444.0079999999999	1558
2	NGC6720	MIRI/IMAGE	F1280W	444.0079999999999	1558
3	NGC6720	MIRI/IMAGE	F1500W	444.0079999999999	1558
4	NGC6720	MIRI/IMAGE	F1800W	444.0079999999999	1558
5	NGC6720	MIRI/IMAGE	F2100W	444.0079999999999	1558
6	NGC6720	MIRI/IMAGE	F2550W	444.0079999999999	1558
7	NGC6720	MIRI/IMAGE	F560W	444.0079999999999	1558
8	NGC6720	MIRI/IMAGE	F770W	444.0079999999999	1558
9	NGC6720	NIRCAM/IMAGE	F150W2;F162M	483.1559999999998	1558
10	NGC6720	NIRCAM/IMAGE	F212N	483.1559999999998	1558
11	NGC6720	NIRCAM/IMAGE	F300M	483.1559999999998	1558
12	NGC6720	NIRCAM/IMAGE	F335M	483.1559999999998	1558
13	NGC6720-MRS-POSITION-1	MIRI/IFU	CH1	2763.936	1558
14	NGC6720-MRS-POSITION-1	MIRI/IFU	CH2	2763.936	1558
15	NGC6720-MRS-POSITION-1	MIRI/IFU	CH3	2763.936	1558
16	NGC6720-MRS-POSITION-1	MIRI/IFU	CH4	2763.936	1558
17	NGC6720-MRS-POSITION-1	MIRI/IMAGE	F1000W	921.312	1558
18	NGC6720-MRS-POSITION-1	MIRI/IMAGE	F1130W	921.312	1558
19	NGC6720-MRS-POSITION-1	MIRI/IMAGE	F770W	921.312	1558
20	NGC6720-MRS-POSITION-2	MIRI/IFU	CH1	2763.936	1558
21	NGC6720-MRS-POSITION-2	MIRI/IFU	CH2	2763.936	1558
22	NGC6720-MRS-POSITION-2	MIRI/IFU	CH3	2763.936	1558
23	NGC6720-MRS-POSITION-2	MIRI/IFU	CH4	2763.936	1558
24	NGC6720-MRS-POSITION-2	MIRI/IMAGE	F1000W	921.312	1558
25	NGC6720-MRS-POSITION-2	MIRI/IMAGE	F1130W	921.312	1558
26	NGC6720-MRS-POSITION-2	MIRI/IMAGE	F770W	921.312	1558
27	NGC6720-NIRSPEC-POSITION-1	NIRSPEC/IFU	F070LP;G140M	145.889	1558
28	NGC6720-NIRSPEC-POSITION-1	NIRSPEC/IFU	F070LP;G140M	583.556	1558
29	NGC6720-NIRSPEC-POSITION-1	NIRSPEC/IFU	F100LP;G140M	145.889	1558
30	NGC6720-NIRSPEC-POSITION-1	NIRSPEC/IFU	F100LP;G140M	583.556	1558
31	NGC6720-NIRSPEC-POSITION-1	NIRSPEC/IFU	F170LP;G235M	145.889	1558
32	NGC6720-NIRSPEC-POSITION-1	NIRSPEC/IFU	F170LP;G235M	583.556	1558
33	NGC6720-NIRSPEC-POSITION-1	NIRSPEC/IFU	F290LP;G395M	145.889	1558
34	NGC6720-NIRSPEC-POSITION-1	NIRSPEC/IFU	F290LP;G395M	583.556	1558
35	NGC6720-NIRSPEC-POSITION-2	NIRSPEC/IFU	F070LP;G140M	145.889	1558
36	NGC6720-NIRSPEC-POSITION-2	NIRSPEC/IFU	F070LP;G140M	583.556	1558
37	NGC6720-NIRSPEC-POSITION-2	NIRSPEC/IFU	F100LP;G140M	145.889	1558
38	NGC6720-NIRSPEC-POSITION-2	NIRSPEC/IFU	F100LP;G140M	583.556	1558
39	NGC6720-NIRSPEC-POSITION-2	NIRSPEC/IFU	F170LP;G235M	145.889	1558
40	NGC6720-NIRSPEC-POSITION-2	NIRSPEC/IFU	F170LP;G235M	583.556	1558
41	NGC6720-NIRSPEC-POSITION-2	NIRSPEC/IFU	F290LP;G395M	145.889	1558
42	NGC6720-NIRSPEC-POSITION-2	NIRSPEC/IFU	F290LP;G395M	583.556	1558

There are both MIRI and NIRCam images in many passbands, and MIRI and NIRSpec spectroscopic IFU Observations of this target that could conflict with the intended proposal.

To retain this target it would be necessary to obtain spectra in a different location, or to use a different instrument/configuration/data-taking mode (e.g., time-series of the central star). It may be helpful to see the footprints of these Observations in the MAST Portal, or to view the observation specifications in Program GO-1558.

Appendix: Caveats#

It is possible to search MAST for individual sources by name, in one of two ways: Object Name or by Target Name.

Object Name: MAST invokes an astrophysical name resolver (e.g., NED or Simbad) to look up source coordinates. Not all object names are recognized by resolvers. Also, different resolvers sometimes return substantially different coordinates; MAST will pick one of them (but not tell you which one). Sometimes the coordinates returned by the resolvers differ by substantial amounts, often because they refer to different sources.
- A classic case is the bright star μ Eri from the Bayer Greek letter system, and the star MU Eri in the general catalog of variable stars. in MAST,
  - Searching for * mu. eri (or just plain mu eri matches μ Eri (RA = 4:45:30.15, Dec = -3:15:16.777)
  - Searching for V* mu eri matches MU Eri (RA = 2:48:10.566, Dec = -15:18:04.03)
Target Name: This search matches to target names specified by Investigators in observing proposals to refer to sources. These are not guaranteed to match standard names for astrophysical sources, and often do not.

Searching by source names is not recommended when checking for potential duplications with fixed targets because the matches are not reliable, consistent, or complete. The exception would be for matching Object names of well known solar system bodies.