Find Existing and Planned JWST Observations#

Introduction#

As with HST, JWST observations that duplicate existing, planned, or approved observations will not be allowed unless investigators provide a scientific justification in their proposal and that request is approved. Consult the JWST Duplicate Observations Policy for details. Broadly speaking, observations you are considering might duplicate observations in the current or prior cycles if your target is of the same astrophysical source, or there is significant spatial overlap of fields, and the following apply:

  • Similar imaging passband, or overlapping spectral range

  • Similar (spectral) resolution

  • Similar exposure depth

Observations with a different scientific instrument than one you are considering might still duplicate if the observing configuration and capabilities are similar (e.g., NIRCam and NIRISS imaging, or NIRCam and NIRSpec spectroscopy).

This notebook illustrates how to use the python package astroquery.mast to search the Mikulski Archive for Space Telescopes (MAST) for potential duplicate observations.

Special Disclaimer#

The capabilities described here will help identify potential duplications between your intended JWST observations and those that have been approved, planned, or that have already executed.

The complete footprint of approved (but not executed) mosaicked or parallel observations is only approximate. That is, while the primary location is reported for an observation, the exact orientation is not known until the observation is scheduled for execution. Moreover metadata in MAST about planned/approved observations may not suffice to determine whether your intended observation is a genuine duplication, particularly for slit or MOS spectroscopy. You are responsible for evaluating the details of the planned observations by using the accepted program's APT file (and/or the Aladin display in APT, as appropriate) to determine if the potential duplications are genuine.

Table of contents#

Strategy#

The following strategy, applied to each target or field, will identify potentially duplicative observations relatively efficiently.

  1. Search for common targets or fields. If no existing or planned observation coincides with your intended target, you are done.

  2. If there is a spatial overlap, determine if the instrument(s) and observing configurations you plan to use are the same or similar to the existing observations. If there is no commonality, you are done.

  3. If there is spatial overlap and common instruments/configuration, determine in detail the overlap in passband/spectral coverage, and exposure depth.

  4. If there is a likely duplication for an intended target, do one of the following:

    • Include in your proposal a scientific case for the duplicating observation(s)

    • Alter your intended observation(s) in a way that does not duplicate

    • Choose a different target

Setup#

Begin by importing some essential python packages: general utilities in astropy, and query services in astroquery.mast.

# Give the notebook cells more of the available width
from IPython.display import display, HTML
display(HTML("<style>.container { width:99% !important; }</style>"))
import astropy
from astropy import table
from astropy.table import Table, unique
from astropy import units as u
from astropy.coordinates import Angle
from astroquery.mast import Mast
from astroquery.mast import Observations
import numpy as np

Utility Routines#

The following utility routine will create URLs to the parent programs of matching observations.

APT_LINK = '(http://www.stsci.edu/cgi-bin/get-proposal-info?id={}&observatory=JWST)'

def get_program_URL(program_id):
    '''
    Generate the URL for program status information, given a program ID. 
    '''
    #return APT_LINK.format(program_id)
    return APT_LINK.format(program_id)

The following utility routine will construct ranges in RA and Dec to facilitate queries in an area surrounding a given place in the sky.

def coord_ranges(ra, dec, extent_ra, extent_dec):
    '''
    Parameters
    ----------
    ra:         right ascension coordinate (deg)
    dec:        declination coordinate (deg)
    extent_ra:  spatial extent in RA (deg)
    extent_dec: spatial extent in Dec (deg)
    '''
    # correct width in ra for declination
    half_width = np.abs(extent_ra / 2. / np.cos(np.deg2rad(dec)))
    ra_range = [ra-half_width, ra+half_width]

    half_width = extent_dec / 2.
    dec_range = [dec-half_width, dec+half_width]
    return (ra_range, dec_range)

Queries by Target/Postion#

All of the queries below search for JWST observations, using a search radius larger than fields of view (FoV) of JWST apertures, to allow for the possibility that the FoV may be rotated when approved-but-unexecuted observations are actually scheduled. Restrict the search to JWST by including the parameter: obs_collection = "JWST".

Searching for source names is not recommended when checking for potential duplications with fixed targets because the matches are not reliable, consistent, or complete. The exception would be for matching Object names of well known solar system bodies.

The best approach is to use an independent source for the coordinates of your desired target. Failing that, the Mast.resolve_object() method will return coordinates in the ICRS reference frame, which you can use to verify whether a target name resolves to the coordinates you intend. See the Appendix for details.

Search by Target Name#

This example shows how to query for a single target with a standard name, HD 104237 which is a T-Tauri star. We first resolve the name and verify that the coordinates are correct.

# Resolve the coordinates if necessary
coords = Mast.resolve_object('HD 104237')
coords
<SkyCoord (ICRS): (ra, dec) in deg
    (180.02119525, -78.19293492)>

In this case the resolved coordinates agree with those in (see Simbad), so it is safe to execute a simple cone search (a search within a specified radius of a place on the sky) to find JWST existing or planned observations. The astroquery.mast package provides the method Observations.query_criteria() to specify the parameters for the search; provide them as key=value pairs. The full set of query parameters for this method may be found on CAOM Field Descriptions.

# If the coordinates are ok
obs = Observations.query_criteria(
        obs_collection='JWST'
        ,objectname='HD 104237'
        ,radius='20s'
        )
print('Number of matching observations: {}'.format(len(obs)))
Number of matching observations: 0
WARNING: NoResultsWarning: Query returned no results. [astroquery.mast.discovery_portal]
The number of matches is zero, so there are no potential duplications with this target.

Single Moving Target#

Moving targets, by definition, do not lend themselves to searches by position. This kind of search is limited to a modest set of solar system bodies with recognized names. Note the use of a wildcard character (*) in case the target name includes other text.

obs = Observations.query_criteria(
        target_name="Io*",
        obs_collection="JWST"
        )
    
print('Number of matching observations: {}'.format(len(obs)))
Number of matching observations: 28

There are some JWST observations of Io, including some that are planned but have not executed as of the beginning of Cy-2 (i.e., calib_level = -1). The details are in the returned table of results, the most essential of which can be viewed below.

out_cols = ['target_name','instrument_name','filters','calib_level','t_exptime','proposal_id']
obs[out_cols].show_in_notebook()
Table length=28
idxtarget_nameinstrument_namefilterscalib_levelt_exptimeproposal_id
0IOMIRI/IFU---1222.0034078
1IOMIRI/IFU---1222.0034078
2IOMIRI/IFU---1222.0034078
3IOMIRI/IFU---1222.0034078
4IOMIRI/IFU---1222.0034078
5IOMIRI/IFU---1222.0034078
6IOMIRI/IFU---1222.0034078
7IOMIRI/IFU---1222.0034078
8IOMIRI/IFU---1222.0034078
9IOMIRI/IFU---1222.0034078
10IOMIRI/IFU---1222.0034078
11IOMIRI/IFU---1222.0034078
12IOMIRI/IFU---1222.0034078
13IOMIRI/IFU---1222.0034078
14IONIRISS/AMIF430M;NRM31697.41373
15IOMIRI/IFUCH43790.88700000000011373
16IOMIRI/IFUCH33790.88700000000011373
17IOMIRI/IFUCH23790.88700000000011373
18IOMIRI/IFUCH13790.88700000000011373
19IONIRSPEC/IFUF170LP;G235H3687.1521373
20IONIRSPEC/IFUF290LP;G395H3343.5761373
21IONIRSPEC/IFUF290LP;G395H3343.5761373
22IONIRSPEC/IFUF170LP;G235H3687.1521373
23IONIRSPEC/IFUF100LP;G140H35135.2881373
24IONIRSPEC/IFUF100LP;G140H-1583.5561373
25IONIRSPEC/IFUF100LP;G140H-1583.5561373
26IONIRSPEC/IFUF100LP;G140H-1583.5561373
27IONIRSPEC/IFUF100LP;G140H-1583.5561373

You may need to examine the specifics of the programs that obtained the observations to know whether your intended observations would duplicate. Here are the unique Program Titles and URLs for the program status pages, which may offer clues:

obs['title'] = [x[:80] for x in obs['obs_title']]
unique(obs['proposal_id','title']).pprint(max_width=-1)
obs['proposal_URL'] = [get_program_URL(x) for x in obs['proposal_id']]
unique(obs['proposal_id','proposal_URL']).pprint(max_width=-1)
proposal_id                                      title                                      
----------- --------------------------------------------------------------------------------
       1373 ERS observations of the Jovian System as a demonstration of JWST capabilities fo
       4078 Mass-loss from Ios volcanic atmosphere: A unique synergy with the Juno Io fly-by
proposal_id                                proposal_URL                              
----------- -------------------------------------------------------------------------
       1373 (http://www.stsci.edu/cgi-bin/get-proposal-info?id=1373&observatory=JWST)
       4078 (http://www.stsci.edu/cgi-bin/get-proposal-info?id=4078&observatory=JWST)
There are images, spectra and IFU observations of this target that could conflict with your intended proposal. Proposed images in passbands that overlap the IFU data would have to be examined closely for duplication.

Checking a List of Targets#

It is often useful to search for individual targets with the MAST Portal because the results are easily visualized. But it is more efficient to search over a large number of targets using astroquery.mast.

Loading Targets from CSV#

For efficiency, make a CSV (comma-separated variable) list of your targets, one per row. The list can alternatively be read from a local file: just substitute the file name for targ_table in the first argument of the Table.read() method. The list can contain many fields, but at a minimum must contain the target name and the equitorial coordinates.

The first row of the file will be interpreted as a column name in the table. This is important.

targ_table = '''
target_name, RA, DEC, Range, Description
Trappist-1, 23:06:29.3684948589, -05:02:29.037301866, 20s, Exoplanet host star
V* XX Cha, 11:11:39.559, -76:20:15.04, 20s, Variable star with debris disk
M 31, 00:42:44.330, +41:16:07.50, 24.0m, Andromeda Galaxy
M 57, 18:53:35.0967659112, +33:01:44.883287544, 4m, Planetary nebula
'''
targets = Table.read(targ_table, format='ascii.csv')

Next, make new columns to hold the coordinates and ranges in units of degrees.

targets['ra_deg'] = [Angle(x+' hours').degree for x in targets['RA']]
targets['dec_deg'] = [Angle(x+' degree').degree for x in targets['DEC']]
targets['range_deg'] = [Angle(x).degree for x in targets['Range']]
targets['N_obs'] = 0  # field to hold the count of matched observations 
targets
Table length=4
target_nameRADECRangeDescriptionra_degdec_degrange_degN_obs
str10str19str19str5str30float64float64float64int64
Trappist-123:06:29.3684948589-05:02:29.03730186620sExoplanet host star346.62236872857875-5.0413992505183330.0055555555555555560
V* XX Cha11:11:39.559-76:20:15.0420sVariable star with debris disk167.91482916666666-76.337511111111110.0055555555555555560
M 3100:42:44.330+41:16:07.5024.0mAndromeda Galaxy10.68470833333333141.268750.40
M 5718:53:35.0967659112+33:01:44.8832875444mPlanetary nebula283.3962365246299633.0291342465399960.066666666666666670

It can save some effort to determine the number of JWST Observations that match each of your targets using the query_criteria_count() method. Note that the coordinate parameters must be specified as lower and upper bounds of a range (i.e., a python list), so use the coord_ranges() utility function.

It is good practice to first check the number of matching observations before fetching the results themselves, in case the number of results is very large. This is more important when querying MAST missions with a very large number of observations, such as HST. This type of search is typically fairly fast.
# get the counts of each target
for t in targets:
    (ra_range, dec_range) = coord_ranges(t['ra_deg'],t['dec_deg'], t['range_deg'], t['range_deg'])
    t['N_obs'] = Observations.query_criteria_count(
            obs_collection='JWST'
            ,s_ra=ra_range
            ,s_dec=dec_range
            )

targets['target_name','N_obs'].show_in_notebook()
Table length=4
idxtarget_nameN_obs
0Trappist-147
1V* XX Cha13
2M 310
3M 57147
For targets where no JWST Observations exist, there is no potential for duplication.

Evaluating Potential Duplications#

To examine the matching Observations for the other targets in more detail, use the query_criteria() method.

targets['obs'] = None
for t in targets:
    if t['N_obs'] > 0:
        (ra_range, dec_range) = coord_ranges(t['ra_deg'],t['dec_deg'], t['range_deg'], t['range_deg'])
        t['obs'] = Observations.query_criteria(
            obs_collection="JWST"
            ,s_ra=ra_range
            ,s_dec=dec_range
            )

Targets with matching JWST Observations are examined in the following sub-sections.

Trappist-1#

Trappist-1 is a well known exo-planet host star. If the intended observations are timeseries spectroscopy, the search would naturally be limited to a small area of sky. There are quite a few JWST observations of this target, so it is important to examine the relevant details.

out_cols = ['target_name','instrument_name','filters','t_exptime','proposal_id']
obs = targets[0]['obs']
unique(obs[out_cols]).show_in_notebook()
Table length=3
idxtarget_nameinstrument_namefilterst_exptimeproposal_id
0TRAPPIST-1MIRI/IMAGEF1500W72540.7493077
1TRAPPIST-1MIRI/IMAGEF1500W135230.2853077
2TRAPPIST-1BMIRI/IMAGEF1280W13983.4271279

You will need to examine the specifics of the programs that obtained the observations to know for certain which of the 7 known exoplanets was being observed, and therefore whether your intended observations would duplicate. Here are the unique Program Titles, which may offer clues:

obs['title'] = [x[:80] for x in obs['obs_title']]
unique(obs['proposal_id','title']).pprint(max_width=-1)
proposal_id                  title                 
----------- ---------------------------------------
       1279      Thermal emission from Trappist-1 b
       3077 TRAPPIST-1 Planets: Atmospheres Or Not?

The Program Status Pages offer the complete specifications for each existing program:

obs['proposal_URL'] = [get_program_URL(x) for x in obs['proposal_id']]
unique(obs['proposal_id','proposal_URL']).pprint(max_width=-1)
proposal_id                                proposal_URL                              
----------- -------------------------------------------------------------------------
       1279 (http://www.stsci.edu/cgi-bin/get-proposal-info?id=1279&observatory=JWST)
       3077 (http://www.stsci.edu/cgi-bin/get-proposal-info?id=3077&observatory=JWST)
There are both NIRISS and NIRSpec spectroscopic observations of this target that could conflict with your intended proposal.

V* XX Cha#

This pre-main-sequence star has a debris disk, and might be worth a coronagraphic observation. The search area is small (20 arcsec) to exclude nearby targets.

out_cols = ['target_name','instrument_name','filters','t_exptime','calib_level','proposal_id']
obs = targets[1]['obs']
unique(obs[out_cols]).show_in_notebook()
Table length=7
idxtarget_nameinstrument_namefilterst_exptimecalib_levelproposal_id
0V-XX-CHA-2MIRI/IFUCH13696.34831282
1V-XX-CHA-2MIRI/IFUCH23696.34831282
2V-XX-CHA-2MIRI/IFUCH33696.34831282
3V-XX-CHA-2MIRI/IFUCH43696.34831282
4V-XX-CHA-2MIRI/IMAGEF1280W3696.34831282
5V-XX-CHA-2NIRSPEC/IFUF290LP;G395H32.21-11282
6V-XX-CHA-2NIRSPEC/IFUF290LP;G395H161.052-11282

Note that some of the observations for this target had not been observed (at the beginning of Cy-2).

While JWST spectra from MIRI and NIRSpec exist or are planned, there is no potential for duplication with a coronagraphic observation.

M 57#

The Ring Nebula (NGC 6720) is well observed, particularly with HST and, recently, with JWST. We searched a 4-arcmin region around the source to include potential spectral Observations in the nebular periphery. The following table shows the unique combinations of instrument configurations.

out_cols = ['target_name','instrument_name','filters','t_exptime','proposal_id']
obs = targets[3]['obs']
unique(obs[out_cols]).show_in_notebook()
Table length=43
idxtarget_nameinstrument_namefilterst_exptimeproposal_id
0NGC6720MIRI/IMAGEF1000W444.00799999999991558
1NGC6720MIRI/IMAGEF1130W444.00799999999991558
2NGC6720MIRI/IMAGEF1280W444.00799999999991558
3NGC6720MIRI/IMAGEF1500W444.00799999999991558
4NGC6720MIRI/IMAGEF1800W444.00799999999991558
5NGC6720MIRI/IMAGEF2100W444.00799999999991558
6NGC6720MIRI/IMAGEF2550W444.00799999999991558
7NGC6720MIRI/IMAGEF560W444.00799999999991558
8NGC6720MIRI/IMAGEF770W444.00799999999991558
9NGC6720NIRCAM/IMAGEF150W2;F162M483.15599999999981558
10NGC6720NIRCAM/IMAGEF212N483.15599999999981558
11NGC6720NIRCAM/IMAGEF300M483.15599999999981558
12NGC6720NIRCAM/IMAGEF335M483.15599999999981558
13NGC6720-MRS-POSITION-1MIRI/IFUCH12763.9361558
14NGC6720-MRS-POSITION-1MIRI/IFUCH22763.9361558
15NGC6720-MRS-POSITION-1MIRI/IFUCH32763.9361558
16NGC6720-MRS-POSITION-1MIRI/IFUCH42763.9361558
17NGC6720-MRS-POSITION-1MIRI/IMAGEF1000W921.3121558
18NGC6720-MRS-POSITION-1MIRI/IMAGEF1130W921.3121558
19NGC6720-MRS-POSITION-1MIRI/IMAGEF770W921.3121558
20NGC6720-MRS-POSITION-2MIRI/IFUCH12763.9361558
21NGC6720-MRS-POSITION-2MIRI/IFUCH22763.9361558
22NGC6720-MRS-POSITION-2MIRI/IFUCH32763.9361558
23NGC6720-MRS-POSITION-2MIRI/IFUCH42763.9361558
24NGC6720-MRS-POSITION-2MIRI/IMAGEF1000W921.3121558
25NGC6720-MRS-POSITION-2MIRI/IMAGEF1130W921.3121558
26NGC6720-MRS-POSITION-2MIRI/IMAGEF770W921.3121558
27NGC6720-NIRSPEC-POSITION-1NIRSPEC/IFUF070LP;G140M145.8891558
28NGC6720-NIRSPEC-POSITION-1NIRSPEC/IFUF070LP;G140M583.5561558
29NGC6720-NIRSPEC-POSITION-1NIRSPEC/IFUF100LP;G140M145.8891558
30NGC6720-NIRSPEC-POSITION-1NIRSPEC/IFUF100LP;G140M583.5561558
31NGC6720-NIRSPEC-POSITION-1NIRSPEC/IFUF170LP;G235M145.8891558
32NGC6720-NIRSPEC-POSITION-1NIRSPEC/IFUF170LP;G235M583.5561558
33NGC6720-NIRSPEC-POSITION-1NIRSPEC/IFUF290LP;G395M145.8891558
34NGC6720-NIRSPEC-POSITION-1NIRSPEC/IFUF290LP;G395M583.5561558
35NGC6720-NIRSPEC-POSITION-2NIRSPEC/IFUF070LP;G140M145.8891558
36NGC6720-NIRSPEC-POSITION-2NIRSPEC/IFUF070LP;G140M583.5561558
37NGC6720-NIRSPEC-POSITION-2NIRSPEC/IFUF100LP;G140M145.8891558
38NGC6720-NIRSPEC-POSITION-2NIRSPEC/IFUF100LP;G140M583.5561558
39NGC6720-NIRSPEC-POSITION-2NIRSPEC/IFUF170LP;G235M145.8891558
40NGC6720-NIRSPEC-POSITION-2NIRSPEC/IFUF170LP;G235M583.5561558
41NGC6720-NIRSPEC-POSITION-2NIRSPEC/IFUF290LP;G395M145.8891558
42NGC6720-NIRSPEC-POSITION-2NIRSPEC/IFUF290LP;G395M583.5561558
There are both MIRI and NIRCam images in many passbands, and MIRI and NIRSpec spectroscopic IFU Observations of this target that could conflict with the intended proposal.

To retain this target it would be necessary to obtain spectra in a different location, or to use a different instrument/configuration/data-taking mode (e.g., time-series of the central star). It may be helpful to see the footprints of these Observations in the MAST Portal, or to view the observation specifications in Program GO-1558.

Appendix: Caveats#

It is possible to search MAST for individual sources by name, in one of two ways: Object Name or by Target Name.

  • Object Name: MAST invokes an astrophysical name resolver (e.g., NED or Simbad) to look up source coordinates. Not all object names are recognized by resolvers. Also, different resolvers sometimes return substantially different coordinates; MAST will pick one of them (but not tell you which one). Sometimes the coordinates returned by the resolvers differ by substantial amounts, often because they refer to different sources.

    • A classic case is the bright star μ Eri from the Bayer Greek letter system, and the star MU Eri in the general catalog of variable stars. in MAST,

      • Searching for * mu. eri (or just plain mu eri matches μ Eri (RA = 4:45:30.15, Dec = -3:15:16.777)

      • Searching for V* mu eri matches MU Eri (RA = 2:48:10.566, Dec = -15:18:04.03)

  • Target Name: This search matches to target names specified by Investigators in observing proposals to refer to sources. These are not guaranteed to match standard names for astrophysical sources, and often do not.

Searching by source names is not recommended when checking for potential duplications with fixed targets because the matches are not reliable, consistent, or complete. The exception would be for matching Object names of well known solar system bodies.

The best approach is to use an independent source for the coordinates of your desired target. Failing that, the Mast.resolve_object() method will return coordinates in the ICRS reference frame, which you can use to verify whether a target name resolves to the coordinates you intend. See the Appendix for details.

Additional Resources#

About this notebook#

This notebook was developed by Archive Sciences Branch staff. For support, please contact the Archive HelpDesk at archive@stsci.edu, or through the JWST HelpDesk Portal. Space Telescope Logo

Last update: 2023 Aug