Downloading Subscription Products with Astroquery

Downloading Subscription Products with Astroquery#

This is a short notebook, focused on processing the results of a MAST subscription email notification. By the end of this notebook, you will be able to:

Use astroquery to download products from a subscription email notification
Categorize subscriptions, based on the contents of the JSON blob

Imports#

There is only one import needed for this notebook:

astroquery.mast to download products. For the latest functionality (and to run the two-line download), you’ll need to install directly from the repo: pip install git+https://github.com/astropy/astroquery.git. Once this feature is released as part of the next astroquery update, you’ll be able to install this directly.

Quick Run: Two-Line Download#

If you are simply interested in getting your products as quickly as possible, this can be done in just two lines of code. The MissionsMast module makes this straightfoward: point download_products at the file you received in your email.

Note: for the sake of simplicity in this notebook, we have named the file JWST.json. In your email, you have likely received a file with a name like event_metadata_jwst-program-5105_hourly_20260422_143523.json.

from astroquery.mast import MastMissions
MastMissions.download_products("./JWST.json")

# if downloading EAP data, you must login
# MastMissions.login()

Downloading URL https://mast.stsci.edu/search/jwst/api/v0.1/retrieve_product?product_name=jw05893001001_02201_00001_mirimage_o001_crf.fits to mastDownload/jwst/jw05893001001_02201_00001/jw05893001001_02201_00001_mirimage_o001_crf.fits ...

 [Done]

Downloading URL https://mast.stsci.edu/search/jwst/api/v0.1/retrieve_product?product_name=jw05105001001_02201_00001_mirimage_o001_crf.fits to mastDownload/jwst/jw05105001001_02201_00001/jw05105001001_02201_00001_mirimage_o001_crf.fits ...

 [Done]

Downloading URL https://mast.stsci.edu/search/jwst/api/v0.1/retrieve_product?product_name=jw05105012002_02201_00001_mirimage_rateints.fits to mastDownload/jwst/jw05105012002_02201_00001/jw05105012002_02201_00001_mirimage_rateints.fits ...

 [Done]

Table length=3

Local Path	Status	Message	URL
object	str8	object	object
mastDownload/jwst/jw05893001001_02201_00001/jw05893001001_02201_00001_mirimage_o001_crf.fits	COMPLETE	None	None
mastDownload/jwst/jw05105001001_02201_00001/jw05105001001_02201_00001_mirimage_o001_crf.fits	COMPLETE	None	None
mastDownload/jwst/jw05105012002_02201_00001/jw05105012002_02201_00001_mirimage_rateints.fits	COMPLETE	None	None

As with a standard astroquery download, the display prints download progress notifications along with a status table of all products after completion.

That’s it! You’re done. Your products are downloaded. This only includes products you’ve subscribed to, so if you want to search for additional products you’ll need to perform a full search using astroquery.

A Little More Detail#

Although it is easy to download your products, it is useful to know more about how the download proceeds and the contents of the JSON file.

Examining the JSON Contents#

Let’s a look at the information in the JSON itself. We’ll need one more import to accomplish this: astropy.table will read our file into an easy-to-read table.

# we need this to properly read in the file
from astropy.table import Table

# we need to specify pandas.json as the format
subscription = Table.read("./JWST.json", format="pandas.json")
subscription

Table length=3

event_id	mission	fileset	filename	program	event_type	archived_date	public_release_date
str36	str4	str25	str48	int64	str11	str27	str23
a1175c61-00ae-484b-8d82-f02ccf17a0b3	JWST	jw05893001001_02201_00001	jw05893001001_02201_00001_mirimage_o001_crf.fits	5893	Reprocessed	2025-12-03T19:44:10.707	2025-04-30T18:50:03.000
17a653f9-c2fe-4d71-a3ca-2590123615da	JWST	jw05105001001_02201_00001	jw05105001001_02201_00001_mirimage_o001_crf.fits	5105	New	2026-03-30T17:53:19.695	2026-03-30T17:52:27.000
09498e44-08b3-4f2e-849a-263301f942a1	JWST	jw05105012002_02201_00001	jw05105012002_02201_00001_mirimage_rateints.fits	5105	Public	2026-03-30T17:54:12.7510000	2026-03-30T17:53:13

As we would expect from our download., there are three files in this list. Each file has eight metadata fields. Of these eight fields, it is particularly important to understand the event_type, which represents the reason for the subscription notification:

Reprocessed: the file was reprocessed by an updated version of a calibration pipeline
New: this file is new to the archive
Public: the file was previously under an exclusive access period (EAP), but is now publicly available

In the case where an investigator waives their EAP, files will be considered both new and public; notifications for either category will trigger.

A Note on the Download Process#

Normally, when using the MissionMast module, you need to specify the mission to complete a download. For example, if we try to download the first file in our set (jwst.download_file("jw05893001001_02201_00001_mirimage_o001_crf.fits):

# specify mission first
jwst = MastMissions(mission='jwst')
# now download file
jwst.download_file("jw05893001001_02201_00001_mirimage_o001_crf.fits")

Downloading URL https://mast.stsci.edu/search/jwst/api/v0.1/retrieve_product?product_name=jw05893001001_02201_00001_mirimage_o001_crf.fits to jw05893001001_02201_00001_mirimage_o001_crf.fits ...

 [Done]

('COMPLETE', None, None)

Since one of the JSON keys is mission, MAST has designed the download_products function to read from this directly; no need to add this value manually.

About this Notebook#

Issues with this notebook or with your subscription data? Contact archive@stsci.edu or use the MAST help desk portal.

Author: Thomas Dutkiewicz
Keywords: subscriptions, MastMissions, astroquery

Top of Page