Roman Research Nexus Data Discovery and Access in the Cloud#
Kernel Information and Read-Only Status#
To run this notebook, please select the “Roman Research Nexus” kernel at the top right of your window.
This notebook is read-only. You can run cells and make edits, but you must save changes to a different location. We recommend saving the notebook within your home directory, or to a new folder within your home (e.g. file > save notebook as > my-nbs/nb.ipynb). Note that a directory must exist before you attempt to add a notebook to it.
Imports#
Here we import the required packages for our data access examples including:
asdf for accessing ASDF files
astropy.io fits for accessing FITS files
astropy.mast Observations for accessing, searching, and selecting data from other missions
s3fs for streaming in data directly from the cloud
roman_datamodels for opening Roman ASDF files. You can find additional information on how to work with ASDF files in the Working with ASDF notebook tutorial.
import asdf
from astropy.io import fits
from astroquery.mast import Observations
import s3fs
import roman_datamodels as rdm
Introduction#
This notebook is designed to provide examples of accessing data from the Research Nexus. Due to its survey nature, the Roman Space Telescope will produce large volumes of data that will need to be easily and quickly accessed to perform scientific tasks like creating catalogs, performing difference imaging, generating light curves, etc. Downloading all the required data would burden most users by requiring excessive data storage solutions (likely >10TB).
This notebook demonstrates how to stream data from the cloud directly into memory, bypassing the need to download the data locally and use excess storage. This method of cloud-based data access is HIGHLY recommended. However, we understand that some use-cases will require downloading the data locally, so we provide an example at the end of the notebook.
During operations, each Roman data file will be given a Unique Resource Identifier (URI), an analog to an online filepath that is similar to a URL, which points to where the data is hosted on the AWS cloud. Users will retrieve these URIs from one of several sources including MAST (see Accessing WFI Data for more information) and will be able to use the URI to access the desired data from the cloud.
Here-in we examine how to download data from two types of sources:
The STScI MAST server which hosts data for in-flight telescopes including Hubble, TESS, and JWST and will host Roman data in the future
Simulated Roman Space Telescope data hosted in storage containers on the AWS cloud
Defining terms#
Cloud computing: the practice of using a network of remote servers hosted on the internet to store, manage, and process data, rather than using a local server or a personal computer.
AWS: Amazon Web Services (AWS) is the cloud computing platform provided by Amazon.
URI: a Universal Resource Identifier (URI) is a sequence of characters that identifies a name or a unique resource on the Internet. URLs for websites are a subclass of URIs.
AWS S3: Amazon Simple Storage Service (S3) is a scalable and cost-effective object storage service on the AWS cloud platform. Storage containers within S3 are knwon as “buckets,” so we often refer to these storage devices as “S3 buckets” or “S3 servers”.
Accessing MAST Data#
In this section, we will go through the steps to retreive archived MAST data from the cloud including how to query the archive and stream the files directly from the cloud, as well as download them locally.
Enabling Cloud Access#
The most important step for accessing data from the cloud is to enable astroquery to retreive URIs and other relevant cloud information. Even if we are working locally and plan to download the data files (not recommended for Roman data), we need to use this command to copy the file locations.
Observations.enable_cloud_dataset()
INFO: Using the S3 STScI public dataset [astroquery.mast.cloud]
Querying MAST#
Now we are ready to begin our query. This example is rather simple, but it is quick and easy to reproduce. We will be querying HST WFC3/IR data of M85. In practice, the science platform should primarily be used for analyzing and exploring Roman data products. However due to the smaller file sizes, HST WFC3/IR data provides a nice example. The process is identical regardless of which space telescope is used.
In our query, we specify that we want to look at HST data using the F160W filter and WFC3/IR. We also specify the proposal id to easily get the data of interest. Once we get the desired observations, we gather the list of products that go into the observations. We then filter the products to gather all the level 3 science data products associated with a specific project which still leaves us with 60 data products.
# query MAST for matching observations
obs = Observations.query_criteria(obs_collection='HST',
filters='F160W',
instrument_name='WFC3/IR',
proposal_id=['11360'],
dataRights='PUBLIC')
# get the list of products (files)
products = Observations.get_product_list(obs)
# filter the products
filtered = Observations.filter_products(products,
calib_level=[3],
productType=['SCIENCE'],
dataproduct_type=['image'],
project=['CALWF3'])
print('Filtered data products:\n', filtered, '\n')
# filter for just one product
single = Observations.filter_products(filtered,
obsID='24797441')
print('Single data product:\n', single, '\n')
Filtered data products:
obsID obs_collection dataproduct_type ... dataRights calib_level filters
-------- -------------- ---------------- ... ---------- ----------- -------
23831959 HST image ... PUBLIC 3 F160W
23831959 HST image ... PUBLIC 3 F160W
23831961 HST image ... PUBLIC 3 F160W
23831961 HST image ... PUBLIC 3 F160W
23831988 HST image ... PUBLIC 3 F160W
23831988 HST image ... PUBLIC 3 F160W
23831990 HST image ... PUBLIC 3 F160W
23831990 HST image ... PUBLIC 3 F160W
23832009 HST image ... PUBLIC 3 F160W
23832009 HST image ... PUBLIC 3 F160W
... ... ... ... ... ... ...
24797437 HST image ... PUBLIC 3 F160W
24797437 HST image ... PUBLIC 3 F160W
24797441 HST image ... PUBLIC 3 F160W
24797441 HST image ... PUBLIC 3 F160W
24797446 HST image ... PUBLIC 3 F160W
24797446 HST image ... PUBLIC 3 F160W
24797447 HST image ... PUBLIC 3 F160W
24797447 HST image ... PUBLIC 3 F160W
24797448 HST image ... PUBLIC 3 F160W
24797448 HST image ... PUBLIC 3 F160W
Length = 60 rows
Single data product:
obsID obs_collection dataproduct_type ... dataRights calib_level filters
-------- -------------- ---------------- ... ---------- ----------- -------
24797441 HST image ... PUBLIC 3 F160W
24797441 HST image ... PUBLIC 3 F160W
Now that we have our desired products, we can gather the URIs for each of the files which indicate their locations in the MAST AWS S3 servers.
uris = Observations.get_cloud_uris(filtered)
uris
INFO: 30 of 60 products were duplicates. Only returning 30 unique product(s). [astroquery.mast.utils]
['s3://stpubdata/hst/public/ib6w/ib6wd4lrq/ib6wd4lrq_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6wd4ltq/ib6wd4ltq_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6wd6f5q/ib6wd6f5q_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6wd6f7q/ib6wd6f7q_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6we1p9q/ib6we1p9q_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6we1paq/ib6we1paq_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6wr8kdq/ib6wr8kdq_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6wr8kfq/ib6wr8kfq_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6w11050/ib6w11050_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6w21050/ib6w21050_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6w37040/ib6w37040_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6w39010/ib6w39010_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6w41050/ib6w41050_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6w61070/ib6w61070_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6w71040/ib6w71040_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6w810b0/ib6w810b0_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6w810c0/ib6w810c0_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6w810d0/ib6w810d0_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6w810e0/ib6w810e0_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6w91020/ib6w91020_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6wa1030/ib6wa1030_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6wb1070/ib6wb1070_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6wd4090/ib6wd4090_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6wd6080/ib6wd6080_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6wr1050/ib6wr1050_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6wr80a0/ib6wr80a0_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6wr9040/ib6wr9040_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6wra070/ib6wra070_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6wra080/ib6wra080_drz.fits',
's3://stpubdata/hst/public/ib6w/ib6wra090/ib6wra090_drz.fits']
The get_cloud_uris method checks for duplicates in the provided products to minimize the data access volume. It is also important to note that get_cloud_uris will always return a list. Thus, we need to extract an individual URI string to access the file. Here we choose the first URI, but in practice you would select the URI associated with the desired file.
uri = uris[0]
Streaming files directly into memory#
Here, we will use fsspec to directly access the data stored in the AWS S3 servers. Because the URI points to a FITS file, we can use fits.open to access the information in the file.
with fits.open(uri, 'readonly', fsspec_kwargs={"anon":True}) as HDUlist:
HDUlist.info()
sci = HDUlist[1].data
type(sci)
Filename: <class 's3fs.core.S3File'>
No. Name Ver Type Cards Dimensions Format
0 PRIMARY 1 PrimaryHDU 770 ()
1 SCI 1 ImageHDU 90 (543, 484) float32
2 WHT 1 ImageHDU 45 (543, 484) float32
3 CTX 1 ImageHDU 40 (543, 484) int32
4 HDRTAB 1 BinTableHDU 561 1R x 276C [9A, 3A, K, D, D, D, D, D, D, D, D, D, D, D, D, K, 8A, 4A, 1A, 4A, D, D, D, D, D, 3A, D, D, D, D, D, D, D, D, D, D, D, D, K, K, D, 3A, D, D, D, D, K, K, 8A, 23A, 11A, 19A, 4A, D, D, K, K, D, D, D, D, 23A, D, D, D, D, K, K, D, 3A, 8A, L, D, D, D, 23A, 1A, D, D, D, D, D, D, 12A, 12A, 8A, 23A, D, D, 10A, 10A, D, D, D, 2A, 23A, 3A, 4A, 8A, 7A, D, K, D, 6A, 9A, D, D, D, 4A, 18A, 3A, K, 5A, D, D, D, 8A, D, 3A, D, D, D, 3A, 1A, D, 23A, D, D, D, 3A, L, 1A, 4A, D, 3A, 6A, D, D, D, D, D, 23A, D, D, D, D, D, 1A, K, K, K, K, 8A, 23A, K, K, 10A, 7A, D, D, D, D, D, D, D, D, D, D, D, D, D, D, D, D, D, 13A, D, 24A, 23A, D, 1A, 1A, D, K, D, D, 1A, 1A, D, D, D, D, D, D, D, D, D, D, D, 4A, D, K, 23A, D, 8A, D, D, 28A, D, D, L, D, D, 6A, 9A, 2A, 4A, D, 1A, K, 1A, 1A, 1A, 1A, D, D, D, D, D, D, 4A, D, D, 4A, 3A, 1A, K, 3A, 5A, K, D, D, D, 4A, K, D, D, D, L, K, K, 11A, 1A, D, D, D, D, K, L, 6A, L, 8A, D, D, 1A, 10A, K, 3A, 8A, 8A, D, K, D, 30A, 8A, 8A]
numpy.ndarray
Streaming from the Roman Research Nexus S3 Bucket#
Though Roman data will eventually be available through MAST, we currently offer a small set of simulated data available via an AWS Open Data Program S3 bucket. These files can be streamed in exactly the same way as the HST FITS file above. Additionally, we can browse the available files similarly to a Unix terminal. A full list of commands can be found in the s3fs documentation here.
We can view the files in the S3 bucket by performing a list command (ls) on the main Nexus directory:
fs = s3fs.S3FileSystem(anon=True)
asdf_dir_uri = 's3://stpubdata/roman/nexus/soc_simulations/tutorial_data/'
fs.ls(asdf_dir_uri)
['stpubdata/roman/nexus/soc_simulations/tutorial_data/._Roman_OpticalModel_v0.5.yml',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._r9999901001001001001_0001_wfi01_f129_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._r9999901001001001001_0001_wfi01_f129_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._r9999902002002002002_0001_wfi01_f213_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._r9999902002002002002_0001_wfi01_f213_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61524_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61524_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61529_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61529_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61534_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61534_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61539_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61539_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61544_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61544_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61549_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61549_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61554_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61554_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61559_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61559_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61564_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61564_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61569_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61569_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61574_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61574_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61579_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61579_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61584_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61584_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61589_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61589_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61594_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61594_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61599_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61599_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61604_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61604_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61609_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61609_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61614_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61614_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61619_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61619_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61624_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61624_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61629_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61629_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61634_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61634_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61639_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61639_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61644_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61644_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61649_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61649_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61654_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61654_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61659_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61659_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61664_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61664_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61669_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61669_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61674_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/._roman_sn1a_61674_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/Roman_OpticalModel_v0.5.yml',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/full_catalog.ecsv',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/my_roman_mosaic_asn.json',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/my_roman_mosaic_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/my_roman_mosaic_coadd.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/my_roman_mosaic_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi01_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi01_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi01_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi01_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi02_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi02_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi02_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi02_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi03_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi03_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi03_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi03_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi04_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi04_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi04_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi04_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi05_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi05_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi05_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi05_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi06_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi06_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi06_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi06_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi07_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi07_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi07_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi07_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi08_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi08_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi08_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi08_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi09_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi09_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi09_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi09_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi10_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi10_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi10_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi10_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi11_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi11_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi11_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi11_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi12_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi12_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi12_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi12_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi13_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi13_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi13_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi13_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi14_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi14_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi14_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi14_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi15_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi15_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi15_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi15_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi16_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi16_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi16_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi16_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi17_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi17_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi17_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi17_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi18_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi18_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi18_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi18_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi01_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi01_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi01_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi01_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi02_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi02_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi02_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi02_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi03_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi03_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi03_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi03_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi04_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi04_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi04_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi04_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi05_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi05_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi05_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi05_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi06_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi06_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi06_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi06_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi07_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi07_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi07_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi07_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi08_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi08_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi08_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi08_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi09_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi09_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi09_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi09_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi10_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi10_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi10_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi10_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi11_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi11_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi11_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi11_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi12_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi12_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi12_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi12_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi13_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi13_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi13_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi13_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi14_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi14_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi14_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi14_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi15_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi15_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi15_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi15_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi16_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi16_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi16_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi16_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi17_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi17_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi17_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi17_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi18_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi18_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi18_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0002_wfi18_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi01_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi01_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi01_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi01_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi02_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi02_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi02_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi02_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi03_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi03_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi03_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi03_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi04_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi04_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi04_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi04_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi05_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi05_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi05_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi05_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi06_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi06_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi06_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi06_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi07_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi07_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi07_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi07_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi08_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi08_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi08_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi08_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi09_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi09_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi09_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi09_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi10_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi10_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi10_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi10_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi11_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi11_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi11_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi11_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi12_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi12_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi12_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi12_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi13_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi13_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi13_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi13_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi14_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi14_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi14_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi14_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi15_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi15_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi15_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi15_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi16_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi16_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi16_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi16_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi17_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi17_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi17_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi17_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi18_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi18_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi18_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0003_wfi18_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi01_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi01_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi01_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi01_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi02_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi02_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi02_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi02_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi03_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi03_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi03_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi03_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi04_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi04_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi04_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi04_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi05_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi05_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi05_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi05_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi06_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi06_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi06_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi06_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi07_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi07_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi07_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi07_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi08_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi08_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi08_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi08_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi09_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi09_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi09_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi09_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi10_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi10_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi10_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi10_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi11_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi11_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi11_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi11_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi12_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi12_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi12_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi12_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi13_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi13_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi13_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi13_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi14_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi14_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi14_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi14_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi15_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi15_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi15_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi15_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi16_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi16_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi16_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi16_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi17_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi17_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi17_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi17_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi18_f106_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi18_f106_cat.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi18_f106_segm.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0004_wfi18_f106_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0007601002004013001_0001_catalog.fits',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0007601002004013001_0001_qso.fits',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0007601002004013001_0001_wfi01_f158_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r0007601002004013002_0001_wfi01_grism_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r9999901001001001001_0001_wfi01_f129_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r9999901001001001001_0001_wfi01_f129_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r9999902002002002002_0001_wfi01_f213_cal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/r9999902002002002002_0001_wfi01_f213_uncal.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61524_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61524_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61529_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61529_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61534_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61534_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61539_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61539_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61544_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61544_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61549_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61549_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61554_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61554_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61559_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61559_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61564_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61564_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61569_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61569_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61574_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61574_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61579_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61579_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61584_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61584_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61589_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61589_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61594_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61594_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61599_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61599_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61604_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61604_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61609_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61609_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61614_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61614_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61619_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61619_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61624_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61624_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61629_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61629_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61634_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61634_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61639_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61639_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61644_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61644_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61649_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61649_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61654_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61654_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61659_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61659_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61664_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61664_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61669_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61669_f213.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61674_f129.asdf',
'stpubdata/roman/nexus/soc_simulations/tutorial_data/roman_sn1a_61674_f213.asdf']
The fs.ls() command allows us to list the contents of the URI. In the above example, the s3://stpubdata/roman/nexus/soc_simulations/tutorial_data/ bucket contains numerous files for the notebook tutorials.
In the next subsection, we will explore opening data files made using the Roman image simulator “Roman I-Sim.” These simulations are saved in the same file formats as Roman data and are useful to help develop file ingestion pipelines. More Roman I-Sim simulated data will be made available in the future.
For more information on the available data products, please visit the Simulated Data Products documentation.
Opening Roman I-Sim Data#
Diving into the S3 bucket, we find several different files:
*_uncal.asdf: Level 1 (L1; ucalibrated ramp cube) files.*_cal.asdf: Level 2 (L2; calibrated rate image) files.*_coadd.asdf: Level 3 (L3; resampled image) files.Some additional miscellaneous files for various tutorials.
To learn how these files were generated, please see the Roman I-Sim tutorial notebook.
As you can see, Roman WFI data are stored in Advanced Scientific Data Format (ASDF) files. See the tutorial notebook Working with ASDF for more information. Regarding file names, note that the first element (separated by underscores) of the file name string of L1* and L2* files denotes programmatic information (e.g., program ID, visit ID, etc.), while the second element gives the exposure number within the visit. Thus, r0003201001001001004_0001_wfi02_f106_cal.asdf and r0003201001001001004_0002_wfi02_f106_cal.asdf are exposure level (L2) files from the same visit but represent exposures 1 and 2, respectively, of the detector WFI02. For simulated data, such as the files used for these tutorials, the first element of the file name string has been chosen simply as an example. The file naming convention for Roman is quite elaborate as each includes all the relevant information about the observation. For more information on the file naming conventions, please see the Data Levels and Products Roman documentation page.
Note: Archival file names for the mosaic images (L3 products) are still being finalized. The L3 file in the S3 bucket has a generic file name (my_roman_mosaic_coadd.asdf) that does not follow any naming convention.
Below, we use roman_datamodels to read the ASDF file corresponding to a dense region. To simplify the workflow, we are providing a URI to the data. Once the data will be available through MAST during operations, users will need to retrieve the URIs using astroquery.
asdf_file_uri = asdf_dir_uri + 'r0003201001001001004_0001_wfi11_f106_cal.asdf'
with fs.open(asdf_file_uri, 'rb') as f:
dm = rdm.open(f)
dm.info()
root (AsdfObject)
├─asdf_library (Software)
│ ├─author (str): The ASDF Developers
│ ├─homepage (str): http://github.com/asdf-format/asdf
│ ├─name (str): asdf
│ └─version (str): 4.1.0
├─history (AsdfDictNode)
│ └─extensions (AsdfListNode) ...
└─roman (WfiImage) # Level 2 (L2) Calibrated Roman Wide Field Instrument (WFI) Rate Image.
├─meta (AsdfDictNode) ...
├─data (NDArrayType) # Science Data (DN/s) or (MJy/sr) ...
├─dq (NDArrayType) # Data Quality ...
├─err (NDArrayType) # Error (DN / s) or (MJy / sr) ...
├─var_poisson (NDArrayType) # Poisson Variance (DN^2/s^2) or (MJy^2/sr^2) ...
├─var_rnoise (NDArrayType) # Read Noise Variance (DN^2/s^2) or (MJy^2/sr^2) ...
├─var_flat (NDArrayType) # Flat Field Variance (DN^2/s^2) or (MJy^2/sr^2) ...
├─amp33 (NDArrayType) # Amplifier 33 Reference Pixel Data (DN) ...
├─border_ref_pix_left (NDArrayType) # Left-Edge Border Reference Pixels (DN) ...
├─border_ref_pix_right (NDArrayType) # Right-Edge Border Reference Pixels (DN) ...
├─border_ref_pix_top (NDArrayType) # Border Reference Pixels on the Top of the Detector (DN) ...
├─border_ref_pix_bottom (NDArrayType) # Bottom-Edge Border Reference Pixels (DN) ...
├─dq_border_ref_pix_left (NDArrayType) # Left-Edge Border Reference Pixel Data Quality (DN) ...
└─3 not shown
Some nodes not shown.
Opening OpenUniverse Simulated Data#
The OpenUniverse data is hosted in its own S3 bucket (see the OpenUniverse AWS Open Data website for more information). Additionally, IPAC has created two OpenUniverse notebooks showcasing how to interact with the original data and catalog files. In this notebook, we focus on how to access the files.
The simulations are natively saved as FITS files and are divided by survey (the Wide Area Survey (WAS) or the Time Domain Survey (TDS)), optical element, and HEALPix cell (HEALPix is a commonly used way to uniformly discretize the area of a sphere). Please see Simulated Data Products for more information about specific OpenUniverse data products.
Below we provide an example of streaming a simulated “calibrated” image FITS file from an S3 bucket using an alternate method. Instead of initializing our own S3FileSystem, we pass the credentials (anonymous credentails in this case, as the data is public) directly to fits.open, allowing it to create the file system. This shorthand is covenient when the URI is explicitly provided, but it does not allow exploration of the S3 directory structure without intitializing the S3FileSystem separately.
s3bucket = 's3://nasa-irsa-simulations/openuniverse2024/roman/preview/RomanWAS/images/simple_model'
band = 'F184'
hpix = '15297'
sensor = 11
s3fpath = s3bucket+f'/{band}/{hpix}/Roman_WAS_simple_model_{band}_{hpix}_{sensor}.fits.gz'
fits_file = fits.open(s3fpath, fsspec_kwargs={'anon':True})
print(fits_file.info())
Filename: <class 's3fs.core.S3File'>
No. Name Ver Type Cards Dimensions Format
0 PRIMARY 1 PrimaryHDU 63 ()
1 SCI 1 ImageHDU 68 (4088, 4088) float64
2 ERR 1 ImageHDU 68 (4088, 4088) float32
3 DQ 1 ImageHDU 70 (4088, 4088) int32 (rescales to uint32)
None
Downloading Files (not recommended)#
Downloading Roman data products is not recommended due to their large file sizes and the high volume expected from the mission’s survey nature. Instead, users are encouraged to adopt workflows that utilize the file streaming services described above for an optimal experience.
However, in specific science cases, downloading files may be necessary. To do so, you can use the URIs along with the S3FileSystem.get function (documentation available here). The code snippet below demonstrates how to download data to your personal instance of the Research Nexus.
# commented out as this use case is not recommended and should only be needed in rare circumstances
# from pathlib import Path
# URI = ## Set this to the URI string you want to download.
# local_file_path = Path('data/')
# local_file_path.mkdir(parents=True, exist_ok=True)
# fs = s3fs.S3FileSystem()
# fs.get(URI, local_file_path)
Additional Resources#
Additional information can be found at the following links:
About this Notebook#
The data streaming information from this notebook largely builds off of the TIKE data-acces notebook by Thomas Dutkiewicz.
Author: Will C. Schultz, Tyler Desjardins
Updated On: 2025-09-30
| ↑ Top of page |
|
|