Running Individual Pipeline Steps#
This notebook walks through calibrating the data with individual pipeline steps (AssignWCS and FlatField) rather than running the entire pipeline stage.
Use case: When using a package outside of the standard JWST pipeline, there may be certain steps that are still helpful to utilize within the JWST pipeline. Here we show the most commonly run individual steps, AssignWCS and FlatField. AssignWCS determines and stores the WCS (World Coordinate System) information, and FlatField removes detector features.
Data: JWST/NIRISS images and spectra from program 2079.
Tools: astropy, crds, glob, jwst, matplotlib, numpy, os, urllib, zipfile
Cross-instrument: NIRISS
Content
Author: Rachel Plesha (rplesha@stsci.edu), Camilla Pacifici (cpacifici@stsci.edu)
Last modified: May 2024
Last tested: This notebook was last tested with JWST pipeline version 1.12.5 and the CRDS context jwst_1229.pmap.
Imports & Data Setup#
# Update the CRDS path to your local directory
%env CRDS_PATH=crds_cache
%env CRDS_SERVER_URL=https://jwst-crds.stsci.edu
env: CRDS_PATH=crds_cache
env: CRDS_SERVER_URL=https://jwst-crds.stsci.edu
import os
import glob
import urllib
import zipfile
import numpy as np
from astropy.io import fits
from matplotlib import pyplot as plt
%matplotlib inline
from jwst.assign_wcs import AssignWcsStep
from jwst.flatfield import FlatFieldStep
Check what version of the JWST pipeline you are using. To see what the latest version of the pipeline is available or install a previous version, check GitHub. Also verify what CRDS context you are using. CRDS documentation explains how to set a specific context to use in the JWST pipeline. If either of these values are different from the last tested note above there may be differences in this notebook.
import jwst
import crds
print('JWST Pipeliene Version:', jwst.__version__)
print('CRDS Context:', crds.get_context_name('jwst'))
JWST Pipeliene Version: 1.18.0
CRDS - INFO - Calibration SW Found: jwst 1.18.0 (/opt/hostedtoolcache/Python/3.11.12/x64/lib/python3.11/site-packages/jwst-1.18.0.dist-info)
CRDS Context: jwst_1364.pmap
Data setup#
Here we download and open the zip file that contains all of the rate files, and we also create an output directory for the calibrated files if it does not already exist
data_dir_in = 'data' # directory where the rate files should be
data_dir_out = 'data/calibrated_steps/' # directory where to save the calibrate files
# if the directory does not exist that you want to save out to, make that directory first
for datadir in [data_dir_in, data_dir_out]:
if not os.path.exists(datadir):
os.makedirs(datadir)
# Download uncalibrated data from Box into the data directory:
boxlink = 'https://data.science.stsci.edu/redirect/JWST/jwst-data_analysis_tools/niriss_wfss_advanced/niriss_wfss_extra_input.zip'
boxfile = os.path.basename(boxlink)
urllib.request.urlretrieve(boxlink, boxfile)
zf = zipfile.ZipFile(boxfile, 'r')
zf.extractall(path=data_dir_in)
# move the files downloaded from the box file into the top level data directory
box_download_dir = os.path.join(data_dir_in, boxfile.split('.zip')[0])
for filename in glob.glob(os.path.join(box_download_dir, '*')):
if '.csv' in filename:
# move to the current directory
os.rename(filename, os.path.basename(filename))
else:
# move to the data directory
os.rename(filename, os.path.join(data_dir_in, os.path.basename(filename)))
# remove unnecessary files now
os.remove(boxfile)
os.rmdir(box_download_dir)
Running Individual Pipeline Steps#
While you could look at the rate images, instead consider running the files through the assign_wcs and flat_field steps of the pipeline to clean up detector artifacts.
Assign WCS Step#
The assign_wcs step of the pipeline is a critical part to obtaining the correct spectral trace cutouts for WFSS images. To read more about the step, visit the AssignWCS description page.
# Run assign_wcs; we are only running on one file for demonstration here
ratefile = os.path.join(data_dir_in, 'jw02079004002_02101_00001_nis_rate.fits')
result = AssignWcsStep.call(ratefile, output_dir=data_dir_out, save_results=True)
2025-05-15 19:16:45,341 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_system_datalvl_0002.rmap 694 bytes (1 / 204 files) (0 / 741.0 K bytes)
2025-05-15 19:16:45,448 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_system_calver_0048.rmap 5.3 K bytes (2 / 204 files) (694 / 741.0 K bytes)
2025-05-15 19:16:45,540 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_system_0047.imap 385 bytes (3 / 204 files) (6.0 K / 741.0 K bytes)
2025-05-15 19:16:45,635 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_wavelengthrange_0024.rmap 1.4 K bytes (4 / 204 files) (6.4 K / 741.0 K bytes)
2025-05-15 19:16:45,726 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_wavecorr_0005.rmap 884 bytes (5 / 204 files) (7.8 K / 741.0 K bytes)
2025-05-15 19:16:45,806 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_superbias_0079.rmap 36.0 K bytes (6 / 204 files) (8.6 K / 741.0 K bytes)
2025-05-15 19:16:45,938 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_sirskernel_0001.rmap 630 bytes (7 / 204 files) (44.6 K / 741.0 K bytes)
2025-05-15 19:16:46,100 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_sflat_0026.rmap 20.6 K bytes (8 / 204 files) (45.3 K / 741.0 K bytes)
2025-05-15 19:16:46,257 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_saturation_0018.rmap 2.0 K bytes (9 / 204 files) (65.9 K / 741.0 K bytes)
2025-05-15 19:16:46,346 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_refpix_0015.rmap 1.6 K bytes (10 / 204 files) (67.9 K / 741.0 K bytes)
2025-05-15 19:16:46,452 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_readnoise_0025.rmap 2.6 K bytes (11 / 204 files) (69.5 K / 741.0 K bytes)
2025-05-15 19:16:46,582 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_pictureframe_0001.rmap 675 bytes (12 / 204 files) (72.0 K / 741.0 K bytes)
2025-05-15 19:16:46,694 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_photom_0013.rmap 958 bytes (13 / 204 files) (72.7 K / 741.0 K bytes)
2025-05-15 19:16:46,786 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_pathloss_0008.rmap 1.2 K bytes (14 / 204 files) (73.7 K / 741.0 K bytes)
2025-05-15 19:16:46,880 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_pars-whitelightstep_0001.rmap 777 bytes (15 / 204 files) (74.9 K / 741.0 K bytes)
2025-05-15 19:16:46,977 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_pars-spec2pipeline_0013.rmap 2.1 K bytes (16 / 204 files) (75.6 K / 741.0 K bytes)
2025-05-15 19:16:47,057 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_pars-resamplespecstep_0002.rmap 709 bytes (17 / 204 files) (77.8 K / 741.0 K bytes)
2025-05-15 19:16:47,172 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_pars-outlierdetectionstep_0005.rmap 1.1 K bytes (18 / 204 files) (78.5 K / 741.0 K bytes)
2025-05-15 19:16:47,249 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_pars-jumpstep_0005.rmap 810 bytes (19 / 204 files) (79.6 K / 741.0 K bytes)
2025-05-15 19:16:47,332 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_pars-image2pipeline_0008.rmap 1.0 K bytes (20 / 204 files) (80.4 K / 741.0 K bytes)
2025-05-15 19:16:47,415 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_pars-detector1pipeline_0003.rmap 1.1 K bytes (21 / 204 files) (81.4 K / 741.0 K bytes)
2025-05-15 19:16:47,583 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_pars-darkpipeline_0003.rmap 872 bytes (22 / 204 files) (82.5 K / 741.0 K bytes)
2025-05-15 19:16:47,683 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_pars-darkcurrentstep_0003.rmap 1.8 K bytes (23 / 204 files) (83.4 K / 741.0 K bytes)
2025-05-15 19:16:47,780 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_ote_0030.rmap 1.3 K bytes (24 / 204 files) (85.2 K / 741.0 K bytes)
2025-05-15 19:16:47,860 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_msaoper_0016.rmap 1.5 K bytes (25 / 204 files) (86.4 K / 741.0 K bytes)
2025-05-15 19:16:47,940 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_msa_0027.rmap 1.3 K bytes (26 / 204 files) (87.9 K / 741.0 K bytes)
2025-05-15 19:16:48,016 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_mask_0043.rmap 3.5 K bytes (27 / 204 files) (89.2 K / 741.0 K bytes)
2025-05-15 19:16:48,136 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_linearity_0017.rmap 1.6 K bytes (28 / 204 files) (92.7 K / 741.0 K bytes)
2025-05-15 19:16:48,244 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_ipc_0006.rmap 876 bytes (29 / 204 files) (94.3 K / 741.0 K bytes)
2025-05-15 19:16:48,321 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_ifuslicer_0017.rmap 1.5 K bytes (30 / 204 files) (95.2 K / 741.0 K bytes)
2025-05-15 19:16:48,446 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_ifupost_0019.rmap 1.5 K bytes (31 / 204 files) (96.7 K / 741.0 K bytes)
2025-05-15 19:16:48,535 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_ifufore_0017.rmap 1.5 K bytes (32 / 204 files) (98.2 K / 741.0 K bytes)
2025-05-15 19:16:48,617 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_gain_0023.rmap 1.8 K bytes (33 / 204 files) (99.7 K / 741.0 K bytes)
2025-05-15 19:16:48,700 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_fpa_0028.rmap 1.3 K bytes (34 / 204 files) (101.5 K / 741.0 K bytes)
2025-05-15 19:16:48,807 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_fore_0026.rmap 5.0 K bytes (35 / 204 files) (102.7 K / 741.0 K bytes)
2025-05-15 19:16:48,889 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_flat_0015.rmap 3.8 K bytes (36 / 204 files) (107.7 K / 741.0 K bytes)
2025-05-15 19:16:48,966 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_fflat_0026.rmap 7.2 K bytes (37 / 204 files) (111.5 K / 741.0 K bytes)
2025-05-15 19:16:49,048 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_extract1d_0018.rmap 2.3 K bytes (38 / 204 files) (118.7 K / 741.0 K bytes)
2025-05-15 19:16:49,135 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_disperser_0028.rmap 5.7 K bytes (39 / 204 files) (121.0 K / 741.0 K bytes)
2025-05-15 19:16:49,226 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_dflat_0007.rmap 1.1 K bytes (40 / 204 files) (126.7 K / 741.0 K bytes)
2025-05-15 19:16:49,343 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_dark_0074.rmap 34.2 K bytes (41 / 204 files) (127.9 K / 741.0 K bytes)
2025-05-15 19:16:49,430 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_cubepar_0015.rmap 966 bytes (42 / 204 files) (162.1 K / 741.0 K bytes)
2025-05-15 19:16:49,531 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_collimator_0026.rmap 1.3 K bytes (43 / 204 files) (163.1 K / 741.0 K bytes)
2025-05-15 19:16:49,610 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_camera_0026.rmap 1.3 K bytes (44 / 204 files) (164.4 K / 741.0 K bytes)
2025-05-15 19:16:49,692 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_barshadow_0007.rmap 1.8 K bytes (45 / 204 files) (165.7 K / 741.0 K bytes)
2025-05-15 19:16:49,777 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_area_0018.rmap 6.3 K bytes (46 / 204 files) (167.5 K / 741.0 K bytes)
2025-05-15 19:16:49,857 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_apcorr_0009.rmap 5.6 K bytes (47 / 204 files) (173.8 K / 741.0 K bytes)
2025-05-15 19:16:49,938 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nirspec_0398.imap 5.8 K bytes (48 / 204 files) (179.3 K / 741.0 K bytes)
2025-05-15 19:16:50,026 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_wfssbkg_0010.rmap 3.1 K bytes (49 / 204 files) (185.1 K / 741.0 K bytes)
2025-05-15 19:16:50,137 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_wavelengthrange_0006.rmap 862 bytes (50 / 204 files) (188.2 K / 741.0 K bytes)
2025-05-15 19:16:50,220 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_trappars_0004.rmap 753 bytes (51 / 204 files) (189.1 K / 741.0 K bytes)
2025-05-15 19:16:50,313 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_trapdensity_0005.rmap 705 bytes (52 / 204 files) (189.9 K / 741.0 K bytes)
2025-05-15 19:16:50,393 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_throughput_0005.rmap 1.3 K bytes (53 / 204 files) (190.6 K / 741.0 K bytes)
2025-05-15 19:16:50,476 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_superbias_0030.rmap 7.4 K bytes (54 / 204 files) (191.8 K / 741.0 K bytes)
2025-05-15 19:16:50,557 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_specwcs_0014.rmap 3.1 K bytes (55 / 204 files) (199.2 K / 741.0 K bytes)
2025-05-15 19:16:50,648 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_specprofile_0008.rmap 2.4 K bytes (56 / 204 files) (202.4 K / 741.0 K bytes)
2025-05-15 19:16:50,730 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_speckernel_0006.rmap 1.0 K bytes (57 / 204 files) (204.7 K / 741.0 K bytes)
2025-05-15 19:16:50,829 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_sirskernel_0001.rmap 627 bytes (58 / 204 files) (205.8 K / 741.0 K bytes)
2025-05-15 19:16:50,924 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_saturation_0015.rmap 829 bytes (59 / 204 files) (206.4 K / 741.0 K bytes)
2025-05-15 19:16:51,007 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_readnoise_0011.rmap 987 bytes (60 / 204 files) (207.2 K / 741.0 K bytes)
2025-05-15 19:16:51,085 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_photom_0036.rmap 1.3 K bytes (61 / 204 files) (208.2 K / 741.0 K bytes)
2025-05-15 19:16:51,170 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_persat_0007.rmap 674 bytes (62 / 204 files) (209.5 K / 741.0 K bytes)
2025-05-15 19:16:51,254 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_pathloss_0003.rmap 758 bytes (63 / 204 files) (210.1 K / 741.0 K bytes)
2025-05-15 19:16:51,338 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_pastasoss_0004.rmap 818 bytes (64 / 204 files) (210.9 K / 741.0 K bytes)
2025-05-15 19:16:51,420 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_pars-undersamplecorrectionstep_0001.rmap 904 bytes (65 / 204 files) (211.7 K / 741.0 K bytes)
2025-05-15 19:16:51,510 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_pars-tweakregstep_0012.rmap 3.1 K bytes (66 / 204 files) (212.6 K / 741.0 K bytes)
2025-05-15 19:16:51,616 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_pars-spec2pipeline_0008.rmap 984 bytes (67 / 204 files) (215.8 K / 741.0 K bytes)
2025-05-15 19:16:51,704 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_pars-sourcecatalogstep_0002.rmap 2.3 K bytes (68 / 204 files) (216.7 K / 741.0 K bytes)
2025-05-15 19:16:51,784 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_pars-resamplestep_0002.rmap 687 bytes (69 / 204 files) (219.1 K / 741.0 K bytes)
2025-05-15 19:16:51,869 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_pars-outlierdetectionstep_0004.rmap 2.7 K bytes (70 / 204 files) (219.7 K / 741.0 K bytes)
2025-05-15 19:16:51,960 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_pars-jumpstep_0007.rmap 6.4 K bytes (71 / 204 files) (222.4 K / 741.0 K bytes)
2025-05-15 19:16:52,039 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_pars-image2pipeline_0005.rmap 1.0 K bytes (72 / 204 files) (228.8 K / 741.0 K bytes)
2025-05-15 19:16:52,117 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_pars-detector1pipeline_0002.rmap 1.0 K bytes (73 / 204 files) (229.8 K / 741.0 K bytes)
2025-05-15 19:16:52,197 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_pars-darkpipeline_0002.rmap 868 bytes (74 / 204 files) (230.8 K / 741.0 K bytes)
2025-05-15 19:16:52,291 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_pars-darkcurrentstep_0001.rmap 591 bytes (75 / 204 files) (231.7 K / 741.0 K bytes)
2025-05-15 19:16:52,381 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_pars-chargemigrationstep_0004.rmap 5.7 K bytes (76 / 204 files) (232.3 K / 741.0 K bytes)
2025-05-15 19:16:52,462 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_nrm_0005.rmap 663 bytes (77 / 204 files) (237.9 K / 741.0 K bytes)
2025-05-15 19:16:52,541 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_mask_0022.rmap 1.3 K bytes (78 / 204 files) (238.6 K / 741.0 K bytes)
2025-05-15 19:16:52,617 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_linearity_0022.rmap 961 bytes (79 / 204 files) (239.9 K / 741.0 K bytes)
2025-05-15 19:16:52,695 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_ipc_0007.rmap 651 bytes (80 / 204 files) (240.9 K / 741.0 K bytes)
2025-05-15 19:16:52,798 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_gain_0011.rmap 797 bytes (81 / 204 files) (241.5 K / 741.0 K bytes)
2025-05-15 19:16:52,878 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_flat_0023.rmap 5.9 K bytes (82 / 204 files) (242.3 K / 741.0 K bytes)
2025-05-15 19:16:52,971 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_filteroffset_0010.rmap 853 bytes (83 / 204 files) (248.2 K / 741.0 K bytes)
2025-05-15 19:16:53,063 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_extract1d_0007.rmap 905 bytes (84 / 204 files) (249.0 K / 741.0 K bytes)
2025-05-15 19:16:53,153 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_drizpars_0004.rmap 519 bytes (85 / 204 files) (249.9 K / 741.0 K bytes)
2025-05-15 19:16:53,230 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_distortion_0025.rmap 3.4 K bytes (86 / 204 files) (250.4 K / 741.0 K bytes)
2025-05-15 19:16:53,314 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_dark_0034.rmap 7.5 K bytes (87 / 204 files) (253.9 K / 741.0 K bytes)
2025-05-15 19:16:53,397 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_bkg_0002.rmap 2.9 K bytes (88 / 204 files) (261.4 K / 741.0 K bytes)
2025-05-15 19:16:53,475 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_area_0014.rmap 2.7 K bytes (89 / 204 files) (264.3 K / 741.0 K bytes)
2025-05-15 19:16:53,559 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_apcorr_0010.rmap 4.3 K bytes (90 / 204 files) (267.0 K / 741.0 K bytes)
2025-05-15 19:16:53,639 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_abvegaoffset_0004.rmap 1.4 K bytes (91 / 204 files) (271.3 K / 741.0 K bytes)
2025-05-15 19:16:53,718 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_niriss_0272.imap 5.8 K bytes (92 / 204 files) (272.7 K / 741.0 K bytes)
2025-05-15 19:16:53,796 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_wfssbkg_0004.rmap 7.2 K bytes (93 / 204 files) (278.5 K / 741.0 K bytes)
2025-05-15 19:16:53,871 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_wavelengthrange_0010.rmap 996 bytes (94 / 204 files) (285.7 K / 741.0 K bytes)
2025-05-15 19:16:53,951 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_tsophot_0003.rmap 896 bytes (95 / 204 files) (286.7 K / 741.0 K bytes)
2025-05-15 19:16:54,044 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_trappars_0003.rmap 1.6 K bytes (96 / 204 files) (287.6 K / 741.0 K bytes)
2025-05-15 19:16:54,119 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_trapdensity_0003.rmap 1.6 K bytes (97 / 204 files) (289.2 K / 741.0 K bytes)
2025-05-15 19:16:54,197 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_superbias_0019.rmap 18.9 K bytes (98 / 204 files) (290.8 K / 741.0 K bytes)
2025-05-15 19:16:54,299 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_specwcs_0022.rmap 7.1 K bytes (99 / 204 files) (309.7 K / 741.0 K bytes)
2025-05-15 19:16:54,376 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_sirskernel_0002.rmap 671 bytes (100 / 204 files) (316.8 K / 741.0 K bytes)
2025-05-15 19:16:54,450 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_saturation_0011.rmap 2.8 K bytes (101 / 204 files) (317.5 K / 741.0 K bytes)
2025-05-15 19:16:54,532 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_readnoise_0026.rmap 25.9 K bytes (102 / 204 files) (320.3 K / 741.0 K bytes)
2025-05-15 19:16:54,634 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_psfmask_0008.rmap 28.4 K bytes (103 / 204 files) (346.2 K / 741.0 K bytes)
2025-05-15 19:16:54,737 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_photom_0028.rmap 3.4 K bytes (104 / 204 files) (374.6 K / 741.0 K bytes)
2025-05-15 19:16:54,818 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_persat_0005.rmap 1.6 K bytes (105 / 204 files) (377.9 K / 741.0 K bytes)
2025-05-15 19:16:54,904 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_pars-whitelightstep_0004.rmap 2.0 K bytes (106 / 204 files) (379.5 K / 741.0 K bytes)
2025-05-15 19:16:54,980 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_pars-tweakregstep_0003.rmap 4.5 K bytes (107 / 204 files) (381.5 K / 741.0 K bytes)
2025-05-15 19:16:55,056 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_pars-spec2pipeline_0008.rmap 984 bytes (108 / 204 files) (386.0 K / 741.0 K bytes)
2025-05-15 19:16:55,139 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_pars-sourcecatalogstep_0002.rmap 4.6 K bytes (109 / 204 files) (387.0 K / 741.0 K bytes)
2025-05-15 19:16:55,231 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_pars-resamplestep_0002.rmap 687 bytes (110 / 204 files) (391.6 K / 741.0 K bytes)
2025-05-15 19:16:55,309 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_pars-outlierdetectionstep_0003.rmap 940 bytes (111 / 204 files) (392.3 K / 741.0 K bytes)
2025-05-15 19:16:55,398 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_pars-jumpstep_0005.rmap 806 bytes (112 / 204 files) (393.2 K / 741.0 K bytes)
2025-05-15 19:16:55,473 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_pars-image2pipeline_0004.rmap 1.1 K bytes (113 / 204 files) (394.0 K / 741.0 K bytes)
2025-05-15 19:16:55,546 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_pars-detector1pipeline_0005.rmap 1.3 K bytes (114 / 204 files) (395.2 K / 741.0 K bytes)
2025-05-15 19:16:55,642 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_pars-darkpipeline_0002.rmap 868 bytes (115 / 204 files) (396.4 K / 741.0 K bytes)
2025-05-15 19:16:55,724 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_pars-darkcurrentstep_0001.rmap 618 bytes (116 / 204 files) (397.3 K / 741.0 K bytes)
2025-05-15 19:16:55,812 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_mask_0012.rmap 4.1 K bytes (117 / 204 files) (397.9 K / 741.0 K bytes)
2025-05-15 19:16:55,888 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_linearity_0011.rmap 2.4 K bytes (118 / 204 files) (402.1 K / 741.0 K bytes)
2025-05-15 19:16:55,968 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_ipc_0003.rmap 2.0 K bytes (119 / 204 files) (404.5 K / 741.0 K bytes)
2025-05-15 19:16:56,045 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_gain_0016.rmap 2.1 K bytes (120 / 204 files) (406.4 K / 741.0 K bytes)
2025-05-15 19:16:56,121 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_flat_0028.rmap 51.7 K bytes (121 / 204 files) (408.6 K / 741.0 K bytes)
2025-05-15 19:16:56,234 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_filteroffset_0004.rmap 1.4 K bytes (122 / 204 files) (460.2 K / 741.0 K bytes)
2025-05-15 19:16:56,310 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_extract1d_0005.rmap 1.2 K bytes (123 / 204 files) (461.7 K / 741.0 K bytes)
2025-05-15 19:16:56,387 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_drizpars_0001.rmap 519 bytes (124 / 204 files) (462.9 K / 741.0 K bytes)
2025-05-15 19:16:56,476 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_distortion_0033.rmap 53.4 K bytes (125 / 204 files) (463.4 K / 741.0 K bytes)
2025-05-15 19:16:56,574 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_dark_0047.rmap 29.0 K bytes (126 / 204 files) (516.7 K / 741.0 K bytes)
2025-05-15 19:16:56,672 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_area_0012.rmap 33.5 K bytes (127 / 204 files) (545.7 K / 741.0 K bytes)
2025-05-15 19:16:56,783 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_apcorr_0008.rmap 4.3 K bytes (128 / 204 files) (579.2 K / 741.0 K bytes)
2025-05-15 19:16:56,863 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_abvegaoffset_0003.rmap 1.3 K bytes (129 / 204 files) (583.5 K / 741.0 K bytes)
2025-05-15 19:16:56,954 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_nircam_0314.imap 5.6 K bytes (130 / 204 files) (584.8 K / 741.0 K bytes)
2025-05-15 19:16:57,043 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_wavelengthrange_0027.rmap 929 bytes (131 / 204 files) (590.4 K / 741.0 K bytes)
2025-05-15 19:16:57,119 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_tsophot_0004.rmap 882 bytes (132 / 204 files) (591.3 K / 741.0 K bytes)
2025-05-15 19:16:57,196 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_straymask_0009.rmap 987 bytes (133 / 204 files) (592.2 K / 741.0 K bytes)
2025-05-15 19:16:57,270 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_specwcs_0043.rmap 5.8 K bytes (134 / 204 files) (593.2 K / 741.0 K bytes)
2025-05-15 19:16:57,345 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_saturation_0015.rmap 1.2 K bytes (135 / 204 files) (599.0 K / 741.0 K bytes)
2025-05-15 19:16:57,423 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_rscd_0008.rmap 1.0 K bytes (136 / 204 files) (600.1 K / 741.0 K bytes)
2025-05-15 19:16:57,507 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_resol_0006.rmap 790 bytes (137 / 204 files) (601.2 K / 741.0 K bytes)
2025-05-15 19:16:57,584 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_reset_0026.rmap 3.9 K bytes (138 / 204 files) (602.0 K / 741.0 K bytes)
2025-05-15 19:16:57,660 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_regions_0034.rmap 5.2 K bytes (139 / 204 files) (605.8 K / 741.0 K bytes)
2025-05-15 19:16:57,740 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_readnoise_0023.rmap 1.6 K bytes (140 / 204 files) (611.0 K / 741.0 K bytes)
2025-05-15 19:16:57,820 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_psfmask_0009.rmap 2.1 K bytes (141 / 204 files) (612.7 K / 741.0 K bytes)
2025-05-15 19:16:57,921 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_psf_0003.rmap 839 bytes (142 / 204 files) (614.8 K / 741.0 K bytes)
2025-05-15 19:16:57,997 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_photom_0056.rmap 3.7 K bytes (143 / 204 files) (615.6 K / 741.0 K bytes)
2025-05-15 19:16:58,094 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_pathloss_0005.rmap 866 bytes (144 / 204 files) (619.4 K / 741.0 K bytes)
2025-05-15 19:16:58,178 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_pars-whitelightstep_0003.rmap 912 bytes (145 / 204 files) (620.2 K / 741.0 K bytes)
2025-05-15 19:16:58,258 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_pars-tweakregstep_0003.rmap 1.8 K bytes (146 / 204 files) (621.2 K / 741.0 K bytes)
2025-05-15 19:16:58,337 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_pars-spec3pipeline_0009.rmap 816 bytes (147 / 204 files) (623.0 K / 741.0 K bytes)
2025-05-15 19:16:58,416 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_pars-spec2pipeline_0012.rmap 1.3 K bytes (148 / 204 files) (623.8 K / 741.0 K bytes)
2025-05-15 19:16:58,519 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_pars-sourcecatalogstep_0003.rmap 1.9 K bytes (149 / 204 files) (625.1 K / 741.0 K bytes)
2025-05-15 19:16:58,611 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_pars-resamplestep_0002.rmap 677 bytes (150 / 204 files) (627.0 K / 741.0 K bytes)
2025-05-15 19:16:58,692 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_pars-resamplespecstep_0002.rmap 706 bytes (151 / 204 files) (627.7 K / 741.0 K bytes)
2025-05-15 19:16:58,792 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_pars-outlierdetectionstep_0020.rmap 3.4 K bytes (152 / 204 files) (628.4 K / 741.0 K bytes)
2025-05-15 19:16:58,885 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_pars-jumpstep_0011.rmap 1.6 K bytes (153 / 204 files) (631.8 K / 741.0 K bytes)
2025-05-15 19:16:58,975 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_pars-image2pipeline_0010.rmap 1.1 K bytes (154 / 204 files) (633.4 K / 741.0 K bytes)
2025-05-15 19:16:59,052 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_pars-extract1dstep_0003.rmap 807 bytes (155 / 204 files) (634.5 K / 741.0 K bytes)
2025-05-15 19:16:59,128 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_pars-emicorrstep_0003.rmap 796 bytes (156 / 204 files) (635.3 K / 741.0 K bytes)
2025-05-15 19:16:59,220 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_pars-detector1pipeline_0010.rmap 1.6 K bytes (157 / 204 files) (636.1 K / 741.0 K bytes)
2025-05-15 19:16:59,298 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_pars-darkpipeline_0002.rmap 860 bytes (158 / 204 files) (637.7 K / 741.0 K bytes)
2025-05-15 19:16:59,376 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_pars-darkcurrentstep_0002.rmap 683 bytes (159 / 204 files) (638.5 K / 741.0 K bytes)
2025-05-15 19:16:59,454 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_mrsxartcorr_0002.rmap 2.2 K bytes (160 / 204 files) (639.2 K / 741.0 K bytes)
2025-05-15 19:16:59,530 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_mrsptcorr_0005.rmap 2.0 K bytes (161 / 204 files) (641.4 K / 741.0 K bytes)
2025-05-15 19:16:59,606 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_mask_0026.rmap 4.3 K bytes (162 / 204 files) (643.3 K / 741.0 K bytes)
2025-05-15 19:16:59,683 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_linearity_0018.rmap 2.8 K bytes (163 / 204 files) (647.6 K / 741.0 K bytes)
2025-05-15 19:16:59,763 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_ipc_0008.rmap 700 bytes (164 / 204 files) (650.4 K / 741.0 K bytes)
2025-05-15 19:16:59,841 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_gain_0013.rmap 3.9 K bytes (165 / 204 files) (651.1 K / 741.0 K bytes)
2025-05-15 19:16:59,926 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_fringefreq_0003.rmap 1.4 K bytes (166 / 204 files) (655.0 K / 741.0 K bytes)
2025-05-15 19:17:00,001 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_fringe_0019.rmap 3.9 K bytes (167 / 204 files) (656.5 K / 741.0 K bytes)
2025-05-15 19:17:00,077 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_flat_0066.rmap 15.7 K bytes (168 / 204 files) (660.4 K / 741.0 K bytes)
2025-05-15 19:17:00,178 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_filteroffset_0025.rmap 2.5 K bytes (169 / 204 files) (676.1 K / 741.0 K bytes)
2025-05-15 19:17:00,267 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_extract1d_0020.rmap 1.4 K bytes (170 / 204 files) (678.6 K / 741.0 K bytes)
2025-05-15 19:17:00,354 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_emicorr_0003.rmap 663 bytes (171 / 204 files) (679.9 K / 741.0 K bytes)
2025-05-15 19:17:00,447 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_drizpars_0002.rmap 511 bytes (172 / 204 files) (680.6 K / 741.0 K bytes)
2025-05-15 19:17:00,522 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_distortion_0040.rmap 4.9 K bytes (173 / 204 files) (681.1 K / 741.0 K bytes)
2025-05-15 19:17:00,609 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_dark_0036.rmap 4.4 K bytes (174 / 204 files) (686.0 K / 741.0 K bytes)
2025-05-15 19:17:00,694 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_cubepar_0017.rmap 800 bytes (175 / 204 files) (690.4 K / 741.0 K bytes)
2025-05-15 19:17:00,769 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_area_0015.rmap 866 bytes (176 / 204 files) (691.2 K / 741.0 K bytes)
2025-05-15 19:17:00,864 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_apcorr_0019.rmap 5.0 K bytes (177 / 204 files) (692.0 K / 741.0 K bytes)
2025-05-15 19:17:00,952 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_abvegaoffset_0003.rmap 1.3 K bytes (178 / 204 files) (697.0 K / 741.0 K bytes)
2025-05-15 19:17:01,028 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_miri_0437.imap 5.8 K bytes (179 / 204 files) (698.3 K / 741.0 K bytes)
2025-05-15 19:17:01,104 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_fgs_trappars_0004.rmap 903 bytes (180 / 204 files) (704.1 K / 741.0 K bytes)
2025-05-15 19:17:01,181 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_fgs_trapdensity_0006.rmap 930 bytes (181 / 204 files) (705.0 K / 741.0 K bytes)
2025-05-15 19:17:01,263 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_fgs_superbias_0017.rmap 3.8 K bytes (182 / 204 files) (706.0 K / 741.0 K bytes)
2025-05-15 19:17:01,341 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_fgs_saturation_0009.rmap 779 bytes (183 / 204 files) (709.7 K / 741.0 K bytes)
2025-05-15 19:17:01,420 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_fgs_readnoise_0014.rmap 1.3 K bytes (184 / 204 files) (710.5 K / 741.0 K bytes)
2025-05-15 19:17:01,511 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_fgs_photom_0014.rmap 1.1 K bytes (185 / 204 files) (711.8 K / 741.0 K bytes)
2025-05-15 19:17:01,587 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_fgs_persat_0006.rmap 884 bytes (186 / 204 files) (712.9 K / 741.0 K bytes)
2025-05-15 19:17:01,672 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_fgs_pars-tweakregstep_0002.rmap 850 bytes (187 / 204 files) (713.8 K / 741.0 K bytes)
2025-05-15 19:17:01,755 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_fgs_pars-sourcecatalogstep_0001.rmap 636 bytes (188 / 204 files) (714.6 K / 741.0 K bytes)
2025-05-15 19:17:01,834 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_fgs_pars-outlierdetectionstep_0001.rmap 654 bytes (189 / 204 files) (715.3 K / 741.0 K bytes)
2025-05-15 19:17:01,915 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_fgs_pars-image2pipeline_0005.rmap 974 bytes (190 / 204 files) (715.9 K / 741.0 K bytes)
2025-05-15 19:17:01,993 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_fgs_pars-detector1pipeline_0002.rmap 1.0 K bytes (191 / 204 files) (716.9 K / 741.0 K bytes)
2025-05-15 19:17:02,078 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_fgs_pars-darkpipeline_0002.rmap 856 bytes (192 / 204 files) (717.9 K / 741.0 K bytes)
2025-05-15 19:17:02,166 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_fgs_mask_0023.rmap 1.1 K bytes (193 / 204 files) (718.8 K / 741.0 K bytes)
2025-05-15 19:17:02,244 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_fgs_linearity_0015.rmap 925 bytes (194 / 204 files) (719.8 K / 741.0 K bytes)
2025-05-15 19:17:02,330 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_fgs_ipc_0003.rmap 614 bytes (195 / 204 files) (720.8 K / 741.0 K bytes)
2025-05-15 19:17:02,428 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_fgs_gain_0010.rmap 890 bytes (196 / 204 files) (721.4 K / 741.0 K bytes)
2025-05-15 19:17:02,517 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_fgs_flat_0009.rmap 1.1 K bytes (197 / 204 files) (722.3 K / 741.0 K bytes)
2025-05-15 19:17:02,606 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_fgs_distortion_0011.rmap 1.2 K bytes (198 / 204 files) (723.4 K / 741.0 K bytes)
2025-05-15 19:17:02,681 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_fgs_dark_0017.rmap 4.3 K bytes (199 / 204 files) (724.6 K / 741.0 K bytes)
2025-05-15 19:17:02,757 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_fgs_area_0010.rmap 1.2 K bytes (200 / 204 files) (728.9 K / 741.0 K bytes)
2025-05-15 19:17:02,832 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_fgs_apcorr_0004.rmap 4.0 K bytes (201 / 204 files) (730.1 K / 741.0 K bytes)
2025-05-15 19:17:02,920 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_fgs_abvegaoffset_0002.rmap 1.3 K bytes (202 / 204 files) (734.0 K / 741.0 K bytes)
2025-05-15 19:17:03,001 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_fgs_0123.imap 5.1 K bytes (203 / 204 files) (735.3 K / 741.0 K bytes)
2025-05-15 19:17:03,089 - CRDS - INFO - Fetching crds_cache/mappings/jwst/jwst_1364.pmap 580 bytes (204 / 204 files) (740.4 K / 741.0 K bytes)
2025-05-15 19:17:03,419 - stpipe.AssignWcsStep - INFO - AssignWcsStep instance created.
2025-05-15 19:17:03,513 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep running with args ('data/jw02079004002_02101_00001_nis_rate.fits',).
2025-05-15 19:17:03,516 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep parameters are:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: data/calibrated_steps/
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: True
skip: False
suffix: None
search_output_file: True
input_dir: ''
sip_approx: True
sip_max_pix_error: 0.01
sip_degree: None
sip_max_inv_pix_error: 0.01
sip_inv_degree: None
sip_npoints: 12
slit_y_low: -0.55
slit_y_high: 0.55
2025-05-15 19:17:03,579 - CRDS - INFO - Fetching crds_cache/references/jwst/niriss/jwst_niriss_distortion_0036.asdf 9.9 K bytes (1 / 1 files) (0 / 9.9 K bytes)
2025-05-15 19:17:03,666 - CRDS - INFO - Fetching crds_cache/references/jwst/niriss/jwst_niriss_filteroffset_0006.asdf 5.4 K bytes (1 / 1 files) (0 / 5.4 K bytes)
2025-05-15 19:17:03,899 - stpipe.AssignWcsStep - INFO - Offsets from filteroffset file are 0.0397, -0.0651
2025-05-15 19:17:03,962 - stpipe.AssignWcsStep - INFO - Update S_REGION to POLYGON ICRS 53.148020701 -27.806991890 53.187895566 -27.794536632 53.173722708 -27.759581559 53.133883845 -27.772015679
2025-05-15 19:17:03,963 - stpipe.AssignWcsStep - INFO - assign_wcs updated S_REGION to POLYGON ICRS 53.148020701 -27.806991890 53.187895566 -27.794536632 53.173722708 -27.759581559 53.133883845 -27.772015679
2025-05-15 19:17:03,964 - stpipe.AssignWcsStep - INFO - COMPLETED assign_wcs
2025-05-15 19:17:04,022 - CRDS - INFO - Calibration SW Found: jwst 1.18.0 (/opt/hostedtoolcache/Python/3.11.12/x64/lib/python3.11/site-packages/jwst-1.18.0.dist-info)
2025-05-15 19:17:04,108 - stpipe.AssignWcsStep - INFO - Results used CRDS context: jwst_1364.pmap
2025-05-15 19:17:04,270 - stpipe.AssignWcsStep - INFO - Saved model in data/calibrated_steps/jw02079004002_02101_00001_nis_assignwcsstep.fits
2025-05-15 19:17:04,270 - stpipe.AssignWcsStep - INFO - Step AssignWcsStep done
2025-05-15 19:17:04,271 - stpipe - INFO - Results used jwst version: 1.18.0
# A quick sanity check to ensure that the files were calibrated.
# if this is zero, check the log message above for any errors that may have occurred during the calibration
wcsstep_files = glob.glob(os.path.join(data_dir_out, '*assignwcsstep*'))
print(len(wcsstep_files), 'assignwcsstep files written') # 1 file should have been written
1 assignwcsstep files written
Flat Field Step#
After the assignwcs file is run, we then want to run the flat_field step of the pipeline which removes detector artifacts using the flat field reference files. To read more about the step, visit the FlatField description page.
# Run flat_field
for wcsfile in wcsstep_files:
result = FlatFieldStep.call(wcsfile, output_dir=data_dir_out, save_results=True)
2025-05-15 19:17:04,410 - stpipe.FlatFieldStep - INFO - FlatFieldStep instance created.
2025-05-15 19:17:04,525 - stpipe.FlatFieldStep - INFO - Step FlatFieldStep running with args ('data/calibrated_steps/jw02079004002_02101_00001_nis_assignwcsstep.fits',).
2025-05-15 19:17:04,527 - stpipe.FlatFieldStep - INFO - Step FlatFieldStep parameters are:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: data/calibrated_steps/
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: True
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_interpolated_flat: False
user_supplied_flat: None
inverse: False
2025-05-15 19:17:04,649 - CRDS - INFO - Fetching crds_cache/references/jwst/niriss/jwst_niriss_flat_0261.fits 67.1 M bytes (1 / 1 files) (0 / 67.1 M bytes)
2025-05-15 19:17:05,469 - stpipe.FlatFieldStep - INFO - Using FLAT reference file: crds_cache/references/jwst/niriss/jwst_niriss_flat_0261.fits
2025-05-15 19:17:05,470 - stpipe.FlatFieldStep - INFO - No reference found for type FFLAT
2025-05-15 19:17:05,471 - stpipe.FlatFieldStep - INFO - No reference found for type SFLAT
2025-05-15 19:17:05,471 - stpipe.FlatFieldStep - INFO - No reference found for type DFLAT
2025-05-15 19:17:05,649 - stpipe.FlatFieldStep - INFO - Results used CRDS context: jwst_1364.pmap
2025-05-15 19:17:05,818 - stpipe.FlatFieldStep - INFO - Saved model in data/calibrated_steps/jw02079004002_02101_00001_nis_flatfieldstep.fits
2025-05-15 19:17:05,819 - stpipe.FlatFieldStep - INFO - Step FlatFieldStep done
2025-05-15 19:17:05,819 - stpipe - INFO - Results used jwst version: 1.18.0
# A quick sanity check to ensure that the files were calibrated.
# if this is zero, check the log message above for any errors that may have occurred during the calibration
flatfield_files = glob.glob(os.path.join(data_dir_out, '*flatfieldstep*'))
print(len(flatfield_files), 'flatfieldstep files written') # 1 file should have been written (matching the wcs step)
1 flatfieldstep files written
Compare Rate vs. Flat fielded Data#
Running the cell below shows the same direct image from just the rate file versus when the flat_field step of the pipeline is run. Some detector artifacts are noticably gone such as the cross-hatching in the bottom right and middle of the detector.
There are remaining optical artifacts to be aware of on the NIRISS known issues page such as the 1/f noise.
test_rate_file = ratefile # look at a direct image for this comparision
test_flat_file = os.path.join(data_dir_out, os.path.basename(test_rate_file).replace('rate.fits', 'flatfieldstep.fits'))
plot_files = [test_rate_file, test_flat_file]
plot_titles = ['Rate File', 'Flat Corrected File']
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(15, 15))
fig.suptitle(f"{os.path.basename(test_rate_file).split('_rate')[0]}\n{fits.getval(test_rate_file, 'PUPIL')}", x=0.5, y=0.72)
for filename, title, ax in zip(plot_files, plot_titles, [ax1, ax2]):
with fits.open(filename) as hdu:
# fill in the nan values from the bad pixels with zero; otherwise a single, non-dithered image is impossible to really see
data = hdu[1].data
data[np.isnan(data)] = 0
ax.imshow(data, vmin=0.2, vmax=1.2, origin='lower')
ax.set_title(title)
