NIRCam Grism Time Series Pipeline Notebook#
Authors: Brian Brooks, Bryan Hilbert, Achrène Dyrek based on the work by Nikolay Nikolov
Last Update: April 15, 2026
Pipeline Version: 2.0.0 (Build 12.3)
Purpose:
This notebook provides a framework for processing Near-Infrared
Camera (NIRCam) Grism Time Series data through all three James Webb Space Telescope
(JWST) pipeline stages. Data is assumed to be located in a folder structure
following the paths set up below. It should not be necessary to edit
any cells other than in the Configuration section unless
modifying the standard pipeline processing options.
Data:
The example data for this notebook is a transit of WASP-39b, observed using grism time-series observing mode with the F322W2 filter. The data set belongs to the JWST Early Release Science Program ID 1366 (PI: Annabella Meech), observation 2, and includes 366 integrations of 12 groups each with the SUBGRISM256 subarray, covering a total of 10.45 hours. The data set consists of 4 segments of uncal files, each with a size of 1.25 GB. Example input data to use will be downloaded automatically unless disabled (i.e., to use local files instead).
JWST pipeline version and CRDS context:
This notebook was written for the above-specified pipeline version and associated build context for this version of the JWST Calibration Pipeline. Information about this and other contexts can be found in the JWST Calibration Reference Data System (CRDS server). If you use different pipeline versions, please refer to the table here to determine what context to use. To learn more about the differences for the pipeline, read the relevant documentation.
Please note that pipeline software development is a continuous process, so results in some cases may be slightly different if a subsequent version is used. For optimal results, users are strongly encouraged to reprocess their data using the most recent pipeline version and associated CRDS context, taking advantage of bug fixes and algorithm improvements. Any known issues for this build are noted in the notebook.
Updates:
This notebook is regularly updated as improvements are made to the
pipeline. Find the most up to date version of this notebook at:
https://github.com/spacetelescope/jwst-pipeline-notebooks/
Recent Changes:
December 10, 2025: Updated notebook to be compatible with JWST pipeline version 1.20.2
March 10, 2026: Updated to fix broken links and to be compatible with segemented uncal file changes
April 15, 2026: Updated to be compatible with JWST pipeline version 2.0.0
Table of Contents#
1. Configuration#
1.1 Install dependencies#
To make sure that the pipeline version is compatabile with this notebook and the required dependencies and taht packages are installed,
it is recommended that users create a new dedicated conda environment and install the provided
requirements.txt file before starting this notebook:
conda create -n nircam_gts_pipeline python=3.13
conda activate nircam_gts_pipeline
pip install -r requirements.txt
Note that
demo_mode must be set appropriately below.
Set the basic parameters to use with this notebook. These parameters will affect what data is used, where data is located (if already on disk), and which pipeline modules will run on this data. The list of parameters are:
demo_mode:True: To run in demonstration mode. In this mode this notebook will download example data from the Barbara A. Mikulski Archive for Space Telescopes (MAST) and process it through the pipeline. This will all happen in a local directory unless modified in Section 3 below.False: If you want to process your own data that has already been downloaded and provide the location of the data.
Directories with data:
sci_dir: Directory where science observation data are stored.
pipeline modules:
do_det1 = Trueruns the Detector1 module on the selected data.do_spec2 = Trueruns calwebb_spec2 module on the selected data.do_tso3 = Trueruns calwebb_tso3 module on the selected data.do_viz = TrueCreates plots to visualize calwebb_tso3 results.
# Basic import necessary for configuration
import os
# Set parameters for demo_mode, data directory, and processing steps
# -----------------------------Demo Mode---------------------------------
demo_mode = True
if demo_mode:
print('Running in demonstration mode using online example data!')
# --------------------------User Mode Directories------------------------
# If demo_mode = False, look for user data in these paths
if not demo_mode:
# Set directory paths for processing specific data; these will need
# to be changed to the user's local directory setup (paths below are given as
# examples).
basedir = os.path.join(os.getcwd(), '')
# Point to where the observation data are stored.
# Assumes uncalibrated data in sci_dir/uncal/ with results stored in stage1, stage2, stage3 directories.
sci_dir = os.path.join(basedir, 'PID01366/Obs002/')
# --------------------------Set Processing Steps--------------------------
# Individual pipeline stages can be turned on/off here. Note that a later
# stage won't be able to run unless data products have already been
# produced from the prior stage.
# Pipeline processing
do_det1 = True # calwebb_detector1
do_spec2 = True # calwebb_spec2
do_tso3 = True # calwebb_tso3
do_viz = True # Visualize calwebb_tso3 results
Running in demonstration mode using online example data!
1.2 Set CRDS context and server#
Before importing CRDS and JWST modules, we need
to configure our environment. This includes defining a CRDS cache
directory in which to keep the reference files that will be used by the
calibration pipeline.
If the local CRDS cache directory has not been set, it will automatically be created in the home directory.
The Build Context Table lists the CRDS context file that is associated with each pipeline release version.
# ------------------------Set CRDS context and paths----------------------
# Each version of the calibration pipeline is associated with a specific CRDS
# context file. The pipeline will select the appropriate context file behind
# the scenes while running. However, if you wish to override the default context
# file and run the pipeline with a different context, you can set that using
# the CRDS_CONTEXT environment variable. Here we show how this is done,
# although we leave the line commented out in order to use the default context.
# If you wish to specify a different context, uncomment the line below.
#%env CRDS_CONTEXT jwst_1535.pmap
#os.environ['CRDS_CONTEXT'] = 'jwst_1535.pmap' # CRDS context for pipeline version 2.0.0
# Check whether the local CRDS cache directory has been set.
# If not, set it to the user home directory
if (os.getenv('CRDS_PATH') is None):
os.environ['CRDS_PATH'] = os.path.join(os.path.expanduser('~'), 'crds')
# Check whether the CRDS server URL has been set. If not, set it.
if (os.getenv('CRDS_SERVER_URL') is None):
os.environ['CRDS_SERVER_URL'] = 'https://jwst-crds.stsci.edu'
# Echo CRDS path in use
print(f"CRDS local filepath: {os.environ['CRDS_PATH']}")
print(f"CRDS file server: {os.environ['CRDS_SERVER_URL']}")
if os.getenv('CRDS_CONTEXT'):
print(f"CRDS CONTEXT: {os.environ['CRDS_CONTEXT']}")
CRDS local filepath: /home/runner/crds
CRDS file server: https://jwst-crds.stsci.edu
2. Package Imports#
# Use the entire available screen width for this notebook
from IPython.display import display, HTML, JSON
display(HTML("<style>.container { width:95% !important; }</style>"))
# Basic system utilities for interacting with files
# ----------------------General Imports------------------------------------
import glob
import time
import json
from pathlib import Path
# Numpy for doing calculations
import numpy as np
# To display full output of cell, not just the last result
from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all"
# -----------------------Astropy Imports-----------------------------------
# Astropy utilities for opening FITS and downloading demo files, etc.
from astropy.io import fits
from astroquery.mast import Observations
from astropy.table import Table
# ----------------------Plotting Imports---------------------
import matplotlib.pyplot as plt
from astropy.stats import sigma_clip
from astropy.visualization import ImageNormalize
from astropy.visualization import AsinhStretch
import pandas as pd
Installation instructions for the JWST pipeline found here: JDox • ReadtheDocs • Github
# --------------JWST Calibration Pipeline Imports---------------------------
# Import the base JWST and calibration reference data packages
import jwst
import crds
# JWST pipelines (each encompassing many steps)
from jwst.pipeline import Detector1Pipeline
from jwst.pipeline import Spec2Pipeline
from jwst.pipeline import Tso3Pipeline
# JWST pipeline utilities
from jwst import datamodels
from jwst.associations import asn_from_list as afl # Tools for creating association files
from jwst.associations.lib.rules_level3_base import DMS_Level3_Base # Definition of a Lvl3 association file
# Print out pipeline version and CRDS context that will be used
print("JWST Calibration Pipeline Version = {}".format(jwst.__version__))
print("Using CRDS Context = {}".format(crds.get_context_name('jwst')))
JWST Calibration Pipeline Version = 2.0.0
CRDS - INFO - Calibration SW Found: jwst 2.0.0 (/home/runner/micromamba/envs/ci-env/lib/python3.11/site-packages/jwst-2.0.0.dist-info)
Using CRDS Context = jwst_1535.pmap
# Start a timer to keep track of runtime
time0 = time.perf_counter()
3. Demo Mode Setup (ignore if not using demo data)#
If running in demonstration mode, set up the program information to
retrieve the uncalibrated data automatically from MAST using
astroquery.
MAST allows for flexibility of searching by the proposal ID and the
observation ID instead of just filenames.
The JWST file naming conventions page provides information about the JWST file naming conventions and can help to decode filenames output from various pipelines and steps.
3.1 Create directory strucutre#
# Set up the program information and paths for demo program
if demo_mode:
print('Running in demonstration mode and will download example data from MAST!')
program = "01366"
sci_obs = "002"
# NOTE:
# The data in this notebook is public and does not require a token.
# For other data sets, you may need to provide a token.
# Observations.login(token=None)
data_dir = os.path.join('.', 'nrc_tso_demo_data')
download_dir = os.path.join(data_dir, program)
sci_dir = os.path.join(download_dir, 'Obs' + sci_obs)
uncal_dir = os.path.join(sci_dir, 'uncal')
# Ensure filepaths for input data exist
if not os.path.exists(uncal_dir):
os.makedirs(uncal_dir, exist_ok=True)
# Create directory if it does not exist
if not os.path.isdir(data_dir):
os.mkdir(data_dir, exist_ok=True)
Running in demonstration mode and will download example data from MAST!
3.2 Query MAST for data#
Compile tables of files from MAST associated with the science (SCI) observations.
# Obtain a list of observation IDs for the specified demo program
if demo_mode:
sci_obs_id_table = Observations.query_criteria(instrument_name=["NIRCAM/*"],
provenance_name=["CALJWST"], # Executed observations
obs_id=['jw' + program + '-o' + sci_obs + '*']
)
if demo_mode:
sci_obs_id_table
Table masked=True length=1
| intentType | obs_collection | provenance_name | instrument_name | project | filters | wave_region | target_name | target_classification | obs_id | s_ra | s_dec | dataproduct_type | proposal_pi | calib_level | t_min | t_max | t_exptime | wavelength_region | em_min | em_max | obs_title | t_obs_release | proposal_id | proposal_type | sequence_number | s_region | jpegURL | dataURL | dataRights | mtFlag | srcDen | obsid | objID | wave_min | wave_max |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| str7 | str4 | str7 | str12 | str4 | str13 | str8 | str7 | str23 | str50 | float64 | float64 | str10 | str16 | int64 | float64 | float64 | float64 | str8 | float64 | float64 | str64 | float64 | str4 | str3 | int64 | str132 | str1 | str81 | str6 | bool | float64 | str8 | str10 | float64 | float64 |
| science | JWST | CALJWST | NIRCAM/GRISM | JWST | F322W2;GRISMR | INFRARED | WASP-39 | Star; Exoplanet Systems | jw01366-o002_t001_nircam_f322w2-grismr-subgrism256 | 217.32661050118682 | -3.44449932372744 | timeseries | Meech, Annabella | 3 | 59782.783412233795 | 59783.15175280093 | 29080.424 | INFRARED | 2430.0 | 4013.0000000000005 | The Transiting Exoplanet Community Early Release Science Program | 59783.35921298 | 1366 | ERS | -- | POLYGON 217.35463336 -3.44487113 217.31827043 -3.44512151 217.31832056 -3.44053818 217.35454974 -3.44034058 217.35463336 -3.44487113 | -- | mast:JWST/product/jw01366-o002_t001_nircam_f322w2-grismr-subgrism256_x1dints.fits | PUBLIC | False | nan | 87682152 | 1083724673 | 2430.0 | 4013.0000000000005 |
NIRCam Grism Time Series mode allows simultaneous imaging with the weak lens (WL), or spectroscopy via the Dispersed Hartmann Sensor (DHS) in the short-wavelength (SW) channel, along with spectroscopy using the grism in the long-wavelength (LW) channel. The process of Target Acquisition (TA) also produces an imaging mode file using the TA subarray. In this notebook, we will focus only on the LW grism data which uses the Spec2Pipeline. Further, the example data in this notebook are from a program where the LW data were obtained in concert with weak lens data (imaging Time Series mode) in the SW channel. Note that LW Grism Time Series data obtained in concert with DHS observations in the SW channel have a different file format, and will be covered in a separate pipeline notebook. SW time series data, including observations using the weak lens and the DHS, will also be covered in separate notebooks.
Since Time Series Observations often result in a large amount of data, the data is stored as segemented products. To mitigate download crashes and processing issues, it is recommended to download and work with one NIRCam channel (short or long) at a time.
# Turn the list of visits into a list of uncalibrated data files
if demo_mode:
# Define types of files to select
file_dict = {'uncal': {'product_type': 'SCIENCE',
'productSubGroupDescription': 'UNCAL',
'calib_level': [1]}}
# Science files
sci_files_to_download = []
# Loop over visits identifying uncalibrated files that are associated with them
for exposure in (sci_obs_id_table):
products = Observations.get_product_list(exposure)
for filetype, query_dict in file_dict.items():
filtered_products = Observations.filter_products(products, productType=query_dict['product_type'],
productSubGroupDescription=query_dict['productSubGroupDescription'],
calib_level=query_dict['calib_level'])
sci_files_to_download.extend(filtered_products['dataURI'])
# To limit data volume, we will only download LW Grism data
# lw detector
sci_files_to_download_lw = np.unique([i for i in sci_files_to_download if 'nrcalong' in i])
# sw detectors
#detectors = ['nrca1', 'nrca3']
#sci_files_to_download_sw = np.unique([i for i in sci_files_to_download if any(d in i for d in detectors)])
# All data
#sci_files_to_download = sorted(sci_files_to_download)
print(f"Long-wave science files selected for downloading: {len(sci_files_to_download_lw)}")
#print(f"Short-wave science files selected for downloading: {len(sci_files_to_download_sw)}")
#print(f"Science files selected for downloading: {len(sci_files_to_download)}")
Long-wave science files selected for downloading: 5
3.3 Download the data#
Warning: If this notebook is halted during this step, the downloaded file may be incomplete and corrupted, and cause crashes later on!
if demo_mode:
for filename in sci_files_to_download_lw:
obs_manifest = Observations.download_file(filename, local_path=os.path.join(uncal_dir, Path(filename).name))
# Example that would download the SW files
#if demo_mode:
# for filename in sci_files_to_download_sw:
# obs_manifest = Observations.download_file(filename, local_path=os.path.join(uncal_dir, Path(filename).name))
# Example that would download both LW and SW files
#if demo_mode:
# for filename in sci_files_to_download:
# obs_manifest = Observations.download_file(filename, local_path=os.path.join(uncal_dir, Path(filename).name))
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw01366002001_02102_00001-seg001_nrcalong_uncal.fits to ./nrc_tso_demo_data/01366/Obs002/uncal/jw01366002001_02102_00001-seg001_nrcalong_uncal.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw01366002001_04103_00001-seg001_nrcalong_uncal.fits to ./nrc_tso_demo_data/01366/Obs002/uncal/jw01366002001_04103_00001-seg001_nrcalong_uncal.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw01366002001_04103_00001-seg002_nrcalong_uncal.fits to ./nrc_tso_demo_data/01366/Obs002/uncal/jw01366002001_04103_00001-seg002_nrcalong_uncal.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw01366002001_04103_00001-seg003_nrcalong_uncal.fits to ./nrc_tso_demo_data/01366/Obs002/uncal/jw01366002001_04103_00001-seg003_nrcalong_uncal.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw01366002001_04103_00001-seg004_nrcalong_uncal.fits to ./nrc_tso_demo_data/01366/Obs002/uncal/jw01366002001_04103_00001-seg004_nrcalong_uncal.fits ...
[Done]
4. Directory Setup#
Set up detailed paths to input/output stages here. When running this notebook outside of demo mode, the uncalibrated pipeline input files must be placed into the appropriate directories before proceeding to the JWST pipeline processing.
# Define output subdirectories to keep science data products organized
uncal_dir = os.path.join(sci_dir, 'uncal') # Uncalibrated pipeline inputs should be here
det1_dir = os.path.join(sci_dir, 'stage1') # calwebb_detector1 pipeline outputs will go here
spec2_dir = os.path.join(sci_dir, 'stage2') # calwebb_spec2 pipeline outputs will go here
tso3_dir = os.path.join(sci_dir, 'stage3') # calwebb_tso3 pipeline outputs will go here
# Create desired output directories, if needed
os.makedirs(det1_dir, exist_ok=True)
os.makedirs(spec2_dir, exist_ok=True)
os.makedirs(tso3_dir, exist_ok=True)
# List uncal files
uncal_files = sorted(glob.glob(os.path.join(uncal_dir, '*_uncal.fits')))
# Separate TA file
lw_ta_file = []
sw_uncal_files = []
lw_uncal_files = []
for unc_fi in uncal_files:
fi = datamodels.open(unc_fi)
# SW files are not downloaded by default, but we leave
# SW-related lines here for cases where the user provides
# their own data.
if fi.meta.exposure.type == 'NRC_TSIMAGE':
sw_uncal_files.append(unc_fi)
if fi.meta.exposure.type == 'NRC_TSGRISM':
lw_uncal_files.append(unc_fi)
if fi.meta.exposure.type == 'NRC_TACQ':
lw_ta_file.append(unc_fi)
colnames = ('Instrument', 'Filter', 'Pupil', 'Number of Integrations', 'Number of Groups',
'Readout pattern', 'Exposure Type')
dtypes = ('S7', 'S10', 'S10', 'i4', 'i4', 'S15', 'S11')
meta_check = Table(names=(colnames), dtype=dtypes)
# Open example files and get metadata for display
for sw_f in sw_uncal_files:
sw_examine = datamodels.open(sw_f)
sw_row = [sw_examine.meta.instrument.name, sw_examine.meta.instrument.filter,
sw_examine.meta.instrument.pupil, sw_examine.meta.exposure.nints,
sw_examine.meta.exposure.ngroups, sw_examine.meta.exposure.readpatt, sw_examine.meta.exposure.type]
meta_check.add_row(sw_row)
for lw_f in lw_uncal_files:
lw_examine = datamodels.open(lw_f)
lw_row = [lw_examine.meta.instrument.name, lw_examine.meta.instrument.filter,
lw_examine.meta.instrument.pupil, lw_examine.meta.exposure.nints,
lw_examine.meta.exposure.ngroups, lw_examine.meta.exposure.readpatt, lw_examine.meta.exposure.type]
meta_check.add_row(lw_row)
if len(lw_ta_file) == 1:
ta_examine = datamodels.open(lw_ta_file[0])
TA_row = [ta_examine.meta.instrument.name, ta_examine.meta.instrument.filter,
ta_examine.meta.instrument.pupil, ta_examine.meta.exposure.nints,
ta_examine.meta.exposure.ngroups, ta_examine.meta.exposure.readpatt, ta_examine.meta.exposure.type]
meta_check.add_row(TA_row)
# Print out exposure info
meta_check
Table length=5
| Instrument | Filter | Pupil | Number of Integrations | Number of Groups | Readout pattern | Exposure Type |
|---|---|---|---|---|---|---|
| bytes7 | bytes10 | bytes10 | int32 | int32 | bytes15 | bytes11 |
| NIRCAM | F322W2 | GRISMR | 366 | 12 | SHALLOW4 | NRC_TSGRISM |
| NIRCAM | F322W2 | GRISMR | 366 | 12 | SHALLOW4 | NRC_TSGRISM |
| NIRCAM | F322W2 | GRISMR | 366 | 12 | SHALLOW4 | NRC_TSGRISM |
| NIRCAM | F322W2 | GRISMR | 366 | 12 | SHALLOW4 | NRC_TSGRISM |
| NIRCAM | F335M | CLEAR | 1 | 17 | RAPID | NRC_TACQ |
# Print out the time benchmark
time_det1 = time.perf_counter()
print(f"Runtime so far: {time_det1 - time0:0.0f} seconds")
Runtime so far: 58 seconds
5. Stage 1: Detector1Pipeline(calwebb_detector1)#
In this section, we process the data through the calwebb_detector1 pipeline to create Stage 1 data products.
Input: Raw exposure (
_uncal.fits) containing original data from all detector readouts (ncols x nrows x ngroups x nintegrations).Output: Uncalibrated countrate (slope) image in units of DN/s:
_rate.fits: A single countrate image averaged over multiple integrations (if available)._rateints.fits: Countrate images for each integration, saved in multiple extensions.
The Detector1Pipeline applies basic detector-level corrections on a group-by-group basis, followed by ramp fitting for all exposure types, commonly referred to as “ramps-to-slopes” processing.
5.1 Configure Detector1Pipeline#
Below, we set up a dictionary that defines how the Detector1Pipeline should be configured.
For more information about each step and a full list of step arguments, please refer to the official documentation: JDox • ReadtheDocs
By default, all steps in the Detector1 stage of the pipeline are run for
NIRCam except the ipc, persistence, and charge_migration steps.
# Set up a dictionary to define how the Detector1 pipeline should be configured
# Boilerplate dictionary setup
det1dict = {}
det1dict['group_scale'], det1dict['dq_init'], det1dict['saturation'] = {}, {}, {}
det1dict['ipc'], det1dict['superbias'], det1dict['refpix'] = {}, {}, {}
det1dict['linearity'], det1dict['persistence'], det1dict['dark_current'], = {}, {}, {}
det1dict['charge_migration'], det1dict['jump'], det1dict['clean_flicker_noise'], det1dict['ramp_fit'] = {}, {}, {}, {}
# Overrides for various reference files
# Files should be in the base local directory or provide full path
#det1dict['dq_init']['override_mask'] = 'myfile.fits' # Bad pixel mask
#det1dict['saturation']['override_saturation'] = 'myfile.fits' # Saturation
#det1dict['linearity']['override_linearity'] = 'myfile.fits' # Linearity
#det1dict['dark_current']['override_dark'] = 'myfile.fits' # Dark current subtraction
#det1dict['jump']['override_gain'] = 'myfile.fits' # Gain used by jump step
#det1dict['ramp_fit']['override_gain'] = 'myfile.fits' # Gain used by ramp fitting step
#det1dict['jump']['override_readnoise'] = 'myfile.fits' # Read noise used by jump step
#det1dict['ramp_fit']['override_readnoise'] = 'myfile.fits' # Read noise used by ramp fitting step
# Turn on multi-core processing (This is off by default). Choose what fraction
# of cores to use (quarter, half, all, or an integer number)
det1dict['jump']['maximum_cores'] = 'half'
det1dict['ramp_fit']['maximum_cores'] = 'half'
# Adjust the flagging threshold for cosmic rays (default is 3.0)
det1dict['jump']['rejection_threshold'] = 6.0
Note that the persistence step
has been turned off by default starting with CRDS context jwst_1264.pmap (with pipeline version 1.15.0 being the subsequent release version). This step can be manually turned on when running the pipeline by setting skip = False in the cell below using a dictionary. However, the persistence behavior for NIRCam is not well calibrated at the moment. Therefore, the associated NIRCam persistence reference files are such that even if the step is turned on, it will not make any changes to the data. The persistence step will run and provide input for subsequent pipeline steps but it will not do anything beyond creating an empty *_trapsfilled.fits file. We therefore recommend that users do not turn this step on at this time.
# The persistence step will be skipped by default. Therefore, this line is redundant,
# and is here simply to emphasize that persistence correction will not be run.
det1dict['persistence']['skip'] = True
JWST detector readout electronics (a.k.a. SIDECAR ASICs) generate significant 1/f noise during detector operations and signal digitization. This noise manifests as banding along the detector’s fast-read direction (along detector rows). If not handled properly, the 1/f noise can introduce systematic errors and extra scatter in Grism Time Series light curves. For more information, please visit JWST Time-Series Observations Noise Sources.
An 1/f noise-cleaning algorithm, clean_flicker_noise, has been implemented at the group stage in the Detector1Pipeline. This step is released and can be used but is turned off by default. It is generally not recommended to run the clean_flicker_noise step on Grism Time Series data as the horizontal banding from the 1/f noise is parallel to the spectroscopic trace, making it very difficult for the routine to distinguish between the two.
The cell bellow provides an example on how to turn it on and change parameters. For more information, please visit 1/f Noise and NIRCam 1/f Noise Removal Methods.
# Turn off/on 1/f correction if desired
# For guidance see https://jwst-docs.stsci.edu/known-issues-with-jwst-data/1-f-noise
det1dict['clean_flicker_noise']['skip'] = True
#det1dict['clean_flicker_noise']['fit_method'] = 'median' # 'median' or 'fft'
#det1dict['clean_flicker_noise']['background_method'] = 'median' # 'median' or 'model'
#det1dict['clean_flicker_noise']['fit_by_channel'] = False
As of CRDS context jwst_1155.pmap and later, the
jump step
of the Detector1 stage of the pipeline will remove signal associated
with snowballs
in the NIRCam imaging and coronagraphy modes. This correction is turned on using the parameter
expand_large_events=True. This and other parameters related to the snowball correction
are specified in the pars-jumpstep parameter reference file. Users may wish to alter
parameters to optimize removal of snowball residuals. Available parameters are discussed
in the Detection and Flagging of Showers and Snowballs in JWST Technical Report (Regan 2023).
# Explicitly skip snowball correction. (Even though it is on by default)
det1dict['jump']['expand_large_events'] = False
5.2 Run Detector1Pipeline#
Run the science files through the calwebb_detector1 pipeline using the recommended .call() method.
Run the datasets through the
Detector1
stage of the pipeline to apply detector level calibrations and create a
countrate data product where slopes are fitted to the integration ramps.
These *_rate.fits products are 2D (nrows x ncols), averaged over all
integrations. 3D countrate data products (*_rateints.fits) are also
created (nintegrations x nrows x ncols) which have the fitted ramp slopes
for each integration. The latter are the primary output used in Time Series observations.
# Run Detector1 stage of pipeline, setting the
# output directory to the location to save the *_rateints.fits files
# and the save_results flag to True so the rateints files are saved
# Note that this cell may take a while to run.
start = time.time()
if do_det1:
for lw_uncal in lw_uncal_files:
rate_result = Detector1Pipeline.call(lw_uncal, output_dir=det1_dir, steps=det1dict, save_results=True)
else:
print('Skipping Detector1 processing')
end = time.time()
print("Run time: ", round(end-start, 1)/60.0, " min")
2026-04-17 15:18:39,240 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_system_datalvl_0002.rmap 694 bytes (1 / 224 files) (0 / 796.2 K bytes)
2026-04-17 15:18:39,358 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_system_calver_0069.rmap 5.8 K bytes (2 / 224 files) (694 / 796.2 K bytes)
2026-04-17 15:18:39,453 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_system_0064.imap 385 bytes (3 / 224 files) (6.5 K / 796.2 K bytes)
2026-04-17 15:18:39,540 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_wavelengthrange_0024.rmap 1.4 K bytes (4 / 224 files) (6.9 K / 796.2 K bytes)
2026-04-17 15:18:39,633 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_wavecorr_0005.rmap 884 bytes (5 / 224 files) (8.3 K / 796.2 K bytes)
2026-04-17 15:18:39,730 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_superbias_0089.rmap 39.4 K bytes (6 / 224 files) (9.1 K / 796.2 K bytes)
2026-04-17 15:18:39,845 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_sirskernel_0002.rmap 704 bytes (7 / 224 files) (48.5 K / 796.2 K bytes)
2026-04-17 15:18:39,942 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_sflat_0027.rmap 20.6 K bytes (8 / 224 files) (49.2 K / 796.2 K bytes)
2026-04-17 15:18:40,058 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_saturation_0018.rmap 2.0 K bytes (9 / 224 files) (69.8 K / 796.2 K bytes)
2026-04-17 15:18:40,141 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_refpix_0015.rmap 1.6 K bytes (10 / 224 files) (71.9 K / 796.2 K bytes)
2026-04-17 15:18:40,327 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_readnoise_0025.rmap 2.6 K bytes (11 / 224 files) (73.4 K / 796.2 K bytes)
2026-04-17 15:18:40,417 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_psf_0002.rmap 687 bytes (12 / 224 files) (76.0 K / 796.2 K bytes)
2026-04-17 15:18:40,500 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pictureframe_0002.rmap 886 bytes (13 / 224 files) (76.7 K / 796.2 K bytes)
2026-04-17 15:18:40,609 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_photom_0013.rmap 958 bytes (14 / 224 files) (77.6 K / 796.2 K bytes)
2026-04-17 15:18:40,695 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pathloss_0011.rmap 1.2 K bytes (15 / 224 files) (78.5 K / 796.2 K bytes)
2026-04-17 15:18:40,784 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-whitelightstep_0001.rmap 777 bytes (16 / 224 files) (79.7 K / 796.2 K bytes)
2026-04-17 15:18:40,875 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-tso3pipeline_0001.rmap 786 bytes (17 / 224 files) (80.5 K / 796.2 K bytes)
2026-04-17 15:18:40,963 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-spec2pipeline_0013.rmap 2.1 K bytes (18 / 224 files) (81.3 K / 796.2 K bytes)
2026-04-17 15:18:41,048 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-resamplespecstep_0002.rmap 709 bytes (19 / 224 files) (83.4 K / 796.2 K bytes)
2026-04-17 15:18:41,175 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-refpixstep_0003.rmap 910 bytes (20 / 224 files) (84.1 K / 796.2 K bytes)
2026-04-17 15:18:41,298 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-pixelreplacestep_0001.rmap 818 bytes (21 / 224 files) (85.0 K / 796.2 K bytes)
2026-04-17 15:18:41,394 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-pictureframestep_0001.rmap 818 bytes (22 / 224 files) (85.8 K / 796.2 K bytes)
2026-04-17 15:18:41,486 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-outlierdetectionstep_0005.rmap 1.1 K bytes (23 / 224 files) (86.7 K / 796.2 K bytes)
2026-04-17 15:18:41,575 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-jumpstep_0006.rmap 810 bytes (24 / 224 files) (87.8 K / 796.2 K bytes)
2026-04-17 15:18:41,662 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-image2pipeline_0008.rmap 1.0 K bytes (25 / 224 files) (88.6 K / 796.2 K bytes)
2026-04-17 15:18:41,755 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-extract1dstep_0001.rmap 794 bytes (26 / 224 files) (89.6 K / 796.2 K bytes)
2026-04-17 15:18:41,836 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-detector1pipeline_0004.rmap 1.1 K bytes (27 / 224 files) (90.4 K / 796.2 K bytes)
2026-04-17 15:18:41,949 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-darkpipeline_0003.rmap 872 bytes (28 / 224 files) (91.5 K / 796.2 K bytes)
2026-04-17 15:18:42,032 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-darkcurrentstep_0003.rmap 1.8 K bytes (29 / 224 files) (92.4 K / 796.2 K bytes)
2026-04-17 15:18:42,134 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-cubebuildstep_0001.rmap 862 bytes (30 / 224 files) (94.2 K / 796.2 K bytes)
2026-04-17 15:18:42,228 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-cleanflickernoisestep_0002.rmap 983 bytes (31 / 224 files) (95.1 K / 796.2 K bytes)
2026-04-17 15:18:42,321 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_pars-adaptivetracemodelstep_0002.rmap 997 bytes (32 / 224 files) (96.1 K / 796.2 K bytes)
2026-04-17 15:18:42,415 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_ote_0030.rmap 1.3 K bytes (33 / 224 files) (97.1 K / 796.2 K bytes)
2026-04-17 15:18:42,522 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_msaoper_0018.rmap 1.6 K bytes (34 / 224 files) (98.3 K / 796.2 K bytes)
2026-04-17 15:18:42,618 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_msa_0027.rmap 1.3 K bytes (35 / 224 files) (100.0 K / 796.2 K bytes)
2026-04-17 15:18:42,701 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_mask_0045.rmap 4.9 K bytes (36 / 224 files) (101.2 K / 796.2 K bytes)
2026-04-17 15:18:42,791 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_linearity_0017.rmap 1.6 K bytes (37 / 224 files) (106.2 K / 796.2 K bytes)
2026-04-17 15:18:42,874 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_ipc_0006.rmap 876 bytes (38 / 224 files) (107.7 K / 796.2 K bytes)
2026-04-17 15:18:42,957 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_ifuslicer_0018.rmap 1.5 K bytes (39 / 224 files) (108.6 K / 796.2 K bytes)
2026-04-17 15:18:43,047 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_ifupost_0020.rmap 1.5 K bytes (40 / 224 files) (110.1 K / 796.2 K bytes)
2026-04-17 15:18:43,140 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_ifufore_0017.rmap 1.5 K bytes (41 / 224 files) (111.6 K / 796.2 K bytes)
2026-04-17 15:18:43,248 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_gain_0023.rmap 1.8 K bytes (42 / 224 files) (113.1 K / 796.2 K bytes)
2026-04-17 15:18:43,344 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_fpa_0028.rmap 1.3 K bytes (43 / 224 files) (114.9 K / 796.2 K bytes)
2026-04-17 15:18:43,434 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_fore_0026.rmap 5.0 K bytes (44 / 224 files) (116.2 K / 796.2 K bytes)
2026-04-17 15:18:43,538 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_flat_0015.rmap 3.8 K bytes (45 / 224 files) (121.1 K / 796.2 K bytes)
2026-04-17 15:18:43,624 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_fflat_0030.rmap 7.2 K bytes (46 / 224 files) (124.9 K / 796.2 K bytes)
2026-04-17 15:18:43,707 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_extract1d_0018.rmap 2.3 K bytes (47 / 224 files) (132.1 K / 796.2 K bytes)
2026-04-17 15:18:43,810 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_disperser_0028.rmap 5.7 K bytes (48 / 224 files) (134.4 K / 796.2 K bytes)
2026-04-17 15:18:43,902 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_dflat_0007.rmap 1.1 K bytes (49 / 224 files) (140.1 K / 796.2 K bytes)
2026-04-17 15:18:44,009 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_dark_0085.rmap 37.4 K bytes (50 / 224 files) (141.3 K / 796.2 K bytes)
2026-04-17 15:18:44,135 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_cubepar_0015.rmap 966 bytes (51 / 224 files) (178.7 K / 796.2 K bytes)
2026-04-17 15:18:44,231 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_collimator_0026.rmap 1.3 K bytes (52 / 224 files) (179.6 K / 796.2 K bytes)
2026-04-17 15:18:44,316 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_camera_0026.rmap 1.3 K bytes (53 / 224 files) (181.0 K / 796.2 K bytes)
2026-04-17 15:18:44,400 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_barshadow_0007.rmap 1.8 K bytes (54 / 224 files) (182.3 K / 796.2 K bytes)
2026-04-17 15:18:44,500 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_area_0019.rmap 6.8 K bytes (55 / 224 files) (184.1 K / 796.2 K bytes)
2026-04-17 15:18:44,582 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_apcorr_0009.rmap 5.6 K bytes (56 / 224 files) (190.9 K / 796.2 K bytes)
2026-04-17 15:18:44,665 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nirspec_0432.imap 6.2 K bytes (57 / 224 files) (196.5 K / 796.2 K bytes)
2026-04-17 15:18:44,764 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_wavelengthrange_0008.rmap 897 bytes (58 / 224 files) (202.6 K / 796.2 K bytes)
2026-04-17 15:18:44,865 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_trappars_0004.rmap 753 bytes (59 / 224 files) (203.5 K / 796.2 K bytes)
2026-04-17 15:18:44,970 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_trapdensity_0005.rmap 705 bytes (60 / 224 files) (204.3 K / 796.2 K bytes)
2026-04-17 15:18:45,075 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_throughput_0005.rmap 1.3 K bytes (61 / 224 files) (205.0 K / 796.2 K bytes)
2026-04-17 15:18:45,167 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_superbias_0035.rmap 8.3 K bytes (62 / 224 files) (206.2 K / 796.2 K bytes)
2026-04-17 15:18:45,252 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_specwcs_0017.rmap 3.1 K bytes (63 / 224 files) (214.5 K / 796.2 K bytes)
2026-04-17 15:18:45,345 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_specprofile_0010.rmap 2.5 K bytes (64 / 224 files) (217.7 K / 796.2 K bytes)
2026-04-17 15:18:45,433 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_speckernel_0006.rmap 1.0 K bytes (65 / 224 files) (220.2 K / 796.2 K bytes)
2026-04-17 15:18:45,525 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_sirskernel_0002.rmap 700 bytes (66 / 224 files) (221.2 K / 796.2 K bytes)
2026-04-17 15:18:45,615 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_saturation_0015.rmap 829 bytes (67 / 224 files) (221.9 K / 796.2 K bytes)
2026-04-17 15:18:45,705 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_readnoise_0011.rmap 987 bytes (68 / 224 files) (222.7 K / 796.2 K bytes)
2026-04-17 15:18:45,789 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_photom_0041.rmap 1.3 K bytes (69 / 224 files) (223.7 K / 796.2 K bytes)
2026-04-17 15:18:45,874 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_persat_0007.rmap 674 bytes (70 / 224 files) (225.0 K / 796.2 K bytes)
2026-04-17 15:18:45,975 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pathloss_0003.rmap 758 bytes (71 / 224 files) (225.6 K / 796.2 K bytes)
2026-04-17 15:18:46,065 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pastasoss_0006.rmap 818 bytes (72 / 224 files) (226.4 K / 796.2 K bytes)
2026-04-17 15:18:46,156 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-wfsscontamstep_0001.rmap 797 bytes (73 / 224 files) (227.2 K / 796.2 K bytes)
2026-04-17 15:18:46,252 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-undersamplecorrectionstep_0001.rmap 904 bytes (74 / 224 files) (228.0 K / 796.2 K bytes)
2026-04-17 15:18:46,342 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-tweakregstep_0012.rmap 3.1 K bytes (75 / 224 files) (228.9 K / 796.2 K bytes)
2026-04-17 15:18:46,427 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-spec2pipeline_0009.rmap 1.2 K bytes (76 / 224 files) (232.0 K / 796.2 K bytes)
2026-04-17 15:18:46,520 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-sourcecatalogstep_0002.rmap 2.3 K bytes (77 / 224 files) (233.3 K / 796.2 K bytes)
2026-04-17 15:18:46,614 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-resamplestep_0002.rmap 687 bytes (78 / 224 files) (235.6 K / 796.2 K bytes)
2026-04-17 15:18:46,712 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-outlierdetectionstep_0004.rmap 2.7 K bytes (79 / 224 files) (236.3 K / 796.2 K bytes)
2026-04-17 15:18:46,806 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-jumpstep_0007.rmap 6.4 K bytes (80 / 224 files) (239.0 K / 796.2 K bytes)
2026-04-17 15:18:46,900 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-image2pipeline_0005.rmap 1.0 K bytes (81 / 224 files) (245.3 K / 796.2 K bytes)
2026-04-17 15:18:46,986 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-detector1pipeline_0005.rmap 1.5 K bytes (82 / 224 files) (246.3 K / 796.2 K bytes)
2026-04-17 15:18:47,082 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-darkpipeline_0002.rmap 868 bytes (83 / 224 files) (247.9 K / 796.2 K bytes)
2026-04-17 15:18:47,172 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-darkcurrentstep_0001.rmap 591 bytes (84 / 224 files) (248.8 K / 796.2 K bytes)
2026-04-17 15:18:47,269 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-cleanflickernoisestep_0003.rmap 1.2 K bytes (85 / 224 files) (249.3 K / 796.2 K bytes)
2026-04-17 15:18:47,350 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-chargemigrationstep_0005.rmap 5.7 K bytes (86 / 224 files) (250.6 K / 796.2 K bytes)
2026-04-17 15:18:47,435 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_pars-backgroundstep_0003.rmap 822 bytes (87 / 224 files) (256.2 K / 796.2 K bytes)
2026-04-17 15:18:47,529 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_nrm_0005.rmap 663 bytes (88 / 224 files) (257.0 K / 796.2 K bytes)
2026-04-17 15:18:47,620 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_mask_0025.rmap 1.6 K bytes (89 / 224 files) (257.7 K / 796.2 K bytes)
2026-04-17 15:18:47,719 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_linearity_0022.rmap 961 bytes (90 / 224 files) (259.3 K / 796.2 K bytes)
2026-04-17 15:18:47,799 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_ipc_0007.rmap 651 bytes (91 / 224 files) (260.3 K / 796.2 K bytes)
2026-04-17 15:18:47,883 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_gain_0011.rmap 797 bytes (92 / 224 files) (260.9 K / 796.2 K bytes)
2026-04-17 15:18:47,975 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_flat_0023.rmap 5.9 K bytes (93 / 224 files) (261.7 K / 796.2 K bytes)
2026-04-17 15:18:48,079 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_filteroffset_0010.rmap 853 bytes (94 / 224 files) (267.6 K / 796.2 K bytes)
2026-04-17 15:18:48,178 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_extract1d_0007.rmap 905 bytes (95 / 224 files) (268.4 K / 796.2 K bytes)
2026-04-17 15:18:48,273 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_drizpars_0004.rmap 519 bytes (96 / 224 files) (269.3 K / 796.2 K bytes)
2026-04-17 15:18:48,355 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_distortion_0025.rmap 3.4 K bytes (97 / 224 files) (269.9 K / 796.2 K bytes)
2026-04-17 15:18:48,454 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_dark_0039.rmap 8.3 K bytes (98 / 224 files) (273.3 K / 796.2 K bytes)
2026-04-17 15:18:48,543 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_bkg_0005.rmap 3.1 K bytes (99 / 224 files) (281.6 K / 796.2 K bytes)
2026-04-17 15:18:48,625 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_area_0014.rmap 2.7 K bytes (100 / 224 files) (284.7 K / 796.2 K bytes)
2026-04-17 15:18:48,709 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_apcorr_0010.rmap 4.3 K bytes (101 / 224 files) (287.4 K / 796.2 K bytes)
2026-04-17 15:18:48,813 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_abvegaoffset_0004.rmap 1.4 K bytes (102 / 224 files) (291.7 K / 796.2 K bytes)
2026-04-17 15:18:48,899 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_niriss_0308.imap 5.9 K bytes (103 / 224 files) (293.0 K / 796.2 K bytes)
2026-04-17 15:18:48,981 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_wavelengthrange_0012.rmap 996 bytes (104 / 224 files) (299.0 K / 796.2 K bytes)
2026-04-17 15:18:49,085 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_tsophot_0003.rmap 896 bytes (105 / 224 files) (300.0 K / 796.2 K bytes)
2026-04-17 15:18:49,179 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_trappars_0003.rmap 1.6 K bytes (106 / 224 files) (300.9 K / 796.2 K bytes)
2026-04-17 15:18:49,272 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_trapdensity_0003.rmap 1.6 K bytes (107 / 224 files) (302.5 K / 796.2 K bytes)
2026-04-17 15:18:49,353 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_superbias_0022.rmap 25.5 K bytes (108 / 224 files) (304.1 K / 796.2 K bytes)
2026-04-17 15:18:49,462 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_specwcs_0027.rmap 8.0 K bytes (109 / 224 files) (329.6 K / 796.2 K bytes)
2026-04-17 15:18:49,555 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_sirskernel_0003.rmap 671 bytes (110 / 224 files) (337.6 K / 796.2 K bytes)
2026-04-17 15:18:49,645 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_saturation_0011.rmap 2.8 K bytes (111 / 224 files) (338.3 K / 796.2 K bytes)
2026-04-17 15:18:49,726 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_regions_0003.rmap 3.4 K bytes (112 / 224 files) (341.1 K / 796.2 K bytes)
2026-04-17 15:18:49,808 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_readnoise_0028.rmap 27.1 K bytes (113 / 224 files) (344.5 K / 796.2 K bytes)
2026-04-17 15:18:49,919 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_psfmask_0008.rmap 28.4 K bytes (114 / 224 files) (371.7 K / 796.2 K bytes)
2026-04-17 15:18:50,024 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_photom_0031.rmap 3.4 K bytes (115 / 224 files) (400.0 K / 796.2 K bytes)
2026-04-17 15:18:50,124 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_persat_0005.rmap 1.6 K bytes (116 / 224 files) (403.5 K / 796.2 K bytes)
2026-04-17 15:18:50,228 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_pars-whitelightstep_0004.rmap 2.0 K bytes (117 / 224 files) (405.0 K / 796.2 K bytes)
2026-04-17 15:18:50,326 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_pars-wfsscontamstep_0001.rmap 797 bytes (118 / 224 files) (407.0 K / 796.2 K bytes)
2026-04-17 15:18:50,423 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_pars-tweakregstep_0003.rmap 4.5 K bytes (119 / 224 files) (407.8 K / 796.2 K bytes)
2026-04-17 15:18:50,514 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_pars-tsophotometrystep_0003.rmap 1.1 K bytes (120 / 224 files) (412.3 K / 796.2 K bytes)
2026-04-17 15:18:50,607 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_pars-spec2pipeline_0009.rmap 984 bytes (121 / 224 files) (413.4 K / 796.2 K bytes)
2026-04-17 15:18:50,689 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_pars-sourcecatalogstep_0002.rmap 4.6 K bytes (122 / 224 files) (414.4 K / 796.2 K bytes)
2026-04-17 15:18:50,782 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_pars-resamplestep_0002.rmap 687 bytes (123 / 224 files) (419.0 K / 796.2 K bytes)
2026-04-17 15:18:50,897 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_pars-outlierdetectionstep_0003.rmap 940 bytes (124 / 224 files) (419.7 K / 796.2 K bytes)
2026-04-17 15:18:51,002 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_pars-jumpstep_0005.rmap 806 bytes (125 / 224 files) (420.6 K / 796.2 K bytes)
2026-04-17 15:18:51,084 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_pars-image2pipeline_0004.rmap 1.1 K bytes (126 / 224 files) (421.4 K / 796.2 K bytes)
2026-04-17 15:18:51,175 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_pars-detector1pipeline_0007.rmap 1.7 K bytes (127 / 224 files) (422.6 K / 796.2 K bytes)
2026-04-17 15:18:51,270 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_pars-darkpipeline_0002.rmap 868 bytes (128 / 224 files) (424.3 K / 796.2 K bytes)
2026-04-17 15:18:51,367 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_pars-darkcurrentstep_0001.rmap 618 bytes (129 / 224 files) (425.2 K / 796.2 K bytes)
2026-04-17 15:18:51,447 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_pars-backgroundstep_0003.rmap 822 bytes (130 / 224 files) (425.8 K / 796.2 K bytes)
2026-04-17 15:18:51,530 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_mask_0014.rmap 5.4 K bytes (131 / 224 files) (426.6 K / 796.2 K bytes)
2026-04-17 15:18:51,619 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_linearity_0011.rmap 2.4 K bytes (132 / 224 files) (432.0 K / 796.2 K bytes)
2026-04-17 15:18:51,700 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_ipc_0003.rmap 2.0 K bytes (133 / 224 files) (434.4 K / 796.2 K bytes)
2026-04-17 15:18:51,781 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_gain_0016.rmap 2.1 K bytes (134 / 224 files) (436.4 K / 796.2 K bytes)
2026-04-17 15:18:51,895 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_flat_0028.rmap 51.7 K bytes (135 / 224 files) (438.5 K / 796.2 K bytes)
2026-04-17 15:18:52,027 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_filteroffset_0004.rmap 1.4 K bytes (136 / 224 files) (490.2 K / 796.2 K bytes)
2026-04-17 15:18:52,108 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_extract1d_0007.rmap 2.2 K bytes (137 / 224 files) (491.6 K / 796.2 K bytes)
2026-04-17 15:18:52,191 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_drizpars_0001.rmap 519 bytes (138 / 224 files) (493.8 K / 796.2 K bytes)
2026-04-17 15:18:52,285 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_distortion_0034.rmap 53.4 K bytes (139 / 224 files) (494.3 K / 796.2 K bytes)
2026-04-17 15:18:52,420 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_dark_0054.rmap 33.9 K bytes (140 / 224 files) (547.6 K / 796.2 K bytes)
2026-04-17 15:18:52,550 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_bkg_0002.rmap 7.0 K bytes (141 / 224 files) (581.5 K / 796.2 K bytes)
2026-04-17 15:18:52,634 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_area_0012.rmap 33.5 K bytes (142 / 224 files) (588.5 K / 796.2 K bytes)
2026-04-17 15:18:52,739 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_apcorr_0009.rmap 4.3 K bytes (143 / 224 files) (622.0 K / 796.2 K bytes)
2026-04-17 15:18:52,823 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_abvegaoffset_0004.rmap 1.3 K bytes (144 / 224 files) (626.2 K / 796.2 K bytes)
2026-04-17 15:18:52,908 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_nircam_0354.imap 5.8 K bytes (145 / 224 files) (627.5 K / 796.2 K bytes)
2026-04-17 15:18:53,005 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_wavelengthrange_0030.rmap 1.0 K bytes (146 / 224 files) (633.3 K / 796.2 K bytes)
2026-04-17 15:18:53,091 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_tsophot_0004.rmap 882 bytes (147 / 224 files) (634.3 K / 796.2 K bytes)
2026-04-17 15:18:53,190 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_straymask_0009.rmap 987 bytes (148 / 224 files) (635.2 K / 796.2 K bytes)
2026-04-17 15:18:53,291 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_specwcs_0048.rmap 5.9 K bytes (149 / 224 files) (636.2 K / 796.2 K bytes)
2026-04-17 15:18:53,395 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_saturation_0015.rmap 1.2 K bytes (150 / 224 files) (642.1 K / 796.2 K bytes)
2026-04-17 15:18:53,487 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_rscd_0010.rmap 1.0 K bytes (151 / 224 files) (643.3 K / 796.2 K bytes)
2026-04-17 15:18:53,587 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_resol_0006.rmap 790 bytes (152 / 224 files) (644.3 K / 796.2 K bytes)
2026-04-17 15:18:53,668 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_reset_0026.rmap 3.9 K bytes (153 / 224 files) (645.1 K / 796.2 K bytes)
2026-04-17 15:18:53,757 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_regions_0036.rmap 4.4 K bytes (154 / 224 files) (649.0 K / 796.2 K bytes)
2026-04-17 15:18:53,854 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_readnoise_0023.rmap 1.6 K bytes (155 / 224 files) (653.3 K / 796.2 K bytes)
2026-04-17 15:18:53,938 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_psfmask_0009.rmap 2.1 K bytes (156 / 224 files) (655.0 K / 796.2 K bytes)
2026-04-17 15:18:54,032 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_psf_0008.rmap 2.6 K bytes (157 / 224 files) (657.1 K / 796.2 K bytes)
2026-04-17 15:18:54,116 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_photom_0063.rmap 3.9 K bytes (158 / 224 files) (659.7 K / 796.2 K bytes)
2026-04-17 15:18:54,212 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pathloss_0005.rmap 866 bytes (159 / 224 files) (663.6 K / 796.2 K bytes)
2026-04-17 15:18:54,308 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-whitelightstep_0003.rmap 912 bytes (160 / 224 files) (664.4 K / 796.2 K bytes)
2026-04-17 15:18:54,406 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-wfsscontamstep_0001.rmap 787 bytes (161 / 224 files) (665.4 K / 796.2 K bytes)
2026-04-17 15:18:54,493 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-tweakregstep_0003.rmap 1.8 K bytes (162 / 224 files) (666.1 K / 796.2 K bytes)
2026-04-17 15:18:54,590 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-tsophotometrystep_0003.rmap 2.7 K bytes (163 / 224 files) (668.0 K / 796.2 K bytes)
2026-04-17 15:18:54,673 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-spec3pipeline_0011.rmap 886 bytes (164 / 224 files) (670.6 K / 796.2 K bytes)
2026-04-17 15:18:54,764 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-spec2pipeline_0013.rmap 1.4 K bytes (165 / 224 files) (671.5 K / 796.2 K bytes)
2026-04-17 15:18:54,858 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-sourcecatalogstep_0003.rmap 1.9 K bytes (166 / 224 files) (672.9 K / 796.2 K bytes)
2026-04-17 15:18:54,939 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-resamplestep_0002.rmap 677 bytes (167 / 224 files) (674.9 K / 796.2 K bytes)
2026-04-17 15:18:55,023 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-resamplespecstep_0002.rmap 706 bytes (168 / 224 files) (675.5 K / 796.2 K bytes)
2026-04-17 15:18:55,106 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-outlierdetectionstep_0020.rmap 3.4 K bytes (169 / 224 files) (676.2 K / 796.2 K bytes)
2026-04-17 15:18:55,207 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-jumpstep_0011.rmap 1.6 K bytes (170 / 224 files) (679.6 K / 796.2 K bytes)
2026-04-17 15:18:55,292 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-image2pipeline_0010.rmap 1.1 K bytes (171 / 224 files) (681.2 K / 796.2 K bytes)
2026-04-17 15:18:55,385 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-extract1dstep_0003.rmap 807 bytes (172 / 224 files) (682.3 K / 796.2 K bytes)
2026-04-17 15:18:55,491 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-emicorrstep_0003.rmap 796 bytes (173 / 224 files) (683.1 K / 796.2 K bytes)
2026-04-17 15:18:55,578 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-detector1pipeline_0010.rmap 1.6 K bytes (174 / 224 files) (683.9 K / 796.2 K bytes)
2026-04-17 15:18:55,678 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-darkpipeline_0002.rmap 860 bytes (175 / 224 files) (685.5 K / 796.2 K bytes)
2026-04-17 15:18:55,770 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-darkcurrentstep_0002.rmap 683 bytes (176 / 224 files) (686.3 K / 796.2 K bytes)
2026-04-17 15:18:55,867 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-backgroundstep_0003.rmap 814 bytes (177 / 224 files) (687.0 K / 796.2 K bytes)
2026-04-17 15:18:55,950 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_pars-adaptivetracemodelstep_0002.rmap 979 bytes (178 / 224 files) (687.8 K / 796.2 K bytes)
2026-04-17 15:18:56,045 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_mrsxartcorr_0002.rmap 2.2 K bytes (179 / 224 files) (688.8 K / 796.2 K bytes)
2026-04-17 15:18:56,132 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_mrsptcorr_0005.rmap 2.0 K bytes (180 / 224 files) (691.0 K / 796.2 K bytes)
2026-04-17 15:18:56,226 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_mask_0036.rmap 8.6 K bytes (181 / 224 files) (692.9 K / 796.2 K bytes)
2026-04-17 15:18:56,323 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_linearity_0018.rmap 2.8 K bytes (182 / 224 files) (701.6 K / 796.2 K bytes)
2026-04-17 15:18:56,404 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_ipc_0008.rmap 700 bytes (183 / 224 files) (704.4 K / 796.2 K bytes)
2026-04-17 15:18:56,500 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_gain_0013.rmap 3.9 K bytes (184 / 224 files) (705.1 K / 796.2 K bytes)
2026-04-17 15:18:56,598 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_fringefreq_0003.rmap 1.4 K bytes (185 / 224 files) (709.0 K / 796.2 K bytes)
2026-04-17 15:18:56,702 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_fringe_0019.rmap 3.9 K bytes (186 / 224 files) (710.5 K / 796.2 K bytes)
2026-04-17 15:18:56,799 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_flat_0073.rmap 16.5 K bytes (187 / 224 files) (714.4 K / 796.2 K bytes)
2026-04-17 15:18:56,915 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_filteroffset_0029.rmap 2.4 K bytes (188 / 224 files) (730.9 K / 796.2 K bytes)
2026-04-17 15:18:57,011 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_extract1d_0022.rmap 1.0 K bytes (189 / 224 files) (733.3 K / 796.2 K bytes)
2026-04-17 15:18:57,096 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_emicorr_0004.rmap 663 bytes (190 / 224 files) (734.3 K / 796.2 K bytes)
2026-04-17 15:18:57,190 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_drizpars_0002.rmap 511 bytes (191 / 224 files) (735.0 K / 796.2 K bytes)
2026-04-17 15:18:57,286 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_distortion_0043.rmap 4.8 K bytes (192 / 224 files) (735.5 K / 796.2 K bytes)
2026-04-17 15:18:57,375 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_dark_0039.rmap 4.3 K bytes (193 / 224 files) (740.3 K / 796.2 K bytes)
2026-04-17 15:18:57,466 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_cubepar_0017.rmap 800 bytes (194 / 224 files) (744.6 K / 796.2 K bytes)
2026-04-17 15:18:57,558 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_bkg_0004.rmap 712 bytes (195 / 224 files) (745.4 K / 796.2 K bytes)
2026-04-17 15:18:57,647 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_area_0015.rmap 866 bytes (196 / 224 files) (746.1 K / 796.2 K bytes)
2026-04-17 15:18:57,735 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_apcorr_0023.rmap 5.0 K bytes (197 / 224 files) (746.9 K / 796.2 K bytes)
2026-04-17 15:18:57,819 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_abvegaoffset_0003.rmap 1.3 K bytes (198 / 224 files) (752.0 K / 796.2 K bytes)
2026-04-17 15:18:57,900 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_miri_0487.imap 6.0 K bytes (199 / 224 files) (753.2 K / 796.2 K bytes)
2026-04-17 15:18:57,992 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_trappars_0004.rmap 903 bytes (200 / 224 files) (759.3 K / 796.2 K bytes)
2026-04-17 15:18:58,088 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_trapdensity_0006.rmap 930 bytes (201 / 224 files) (760.2 K / 796.2 K bytes)
2026-04-17 15:18:58,189 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_superbias_0017.rmap 3.8 K bytes (202 / 224 files) (761.1 K / 796.2 K bytes)
2026-04-17 15:18:58,292 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_saturation_0009.rmap 779 bytes (203 / 224 files) (764.9 K / 796.2 K bytes)
2026-04-17 15:18:58,388 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_readnoise_0014.rmap 1.3 K bytes (204 / 224 files) (765.7 K / 796.2 K bytes)
2026-04-17 15:18:58,470 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_photom_0014.rmap 1.1 K bytes (205 / 224 files) (766.9 K / 796.2 K bytes)
2026-04-17 15:18:58,559 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_persat_0006.rmap 884 bytes (206 / 224 files) (768.1 K / 796.2 K bytes)
2026-04-17 15:18:58,660 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_pars-tweakregstep_0002.rmap 850 bytes (207 / 224 files) (769.0 K / 796.2 K bytes)
2026-04-17 15:18:58,748 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_pars-sourcecatalogstep_0001.rmap 636 bytes (208 / 224 files) (769.8 K / 796.2 K bytes)
2026-04-17 15:18:58,840 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_pars-outlierdetectionstep_0001.rmap 654 bytes (209 / 224 files) (770.4 K / 796.2 K bytes)
2026-04-17 15:18:58,942 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_pars-image2pipeline_0005.rmap 974 bytes (210 / 224 files) (771.1 K / 796.2 K bytes)
2026-04-17 15:18:59,025 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_pars-detector1pipeline_0002.rmap 1.0 K bytes (211 / 224 files) (772.1 K / 796.2 K bytes)
2026-04-17 15:18:59,110 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_pars-darkpipeline_0002.rmap 856 bytes (212 / 224 files) (773.1 K / 796.2 K bytes)
2026-04-17 15:18:59,205 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_mask_0023.rmap 1.1 K bytes (213 / 224 files) (774.0 K / 796.2 K bytes)
2026-04-17 15:18:59,292 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_linearity_0015.rmap 925 bytes (214 / 224 files) (775.0 K / 796.2 K bytes)
2026-04-17 15:18:59,380 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_ipc_0003.rmap 614 bytes (215 / 224 files) (775.9 K / 796.2 K bytes)
2026-04-17 15:18:59,471 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_gain_0010.rmap 890 bytes (216 / 224 files) (776.5 K / 796.2 K bytes)
2026-04-17 15:18:59,561 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_flat_0009.rmap 1.1 K bytes (217 / 224 files) (777.4 K / 796.2 K bytes)
2026-04-17 15:18:59,652 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_distortion_0011.rmap 1.2 K bytes (218 / 224 files) (778.6 K / 796.2 K bytes)
2026-04-17 15:18:59,746 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_dark_0017.rmap 4.3 K bytes (219 / 224 files) (779.8 K / 796.2 K bytes)
2026-04-17 15:18:59,827 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_area_0010.rmap 1.2 K bytes (220 / 224 files) (784.1 K / 796.2 K bytes)
2026-04-17 15:18:59,958 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_apcorr_0004.rmap 4.0 K bytes (221 / 224 files) (785.2 K / 796.2 K bytes)
2026-04-17 15:19:00,052 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_abvegaoffset_0002.rmap 1.3 K bytes (222 / 224 files) (789.2 K / 796.2 K bytes)
2026-04-17 15:19:00,146 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_fgs_0125.imap 5.1 K bytes (223 / 224 files) (790.5 K / 796.2 K bytes)
2026-04-17 15:19:00,226 - CRDS - INFO - Fetching /home/runner/crds/mappings/jwst/jwst_1535.pmap 580 bytes (224 / 224 files) (795.6 K / 796.2 K bytes)
2026-04-17 15:19:00,612 - CRDS - ERROR - Error determining best reference for 'pars-darkcurrentstep' = No match found.
2026-04-17 15:19:00,617 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-jumpstep_0003.asdf 1.8 K bytes (1 / 1 files) (0 / 1.8 K bytes)
2026-04-17 15:19:00,706 - stpipe.step - INFO - PARS-JUMPSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-jumpstep_0003.asdf
2026-04-17 15:19:00,723 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-detector1pipeline_0001.asdf 1.5 K bytes (1 / 1 files) (0 / 1.5 K bytes)
2026-04-17 15:19:00,821 - stpipe.pipeline - INFO - PARS-DETECTOR1PIPELINE parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-detector1pipeline_0001.asdf
2026-04-17 15:19:00,838 - stpipe.step - INFO - Detector1Pipeline instance created.
2026-04-17 15:19:00,839 - stpipe.step - INFO - GroupScaleStep instance created.
2026-04-17 15:19:00,840 - stpipe.step - INFO - DQInitStep instance created.
2026-04-17 15:19:00,841 - stpipe.step - INFO - EmiCorrStep instance created.
2026-04-17 15:19:00,842 - stpipe.step - INFO - SaturationStep instance created.
2026-04-17 15:19:00,843 - stpipe.step - INFO - IPCStep instance created.
2026-04-17 15:19:00,844 - stpipe.step - INFO - SuperBiasStep instance created.
2026-04-17 15:19:00,845 - stpipe.step - INFO - RefPixStep instance created.
2026-04-17 15:19:00,846 - stpipe.step - INFO - RscdStep instance created.
2026-04-17 15:19:00,847 - stpipe.step - INFO - FirstFrameStep instance created.
2026-04-17 15:19:00,848 - stpipe.step - INFO - LastFrameStep instance created.
2026-04-17 15:19:00,849 - stpipe.step - INFO - LinearityStep instance created.
2026-04-17 15:19:00,850 - stpipe.step - INFO - DarkCurrentStep instance created.
2026-04-17 15:19:00,851 - stpipe.step - INFO - ResetStep instance created.
2026-04-17 15:19:00,852 - stpipe.step - INFO - PersistenceStep instance created.
2026-04-17 15:19:00,853 - stpipe.step - INFO - ChargeMigrationStep instance created.
2026-04-17 15:19:00,855 - stpipe.step - INFO - JumpStep instance created.
2026-04-17 15:19:00,856 - stpipe.step - INFO - PictureFrameStep instance created.
2026-04-17 15:19:00,857 - stpipe.step - INFO - CleanFlickerNoiseStep instance created.
2026-04-17 15:19:00,858 - stpipe.step - INFO - RampFitStep instance created.
2026-04-17 15:19:00,859 - stpipe.step - INFO - GainScaleStep instance created.
2026-04-17 15:19:01,009 - stpipe.step - INFO - Step Detector1Pipeline running with args ('./nrc_tso_demo_data/01366/Obs002/uncal/jw01366002001_04103_00001-seg001_nrcalong_uncal.fits',).
2026-04-17 15:19:01,032 - stpipe.step - INFO - Step Detector1Pipeline parameters are:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: ./nrc_tso_demo_data/01366/Obs002/stage1
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: True
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_calibrated_ramp: True
steps:
group_scale:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dq_init:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
user_supplied_dq: None
emicorr:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
algorithm: joint
nints_to_phase: None
nbins: None
scale_reference: True
onthefly_corr_freq: None
use_n_cycles: 3
fit_ints_separately: False
user_supplied_reffile: None
save_intermediate_results: False
saturation:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
n_pix_grow_sat: 1
use_readpatt: True
ipc:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
superbias:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
refpix:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
odd_even_columns: True
use_side_ref_pixels: True
side_smoothing_length: 11
side_gain: 1.0
odd_even_rows: True
ovr_corr_mitigation_ftr: 3.0
preserve_irs2_refpix: False
irs2_mean_subtraction: False
refpix_algorithm: median
sigreject: 4.0
gaussmooth: 1.0
halfwidth: 30
rscd:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
firstframe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
bright_use_group1: True
lastframe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
linearity:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dark_current:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dark_output: None
average_dark_current: None
reset:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
persistence:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
input_trapsfilled: ''
flag_pers_cutoff: 40.0
save_persistence: False
save_trapsfilled: True
modify_input: False
charge_migration:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
signal_threshold: 25000.0
jump:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
rejection_threshold: 6.0
three_group_rejection_threshold: 6.0
four_group_rejection_threshold: 5.0
maximum_cores: half
flag_4_neighbors: True
max_jump_to_flag_neighbors: 1000
min_jump_to_flag_neighbors: 30
after_jump_flag_dn1: 0
after_jump_flag_time1: 0
after_jump_flag_dn2: 0
after_jump_flag_time2: 0
expand_large_events: False
min_sat_area: 1
min_jump_area: 5
expand_factor: 2
use_ellipses: False
sat_required_snowball: True
min_sat_radius_extend: 2.5
sat_expand: 2
edge_size: 25
mask_snowball_core_next_int: True
snowball_time_masked_next_int: 4000
find_showers: False
max_shower_amplitude: 4.0
extend_snr_threshold: 0.0
extend_min_area: 0
extend_inner_radius: 0
extend_outer_radius: 0.0
extend_ellipse_expand_ratio: 0.0
time_masked_after_shower: 0
min_diffs_single_pass: 10
max_extended_radius: 200
minimum_groups: 3
minimum_sigclip_groups: 100
only_use_ints: True
picture_frame:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
mask_science_regions: True
n_sigma: 2.0
save_mask: False
save_correction: False
clean_flicker_noise:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
autoparam: False
fit_method: median
fit_by_channel: False
background_method: median
background_box_size: None
mask_science_regions: False
apply_flat_field: False
n_sigma: 2.0
fit_histogram: False
single_mask: True
user_mask: None
save_mask: False
save_background: False
save_noise: False
ramp_fit:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
algorithm: OLS_C
int_name: ''
save_opt: False
opt_name: ''
suppress_one_group: True
firstgroup: None
lastgroup: None
maximum_cores: half
gain_scale:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
2026-04-17 15:19:01,059 - stpipe.pipeline - INFO - Prefetching reference files for dataset: 'jw01366002001_04103_00001-seg001_nrcalong_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'readnoise', 'refpix', 'reset', 'saturation', 'sirskernel', 'superbias']
2026-04-17 15:19:01,063 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0444.fits 337.7 M bytes (1 / 8 files) (0 / 565.0 M bytes)
2026-04-17 15:19:04,744 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits 16.8 M bytes (2 / 8 files) (337.7 M / 565.0 M bytes)
2026-04-17 15:19:05,202 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0052.fits 151.0 M bytes (3 / 8 files) (354.5 M / 565.0 M bytes)
2026-04-17 15:19:06,709 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0076.fits 16.8 M bytes (4 / 8 files) (505.5 M / 565.0 M bytes)
2026-04-17 15:19:07,205 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0227.fits 2.1 M bytes (5 / 8 files) (522.3 M / 565.0 M bytes)
2026-04-17 15:19:07,497 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0097.fits 33.6 M bytes (6 / 8 files) (524.4 M / 565.0 M bytes)
2026-04-17 15:19:08,068 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_sirskernel_0002.asdf 661.0 K bytes (7 / 8 files) (558.0 M / 565.0 M bytes)
2026-04-17 15:19:08,304 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0210.fits 6.3 M bytes (8 / 8 files) (558.6 M / 565.0 M bytes)
2026-04-17 15:19:08,674 - stpipe.pipeline - INFO - Prefetch for DARK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0444.fits'.
2026-04-17 15:19:08,675 - stpipe.pipeline - INFO - Prefetch for GAIN reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits'.
2026-04-17 15:19:08,675 - stpipe.pipeline - INFO - Prefetch for LINEARITY reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0052.fits'.
2026-04-17 15:19:08,676 - stpipe.pipeline - INFO - Prefetch for MASK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0076.fits'.
2026-04-17 15:19:08,677 - stpipe.pipeline - INFO - Prefetch for READNOISE reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0227.fits'.
2026-04-17 15:19:08,677 - stpipe.pipeline - INFO - Prefetch for REFPIX reference file is 'N/A'.
2026-04-17 15:19:08,679 - stpipe.pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2026-04-17 15:19:08,679 - stpipe.pipeline - INFO - Prefetch for SATURATION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0097.fits'.
2026-04-17 15:19:08,679 - stpipe.pipeline - INFO - Prefetch for SIRSKERNEL reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_sirskernel_0002.asdf'.
2026-04-17 15:19:08,680 - stpipe.pipeline - INFO - Prefetch for SUPERBIAS reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0210.fits'.
2026-04-17 15:19:08,681 - jwst.pipeline.calwebb_detector1 - INFO - Starting calwebb_detector1 ...
2026-04-17 15:19:10,700 - stpipe.step - INFO - Step group_scale running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg001_nrcalong_uncal.fits>,).
2026-04-17 15:19:10,702 - jwst.group_scale.group_scale_step - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2026-04-17 15:19:10,703 - jwst.group_scale.group_scale_step - INFO - Step will be skipped
2026-04-17 15:19:10,705 - stpipe.step - INFO - Step group_scale done
2026-04-17 15:19:10,843 - stpipe.step - INFO - Step dq_init running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg001_nrcalong_uncal.fits>,).
2026-04-17 15:19:10,854 - jwst.dq_init.dq_init_step - INFO - Using MASK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0076.fits
2026-04-17 15:19:10,925 - stdatamodels.dynamicdq - WARNING - Keyword UNRELIABLE_ERROR does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:19:10,988 - jwst.dq_init.dq_initialization - INFO - Extracting mask subarray to match science data
2026-04-17 15:19:11,189 - CRDS - INFO - Calibration SW Found: jwst 2.0.0 (/home/runner/micromamba/envs/ci-env/lib/python3.11/site-packages/jwst-2.0.0.dist-info)
2026-04-17 15:19:12,780 - stpipe.step - INFO - Step dq_init done
2026-04-17 15:19:12,934 - stpipe.step - INFO - Step saturation running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg001_nrcalong_uncal.fits>,).
2026-04-17 15:19:12,941 - jwst.saturation.saturation_step - INFO - Using SATURATION reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0097.fits
2026-04-17 15:19:12,942 - jwst.saturation.saturation_step - INFO - Using SUPERBIAS reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0210.fits
2026-04-17 15:19:13,004 - stdatamodels.dynamicdq - WARNING - Keyword RESERVED_4 does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:19:13,005 - stdatamodels.dynamicdq - WARNING - Keyword UNRELIABLE_ERROR does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:19:13,046 - stdatamodels.dynamicdq - WARNING - Keyword UNRELIABLE_RESET does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:19:13,103 - stdatamodels.dynamicdq - WARNING - Keyword UNRELIABLE_ERROR does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:19:13,118 - jwst.saturation.saturation - INFO - Extracting reference file subarray to match science data
2026-04-17 15:19:13,137 - jwst.saturation.saturation - INFO - Using read_pattern with nframes 4
2026-04-17 15:19:18,524 - stcal.saturation.saturation - INFO - Detected 3043 saturated pixels
2026-04-17 15:19:18,632 - stcal.saturation.saturation - INFO - Detected 0 A/D floor pixels
2026-04-17 15:19:18,651 - stpipe.step - INFO - Step saturation done
2026-04-17 15:19:18,808 - stpipe.step - INFO - Step ipc running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg001_nrcalong_uncal.fits>,).
2026-04-17 15:19:18,809 - stpipe.step - INFO - Step skipped.
2026-04-17 15:19:18,957 - stpipe.step - INFO - Step superbias running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg001_nrcalong_uncal.fits>,).
2026-04-17 15:19:18,967 - jwst.superbias.superbias_step - INFO - Using SUPERBIAS reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0210.fits
2026-04-17 15:19:19,008 - stdatamodels.dynamicdq - WARNING - Keyword UNRELIABLE_ERROR does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:19:19,354 - stpipe.step - INFO - Step superbias done
2026-04-17 15:19:19,514 - stpipe.step - INFO - Step refpix running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg001_nrcalong_uncal.fits>,).
2026-04-17 15:19:19,517 - jwst.refpix.reference_pixels - INFO - NIR subarray data
2026-04-17 15:19:19,519 - jwst.refpix.reference_pixels - INFO - 4 readout amplifiers used
2026-04-17 15:19:19,520 - jwst.refpix.reference_pixels - INFO - The following parameters are valid for this mode:
2026-04-17 15:19:19,520 - jwst.refpix.reference_pixels - INFO - odd_even_columns = True
2026-04-17 15:19:19,521 - jwst.refpix.reference_pixels - INFO - use_side_ref_pixels = True
2026-04-17 15:19:19,522 - jwst.refpix.reference_pixels - INFO - side_smoothing_length = 11
2026-04-17 15:19:19,523 - jwst.refpix.reference_pixels - INFO - side_gain = 1.0
2026-04-17 15:19:19,523 - jwst.refpix.reference_pixels - INFO - The following parameters are not applicable and are ignored
2026-04-17 15:19:19,524 - jwst.refpix.reference_pixels - INFO - odd_even_rows = False
2026-04-17 15:21:00,405 - jwst.refpix.reference_pixels - INFO - Processing the zero frame
2026-04-17 15:21:08,960 - stpipe.step - INFO - Step refpix done
2026-04-17 15:21:09,137 - stpipe.step - INFO - Step linearity running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg001_nrcalong_uncal.fits>,).
2026-04-17 15:21:09,146 - jwst.linearity.linearity_step - INFO - Using Linearity reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0052.fits
2026-04-17 15:21:09,211 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_DO_NOT_USE does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:21:09,212 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_NO_LIN_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:21:09,213 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_BAD_LIN_FIT does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:21:09,213 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_NO_WELL_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:21:09,215 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_MODEL_FIT_FAIL does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:21:09,215 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_WELL_NOT_DEFINED does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:21:09,216 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_MASTER_MASK does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:21:11,826 - stpipe.step - INFO - Step linearity done
2026-04-17 15:21:12,001 - stpipe.step - INFO - Step persistence running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg001_nrcalong_uncal.fits>,).
2026-04-17 15:21:12,002 - stpipe.step - INFO - Step skipped.
2026-04-17 15:21:12,175 - stpipe.step - INFO - Step dark_current running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg001_nrcalong_uncal.fits>,).
2026-04-17 15:21:12,185 - jwst.dark_current.dark_current_step - INFO - Using DARK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0444.fits
2026-04-17 15:21:12,269 - stdatamodels.dynamicdq - WARNING - Keyword UNRELIABLE_ERROR does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:21:12,286 - stcal.dark_current.dark_sub - INFO - Science data nints=92, ngroups=12, nframes=4, groupgap=1
2026-04-17 15:21:12,287 - stcal.dark_current.dark_sub - INFO - Dark data nints=1, ngroups=80, nframes=1, groupgap=0
2026-04-17 15:21:13,669 - stpipe.step - INFO - Step dark_current done
2026-04-17 15:21:13,821 - stpipe.step - INFO - Step charge_migration running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg001_nrcalong_uncal.fits>,).
2026-04-17 15:21:13,822 - stpipe.step - INFO - Step skipped.
2026-04-17 15:21:13,968 - stpipe.step - INFO - Step jump running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg001_nrcalong_uncal.fits>,).
2026-04-17 15:21:13,969 - jwst.jump.jump_step - INFO - CR rejection threshold = 6 sigma
2026-04-17 15:21:13,970 - jwst.jump.jump_step - INFO - Maximum cores to use = half
2026-04-17 15:21:13,976 - jwst.jump.jump_step - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits
2026-04-17 15:21:13,979 - jwst.jump.jump_step - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0227.fits
2026-04-17 15:21:14,011 - jwst.jump.jump_step - INFO - Extracting gain subarray to match science data
2026-04-17 15:21:14,312 - stcal.jump.jump - INFO - Executing two-point difference method
2026-04-17 15:21:14,313 - stcal.jump.jump - INFO - Creating 2 processes for jump detection
2026-04-17 15:21:42,470 - stcal.jump.jump - INFO - Total elapsed time = 28.1524 sec
2026-04-17 15:21:42,715 - jwst.jump.jump_step - INFO - The execution time in seconds: 28.746146
2026-04-17 15:21:42,731 - stpipe.step - INFO - Step jump done
2026-04-17 15:21:45,356 - stpipe.step - INFO - Step picture_frame running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg001_nrcalong_uncal.fits>,).
2026-04-17 15:21:45,357 - stpipe.step - INFO - Step skipped.
2026-04-17 15:21:45,517 - stpipe.step - INFO - Step clean_flicker_noise running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg001_nrcalong_uncal.fits>,).
2026-04-17 15:21:45,519 - stpipe.step - INFO - Step skipped.
2026-04-17 15:21:49,320 - stpipe.step - INFO - Saved model in ./nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg001_nrcalong_ramp.fits
2026-04-17 15:21:49,472 - stpipe.step - INFO - Step ramp_fit running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg001_nrcalong_ramp.fits>,).
2026-04-17 15:21:49,487 - jwst.ramp_fitting.ramp_fit_step - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0227.fits
2026-04-17 15:21:49,488 - jwst.ramp_fitting.ramp_fit_step - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits
2026-04-17 15:21:50,476 - jwst.ramp_fitting.ramp_fit_step - INFO - Extracting gain subarray to match science data
2026-04-17 15:21:50,490 - jwst.ramp_fitting.ramp_fit_step - INFO - Using algorithm = OLS_C
2026-04-17 15:21:50,491 - jwst.ramp_fitting.ramp_fit_step - INFO - Using weighting = optimal
2026-04-17 15:21:51,775 - stcal.ramp_fitting.ols_fit - INFO - Number of multiprocessing slices: 2
2026-04-17 15:21:51,783 - stcal.ramp_fitting.ols_fit - INFO - Number of processors used for multiprocessing: 2
2026-04-17 15:22:21,146 - stpipe.step - INFO - Step ramp_fit done
2026-04-17 15:22:23,253 - stpipe.step - INFO - Step gain_scale running with args (<ImageModel(256, 2048) from jw01366002001_04103_00001-seg001_nrcalong_ramp.fits>,).
2026-04-17 15:22:23,261 - jwst.gain_scale.gain_scale_step - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits
2026-04-17 15:22:23,280 - jwst.gain_scale.gain_scale_step - INFO - GAINFACT not found in gain reference file
2026-04-17 15:22:23,281 - jwst.gain_scale.gain_scale_step - INFO - Step will be skipped
2026-04-17 15:22:23,284 - stpipe.step - INFO - Step gain_scale done
2026-04-17 15:22:23,425 - stpipe.step - INFO - Step gain_scale running with args (<CubeModel(92, 256, 2048) from jw01366002001_04103_00001-seg001_nrcalong_ramp.fits>,).
2026-04-17 15:22:23,428 - jwst.gain_scale.gain_scale_step - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits
2026-04-17 15:22:23,445 - jwst.gain_scale.gain_scale_step - INFO - GAINFACT not found in gain reference file
2026-04-17 15:22:23,445 - jwst.gain_scale.gain_scale_step - INFO - Step will be skipped
2026-04-17 15:22:23,449 - stpipe.step - INFO - Step gain_scale done
2026-04-17 15:22:23,827 - stpipe.step - INFO - Saved model in ./nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg001_nrcalong_rateints.fits
2026-04-17 15:22:23,828 - jwst.pipeline.calwebb_detector1 - INFO - ... ending calwebb_detector1
2026-04-17 15:22:23,833 - jwst.stpipe.core - INFO - Results used CRDS context: jwst_1535.pmap
2026-04-17 15:22:23,885 - stpipe.step - INFO - Saved model in ./nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg001_nrcalong_rate.fits
2026-04-17 15:22:23,886 - stpipe.step - INFO - Step Detector1Pipeline done
2026-04-17 15:22:23,887 - jwst.stpipe.core - INFO - Results used jwst version: 2.0.0
2026-04-17 15:22:23,924 - CRDS - ERROR - Error determining best reference for 'pars-darkcurrentstep' = No match found.
2026-04-17 15:22:23,928 - stpipe.step - INFO - PARS-JUMPSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-jumpstep_0003.asdf
2026-04-17 15:22:23,944 - stpipe.pipeline - INFO - PARS-DETECTOR1PIPELINE parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-detector1pipeline_0001.asdf
2026-04-17 15:22:23,966 - stpipe.step - INFO - Detector1Pipeline instance created.
2026-04-17 15:22:23,967 - stpipe.step - INFO - GroupScaleStep instance created.
2026-04-17 15:22:23,969 - stpipe.step - INFO - DQInitStep instance created.
2026-04-17 15:22:23,972 - stpipe.step - INFO - EmiCorrStep instance created.
2026-04-17 15:22:23,974 - stpipe.step - INFO - SaturationStep instance created.
2026-04-17 15:22:23,975 - stpipe.step - INFO - IPCStep instance created.
2026-04-17 15:22:23,977 - stpipe.step - INFO - SuperBiasStep instance created.
2026-04-17 15:22:23,981 - stpipe.step - INFO - RefPixStep instance created.
2026-04-17 15:22:23,982 - stpipe.step - INFO - RscdStep instance created.
2026-04-17 15:22:23,983 - stpipe.step - INFO - FirstFrameStep instance created.
2026-04-17 15:22:23,984 - stpipe.step - INFO - LastFrameStep instance created.
2026-04-17 15:22:23,985 - stpipe.step - INFO - LinearityStep instance created.
2026-04-17 15:22:23,987 - stpipe.step - INFO - DarkCurrentStep instance created.
2026-04-17 15:22:23,988 - stpipe.step - INFO - ResetStep instance created.
2026-04-17 15:22:23,991 - stpipe.step - INFO - PersistenceStep instance created.
2026-04-17 15:22:23,992 - stpipe.step - INFO - ChargeMigrationStep instance created.
2026-04-17 15:22:23,999 - stpipe.step - INFO - JumpStep instance created.
2026-04-17 15:22:24,001 - stpipe.step - INFO - PictureFrameStep instance created.
2026-04-17 15:22:24,004 - stpipe.step - INFO - CleanFlickerNoiseStep instance created.
2026-04-17 15:22:24,005 - stpipe.step - INFO - RampFitStep instance created.
2026-04-17 15:22:24,006 - stpipe.step - INFO - GainScaleStep instance created.
2026-04-17 15:22:24,158 - stpipe.step - INFO - Step Detector1Pipeline running with args ('./nrc_tso_demo_data/01366/Obs002/uncal/jw01366002001_04103_00001-seg002_nrcalong_uncal.fits',).
2026-04-17 15:22:24,180 - stpipe.step - INFO - Step Detector1Pipeline parameters are:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: ./nrc_tso_demo_data/01366/Obs002/stage1
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: True
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_calibrated_ramp: True
steps:
group_scale:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dq_init:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
user_supplied_dq: None
emicorr:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
algorithm: joint
nints_to_phase: None
nbins: None
scale_reference: True
onthefly_corr_freq: None
use_n_cycles: 3
fit_ints_separately: False
user_supplied_reffile: None
save_intermediate_results: False
saturation:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
n_pix_grow_sat: 1
use_readpatt: True
ipc:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
superbias:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
refpix:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
odd_even_columns: True
use_side_ref_pixels: True
side_smoothing_length: 11
side_gain: 1.0
odd_even_rows: True
ovr_corr_mitigation_ftr: 3.0
preserve_irs2_refpix: False
irs2_mean_subtraction: False
refpix_algorithm: median
sigreject: 4.0
gaussmooth: 1.0
halfwidth: 30
rscd:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
firstframe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
bright_use_group1: True
lastframe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
linearity:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dark_current:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dark_output: None
average_dark_current: None
reset:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
persistence:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
input_trapsfilled: ''
flag_pers_cutoff: 40.0
save_persistence: False
save_trapsfilled: True
modify_input: False
charge_migration:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
signal_threshold: 25000.0
jump:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
rejection_threshold: 6.0
three_group_rejection_threshold: 6.0
four_group_rejection_threshold: 5.0
maximum_cores: half
flag_4_neighbors: True
max_jump_to_flag_neighbors: 1000
min_jump_to_flag_neighbors: 30
after_jump_flag_dn1: 0
after_jump_flag_time1: 0
after_jump_flag_dn2: 0
after_jump_flag_time2: 0
expand_large_events: False
min_sat_area: 1
min_jump_area: 5
expand_factor: 2
use_ellipses: False
sat_required_snowball: True
min_sat_radius_extend: 2.5
sat_expand: 2
edge_size: 25
mask_snowball_core_next_int: True
snowball_time_masked_next_int: 4000
find_showers: False
max_shower_amplitude: 4.0
extend_snr_threshold: 0.0
extend_min_area: 0
extend_inner_radius: 0
extend_outer_radius: 0.0
extend_ellipse_expand_ratio: 0.0
time_masked_after_shower: 0
min_diffs_single_pass: 10
max_extended_radius: 200
minimum_groups: 3
minimum_sigclip_groups: 100
only_use_ints: True
picture_frame:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
mask_science_regions: True
n_sigma: 2.0
save_mask: False
save_correction: False
clean_flicker_noise:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
autoparam: False
fit_method: median
fit_by_channel: False
background_method: median
background_box_size: None
mask_science_regions: False
apply_flat_field: False
n_sigma: 2.0
fit_histogram: False
single_mask: True
user_mask: None
save_mask: False
save_background: False
save_noise: False
ramp_fit:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
algorithm: OLS_C
int_name: ''
save_opt: False
opt_name: ''
suppress_one_group: True
firstgroup: None
lastgroup: None
maximum_cores: half
gain_scale:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
2026-04-17 15:22:24,209 - stpipe.pipeline - INFO - Prefetching reference files for dataset: 'jw01366002001_04103_00001-seg002_nrcalong_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'readnoise', 'refpix', 'reset', 'saturation', 'sirskernel', 'superbias']
2026-04-17 15:22:24,213 - stpipe.pipeline - INFO - Prefetch for DARK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0444.fits'.
2026-04-17 15:22:24,213 - stpipe.pipeline - INFO - Prefetch for GAIN reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits'.
2026-04-17 15:22:24,214 - stpipe.pipeline - INFO - Prefetch for LINEARITY reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0052.fits'.
2026-04-17 15:22:24,215 - stpipe.pipeline - INFO - Prefetch for MASK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0076.fits'.
2026-04-17 15:22:24,215 - stpipe.pipeline - INFO - Prefetch for READNOISE reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0227.fits'.
2026-04-17 15:22:24,216 - stpipe.pipeline - INFO - Prefetch for REFPIX reference file is 'N/A'.
2026-04-17 15:22:24,216 - stpipe.pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2026-04-17 15:22:24,217 - stpipe.pipeline - INFO - Prefetch for SATURATION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0097.fits'.
2026-04-17 15:22:24,218 - stpipe.pipeline - INFO - Prefetch for SIRSKERNEL reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_sirskernel_0002.asdf'.
2026-04-17 15:22:24,218 - stpipe.pipeline - INFO - Prefetch for SUPERBIAS reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0210.fits'.
2026-04-17 15:22:24,220 - jwst.pipeline.calwebb_detector1 - INFO - Starting calwebb_detector1 ...
2026-04-17 15:22:27,789 - stpipe.step - INFO - Step group_scale running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg002_nrcalong_uncal.fits>,).
2026-04-17 15:22:27,790 - jwst.group_scale.group_scale_step - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2026-04-17 15:22:27,791 - jwst.group_scale.group_scale_step - INFO - Step will be skipped
2026-04-17 15:22:27,793 - stpipe.step - INFO - Step group_scale done
2026-04-17 15:22:27,933 - stpipe.step - INFO - Step dq_init running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg002_nrcalong_uncal.fits>,).
2026-04-17 15:22:27,936 - jwst.dq_init.dq_init_step - INFO - Using MASK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0076.fits
2026-04-17 15:22:28,765 - stdatamodels.dynamicdq - WARNING - Keyword UNRELIABLE_ERROR does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:22:28,832 - jwst.dq_init.dq_initialization - INFO - Extracting mask subarray to match science data
2026-04-17 15:22:29,031 - stpipe.step - INFO - Step dq_init done
2026-04-17 15:22:29,282 - stpipe.step - INFO - Step saturation running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg002_nrcalong_uncal.fits>,).
2026-04-17 15:22:29,287 - jwst.saturation.saturation_step - INFO - Using SATURATION reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0097.fits
2026-04-17 15:22:29,288 - jwst.saturation.saturation_step - INFO - Using SUPERBIAS reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0210.fits
2026-04-17 15:22:30,067 - stdatamodels.dynamicdq - WARNING - Keyword RESERVED_4 does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:22:30,068 - stdatamodels.dynamicdq - WARNING - Keyword UNRELIABLE_ERROR does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:22:30,117 - stdatamodels.dynamicdq - WARNING - Keyword UNRELIABLE_RESET does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:22:30,271 - stdatamodels.dynamicdq - WARNING - Keyword UNRELIABLE_ERROR does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:22:30,284 - jwst.saturation.saturation - INFO - Extracting reference file subarray to match science data
2026-04-17 15:22:30,303 - jwst.saturation.saturation - INFO - Using read_pattern with nframes 4
2026-04-17 15:22:35,606 - stcal.saturation.saturation - INFO - Detected 3312 saturated pixels
2026-04-17 15:22:35,710 - stcal.saturation.saturation - INFO - Detected 0 A/D floor pixels
2026-04-17 15:22:35,729 - stpipe.step - INFO - Step saturation done
2026-04-17 15:22:35,881 - stpipe.step - INFO - Step ipc running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg002_nrcalong_uncal.fits>,).
2026-04-17 15:22:35,883 - stpipe.step - INFO - Step skipped.
2026-04-17 15:22:36,023 - stpipe.step - INFO - Step superbias running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg002_nrcalong_uncal.fits>,).
2026-04-17 15:22:36,026 - jwst.superbias.superbias_step - INFO - Using SUPERBIAS reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0210.fits
2026-04-17 15:22:36,067 - stdatamodels.dynamicdq - WARNING - Keyword UNRELIABLE_ERROR does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:22:36,408 - stpipe.step - INFO - Step superbias done
2026-04-17 15:22:36,561 - stpipe.step - INFO - Step refpix running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg002_nrcalong_uncal.fits>,).
2026-04-17 15:22:36,564 - jwst.refpix.reference_pixels - INFO - NIR subarray data
2026-04-17 15:22:36,566 - jwst.refpix.reference_pixels - INFO - 4 readout amplifiers used
2026-04-17 15:22:36,567 - jwst.refpix.reference_pixels - INFO - The following parameters are valid for this mode:
2026-04-17 15:22:36,568 - jwst.refpix.reference_pixels - INFO - odd_even_columns = True
2026-04-17 15:22:36,568 - jwst.refpix.reference_pixels - INFO - use_side_ref_pixels = True
2026-04-17 15:22:36,569 - jwst.refpix.reference_pixels - INFO - side_smoothing_length = 11
2026-04-17 15:22:36,570 - jwst.refpix.reference_pixels - INFO - side_gain = 1.0
2026-04-17 15:22:36,571 - jwst.refpix.reference_pixels - INFO - The following parameters are not applicable and are ignored
2026-04-17 15:22:36,571 - jwst.refpix.reference_pixels - INFO - odd_even_rows = False
2026-04-17 15:24:15,902 - jwst.refpix.reference_pixels - INFO - Processing the zero frame
2026-04-17 15:24:24,225 - stpipe.step - INFO - Step refpix done
2026-04-17 15:24:24,379 - stpipe.step - INFO - Step linearity running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg002_nrcalong_uncal.fits>,).
2026-04-17 15:24:24,382 - jwst.linearity.linearity_step - INFO - Using Linearity reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0052.fits
2026-04-17 15:24:24,502 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_DO_NOT_USE does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:24:24,503 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_NO_LIN_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:24:24,504 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_BAD_LIN_FIT does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:24:24,505 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_NO_WELL_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:24:24,506 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_MODEL_FIT_FAIL does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:24:24,508 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_WELL_NOT_DEFINED does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:24:24,509 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_MASTER_MASK does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:24:26,642 - stpipe.step - INFO - Step linearity done
2026-04-17 15:24:26,793 - stpipe.step - INFO - Step persistence running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg002_nrcalong_uncal.fits>,).
2026-04-17 15:24:26,794 - stpipe.step - INFO - Step skipped.
2026-04-17 15:24:26,932 - stpipe.step - INFO - Step dark_current running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg002_nrcalong_uncal.fits>,).
2026-04-17 15:24:26,936 - jwst.dark_current.dark_current_step - INFO - Using DARK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0444.fits
2026-04-17 15:24:27,492 - stdatamodels.dynamicdq - WARNING - Keyword UNRELIABLE_ERROR does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:24:27,509 - stcal.dark_current.dark_sub - INFO - Science data nints=92, ngroups=12, nframes=4, groupgap=1
2026-04-17 15:24:27,510 - stcal.dark_current.dark_sub - INFO - Dark data nints=1, ngroups=80, nframes=1, groupgap=0
2026-04-17 15:24:28,049 - stpipe.step - INFO - Step dark_current done
2026-04-17 15:24:28,208 - stpipe.step - INFO - Step charge_migration running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg002_nrcalong_uncal.fits>,).
2026-04-17 15:24:28,210 - stpipe.step - INFO - Step skipped.
2026-04-17 15:24:28,354 - stpipe.step - INFO - Step jump running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg002_nrcalong_uncal.fits>,).
2026-04-17 15:24:28,355 - jwst.jump.jump_step - INFO - CR rejection threshold = 6 sigma
2026-04-17 15:24:28,356 - jwst.jump.jump_step - INFO - Maximum cores to use = half
2026-04-17 15:24:28,359 - jwst.jump.jump_step - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits
2026-04-17 15:24:28,361 - jwst.jump.jump_step - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0227.fits
2026-04-17 15:24:28,393 - jwst.jump.jump_step - INFO - Extracting gain subarray to match science data
2026-04-17 15:24:28,692 - stcal.jump.jump - INFO - Executing two-point difference method
2026-04-17 15:24:28,693 - stcal.jump.jump - INFO - Creating 2 processes for jump detection
2026-04-17 15:24:54,597 - stcal.jump.jump - INFO - Total elapsed time = 25.9039 sec
2026-04-17 15:24:54,669 - jwst.jump.jump_step - INFO - The execution time in seconds: 26.314385
2026-04-17 15:24:54,673 - stpipe.step - INFO - Step jump done
2026-04-17 15:24:54,827 - stpipe.step - INFO - Step picture_frame running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg002_nrcalong_uncal.fits>,).
2026-04-17 15:24:54,828 - stpipe.step - INFO - Step skipped.
2026-04-17 15:24:54,968 - stpipe.step - INFO - Step clean_flicker_noise running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg002_nrcalong_uncal.fits>,).
2026-04-17 15:24:54,969 - stpipe.step - INFO - Step skipped.
2026-04-17 15:24:58,132 - stpipe.step - INFO - Saved model in ./nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg002_nrcalong_ramp.fits
2026-04-17 15:24:58,290 - stpipe.step - INFO - Step ramp_fit running with args (<RampModel(92, 12, 256, 2048) from jw01366002001_04103_00001-seg002_nrcalong_ramp.fits>,).
2026-04-17 15:24:58,295 - jwst.ramp_fitting.ramp_fit_step - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0227.fits
2026-04-17 15:24:58,296 - jwst.ramp_fitting.ramp_fit_step - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits
2026-04-17 15:24:59,086 - jwst.ramp_fitting.ramp_fit_step - INFO - Extracting gain subarray to match science data
2026-04-17 15:24:59,101 - jwst.ramp_fitting.ramp_fit_step - INFO - Using algorithm = OLS_C
2026-04-17 15:24:59,101 - jwst.ramp_fitting.ramp_fit_step - INFO - Using weighting = optimal
2026-04-17 15:25:00,406 - stcal.ramp_fitting.ols_fit - INFO - Number of multiprocessing slices: 2
2026-04-17 15:25:00,414 - stcal.ramp_fitting.ols_fit - INFO - Number of processors used for multiprocessing: 2
2026-04-17 15:25:27,381 - stpipe.step - INFO - Step ramp_fit done
2026-04-17 15:25:27,552 - stpipe.step - INFO - Step gain_scale running with args (<ImageModel(256, 2048) from jw01366002001_04103_00001-seg002_nrcalong_ramp.fits>,).
2026-04-17 15:25:27,556 - jwst.gain_scale.gain_scale_step - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits
2026-04-17 15:25:27,572 - jwst.gain_scale.gain_scale_step - INFO - GAINFACT not found in gain reference file
2026-04-17 15:25:27,573 - jwst.gain_scale.gain_scale_step - INFO - Step will be skipped
2026-04-17 15:25:27,576 - stpipe.step - INFO - Step gain_scale done
2026-04-17 15:25:27,720 - stpipe.step - INFO - Step gain_scale running with args (<CubeModel(92, 256, 2048) from jw01366002001_04103_00001-seg002_nrcalong_ramp.fits>,).
2026-04-17 15:25:27,723 - jwst.gain_scale.gain_scale_step - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits
2026-04-17 15:25:27,740 - jwst.gain_scale.gain_scale_step - INFO - GAINFACT not found in gain reference file
2026-04-17 15:25:27,740 - jwst.gain_scale.gain_scale_step - INFO - Step will be skipped
2026-04-17 15:25:27,744 - stpipe.step - INFO - Step gain_scale done
2026-04-17 15:25:28,119 - stpipe.step - INFO - Saved model in ./nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg002_nrcalong_rateints.fits
2026-04-17 15:25:28,120 - jwst.pipeline.calwebb_detector1 - INFO - ... ending calwebb_detector1
2026-04-17 15:25:28,125 - jwst.stpipe.core - INFO - Results used CRDS context: jwst_1535.pmap
2026-04-17 15:25:28,177 - stpipe.step - INFO - Saved model in ./nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg002_nrcalong_rate.fits
2026-04-17 15:25:28,178 - stpipe.step - INFO - Step Detector1Pipeline done
2026-04-17 15:25:28,179 - jwst.stpipe.core - INFO - Results used jwst version: 2.0.0
2026-04-17 15:25:28,213 - CRDS - ERROR - Error determining best reference for 'pars-darkcurrentstep' = No match found.
2026-04-17 15:25:28,218 - stpipe.step - INFO - PARS-JUMPSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-jumpstep_0003.asdf
2026-04-17 15:25:28,232 - stpipe.pipeline - INFO - PARS-DETECTOR1PIPELINE parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-detector1pipeline_0001.asdf
2026-04-17 15:25:28,249 - stpipe.step - INFO - Detector1Pipeline instance created.
2026-04-17 15:25:28,250 - stpipe.step - INFO - GroupScaleStep instance created.
2026-04-17 15:25:28,251 - stpipe.step - INFO - DQInitStep instance created.
2026-04-17 15:25:28,252 - stpipe.step - INFO - EmiCorrStep instance created.
2026-04-17 15:25:28,253 - stpipe.step - INFO - SaturationStep instance created.
2026-04-17 15:25:28,254 - stpipe.step - INFO - IPCStep instance created.
2026-04-17 15:25:28,255 - stpipe.step - INFO - SuperBiasStep instance created.
2026-04-17 15:25:28,256 - stpipe.step - INFO - RefPixStep instance created.
2026-04-17 15:25:28,257 - stpipe.step - INFO - RscdStep instance created.
2026-04-17 15:25:28,258 - stpipe.step - INFO - FirstFrameStep instance created.
2026-04-17 15:25:28,259 - stpipe.step - INFO - LastFrameStep instance created.
2026-04-17 15:25:28,260 - stpipe.step - INFO - LinearityStep instance created.
2026-04-17 15:25:28,260 - stpipe.step - INFO - DarkCurrentStep instance created.
2026-04-17 15:25:28,261 - stpipe.step - INFO - ResetStep instance created.
2026-04-17 15:25:28,262 - stpipe.step - INFO - PersistenceStep instance created.
2026-04-17 15:25:28,263 - stpipe.step - INFO - ChargeMigrationStep instance created.
2026-04-17 15:25:28,265 - stpipe.step - INFO - JumpStep instance created.
2026-04-17 15:25:28,265 - stpipe.step - INFO - PictureFrameStep instance created.
2026-04-17 15:25:28,266 - stpipe.step - INFO - CleanFlickerNoiseStep instance created.
2026-04-17 15:25:28,269 - stpipe.step - INFO - RampFitStep instance created.
2026-04-17 15:25:28,270 - stpipe.step - INFO - GainScaleStep instance created.
2026-04-17 15:25:28,424 - stpipe.step - INFO - Step Detector1Pipeline running with args ('./nrc_tso_demo_data/01366/Obs002/uncal/jw01366002001_04103_00001-seg003_nrcalong_uncal.fits',).
2026-04-17 15:25:28,446 - stpipe.step - INFO - Step Detector1Pipeline parameters are:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: ./nrc_tso_demo_data/01366/Obs002/stage1
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: True
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_calibrated_ramp: True
steps:
group_scale:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dq_init:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
user_supplied_dq: None
emicorr:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
algorithm: joint
nints_to_phase: None
nbins: None
scale_reference: True
onthefly_corr_freq: None
use_n_cycles: 3
fit_ints_separately: False
user_supplied_reffile: None
save_intermediate_results: False
saturation:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
n_pix_grow_sat: 1
use_readpatt: True
ipc:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
superbias:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
refpix:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
odd_even_columns: True
use_side_ref_pixels: True
side_smoothing_length: 11
side_gain: 1.0
odd_even_rows: True
ovr_corr_mitigation_ftr: 3.0
preserve_irs2_refpix: False
irs2_mean_subtraction: False
refpix_algorithm: median
sigreject: 4.0
gaussmooth: 1.0
halfwidth: 30
rscd:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
firstframe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
bright_use_group1: True
lastframe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
linearity:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dark_current:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dark_output: None
average_dark_current: None
reset:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
persistence:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
input_trapsfilled: ''
flag_pers_cutoff: 40.0
save_persistence: False
save_trapsfilled: True
modify_input: False
charge_migration:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
signal_threshold: 25000.0
jump:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
rejection_threshold: 6.0
three_group_rejection_threshold: 6.0
four_group_rejection_threshold: 5.0
maximum_cores: half
flag_4_neighbors: True
max_jump_to_flag_neighbors: 1000
min_jump_to_flag_neighbors: 30
after_jump_flag_dn1: 0
after_jump_flag_time1: 0
after_jump_flag_dn2: 0
after_jump_flag_time2: 0
expand_large_events: False
min_sat_area: 1
min_jump_area: 5
expand_factor: 2
use_ellipses: False
sat_required_snowball: True
min_sat_radius_extend: 2.5
sat_expand: 2
edge_size: 25
mask_snowball_core_next_int: True
snowball_time_masked_next_int: 4000
find_showers: False
max_shower_amplitude: 4.0
extend_snr_threshold: 0.0
extend_min_area: 0
extend_inner_radius: 0
extend_outer_radius: 0.0
extend_ellipse_expand_ratio: 0.0
time_masked_after_shower: 0
min_diffs_single_pass: 10
max_extended_radius: 200
minimum_groups: 3
minimum_sigclip_groups: 100
only_use_ints: True
picture_frame:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
mask_science_regions: True
n_sigma: 2.0
save_mask: False
save_correction: False
clean_flicker_noise:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
autoparam: False
fit_method: median
fit_by_channel: False
background_method: median
background_box_size: None
mask_science_regions: False
apply_flat_field: False
n_sigma: 2.0
fit_histogram: False
single_mask: True
user_mask: None
save_mask: False
save_background: False
save_noise: False
ramp_fit:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
algorithm: OLS_C
int_name: ''
save_opt: False
opt_name: ''
suppress_one_group: True
firstgroup: None
lastgroup: None
maximum_cores: half
gain_scale:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
2026-04-17 15:25:28,473 - stpipe.pipeline - INFO - Prefetching reference files for dataset: 'jw01366002001_04103_00001-seg003_nrcalong_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'readnoise', 'refpix', 'reset', 'saturation', 'sirskernel', 'superbias']
2026-04-17 15:25:28,477 - stpipe.pipeline - INFO - Prefetch for DARK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0444.fits'.
2026-04-17 15:25:28,477 - stpipe.pipeline - INFO - Prefetch for GAIN reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits'.
2026-04-17 15:25:28,478 - stpipe.pipeline - INFO - Prefetch for LINEARITY reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0052.fits'.
2026-04-17 15:25:28,479 - stpipe.pipeline - INFO - Prefetch for MASK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0076.fits'.
2026-04-17 15:25:28,479 - stpipe.pipeline - INFO - Prefetch for READNOISE reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0227.fits'.
2026-04-17 15:25:28,480 - stpipe.pipeline - INFO - Prefetch for REFPIX reference file is 'N/A'.
2026-04-17 15:25:28,480 - stpipe.pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2026-04-17 15:25:28,481 - stpipe.pipeline - INFO - Prefetch for SATURATION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0097.fits'.
2026-04-17 15:25:28,481 - stpipe.pipeline - INFO - Prefetch for SIRSKERNEL reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_sirskernel_0002.asdf'.
2026-04-17 15:25:28,483 - stpipe.pipeline - INFO - Prefetch for SUPERBIAS reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0210.fits'.
2026-04-17 15:25:28,484 - jwst.pipeline.calwebb_detector1 - INFO - Starting calwebb_detector1 ...
2026-04-17 15:25:32,588 - stpipe.step - INFO - Step group_scale running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg003_nrcalong_uncal.fits>,).
2026-04-17 15:25:32,589 - jwst.group_scale.group_scale_step - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2026-04-17 15:25:32,589 - jwst.group_scale.group_scale_step - INFO - Step will be skipped
2026-04-17 15:25:32,592 - stpipe.step - INFO - Step group_scale done
2026-04-17 15:25:32,734 - stpipe.step - INFO - Step dq_init running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg003_nrcalong_uncal.fits>,).
2026-04-17 15:25:32,737 - jwst.dq_init.dq_init_step - INFO - Using MASK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0076.fits
2026-04-17 15:25:32,798 - stdatamodels.dynamicdq - WARNING - Keyword UNRELIABLE_ERROR does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:25:32,859 - jwst.dq_init.dq_initialization - INFO - Extracting mask subarray to match science data
2026-04-17 15:25:33,056 - stpipe.step - INFO - Step dq_init done
2026-04-17 15:25:33,214 - stpipe.step - INFO - Step saturation running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg003_nrcalong_uncal.fits>,).
2026-04-17 15:25:33,218 - jwst.saturation.saturation_step - INFO - Using SATURATION reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0097.fits
2026-04-17 15:25:33,219 - jwst.saturation.saturation_step - INFO - Using SUPERBIAS reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0210.fits
2026-04-17 15:25:33,281 - stdatamodels.dynamicdq - WARNING - Keyword RESERVED_4 does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:25:33,283 - stdatamodels.dynamicdq - WARNING - Keyword UNRELIABLE_ERROR does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:25:33,323 - stdatamodels.dynamicdq - WARNING - Keyword UNRELIABLE_RESET does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:25:33,379 - stdatamodels.dynamicdq - WARNING - Keyword UNRELIABLE_ERROR does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:25:33,392 - jwst.saturation.saturation - INFO - Extracting reference file subarray to match science data
2026-04-17 15:25:33,411 - jwst.saturation.saturation - INFO - Using read_pattern with nframes 4
2026-04-17 15:25:38,622 - stcal.saturation.saturation - INFO - Detected 3456 saturated pixels
2026-04-17 15:25:38,720 - stcal.saturation.saturation - INFO - Detected 0 A/D floor pixels
2026-04-17 15:25:38,739 - stpipe.step - INFO - Step saturation done
2026-04-17 15:25:38,893 - stpipe.step - INFO - Step ipc running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg003_nrcalong_uncal.fits>,).
2026-04-17 15:25:38,894 - stpipe.step - INFO - Step skipped.
2026-04-17 15:25:39,034 - stpipe.step - INFO - Step superbias running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg003_nrcalong_uncal.fits>,).
2026-04-17 15:25:39,037 - jwst.superbias.superbias_step - INFO - Using SUPERBIAS reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0210.fits
2026-04-17 15:25:39,078 - stdatamodels.dynamicdq - WARNING - Keyword UNRELIABLE_ERROR does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:25:39,421 - stpipe.step - INFO - Step superbias done
2026-04-17 15:25:39,575 - stpipe.step - INFO - Step refpix running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg003_nrcalong_uncal.fits>,).
2026-04-17 15:25:39,577 - jwst.refpix.reference_pixels - INFO - NIR subarray data
2026-04-17 15:25:39,580 - jwst.refpix.reference_pixels - INFO - 4 readout amplifiers used
2026-04-17 15:25:39,580 - jwst.refpix.reference_pixels - INFO - The following parameters are valid for this mode:
2026-04-17 15:25:39,581 - jwst.refpix.reference_pixels - INFO - odd_even_columns = True
2026-04-17 15:25:39,582 - jwst.refpix.reference_pixels - INFO - use_side_ref_pixels = True
2026-04-17 15:25:39,583 - jwst.refpix.reference_pixels - INFO - side_smoothing_length = 11
2026-04-17 15:25:39,583 - jwst.refpix.reference_pixels - INFO - side_gain = 1.0
2026-04-17 15:25:39,584 - jwst.refpix.reference_pixels - INFO - The following parameters are not applicable and are ignored
2026-04-17 15:25:39,585 - jwst.refpix.reference_pixels - INFO - odd_even_rows = False
2026-04-17 15:27:18,099 - jwst.refpix.reference_pixels - INFO - Processing the zero frame
2026-04-17 15:27:26,346 - stpipe.step - INFO - Step refpix done
2026-04-17 15:27:26,503 - stpipe.step - INFO - Step linearity running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg003_nrcalong_uncal.fits>,).
2026-04-17 15:27:26,507 - jwst.linearity.linearity_step - INFO - Using Linearity reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0052.fits
2026-04-17 15:27:26,604 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_DO_NOT_USE does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:27:26,605 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_NO_LIN_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:27:26,606 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_BAD_LIN_FIT does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:27:26,607 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_NO_WELL_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:27:26,608 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_MODEL_FIT_FAIL does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:27:26,609 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_WELL_NOT_DEFINED does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:27:26,610 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_MASTER_MASK does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:27:28,715 - stpipe.step - INFO - Step linearity done
2026-04-17 15:27:28,870 - stpipe.step - INFO - Step persistence running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg003_nrcalong_uncal.fits>,).
2026-04-17 15:27:28,871 - stpipe.step - INFO - Step skipped.
2026-04-17 15:27:29,013 - stpipe.step - INFO - Step dark_current running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg003_nrcalong_uncal.fits>,).
2026-04-17 15:27:29,016 - jwst.dark_current.dark_current_step - INFO - Using DARK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0444.fits
2026-04-17 15:27:29,600 - stdatamodels.dynamicdq - WARNING - Keyword UNRELIABLE_ERROR does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:27:29,617 - stcal.dark_current.dark_sub - INFO - Science data nints=91, ngroups=12, nframes=4, groupgap=1
2026-04-17 15:27:29,618 - stcal.dark_current.dark_sub - INFO - Dark data nints=1, ngroups=80, nframes=1, groupgap=0
2026-04-17 15:27:30,076 - stpipe.step - INFO - Step dark_current done
2026-04-17 15:27:30,233 - stpipe.step - INFO - Step charge_migration running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg003_nrcalong_uncal.fits>,).
2026-04-17 15:27:30,234 - stpipe.step - INFO - Step skipped.
2026-04-17 15:27:30,374 - stpipe.step - INFO - Step jump running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg003_nrcalong_uncal.fits>,).
2026-04-17 15:27:30,375 - jwst.jump.jump_step - INFO - CR rejection threshold = 6 sigma
2026-04-17 15:27:30,376 - jwst.jump.jump_step - INFO - Maximum cores to use = half
2026-04-17 15:27:30,379 - jwst.jump.jump_step - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits
2026-04-17 15:27:30,382 - jwst.jump.jump_step - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0227.fits
2026-04-17 15:27:30,414 - jwst.jump.jump_step - INFO - Extracting gain subarray to match science data
2026-04-17 15:27:30,711 - stcal.jump.jump - INFO - Executing two-point difference method
2026-04-17 15:27:30,712 - stcal.jump.jump - INFO - Creating 2 processes for jump detection
2026-04-17 15:27:54,234 - stcal.jump.jump - INFO - Total elapsed time = 23.5222 sec
2026-04-17 15:27:54,306 - jwst.jump.jump_step - INFO - The execution time in seconds: 23.930757
2026-04-17 15:27:54,310 - stpipe.step - INFO - Step jump done
2026-04-17 15:27:54,468 - stpipe.step - INFO - Step picture_frame running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg003_nrcalong_uncal.fits>,).
2026-04-17 15:27:54,469 - stpipe.step - INFO - Step skipped.
2026-04-17 15:27:54,611 - stpipe.step - INFO - Step clean_flicker_noise running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg003_nrcalong_uncal.fits>,).
2026-04-17 15:27:54,612 - stpipe.step - INFO - Step skipped.
2026-04-17 15:27:57,822 - stpipe.step - INFO - Saved model in ./nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg003_nrcalong_ramp.fits
2026-04-17 15:27:57,982 - stpipe.step - INFO - Step ramp_fit running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg003_nrcalong_ramp.fits>,).
2026-04-17 15:27:57,988 - jwst.ramp_fitting.ramp_fit_step - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0227.fits
2026-04-17 15:27:57,989 - jwst.ramp_fitting.ramp_fit_step - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits
2026-04-17 15:27:58,458 - jwst.ramp_fitting.ramp_fit_step - INFO - Extracting gain subarray to match science data
2026-04-17 15:27:58,544 - jwst.ramp_fitting.ramp_fit_step - INFO - Using algorithm = OLS_C
2026-04-17 15:27:58,545 - jwst.ramp_fitting.ramp_fit_step - INFO - Using weighting = optimal
2026-04-17 15:27:59,806 - stcal.ramp_fitting.ols_fit - INFO - Number of multiprocessing slices: 2
2026-04-17 15:27:59,815 - stcal.ramp_fitting.ols_fit - INFO - Number of processors used for multiprocessing: 2
2026-04-17 15:28:25,965 - stpipe.step - INFO - Step ramp_fit done
2026-04-17 15:28:26,133 - stpipe.step - INFO - Step gain_scale running with args (<ImageModel(256, 2048) from jw01366002001_04103_00001-seg003_nrcalong_ramp.fits>,).
2026-04-17 15:28:26,136 - jwst.gain_scale.gain_scale_step - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits
2026-04-17 15:28:26,152 - jwst.gain_scale.gain_scale_step - INFO - GAINFACT not found in gain reference file
2026-04-17 15:28:26,153 - jwst.gain_scale.gain_scale_step - INFO - Step will be skipped
2026-04-17 15:28:26,156 - stpipe.step - INFO - Step gain_scale done
2026-04-17 15:28:26,310 - stpipe.step - INFO - Step gain_scale running with args (<CubeModel(91, 256, 2048) from jw01366002001_04103_00001-seg003_nrcalong_ramp.fits>,).
2026-04-17 15:28:26,313 - jwst.gain_scale.gain_scale_step - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits
2026-04-17 15:28:26,329 - jwst.gain_scale.gain_scale_step - INFO - GAINFACT not found in gain reference file
2026-04-17 15:28:26,330 - jwst.gain_scale.gain_scale_step - INFO - Step will be skipped
2026-04-17 15:28:26,333 - stpipe.step - INFO - Step gain_scale done
2026-04-17 15:28:26,701 - stpipe.step - INFO - Saved model in ./nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg003_nrcalong_rateints.fits
2026-04-17 15:28:26,702 - jwst.pipeline.calwebb_detector1 - INFO - ... ending calwebb_detector1
2026-04-17 15:28:26,707 - jwst.stpipe.core - INFO - Results used CRDS context: jwst_1535.pmap
2026-04-17 15:28:26,760 - stpipe.step - INFO - Saved model in ./nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg003_nrcalong_rate.fits
2026-04-17 15:28:26,760 - stpipe.step - INFO - Step Detector1Pipeline done
2026-04-17 15:28:26,761 - jwst.stpipe.core - INFO - Results used jwst version: 2.0.0
2026-04-17 15:28:26,795 - CRDS - ERROR - Error determining best reference for 'pars-darkcurrentstep' = No match found.
2026-04-17 15:28:26,800 - stpipe.step - INFO - PARS-JUMPSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-jumpstep_0003.asdf
2026-04-17 15:28:26,815 - stpipe.pipeline - INFO - PARS-DETECTOR1PIPELINE parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-detector1pipeline_0001.asdf
2026-04-17 15:28:26,831 - stpipe.step - INFO - Detector1Pipeline instance created.
2026-04-17 15:28:26,833 - stpipe.step - INFO - GroupScaleStep instance created.
2026-04-17 15:28:26,834 - stpipe.step - INFO - DQInitStep instance created.
2026-04-17 15:28:26,835 - stpipe.step - INFO - EmiCorrStep instance created.
2026-04-17 15:28:26,836 - stpipe.step - INFO - SaturationStep instance created.
2026-04-17 15:28:26,837 - stpipe.step - INFO - IPCStep instance created.
2026-04-17 15:28:26,838 - stpipe.step - INFO - SuperBiasStep instance created.
2026-04-17 15:28:26,839 - stpipe.step - INFO - RefPixStep instance created.
2026-04-17 15:28:26,840 - stpipe.step - INFO - RscdStep instance created.
2026-04-17 15:28:26,841 - stpipe.step - INFO - FirstFrameStep instance created.
2026-04-17 15:28:26,842 - stpipe.step - INFO - LastFrameStep instance created.
2026-04-17 15:28:26,843 - stpipe.step - INFO - LinearityStep instance created.
2026-04-17 15:28:26,844 - stpipe.step - INFO - DarkCurrentStep instance created.
2026-04-17 15:28:26,845 - stpipe.step - INFO - ResetStep instance created.
2026-04-17 15:28:26,846 - stpipe.step - INFO - PersistenceStep instance created.
2026-04-17 15:28:26,847 - stpipe.step - INFO - ChargeMigrationStep instance created.
2026-04-17 15:28:26,849 - stpipe.step - INFO - JumpStep instance created.
2026-04-17 15:28:26,850 - stpipe.step - INFO - PictureFrameStep instance created.
2026-04-17 15:28:26,851 - stpipe.step - INFO - CleanFlickerNoiseStep instance created.
2026-04-17 15:28:26,852 - stpipe.step - INFO - RampFitStep instance created.
2026-04-17 15:28:26,853 - stpipe.step - INFO - GainScaleStep instance created.
2026-04-17 15:28:27,006 - stpipe.step - INFO - Step Detector1Pipeline running with args ('./nrc_tso_demo_data/01366/Obs002/uncal/jw01366002001_04103_00001-seg004_nrcalong_uncal.fits',).
2026-04-17 15:28:27,027 - stpipe.step - INFO - Step Detector1Pipeline parameters are:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: ./nrc_tso_demo_data/01366/Obs002/stage1
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: True
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_calibrated_ramp: True
steps:
group_scale:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dq_init:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
user_supplied_dq: None
emicorr:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
algorithm: joint
nints_to_phase: None
nbins: None
scale_reference: True
onthefly_corr_freq: None
use_n_cycles: 3
fit_ints_separately: False
user_supplied_reffile: None
save_intermediate_results: False
saturation:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
n_pix_grow_sat: 1
use_readpatt: True
ipc:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
superbias:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
refpix:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
odd_even_columns: True
use_side_ref_pixels: True
side_smoothing_length: 11
side_gain: 1.0
odd_even_rows: True
ovr_corr_mitigation_ftr: 3.0
preserve_irs2_refpix: False
irs2_mean_subtraction: False
refpix_algorithm: median
sigreject: 4.0
gaussmooth: 1.0
halfwidth: 30
rscd:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
firstframe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
bright_use_group1: True
lastframe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
linearity:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dark_current:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
dark_output: None
average_dark_current: None
reset:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
persistence:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
input_trapsfilled: ''
flag_pers_cutoff: 40.0
save_persistence: False
save_trapsfilled: True
modify_input: False
charge_migration:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
signal_threshold: 25000.0
jump:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
rejection_threshold: 6.0
three_group_rejection_threshold: 6.0
four_group_rejection_threshold: 5.0
maximum_cores: half
flag_4_neighbors: True
max_jump_to_flag_neighbors: 1000
min_jump_to_flag_neighbors: 30
after_jump_flag_dn1: 0
after_jump_flag_time1: 0
after_jump_flag_dn2: 0
after_jump_flag_time2: 0
expand_large_events: False
min_sat_area: 1
min_jump_area: 5
expand_factor: 2
use_ellipses: False
sat_required_snowball: True
min_sat_radius_extend: 2.5
sat_expand: 2
edge_size: 25
mask_snowball_core_next_int: True
snowball_time_masked_next_int: 4000
find_showers: False
max_shower_amplitude: 4.0
extend_snr_threshold: 0.0
extend_min_area: 0
extend_inner_radius: 0
extend_outer_radius: 0.0
extend_ellipse_expand_ratio: 0.0
time_masked_after_shower: 0
min_diffs_single_pass: 10
max_extended_radius: 200
minimum_groups: 3
minimum_sigclip_groups: 100
only_use_ints: True
picture_frame:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
mask_science_regions: True
n_sigma: 2.0
save_mask: False
save_correction: False
clean_flicker_noise:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
autoparam: False
fit_method: median
fit_by_channel: False
background_method: median
background_box_size: None
mask_science_regions: False
apply_flat_field: False
n_sigma: 2.0
fit_histogram: False
single_mask: True
user_mask: None
save_mask: False
save_background: False
save_noise: False
ramp_fit:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
algorithm: OLS_C
int_name: ''
save_opt: False
opt_name: ''
suppress_one_group: True
firstgroup: None
lastgroup: None
maximum_cores: half
gain_scale:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
2026-04-17 15:28:27,054 - stpipe.pipeline - INFO - Prefetching reference files for dataset: 'jw01366002001_04103_00001-seg004_nrcalong_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'readnoise', 'refpix', 'reset', 'saturation', 'sirskernel', 'superbias']
2026-04-17 15:28:27,058 - stpipe.pipeline - INFO - Prefetch for DARK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0444.fits'.
2026-04-17 15:28:27,058 - stpipe.pipeline - INFO - Prefetch for GAIN reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits'.
2026-04-17 15:28:27,059 - stpipe.pipeline - INFO - Prefetch for LINEARITY reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0052.fits'.
2026-04-17 15:28:27,059 - stpipe.pipeline - INFO - Prefetch for MASK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0076.fits'.
2026-04-17 15:28:27,060 - stpipe.pipeline - INFO - Prefetch for READNOISE reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0227.fits'.
2026-04-17 15:28:27,061 - stpipe.pipeline - INFO - Prefetch for REFPIX reference file is 'N/A'.
2026-04-17 15:28:27,061 - stpipe.pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2026-04-17 15:28:27,062 - stpipe.pipeline - INFO - Prefetch for SATURATION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0097.fits'.
2026-04-17 15:28:27,062 - stpipe.pipeline - INFO - Prefetch for SIRSKERNEL reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_sirskernel_0002.asdf'.
2026-04-17 15:28:27,063 - stpipe.pipeline - INFO - Prefetch for SUPERBIAS reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0210.fits'.
2026-04-17 15:28:27,063 - jwst.pipeline.calwebb_detector1 - INFO - Starting calwebb_detector1 ...
2026-04-17 15:28:30,594 - stpipe.step - INFO - Step group_scale running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg004_nrcalong_uncal.fits>,).
2026-04-17 15:28:30,595 - jwst.group_scale.group_scale_step - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2026-04-17 15:28:30,596 - jwst.group_scale.group_scale_step - INFO - Step will be skipped
2026-04-17 15:28:30,598 - stpipe.step - INFO - Step group_scale done
2026-04-17 15:28:30,738 - stpipe.step - INFO - Step dq_init running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg004_nrcalong_uncal.fits>,).
2026-04-17 15:28:30,741 - jwst.dq_init.dq_init_step - INFO - Using MASK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0076.fits
2026-04-17 15:28:30,801 - stdatamodels.dynamicdq - WARNING - Keyword UNRELIABLE_ERROR does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:28:30,863 - jwst.dq_init.dq_initialization - INFO - Extracting mask subarray to match science data
2026-04-17 15:28:31,062 - stpipe.step - INFO - Step dq_init done
2026-04-17 15:28:31,220 - stpipe.step - INFO - Step saturation running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg004_nrcalong_uncal.fits>,).
2026-04-17 15:28:31,224 - jwst.saturation.saturation_step - INFO - Using SATURATION reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0097.fits
2026-04-17 15:28:31,225 - jwst.saturation.saturation_step - INFO - Using SUPERBIAS reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0210.fits
2026-04-17 15:28:31,287 - stdatamodels.dynamicdq - WARNING - Keyword RESERVED_4 does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:28:31,288 - stdatamodels.dynamicdq - WARNING - Keyword UNRELIABLE_ERROR does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:28:31,327 - stdatamodels.dynamicdq - WARNING - Keyword UNRELIABLE_RESET does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:28:31,384 - stdatamodels.dynamicdq - WARNING - Keyword UNRELIABLE_ERROR does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:28:31,397 - jwst.saturation.saturation - INFO - Extracting reference file subarray to match science data
2026-04-17 15:28:31,416 - jwst.saturation.saturation - INFO - Using read_pattern with nframes 4
2026-04-17 15:28:36,776 - stcal.saturation.saturation - INFO - Detected 3219 saturated pixels
2026-04-17 15:28:36,873 - stcal.saturation.saturation - INFO - Detected 0 A/D floor pixels
2026-04-17 15:28:36,891 - stpipe.step - INFO - Step saturation done
2026-04-17 15:28:37,048 - stpipe.step - INFO - Step ipc running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg004_nrcalong_uncal.fits>,).
2026-04-17 15:28:37,049 - stpipe.step - INFO - Step skipped.
2026-04-17 15:28:37,190 - stpipe.step - INFO - Step superbias running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg004_nrcalong_uncal.fits>,).
2026-04-17 15:28:37,193 - jwst.superbias.superbias_step - INFO - Using SUPERBIAS reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0210.fits
2026-04-17 15:28:37,235 - stdatamodels.dynamicdq - WARNING - Keyword UNRELIABLE_ERROR does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:28:37,572 - stpipe.step - INFO - Step superbias done
2026-04-17 15:28:37,730 - stpipe.step - INFO - Step refpix running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg004_nrcalong_uncal.fits>,).
2026-04-17 15:28:37,733 - jwst.refpix.reference_pixels - INFO - NIR subarray data
2026-04-17 15:28:37,736 - jwst.refpix.reference_pixels - INFO - 4 readout amplifiers used
2026-04-17 15:28:37,736 - jwst.refpix.reference_pixels - INFO - The following parameters are valid for this mode:
2026-04-17 15:28:37,737 - jwst.refpix.reference_pixels - INFO - odd_even_columns = True
2026-04-17 15:28:37,738 - jwst.refpix.reference_pixels - INFO - use_side_ref_pixels = True
2026-04-17 15:28:37,738 - jwst.refpix.reference_pixels - INFO - side_smoothing_length = 11
2026-04-17 15:28:37,739 - jwst.refpix.reference_pixels - INFO - side_gain = 1.0
2026-04-17 15:28:37,740 - jwst.refpix.reference_pixels - INFO - The following parameters are not applicable and are ignored
2026-04-17 15:28:37,741 - jwst.refpix.reference_pixels - INFO - odd_even_rows = False
2026-04-17 15:30:15,630 - jwst.refpix.reference_pixels - INFO - Processing the zero frame
2026-04-17 15:30:23,909 - stpipe.step - INFO - Step refpix done
2026-04-17 15:30:24,067 - stpipe.step - INFO - Step linearity running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg004_nrcalong_uncal.fits>,).
2026-04-17 15:30:24,070 - jwst.linearity.linearity_step - INFO - Using Linearity reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0052.fits
2026-04-17 15:30:24,206 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_DO_NOT_USE does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:30:24,207 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_NO_LIN_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:30:24,208 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_BAD_LIN_FIT does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:30:24,209 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_NO_WELL_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:30:24,210 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_MODEL_FIT_FAIL does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:30:24,211 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_WELL_NOT_DEFINED does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:30:24,212 - stdatamodels.dynamicdq - WARNING - Keyword UAZ_MASTER_MASK does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:30:26,331 - stpipe.step - INFO - Step linearity done
2026-04-17 15:30:26,485 - stpipe.step - INFO - Step persistence running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg004_nrcalong_uncal.fits>,).
2026-04-17 15:30:26,486 - stpipe.step - INFO - Step skipped.
2026-04-17 15:30:26,625 - stpipe.step - INFO - Step dark_current running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg004_nrcalong_uncal.fits>,).
2026-04-17 15:30:26,629 - jwst.dark_current.dark_current_step - INFO - Using DARK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0444.fits
2026-04-17 15:30:27,201 - stdatamodels.dynamicdq - WARNING - Keyword UNRELIABLE_ERROR does not correspond to an existing DQ mnemonic, so will be ignored
2026-04-17 15:30:27,218 - stcal.dark_current.dark_sub - INFO - Science data nints=91, ngroups=12, nframes=4, groupgap=1
2026-04-17 15:30:27,219 - stcal.dark_current.dark_sub - INFO - Dark data nints=1, ngroups=80, nframes=1, groupgap=0
2026-04-17 15:30:27,662 - stpipe.step - INFO - Step dark_current done
2026-04-17 15:30:27,816 - stpipe.step - INFO - Step charge_migration running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg004_nrcalong_uncal.fits>,).
2026-04-17 15:30:27,817 - stpipe.step - INFO - Step skipped.
2026-04-17 15:30:27,956 - stpipe.step - INFO - Step jump running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg004_nrcalong_uncal.fits>,).
2026-04-17 15:30:27,957 - jwst.jump.jump_step - INFO - CR rejection threshold = 6 sigma
2026-04-17 15:30:27,957 - jwst.jump.jump_step - INFO - Maximum cores to use = half
2026-04-17 15:30:27,960 - jwst.jump.jump_step - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits
2026-04-17 15:30:27,963 - jwst.jump.jump_step - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0227.fits
2026-04-17 15:30:27,994 - jwst.jump.jump_step - INFO - Extracting gain subarray to match science data
2026-04-17 15:30:28,285 - stcal.jump.jump - INFO - Executing two-point difference method
2026-04-17 15:30:28,286 - stcal.jump.jump - INFO - Creating 2 processes for jump detection
2026-04-17 15:30:50,872 - stcal.jump.jump - INFO - Total elapsed time = 22.586 sec
2026-04-17 15:30:50,942 - jwst.jump.jump_step - INFO - The execution time in seconds: 22.985599
2026-04-17 15:30:50,946 - stpipe.step - INFO - Step jump done
2026-04-17 15:30:51,102 - stpipe.step - INFO - Step picture_frame running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg004_nrcalong_uncal.fits>,).
2026-04-17 15:30:51,103 - stpipe.step - INFO - Step skipped.
2026-04-17 15:30:51,242 - stpipe.step - INFO - Step clean_flicker_noise running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg004_nrcalong_uncal.fits>,).
2026-04-17 15:30:51,243 - stpipe.step - INFO - Step skipped.
2026-04-17 15:30:54,355 - stpipe.step - INFO - Saved model in ./nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg004_nrcalong_ramp.fits
2026-04-17 15:30:54,510 - stpipe.step - INFO - Step ramp_fit running with args (<RampModel(91, 12, 256, 2048) from jw01366002001_04103_00001-seg004_nrcalong_ramp.fits>,).
2026-04-17 15:30:54,515 - jwst.ramp_fitting.ramp_fit_step - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0227.fits
2026-04-17 15:30:54,516 - jwst.ramp_fitting.ramp_fit_step - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits
2026-04-17 15:30:54,760 - jwst.ramp_fitting.ramp_fit_step - INFO - Extracting gain subarray to match science data
2026-04-17 15:30:54,774 - jwst.ramp_fitting.ramp_fit_step - INFO - Using algorithm = OLS_C
2026-04-17 15:30:54,775 - jwst.ramp_fitting.ramp_fit_step - INFO - Using weighting = optimal
2026-04-17 15:30:56,024 - stcal.ramp_fitting.ols_fit - INFO - Number of multiprocessing slices: 2
2026-04-17 15:30:56,033 - stcal.ramp_fitting.ols_fit - INFO - Number of processors used for multiprocessing: 2
2026-04-17 15:31:21,988 - stpipe.step - INFO - Step ramp_fit done
2026-04-17 15:31:22,157 - stpipe.step - INFO - Step gain_scale running with args (<ImageModel(256, 2048) from jw01366002001_04103_00001-seg004_nrcalong_ramp.fits>,).
2026-04-17 15:31:22,160 - jwst.gain_scale.gain_scale_step - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits
2026-04-17 15:31:22,176 - jwst.gain_scale.gain_scale_step - INFO - GAINFACT not found in gain reference file
2026-04-17 15:31:22,177 - jwst.gain_scale.gain_scale_step - INFO - Step will be skipped
2026-04-17 15:31:22,180 - stpipe.step - INFO - Step gain_scale done
2026-04-17 15:31:22,328 - stpipe.step - INFO - Step gain_scale running with args (<CubeModel(91, 256, 2048) from jw01366002001_04103_00001-seg004_nrcalong_ramp.fits>,).
2026-04-17 15:31:22,332 - jwst.gain_scale.gain_scale_step - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits
2026-04-17 15:31:22,348 - jwst.gain_scale.gain_scale_step - INFO - GAINFACT not found in gain reference file
2026-04-17 15:31:22,349 - jwst.gain_scale.gain_scale_step - INFO - Step will be skipped
2026-04-17 15:31:22,352 - stpipe.step - INFO - Step gain_scale done
2026-04-17 15:31:22,722 - stpipe.step - INFO - Saved model in ./nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg004_nrcalong_rateints.fits
2026-04-17 15:31:22,723 - jwst.pipeline.calwebb_detector1 - INFO - ... ending calwebb_detector1
2026-04-17 15:31:22,728 - jwst.stpipe.core - INFO - Results used CRDS context: jwst_1535.pmap
2026-04-17 15:31:22,781 - stpipe.step - INFO - Saved model in ./nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg004_nrcalong_rate.fits
2026-04-17 15:31:22,781 - stpipe.step - INFO - Step Detector1Pipeline done
2026-04-17 15:31:22,782 - jwst.stpipe.core - INFO - Results used jwst version: 2.0.0
Run time: 12.728333333333333 min
5.3 Exploring the data#
Identify *_rateints.fits files and verify which pipeline steps were run and
which calibration reference files were applied.
The header contains information about which calibration steps were completed and skipped and which reference files were used to process the data.
# Final list of RATE[INTS] files ready for Stage 2 processing.
rate_files = sorted(glob.glob(det1_dir + '/*nrcalong_rateints.fits'))
print(f"SCIENCE | RATE[INTS] Files:\n{'-' * 20}\n" + "\n".join(rate_files))
SCIENCE | RATE[INTS] Files:
--------------------
./nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg001_nrcalong_rateints.fits
./nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg002_nrcalong_rateints.fits
./nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg003_nrcalong_rateints.fits
./nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg004_nrcalong_rateints.fits
if do_det1:
# Read in file as datamodel
rate_f = datamodels.open(rate_files[0])
# Check which steps were run
rate_f.meta.cal_step.instance
{'charge_migration': 'SKIPPED',
'clean_flicker_noise': 'SKIPPED',
'dark_sub': 'COMPLETE',
'dq_init': 'COMPLETE',
'gain_scale': 'SKIPPED',
'group_scale': 'SKIPPED',
'ipc': 'SKIPPED',
'jump': 'COMPLETE',
'linearity': 'COMPLETE',
'persistence': 'SKIPPED',
'picture_frame': 'SKIPPED',
'ramp_fit': 'COMPLETE',
'refpix': 'COMPLETE',
'saturation': 'COMPLETE',
'superbias': 'COMPLETE'}
For this particular rate file, show which reference files were used to calibrate the dataset. Note that these files will be different for each NIRCam detector.
if do_det1:
rate_f.meta.ref_file.instance
{'crds': {'context_used': 'jwst_1535.pmap', 'sw_version': '13.1.14'},
'dark': {'name': 'crds://jwst_nircam_dark_0444.fits'},
'gain': {'name': 'crds://jwst_nircam_gain_0097.fits'},
'linearity': {'name': 'crds://jwst_nircam_linearity_0052.fits'},
'mask': {'name': 'crds://jwst_nircam_mask_0076.fits'},
'readnoise': {'name': 'crds://jwst_nircam_readnoise_0227.fits'},
'saturation': {'name': 'crds://jwst_nircam_saturation_0097.fits'},
'superbias': {'name': 'crds://jwst_nircam_superbias_0210.fits'}}
# Print out the time benchmark
time1 = time.perf_counter()
print(f"Runtime so far: {(time1 - time0) / 60:0.2f} minutes")
print(f"Runtime for Detector1: {(time1 - time_det1) / 60:0.2f} minutes")
Runtime so far: 13.73 minutes
Runtime for Detector1: 12.76 minutes
6. Stage 2: Spec2Pipeline (calwebb_spec2)#
In this section, we process our countrate (slope) image products from Stage 1 (calwebb_detector1) through the Spec2 (calwebb_spec2) pipeline to create Stage 2 data products.
Input: A single countrate (slope) image (
_rate[ints].fits) or an association file listing multiple inputs.Output: Calibrated products (rectified and unrectified) and 1D spectra.
_cal[ints].fits: Calibrated 2D spectra (nrows x ncols)._x1d[ints].fits: Extracted 1D spectroscopic data (wavelength vs. flux).
The Spec2Pipeline applies additional instrumental corrections and calibrations to countrate products that result in a fully calibrated individual exposure (per segment). If the photom step is enabled, the Spec2Pipeline also converts countrate products from units of DN/s to flux (Jy) for point sources.
6.1 Configure Spec2Pipeline#
Below, we set up a dictionary that defines how the Spec2Pipeline should be configured, but first we will start by creating a customized extract1d reference file. Note that a customized reference file is not necessary for the pipeline to run. If users do not provide an extract1d reference file in a pipeline call, the pipeline will automatically retrieve the default filter-dependent extract1d reference file from the Calibration Reference Database System (CRDS). The values in the reference files from CRDS should give decent results for the extracted spectrum. However, customizing extract1d regions may lead to improved signal to noise.
For more information about each step and a full list of step arguments, please refer to the official documentation: JDox • ReadtheDocs
time_spec2 = time.perf_counter()
To override specific steps and reference files, use the examples below. In this particular example, we will create an extract1d reference file to override with more details for background extraction regions.
The default pipeline spectral extraction uses an aperture width defined in the CRDS extract 1D reference file for a given filter, centered on the expected location of the spectral trace. Users can alter the location and width of the extraction aperture and modify background subtraction strategies by providing a custom extract1d reference file. For details concerning the proper format and available parameter settings for such reference files, consult the extract1d refrence file description.
Below we will create a custom extract1d reference file tailored for the example data in this notebook.
def extract_1d_json(
file,
xstart,
xstop,
ystart,
ystop,
bkg_coeff,
bkg_order,
dispaxis,
bkg_fit,
smoothing_length
):
with datamodels.open(file) as dm:
telescope = dm.meta.telescope
instrument = dm.meta.instrument.name
exp_type = dm.meta.exposure.type
filt = dm.meta.instrument.filter
extract1d_json_data = {
'reftype': 'EXTRACT1D',
'telescope': telescope,
'instrument': instrument,
'exp_type': exp_type,
'filter': filt,
'apertures': [
{'id': 'ANY',
'region_type': 'target',
'xstart': xstart,
'xstop': xstop,
'ystart': ystart,
'ystop': ystop,
'bkg_coeff': bkg_coeff,
'bkg_order': bkg_order,
'dispaxis': dispaxis,
'bkg_fit': bkg_fit,
'smoothing_length': smoothing_length
}
]
}
custom_extract1d_json_file = '{}_{}_extract1d_{}.json'.format(telescope, instrument, filt)
custom_extract1d_json_file = os.path.join(spec2_dir, custom_extract1d_json_file)
json_object = json.dumps(extract1d_json_data, indent=4)
with open(custom_extract1d_json_file, "w") as outfile:
outfile.write(json_object)
return custom_extract1d_json_file
Create a custom extract1d reference file that slightly changes the area used for spectral extraction, as well as those used to calculate background.
In each column of the 2D trace image, sum the signal in rows 31 through 37. Use rows 21 through 31, and 35 through 45 to measure the background signal. Extract only columns 4 through 1704 since the trace fades away to the right of this range.
When working with data from other programs, users may want to tweak these ranges to optimize the spectral extraction. The displayed images in Section 8.1 and Section 8.2 can be useful for determining any changes.
custom_extract1d_reffile = extract_1d_json(
file=rate_files[0],
xstart=4,
xstop=1704,
ystart=31,
ystop=37,
bkg_coeff=[[21], [31], [35], [45]],
bkg_order=0,
dispaxis=1,
bkg_fit="median",
smoothing_length=0
)
# Show contents of extract1d json file
with open(custom_extract1d_reffile) as f_obj:
extract1d_json_data = json.load(f_obj)
# extract1d_json_data
JSON(extract1d_json_data, expanded=True)
<IPython.core.display.JSON object>
# Set up a dictionary to define how the Spec2 pipeline should be configured
# Boilerplate dictionary setup
spec2dict = {}
spec2dict['assign_wcs'], spec2dict['flat_field'] = {}, {}
spec2dict['extract_2d'], spec2dict['srctype'] = {}, {}
spec2dict['photom'], spec2dict['extract_1d'] = {}, {}
# Overrides for whether or not certain steps should be skipped (example)
# In this case, we skip the flux calibration (photom) step
spec2dict['photom']['skip'] = True
# Adjust 2D cutout box extraction height (default is 64)
spec2dict['extract_2d']['tsgrism_extract_height'] = 64
# Overrides for various reference files
# Files should be in the base local directory or provide full path
#spec2dict['assign_wcs']['override_distortion'] = 'myfile.asdf' # Spatial distortion (ASDF file)
#spec2dict['assign_wcs']['override_specwcs'] = 'myfile.asdf' # Spectral distortion (ASDF file)
#spec2dict['flat_field']['override_flat'] = 'myfile.fits' # Pixel flatfield
#spec2dict['photom']['override_photom'] = 'myfile.fits' # Photometric calibration array
# Here we supply the custom extract1d reference file created above.
spec2dict['extract_1d']['override_extract1d'] = custom_extract1d_reffile # Spectral extraction parameters
#spec2dict['extract_1d']['override_apcorr'] = 'myfile.asdf' # Aperture correction parameters
6.2 Run Spec2Pipeline#
Run the science files through the calwebb_spec2 pipeline using the .call() method.
# Generate a list of the rateints files output by Stage 1 of the pipeline
rateints_files_lw = [os.path.abspath(f) for f in sorted(glob.glob(os.path.join(det1_dir, 'jw*long*rateints.fits')))]
# Run the pipeline on the selected science rateints files one by one with the custom parameter dictionary
start = time.time()
if do_spec2:
for rateints in rateints_files_lw:
cal_result = Spec2Pipeline.call(rateints, steps=spec2dict, output_dir=spec2_dir, save_results=True)
else:
print('Skipping Spec2 processing...')
end = time.time()
print("Run time: ", round(end-start, 1)/60.0, " min")
2026-04-17 15:31:25,262 - CRDS - ERROR - Error determining best reference for 'pars-targcentroidstep' = Unknown reference type 'pars-targcentroidstep'
2026-04-17 15:31:25,276 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-spec2pipeline_0006.asdf 2.5 K bytes (1 / 1 files) (0 / 2.5 K bytes)
2026-04-17 15:31:25,543 - stpipe.pipeline - INFO - PARS-SPEC2PIPELINE parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-spec2pipeline_0006.asdf
2026-04-17 15:31:25,569 - stpipe.step - INFO - Spec2Pipeline instance created.
2026-04-17 15:31:25,572 - stpipe.step - INFO - AssignWcsStep instance created.
2026-04-17 15:31:25,574 - stpipe.step - INFO - BadpixSelfcalStep instance created.
2026-04-17 15:31:25,575 - stpipe.step - INFO - MSAFlagOpenStep instance created.
2026-04-17 15:31:25,576 - stpipe.step - INFO - CleanFlickerNoiseStep instance created.
2026-04-17 15:31:25,578 - stpipe.step - INFO - BackgroundStep instance created.
2026-04-17 15:31:25,579 - stpipe.step - INFO - ImprintStep instance created.
2026-04-17 15:31:25,581 - stpipe.step - INFO - Extract2dStep instance created.
2026-04-17 15:31:25,586 - stpipe.step - INFO - MasterBackgroundMosStep instance created.
2026-04-17 15:31:25,588 - stpipe.step - INFO - FlatFieldStep instance created.
2026-04-17 15:31:25,589 - stpipe.step - INFO - PathLossStep instance created.
2026-04-17 15:31:25,590 - stpipe.step - INFO - BarShadowStep instance created.
2026-04-17 15:31:25,591 - stpipe.step - INFO - PhotomStep instance created.
2026-04-17 15:31:25,592 - stpipe.step - INFO - PixelReplaceStep instance created.
2026-04-17 15:31:25,594 - stpipe.step - INFO - ResampleSpecStep instance created.
2026-04-17 15:31:25,599 - stpipe.step - INFO - Extract1dStep instance created.
2026-04-17 15:31:25,600 - stpipe.step - INFO - TargCentroidStep instance created.
2026-04-17 15:31:25,601 - stpipe.step - INFO - WavecorrStep instance created.
2026-04-17 15:31:25,602 - stpipe.step - INFO - FlatFieldStep instance created.
2026-04-17 15:31:25,603 - stpipe.step - INFO - SourceTypeStep instance created.
2026-04-17 15:31:25,604 - stpipe.step - INFO - StraylightStep instance created.
2026-04-17 15:31:25,605 - stpipe.step - INFO - FringeStep instance created.
2026-04-17 15:31:25,606 - stpipe.step - INFO - ResidualFringeStep instance created.
2026-04-17 15:31:25,607 - stpipe.step - INFO - PathLossStep instance created.
2026-04-17 15:31:25,608 - stpipe.step - INFO - BarShadowStep instance created.
2026-04-17 15:31:25,610 - stpipe.step - INFO - WfssContamStep instance created.
2026-04-17 15:31:25,611 - stpipe.step - INFO - PhotomStep instance created.
2026-04-17 15:31:25,612 - stpipe.step - INFO - AdaptiveTraceModelStep instance created.
2026-04-17 15:31:25,613 - stpipe.step - INFO - PixelReplaceStep instance created.
2026-04-17 15:31:25,614 - stpipe.step - INFO - ResampleSpecStep instance created.
2026-04-17 15:31:25,617 - stpipe.step - INFO - CubeBuildStep instance created.
2026-04-17 15:31:25,619 - stpipe.step - INFO - Extract1dStep instance created.
2026-04-17 15:31:25,779 - stpipe.step - INFO - Step Spec2Pipeline running with args ('/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg001_nrcalong_rateints.fits',).
2026-04-17 15:31:25,815 - stpipe.step - INFO - Step Spec2Pipeline parameters are:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: ./nrc_tso_demo_data/01366/Obs002/stage2
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: True
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_bsub: False
fail_on_exception: True
save_wfss_esec: False
steps:
assign_wcs:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
sip_approx: True
sip_max_pix_error: 0.01
sip_degree: None
sip_max_inv_pix_error: 0.01
sip_inv_degree: None
sip_npoints: 12
slit_y_low: -0.55
slit_y_high: 0.55
nrs_ifu_slice_wcs: False
badpix_selfcal:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
flagfrac_lower: 0.001
flagfrac_upper: 0.001
kernel_size: 15
force_single: False
save_flagged_bkg: False
msa_flagging:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
clean_flicker_noise:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
autoparam: False
fit_method: median
fit_by_channel: False
background_method: median
background_box_size: None
mask_science_regions: False
apply_flat_field: False
n_sigma: 2.0
fit_histogram: False
single_mask: True
user_mask: None
save_mask: False
save_background: False
save_noise: False
bkg_subtract:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
bkg_list: None
save_combined_background: False
sigma: 3.0
maxiters: None
soss_source_percentile: 35.0
soss_bkg_percentile: None
wfss_mmag_extract: None
wfss_mask: None
wfss_maxiter: 5
wfss_rms_stop: 0.0
wfss_outlier_percent: 1.0
imprint_subtract:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
extract_2d:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
slit_names: None
source_ids: None
source_ra: None
source_dec: None
source_max_sep: 2.0
extract_orders: None
grism_objects: None
tsgrism_extract_height: 64
wfss_extract_half_height: 5
wfss_mmag_extract: None
wfss_nbright: 1000
master_background_mos:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: True
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
sigma_clip: 3.0
median_kernel: 1
force_subtract: False
save_background: False
user_background: None
inverse: False
steps:
flat_field:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_interpolated_flat: False
user_supplied_flat: None
inverse: False
pathloss:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
user_slit_loc: None
barshadow:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
photom:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
apply_time_correction: True
pixel_replace:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: True
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
algorithm: fit_profile
n_adjacent_cols: 3
resample_spec:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
pixfrac: 1.0
kernel: square
fillval: NAN
weight_type: ivm
output_shape: None
pixel_scale_ratio: 1.0
pixel_scale: None
output_wcs: ''
single: False
blendheaders: True
in_memory: True
propagate_dq: False
extract_1d:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
subtract_background: None
apply_apcorr: True
extraction_type: box
use_source_posn: None
position_offset: 0.0
model_nod_pair: True
optimize_psf_location: True
smoothing_length: None
bkg_fit: None
bkg_order: None
log_increment: 50
save_profile: False
save_scene_model: False
save_residual_image: False
center_xy: None
ifu_autocen: False
bkg_sigma_clip: 3.0
ifu_rfcorr: True
ifu_set_srctype: None
ifu_rscale: None
ifu_covar_scale: 1.0
soss_atoca: True
soss_threshold: 0.01
soss_n_os: 2
soss_wave_grid_in: None
soss_wave_grid_out: None
soss_estimate: None
soss_rtol: 0.0001
soss_max_grid_size: 20000
soss_tikfac: None
soss_width: 40.0
soss_bad_pix: masking
soss_modelname: None
soss_order_3: True
targ_centroid:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
ta_file: None
wavecorr:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
flat_field:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_interpolated_flat: False
user_supplied_flat: None
inverse: False
srctype:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
source_type: None
straylight:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
clean_showers: False
shower_plane: 3
shower_x_stddev: 18.0
shower_y_stddev: 5.0
shower_low_reject: 0.1
shower_high_reject: 99.9
save_shower_model: False
fringe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
residual_fringe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: residual_fringe
search_output_file: False
input_dir: ''
save_intermediate_results: False
ignore_region_min: None
ignore_region_max: None
pathloss:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
user_slit_loc: None
barshadow:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
wfss_contam:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
save_simulated_image: False
save_contam_images: False
maximum_cores: '1'
orders: None
magnitude_limit: None
wl_oversample: 2
max_pixels_per_chunk: 5000
photom:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
apply_time_correction: True
adaptive_trace_model:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: True
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
fit_threshold: 10.0
oversample: 1.0
slope_limit: 0.1
psf_optimal: False
save_intermediate_results: False
pixel_replace:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: True
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
algorithm: fit_profile
n_adjacent_cols: 3
resample_spec:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
pixfrac: 1.0
kernel: square
fillval: NAN
weight_type: ivm
output_shape: None
pixel_scale_ratio: 1.0
pixel_scale: None
output_wcs: ''
single: False
blendheaders: True
in_memory: True
propagate_dq: False
cube_build:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: True
output_use_index: True
save_results: False
skip: True
suffix: s3d
search_output_file: False
input_dir: ''
pipeline: 3
channel: all
band: all
grating: all
filter: all
output_type: None
linear_wave: True
scalexy: 0.0
scalew: 0.0
weighting: drizzle
coord_system: skyalign
ra_center: None
dec_center: None
cube_pa: None
nspax_x: None
nspax_y: None
rois: 0.0
roiw: 0.0
weight_power: 2.0
wavemin: None
wavemax: None
single: False
skip_dqflagging: False
offset_file: None
debug_spaxel: -1 -1 -1
extract_1d:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
subtract_background: None
apply_apcorr: True
extraction_type: box
use_source_posn: None
position_offset: 0.0
model_nod_pair: True
optimize_psf_location: True
smoothing_length: None
bkg_fit: None
bkg_order: None
log_increment: 50
save_profile: False
save_scene_model: False
save_residual_image: False
center_xy: None
ifu_autocen: False
bkg_sigma_clip: 3.0
ifu_rfcorr: True
ifu_set_srctype: None
ifu_rscale: None
ifu_covar_scale: 1.0
soss_atoca: True
soss_threshold: 0.01
soss_n_os: 2
soss_wave_grid_in: None
soss_wave_grid_out: None
soss_estimate: None
soss_rtol: 0.0001
soss_max_grid_size: 20000
soss_tikfac: None
soss_width: 40.0
soss_bad_pix: masking
soss_modelname: None
soss_order_3: True
2026-04-17 15:31:25,846 - stpipe.pipeline - INFO - Prefetching reference files for dataset: 'jw01366002001_04103_00001-seg001_nrcalong_rateints.fits' reftypes = ['apcorr', 'camera', 'collimator', 'dflat', 'disperser', 'distortion', 'fflat', 'filteroffset', 'flat', 'fore', 'fpa', 'fringe', 'ifufore', 'ifupost', 'ifuslicer', 'msa', 'ote', 'pastasoss', 'psf', 'regions', 'sflat', 'speckernel', 'specprofile', 'specwcs', 'wavecorr', 'wavelengthrange']
2026-04-17 15:31:25,853 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_distortion_0259.asdf 11.4 K bytes (1 / 5 files) (0 / 50.4 M bytes)
2026-04-17 15:31:25,974 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_filteroffset_0007.asdf 11.4 K bytes (2 / 5 files) (11.4 K / 50.4 M bytes)
2026-04-17 15:31:26,182 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0756.fits 50.4 M bytes (3 / 5 files) (22.8 K / 50.4 M bytes)
2026-04-17 15:31:26,902 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_specwcs_0259.asdf 9.4 K bytes (4 / 5 files) (50.4 M / 50.4 M bytes)
2026-04-17 15:31:27,008 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_wavelengthrange_0004.asdf 3.1 K bytes (5 / 5 files) (50.4 M / 50.4 M bytes)
2026-04-17 15:31:27,254 - stpipe.pipeline - INFO - Prefetch for APCORR reference file is 'N/A'.
2026-04-17 15:31:27,255 - stpipe.pipeline - INFO - Prefetch for CAMERA reference file is 'N/A'.
2026-04-17 15:31:27,255 - stpipe.pipeline - INFO - Prefetch for COLLIMATOR reference file is 'N/A'.
2026-04-17 15:31:27,256 - stpipe.pipeline - INFO - Prefetch for DFLAT reference file is 'N/A'.
2026-04-17 15:31:27,256 - stpipe.pipeline - INFO - Prefetch for DISPERSER reference file is 'N/A'.
2026-04-17 15:31:27,257 - stpipe.pipeline - INFO - Prefetch for DISTORTION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_distortion_0259.asdf'.
2026-04-17 15:31:27,258 - stpipe.pipeline - INFO - Override for EXTRACT1D reference file is '/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage2/JWST_NIRCAM_extract1d_F322W2.json'.
2026-04-17 15:31:27,258 - stpipe.pipeline - INFO - Prefetch for FFLAT reference file is 'N/A'.
2026-04-17 15:31:27,259 - stpipe.pipeline - INFO - Prefetch for FILTEROFFSET reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_filteroffset_0007.asdf'.
2026-04-17 15:31:27,259 - stpipe.pipeline - INFO - Prefetch for FLAT reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0756.fits'.
2026-04-17 15:31:27,260 - stpipe.pipeline - INFO - Prefetch for FORE reference file is 'N/A'.
2026-04-17 15:31:27,260 - stpipe.pipeline - INFO - Prefetch for FPA reference file is 'N/A'.
2026-04-17 15:31:27,261 - stpipe.pipeline - INFO - Prefetch for FRINGE reference file is 'N/A'.
2026-04-17 15:31:27,262 - stpipe.pipeline - INFO - Prefetch for IFUFORE reference file is 'N/A'.
2026-04-17 15:31:27,262 - stpipe.pipeline - INFO - Prefetch for IFUPOST reference file is 'N/A'.
2026-04-17 15:31:27,263 - stpipe.pipeline - INFO - Prefetch for IFUSLICER reference file is 'N/A'.
2026-04-17 15:31:27,264 - stpipe.pipeline - INFO - Prefetch for MSA reference file is 'N/A'.
2026-04-17 15:31:27,265 - stpipe.pipeline - INFO - Prefetch for OTE reference file is 'N/A'.
2026-04-17 15:31:27,265 - stpipe.pipeline - INFO - Prefetch for PASTASOSS reference file is 'N/A'.
2026-04-17 15:31:27,266 - stpipe.pipeline - INFO - Prefetch for PSF reference file is 'N/A'.
2026-04-17 15:31:27,267 - stpipe.pipeline - INFO - Prefetch for REGIONS reference file is 'N/A'.
2026-04-17 15:31:27,267 - stpipe.pipeline - INFO - Prefetch for SFLAT reference file is 'N/A'.
2026-04-17 15:31:27,268 - stpipe.pipeline - INFO - Prefetch for SPECKERNEL reference file is 'N/A'.
2026-04-17 15:31:27,269 - stpipe.pipeline - INFO - Prefetch for SPECPROFILE reference file is 'N/A'.
2026-04-17 15:31:27,269 - stpipe.pipeline - INFO - Prefetch for SPECWCS reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_specwcs_0259.asdf'.
2026-04-17 15:31:27,270 - stpipe.pipeline - INFO - Prefetch for WAVECORR reference file is 'N/A'.
2026-04-17 15:31:27,271 - stpipe.pipeline - INFO - Prefetch for WAVELENGTHRANGE reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_wavelengthrange_0004.asdf'.
2026-04-17 15:31:27,271 - jwst.pipeline.calwebb_spec2 - INFO - Starting calwebb_spec2 ...
2026-04-17 15:31:27,277 - jwst.pipeline.calwebb_spec2 - INFO - Processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg001_nrcalong
2026-04-17 15:31:27,277 - jwst.pipeline.calwebb_spec2 - INFO - Working on input /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg001_nrcalong_rateints.fits ...
2026-04-17 15:31:27,673 - stpipe.step - INFO - Step assign_wcs running with args (<CubeModel(92, 256, 2048) from jw01366002001_04103_00001-seg001_nrcalong_rateints.fits>,).
2026-04-17 15:31:27,737 - jwst.assign_wcs.nircam - INFO - Added Barycentric velocity correction: 0.9999043045827591
2026-04-17 15:31:27,865 - jwst.assign_wcs.assign_wcs - INFO - COMPLETED assign_wcs
2026-04-17 15:31:27,869 - stpipe.step - INFO - Step assign_wcs done
2026-04-17 15:31:28,017 - stpipe.step - INFO - Step badpix_selfcal running with args (<CubeModel(92, 256, 2048) from jw01366002001_04103_00001-seg001_nrcalong_rateints.fits>, [], []).
2026-04-17 15:31:28,019 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:28,173 - stpipe.step - INFO - Step msa_flagging running with args (<CubeModel(92, 256, 2048) from jw01366002001_04103_00001-seg001_nrcalong_rateints.fits>,).
2026-04-17 15:31:28,174 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:28,318 - stpipe.step - INFO - Step clean_flicker_noise running with args (<CubeModel(92, 256, 2048) from jw01366002001_04103_00001-seg001_nrcalong_rateints.fits>,).
2026-04-17 15:31:28,320 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:28,461 - stpipe.step - INFO - Step imprint_subtract running with args (<CubeModel(92, 256, 2048) from jw01366002001_04103_00001-seg001_nrcalong_rateints.fits>, []).
2026-04-17 15:31:28,462 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:28,616 - stpipe.step - INFO - Step bkg_subtract running with args (<CubeModel(92, 256, 2048) from jw01366002001_04103_00001-seg001_nrcalong_rateints.fits>, []).
2026-04-17 15:31:28,616 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:28,761 - stpipe.step - INFO - Step flat_field running with args (<CubeModel(92, 256, 2048) from jw01366002001_04103_00001-seg001_nrcalong_rateints.fits>,).
2026-04-17 15:31:28,866 - jwst.flatfield.flat_field_step - INFO - Using FLAT reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0756.fits
2026-04-17 15:31:28,867 - jwst.flatfield.flat_field_step - INFO - No reference found for type FFLAT
2026-04-17 15:31:28,868 - jwst.flatfield.flat_field_step - INFO - No reference found for type SFLAT
2026-04-17 15:31:28,868 - jwst.flatfield.flat_field_step - INFO - No reference found for type DFLAT
2026-04-17 15:31:28,869 - jwst.flatfield.flat_field - INFO - Extracting matching subarray from flat
2026-04-17 15:31:29,462 - stpipe.step - INFO - Step flat_field done
2026-04-17 15:31:29,622 - stpipe.step - INFO - Step extract_2d running with args (<CubeModel(92, 256, 2048) from jw01366002001_04103_00001-seg001_nrcalong_rateints.fits>,).
2026-04-17 15:31:29,628 - jwst.extract_2d.extract_2d - INFO - EXP_TYPE is NRC_TSGRISM
2026-04-17 15:31:29,641 - jwst.extract_2d.grisms - INFO - Using default order extraction from reference file
2026-04-17 15:31:29,642 - jwst.extract_2d.grisms - INFO - Extracting order: [1]
2026-04-17 15:31:29,686 - jwst.extract_2d.grisms - INFO - Setting extraction height to 64
2026-04-17 15:31:29,696 - jwst.extract_2d.grisms - INFO - WCS made explicit for order: 1
2026-04-17 15:31:29,697 - jwst.extract_2d.grisms - INFO - Spectral trace extents: (xmin: 104, ymin: 0), (xmax: 1867, ymax: 63)
2026-04-17 15:31:29,698 - jwst.extract_2d.grisms - INFO - Extraction limits: (xmin: 0, ymin: 0), (xmax: 2047, ymax: 63)
2026-04-17 15:31:29,865 - jwst.extract_2d.grisms - INFO - Finished extraction
2026-04-17 15:31:29,869 - stpipe.step - INFO - Step extract_2d done
2026-04-17 15:31:30,042 - stpipe.step - INFO - Step srctype running with args (<SlitModel(92, 64, 2048) from jw01366002001_04103_00001-seg001_nrcalong_rateints.fits>,).
2026-04-17 15:31:30,043 - jwst.srctype.srctype - INFO - Input EXP_TYPE is NRC_TSGRISM
2026-04-17 15:31:30,044 - jwst.srctype.srctype - INFO - Input SRCTYAPT = UNKNOWN
2026-04-17 15:31:30,045 - jwst.srctype.srctype - INFO - Input is a TSO exposure; setting SRCTYPE = POINT
2026-04-17 15:31:30,047 - stpipe.step - INFO - Step srctype done
2026-04-17 15:31:30,192 - stpipe.step - INFO - Step straylight running with args (<SlitModel(92, 64, 2048) from jw01366002001_04103_00001-seg001_nrcalong_rateints.fits>,).
2026-04-17 15:31:30,193 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:30,336 - stpipe.step - INFO - Step fringe running with args (<SlitModel(92, 64, 2048) from jw01366002001_04103_00001-seg001_nrcalong_rateints.fits>,).
2026-04-17 15:31:30,337 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:30,481 - stpipe.step - INFO - Step pathloss running with args (<SlitModel(92, 64, 2048) from jw01366002001_04103_00001-seg001_nrcalong_rateints.fits>,).
2026-04-17 15:31:30,482 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:30,626 - stpipe.step - INFO - Step barshadow running with args (<SlitModel(92, 64, 2048) from jw01366002001_04103_00001-seg001_nrcalong_rateints.fits>,).
2026-04-17 15:31:30,627 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:30,771 - stpipe.step - INFO - Step wfss_contam running with args (<SlitModel(92, 64, 2048) from jw01366002001_04103_00001-seg001_nrcalong_rateints.fits>,).
2026-04-17 15:31:30,772 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:30,916 - stpipe.step - INFO - Step photom running with args (<SlitModel(92, 64, 2048) from jw01366002001_04103_00001-seg001_nrcalong_rateints.fits>,).
2026-04-17 15:31:30,917 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:31,142 - stpipe.step - INFO - Step pixel_replace running with args (<SlitModel(92, 64, 2048) from ./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg001_nrcalong_calints.fits>,).
2026-04-17 15:31:31,143 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:31,367 - stpipe.step - INFO - Step extract_1d running with args (<SlitModel(92, 64, 2048) from ./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg001_nrcalong_calints.fits>,).
2026-04-17 15:31:31,369 - jwst.extract_1d.extract_1d_step - INFO - Using EXTRACT1D reference file /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage2/JWST_NIRCAM_extract1d_F322W2.json
2026-04-17 15:31:31,428 - jwst.extract_1d.extract - INFO - Processing spectral order 1
2026-04-17 15:31:31,490 - jwst.extract_1d.extract - WARNING - The photom step has not been run.
2026-04-17 15:31:31,493 - jwst.extract_1d.extract - INFO - Aperture start/stop: 31.00 -> 37.00 (inclusive)
2026-04-17 15:31:31,496 - jwst.extract_1d.extract - INFO - Mean aperture start/stop from bkg_coeff: 21.50 -> 30.50 (inclusive)
2026-04-17 15:31:31,498 - jwst.extract_1d.extract - INFO - Mean aperture start/stop from bkg_coeff: 35.50 -> 44.50 (inclusive)
2026-04-17 15:31:31,505 - jwst.extract_1d.extract - INFO - Beginning loop over 92 integrations ...
2026-04-17 15:31:33,911 - jwst.extract_1d.extract - INFO - ... 50 integrations done
2026-04-17 15:31:36,195 - jwst.extract_1d.extract - INFO - All 92 integrations done
2026-04-17 15:31:36,426 - stpipe.step - INFO - Saved model in ./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg001_nrcalong_x1dints.fits
2026-04-17 15:31:36,427 - stpipe.step - INFO - Step extract_1d done
2026-04-17 15:31:36,431 - jwst.pipeline.calwebb_spec2 - INFO - Finished processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg001_nrcalong
2026-04-17 15:31:36,443 - jwst.pipeline.calwebb_spec2 - INFO - Ending calwebb_spec2
2026-04-17 15:31:36,443 - jwst.stpipe.core - INFO - Results used CRDS context: jwst_1535.pmap
2026-04-17 15:31:36,629 - stpipe.step - INFO - Saved model in ./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg001_nrcalong_calints.fits
2026-04-17 15:31:36,629 - stpipe.step - INFO - Step Spec2Pipeline done
2026-04-17 15:31:36,630 - jwst.stpipe.core - INFO - Results used jwst version: 2.0.0
2026-04-17 15:31:36,713 - CRDS - ERROR - Error determining best reference for 'pars-targcentroidstep' = Unknown reference type 'pars-targcentroidstep'
2026-04-17 15:31:36,727 - stpipe.pipeline - INFO - PARS-SPEC2PIPELINE parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-spec2pipeline_0006.asdf
2026-04-17 15:31:36,758 - stpipe.step - INFO - Spec2Pipeline instance created.
2026-04-17 15:31:36,760 - stpipe.step - INFO - AssignWcsStep instance created.
2026-04-17 15:31:36,760 - stpipe.step - INFO - BadpixSelfcalStep instance created.
2026-04-17 15:31:36,761 - stpipe.step - INFO - MSAFlagOpenStep instance created.
2026-04-17 15:31:36,763 - stpipe.step - INFO - CleanFlickerNoiseStep instance created.
2026-04-17 15:31:36,764 - stpipe.step - INFO - BackgroundStep instance created.
2026-04-17 15:31:36,765 - stpipe.step - INFO - ImprintStep instance created.
2026-04-17 15:31:36,766 - stpipe.step - INFO - Extract2dStep instance created.
2026-04-17 15:31:36,770 - stpipe.step - INFO - MasterBackgroundMosStep instance created.
2026-04-17 15:31:36,771 - stpipe.step - INFO - FlatFieldStep instance created.
2026-04-17 15:31:36,772 - stpipe.step - INFO - PathLossStep instance created.
2026-04-17 15:31:36,773 - stpipe.step - INFO - BarShadowStep instance created.
2026-04-17 15:31:36,774 - stpipe.step - INFO - PhotomStep instance created.
2026-04-17 15:31:36,775 - stpipe.step - INFO - PixelReplaceStep instance created.
2026-04-17 15:31:36,776 - stpipe.step - INFO - ResampleSpecStep instance created.
2026-04-17 15:31:36,778 - stpipe.step - INFO - Extract1dStep instance created.
2026-04-17 15:31:36,779 - stpipe.step - INFO - TargCentroidStep instance created.
2026-04-17 15:31:36,780 - stpipe.step - INFO - WavecorrStep instance created.
2026-04-17 15:31:36,781 - stpipe.step - INFO - FlatFieldStep instance created.
2026-04-17 15:31:36,782 - stpipe.step - INFO - SourceTypeStep instance created.
2026-04-17 15:31:36,783 - stpipe.step - INFO - StraylightStep instance created.
2026-04-17 15:31:36,784 - stpipe.step - INFO - FringeStep instance created.
2026-04-17 15:31:36,785 - stpipe.step - INFO - ResidualFringeStep instance created.
2026-04-17 15:31:36,786 - stpipe.step - INFO - PathLossStep instance created.
2026-04-17 15:31:36,787 - stpipe.step - INFO - BarShadowStep instance created.
2026-04-17 15:31:36,788 - stpipe.step - INFO - WfssContamStep instance created.
2026-04-17 15:31:36,789 - stpipe.step - INFO - PhotomStep instance created.
2026-04-17 15:31:36,790 - stpipe.step - INFO - AdaptiveTraceModelStep instance created.
2026-04-17 15:31:36,791 - stpipe.step - INFO - PixelReplaceStep instance created.
2026-04-17 15:31:36,792 - stpipe.step - INFO - ResampleSpecStep instance created.
2026-04-17 15:31:36,794 - stpipe.step - INFO - CubeBuildStep instance created.
2026-04-17 15:31:36,795 - stpipe.step - INFO - Extract1dStep instance created.
2026-04-17 15:31:36,989 - stpipe.step - INFO - Step Spec2Pipeline running with args ('/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg002_nrcalong_rateints.fits',).
2026-04-17 15:31:37,024 - stpipe.step - INFO - Step Spec2Pipeline parameters are:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: ./nrc_tso_demo_data/01366/Obs002/stage2
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: True
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_bsub: False
fail_on_exception: True
save_wfss_esec: False
steps:
assign_wcs:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
sip_approx: True
sip_max_pix_error: 0.01
sip_degree: None
sip_max_inv_pix_error: 0.01
sip_inv_degree: None
sip_npoints: 12
slit_y_low: -0.55
slit_y_high: 0.55
nrs_ifu_slice_wcs: False
badpix_selfcal:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
flagfrac_lower: 0.001
flagfrac_upper: 0.001
kernel_size: 15
force_single: False
save_flagged_bkg: False
msa_flagging:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
clean_flicker_noise:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
autoparam: False
fit_method: median
fit_by_channel: False
background_method: median
background_box_size: None
mask_science_regions: False
apply_flat_field: False
n_sigma: 2.0
fit_histogram: False
single_mask: True
user_mask: None
save_mask: False
save_background: False
save_noise: False
bkg_subtract:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
bkg_list: None
save_combined_background: False
sigma: 3.0
maxiters: None
soss_source_percentile: 35.0
soss_bkg_percentile: None
wfss_mmag_extract: None
wfss_mask: None
wfss_maxiter: 5
wfss_rms_stop: 0.0
wfss_outlier_percent: 1.0
imprint_subtract:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
extract_2d:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
slit_names: None
source_ids: None
source_ra: None
source_dec: None
source_max_sep: 2.0
extract_orders: None
grism_objects: None
tsgrism_extract_height: 64
wfss_extract_half_height: 5
wfss_mmag_extract: None
wfss_nbright: 1000
master_background_mos:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: True
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
sigma_clip: 3.0
median_kernel: 1
force_subtract: False
save_background: False
user_background: None
inverse: False
steps:
flat_field:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_interpolated_flat: False
user_supplied_flat: None
inverse: False
pathloss:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
user_slit_loc: None
barshadow:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
photom:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
apply_time_correction: True
pixel_replace:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: True
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
algorithm: fit_profile
n_adjacent_cols: 3
resample_spec:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
pixfrac: 1.0
kernel: square
fillval: NAN
weight_type: ivm
output_shape: None
pixel_scale_ratio: 1.0
pixel_scale: None
output_wcs: ''
single: False
blendheaders: True
in_memory: True
propagate_dq: False
extract_1d:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
subtract_background: None
apply_apcorr: True
extraction_type: box
use_source_posn: None
position_offset: 0.0
model_nod_pair: True
optimize_psf_location: True
smoothing_length: None
bkg_fit: None
bkg_order: None
log_increment: 50
save_profile: False
save_scene_model: False
save_residual_image: False
center_xy: None
ifu_autocen: False
bkg_sigma_clip: 3.0
ifu_rfcorr: True
ifu_set_srctype: None
ifu_rscale: None
ifu_covar_scale: 1.0
soss_atoca: True
soss_threshold: 0.01
soss_n_os: 2
soss_wave_grid_in: None
soss_wave_grid_out: None
soss_estimate: None
soss_rtol: 0.0001
soss_max_grid_size: 20000
soss_tikfac: None
soss_width: 40.0
soss_bad_pix: masking
soss_modelname: None
soss_order_3: True
targ_centroid:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
ta_file: None
wavecorr:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
flat_field:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_interpolated_flat: False
user_supplied_flat: None
inverse: False
srctype:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
source_type: None
straylight:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
clean_showers: False
shower_plane: 3
shower_x_stddev: 18.0
shower_y_stddev: 5.0
shower_low_reject: 0.1
shower_high_reject: 99.9
save_shower_model: False
fringe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
residual_fringe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: residual_fringe
search_output_file: False
input_dir: ''
save_intermediate_results: False
ignore_region_min: None
ignore_region_max: None
pathloss:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
user_slit_loc: None
barshadow:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
wfss_contam:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
save_simulated_image: False
save_contam_images: False
maximum_cores: '1'
orders: None
magnitude_limit: None
wl_oversample: 2
max_pixels_per_chunk: 5000
photom:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
apply_time_correction: True
adaptive_trace_model:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: True
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
fit_threshold: 10.0
oversample: 1.0
slope_limit: 0.1
psf_optimal: False
save_intermediate_results: False
pixel_replace:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: True
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
algorithm: fit_profile
n_adjacent_cols: 3
resample_spec:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
pixfrac: 1.0
kernel: square
fillval: NAN
weight_type: ivm
output_shape: None
pixel_scale_ratio: 1.0
pixel_scale: None
output_wcs: ''
single: False
blendheaders: True
in_memory: True
propagate_dq: False
cube_build:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: True
output_use_index: True
save_results: False
skip: True
suffix: s3d
search_output_file: False
input_dir: ''
pipeline: 3
channel: all
band: all
grating: all
filter: all
output_type: None
linear_wave: True
scalexy: 0.0
scalew: 0.0
weighting: drizzle
coord_system: skyalign
ra_center: None
dec_center: None
cube_pa: None
nspax_x: None
nspax_y: None
rois: 0.0
roiw: 0.0
weight_power: 2.0
wavemin: None
wavemax: None
single: False
skip_dqflagging: False
offset_file: None
debug_spaxel: -1 -1 -1
extract_1d:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
subtract_background: None
apply_apcorr: True
extraction_type: box
use_source_posn: None
position_offset: 0.0
model_nod_pair: True
optimize_psf_location: True
smoothing_length: None
bkg_fit: None
bkg_order: None
log_increment: 50
save_profile: False
save_scene_model: False
save_residual_image: False
center_xy: None
ifu_autocen: False
bkg_sigma_clip: 3.0
ifu_rfcorr: True
ifu_set_srctype: None
ifu_rscale: None
ifu_covar_scale: 1.0
soss_atoca: True
soss_threshold: 0.01
soss_n_os: 2
soss_wave_grid_in: None
soss_wave_grid_out: None
soss_estimate: None
soss_rtol: 0.0001
soss_max_grid_size: 20000
soss_tikfac: None
soss_width: 40.0
soss_bad_pix: masking
soss_modelname: None
soss_order_3: True
2026-04-17 15:31:37,056 - stpipe.pipeline - INFO - Prefetching reference files for dataset: 'jw01366002001_04103_00001-seg002_nrcalong_rateints.fits' reftypes = ['apcorr', 'camera', 'collimator', 'dflat', 'disperser', 'distortion', 'fflat', 'filteroffset', 'flat', 'fore', 'fpa', 'fringe', 'ifufore', 'ifupost', 'ifuslicer', 'msa', 'ote', 'pastasoss', 'psf', 'regions', 'sflat', 'speckernel', 'specprofile', 'specwcs', 'wavecorr', 'wavelengthrange']
2026-04-17 15:31:37,060 - stpipe.pipeline - INFO - Prefetch for APCORR reference file is 'N/A'.
2026-04-17 15:31:37,061 - stpipe.pipeline - INFO - Prefetch for CAMERA reference file is 'N/A'.
2026-04-17 15:31:37,061 - stpipe.pipeline - INFO - Prefetch for COLLIMATOR reference file is 'N/A'.
2026-04-17 15:31:37,062 - stpipe.pipeline - INFO - Prefetch for DFLAT reference file is 'N/A'.
2026-04-17 15:31:37,063 - stpipe.pipeline - INFO - Prefetch for DISPERSER reference file is 'N/A'.
2026-04-17 15:31:37,063 - stpipe.pipeline - INFO - Prefetch for DISTORTION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_distortion_0259.asdf'.
2026-04-17 15:31:37,064 - stpipe.pipeline - INFO - Override for EXTRACT1D reference file is '/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage2/JWST_NIRCAM_extract1d_F322W2.json'.
2026-04-17 15:31:37,064 - stpipe.pipeline - INFO - Prefetch for FFLAT reference file is 'N/A'.
2026-04-17 15:31:37,065 - stpipe.pipeline - INFO - Prefetch for FILTEROFFSET reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_filteroffset_0007.asdf'.
2026-04-17 15:31:37,066 - stpipe.pipeline - INFO - Prefetch for FLAT reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0756.fits'.
2026-04-17 15:31:37,067 - stpipe.pipeline - INFO - Prefetch for FORE reference file is 'N/A'.
2026-04-17 15:31:37,067 - stpipe.pipeline - INFO - Prefetch for FPA reference file is 'N/A'.
2026-04-17 15:31:37,068 - stpipe.pipeline - INFO - Prefetch for FRINGE reference file is 'N/A'.
2026-04-17 15:31:37,068 - stpipe.pipeline - INFO - Prefetch for IFUFORE reference file is 'N/A'.
2026-04-17 15:31:37,069 - stpipe.pipeline - INFO - Prefetch for IFUPOST reference file is 'N/A'.
2026-04-17 15:31:37,070 - stpipe.pipeline - INFO - Prefetch for IFUSLICER reference file is 'N/A'.
2026-04-17 15:31:37,070 - stpipe.pipeline - INFO - Prefetch for MSA reference file is 'N/A'.
2026-04-17 15:31:37,071 - stpipe.pipeline - INFO - Prefetch for OTE reference file is 'N/A'.
2026-04-17 15:31:37,071 - stpipe.pipeline - INFO - Prefetch for PASTASOSS reference file is 'N/A'.
2026-04-17 15:31:37,072 - stpipe.pipeline - INFO - Prefetch for PSF reference file is 'N/A'.
2026-04-17 15:31:37,072 - stpipe.pipeline - INFO - Prefetch for REGIONS reference file is 'N/A'.
2026-04-17 15:31:37,073 - stpipe.pipeline - INFO - Prefetch for SFLAT reference file is 'N/A'.
2026-04-17 15:31:37,073 - stpipe.pipeline - INFO - Prefetch for SPECKERNEL reference file is 'N/A'.
2026-04-17 15:31:37,074 - stpipe.pipeline - INFO - Prefetch for SPECPROFILE reference file is 'N/A'.
2026-04-17 15:31:37,074 - stpipe.pipeline - INFO - Prefetch for SPECWCS reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_specwcs_0259.asdf'.
2026-04-17 15:31:37,075 - stpipe.pipeline - INFO - Prefetch for WAVECORR reference file is 'N/A'.
2026-04-17 15:31:37,075 - stpipe.pipeline - INFO - Prefetch for WAVELENGTHRANGE reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_wavelengthrange_0004.asdf'.
2026-04-17 15:31:37,077 - jwst.pipeline.calwebb_spec2 - INFO - Starting calwebb_spec2 ...
2026-04-17 15:31:37,078 - jwst.pipeline.calwebb_spec2 - INFO - Processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg002_nrcalong
2026-04-17 15:31:37,078 - jwst.pipeline.calwebb_spec2 - INFO - Working on input /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg002_nrcalong_rateints.fits ...
2026-04-17 15:31:39,347 - stpipe.step - INFO - Step assign_wcs running with args (<CubeModel(92, 256, 2048) from jw01366002001_04103_00001-seg002_nrcalong_rateints.fits>,).
2026-04-17 15:31:39,401 - jwst.assign_wcs.nircam - INFO - Added Barycentric velocity correction: 0.9999043045827591
2026-04-17 15:31:39,508 - jwst.assign_wcs.assign_wcs - INFO - COMPLETED assign_wcs
2026-04-17 15:31:39,513 - stpipe.step - INFO - Step assign_wcs done
2026-04-17 15:31:39,674 - stpipe.step - INFO - Step badpix_selfcal running with args (<CubeModel(92, 256, 2048) from jw01366002001_04103_00001-seg002_nrcalong_rateints.fits>, [], []).
2026-04-17 15:31:39,675 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:39,827 - stpipe.step - INFO - Step msa_flagging running with args (<CubeModel(92, 256, 2048) from jw01366002001_04103_00001-seg002_nrcalong_rateints.fits>,).
2026-04-17 15:31:39,828 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:39,978 - stpipe.step - INFO - Step clean_flicker_noise running with args (<CubeModel(92, 256, 2048) from jw01366002001_04103_00001-seg002_nrcalong_rateints.fits>,).
2026-04-17 15:31:39,979 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:40,128 - stpipe.step - INFO - Step imprint_subtract running with args (<CubeModel(92, 256, 2048) from jw01366002001_04103_00001-seg002_nrcalong_rateints.fits>, []).
2026-04-17 15:31:40,130 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:40,280 - stpipe.step - INFO - Step bkg_subtract running with args (<CubeModel(92, 256, 2048) from jw01366002001_04103_00001-seg002_nrcalong_rateints.fits>, []).
2026-04-17 15:31:40,281 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:40,430 - stpipe.step - INFO - Step flat_field running with args (<CubeModel(92, 256, 2048) from jw01366002001_04103_00001-seg002_nrcalong_rateints.fits>,).
2026-04-17 15:31:40,503 - jwst.flatfield.flat_field_step - INFO - Using FLAT reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0756.fits
2026-04-17 15:31:40,504 - jwst.flatfield.flat_field_step - INFO - No reference found for type FFLAT
2026-04-17 15:31:40,504 - jwst.flatfield.flat_field_step - INFO - No reference found for type SFLAT
2026-04-17 15:31:40,505 - jwst.flatfield.flat_field_step - INFO - No reference found for type DFLAT
2026-04-17 15:31:40,506 - jwst.flatfield.flat_field - INFO - Extracting matching subarray from flat
2026-04-17 15:31:41,100 - stpipe.step - INFO - Step flat_field done
2026-04-17 15:31:41,271 - stpipe.step - INFO - Step extract_2d running with args (<CubeModel(92, 256, 2048) from jw01366002001_04103_00001-seg002_nrcalong_rateints.fits>,).
2026-04-17 15:31:41,275 - jwst.extract_2d.extract_2d - INFO - EXP_TYPE is NRC_TSGRISM
2026-04-17 15:31:41,285 - jwst.extract_2d.grisms - INFO - Using default order extraction from reference file
2026-04-17 15:31:41,286 - jwst.extract_2d.grisms - INFO - Extracting order: [1]
2026-04-17 15:31:41,325 - jwst.extract_2d.grisms - INFO - Setting extraction height to 64
2026-04-17 15:31:41,334 - jwst.extract_2d.grisms - INFO - WCS made explicit for order: 1
2026-04-17 15:31:41,334 - jwst.extract_2d.grisms - INFO - Spectral trace extents: (xmin: 104, ymin: 0), (xmax: 1867, ymax: 63)
2026-04-17 15:31:41,335 - jwst.extract_2d.grisms - INFO - Extraction limits: (xmin: 0, ymin: 0), (xmax: 2047, ymax: 63)
2026-04-17 15:31:41,502 - jwst.extract_2d.grisms - INFO - Finished extraction
2026-04-17 15:31:41,506 - stpipe.step - INFO - Step extract_2d done
2026-04-17 15:31:41,689 - stpipe.step - INFO - Step srctype running with args (<SlitModel(92, 64, 2048) from jw01366002001_04103_00001-seg002_nrcalong_rateints.fits>,).
2026-04-17 15:31:41,690 - jwst.srctype.srctype - INFO - Input EXP_TYPE is NRC_TSGRISM
2026-04-17 15:31:41,691 - jwst.srctype.srctype - INFO - Input SRCTYAPT = UNKNOWN
2026-04-17 15:31:41,691 - jwst.srctype.srctype - INFO - Input is a TSO exposure; setting SRCTYPE = POINT
2026-04-17 15:31:41,694 - stpipe.step - INFO - Step srctype done
2026-04-17 15:31:41,847 - stpipe.step - INFO - Step straylight running with args (<SlitModel(92, 64, 2048) from jw01366002001_04103_00001-seg002_nrcalong_rateints.fits>,).
2026-04-17 15:31:41,848 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:41,999 - stpipe.step - INFO - Step fringe running with args (<SlitModel(92, 64, 2048) from jw01366002001_04103_00001-seg002_nrcalong_rateints.fits>,).
2026-04-17 15:31:42,000 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:42,151 - stpipe.step - INFO - Step pathloss running with args (<SlitModel(92, 64, 2048) from jw01366002001_04103_00001-seg002_nrcalong_rateints.fits>,).
2026-04-17 15:31:42,153 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:42,311 - stpipe.step - INFO - Step barshadow running with args (<SlitModel(92, 64, 2048) from jw01366002001_04103_00001-seg002_nrcalong_rateints.fits>,).
2026-04-17 15:31:42,312 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:42,465 - stpipe.step - INFO - Step wfss_contam running with args (<SlitModel(92, 64, 2048) from jw01366002001_04103_00001-seg002_nrcalong_rateints.fits>,).
2026-04-17 15:31:42,466 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:42,635 - stpipe.step - INFO - Step photom running with args (<SlitModel(92, 64, 2048) from jw01366002001_04103_00001-seg002_nrcalong_rateints.fits>,).
2026-04-17 15:31:42,636 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:42,870 - stpipe.step - INFO - Step pixel_replace running with args (<SlitModel(92, 64, 2048) from ./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg002_nrcalong_calints.fits>,).
2026-04-17 15:31:42,871 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:43,105 - stpipe.step - INFO - Step extract_1d running with args (<SlitModel(92, 64, 2048) from ./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg002_nrcalong_calints.fits>,).
2026-04-17 15:31:43,106 - jwst.extract_1d.extract_1d_step - INFO - Using EXTRACT1D reference file /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage2/JWST_NIRCAM_extract1d_F322W2.json
2026-04-17 15:31:43,142 - jwst.extract_1d.extract - INFO - Processing spectral order 1
2026-04-17 15:31:43,204 - jwst.extract_1d.extract - WARNING - The photom step has not been run.
2026-04-17 15:31:43,206 - jwst.extract_1d.extract - INFO - Aperture start/stop: 31.00 -> 37.00 (inclusive)
2026-04-17 15:31:43,209 - jwst.extract_1d.extract - INFO - Mean aperture start/stop from bkg_coeff: 21.50 -> 30.50 (inclusive)
2026-04-17 15:31:43,211 - jwst.extract_1d.extract - INFO - Mean aperture start/stop from bkg_coeff: 35.50 -> 44.50 (inclusive)
2026-04-17 15:31:43,218 - jwst.extract_1d.extract - INFO - Beginning loop over 92 integrations ...
2026-04-17 15:31:45,712 - jwst.extract_1d.extract - INFO - ... 50 integrations done
2026-04-17 15:31:48,042 - jwst.extract_1d.extract - INFO - All 92 integrations done
2026-04-17 15:31:48,274 - stpipe.step - INFO - Saved model in ./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg002_nrcalong_x1dints.fits
2026-04-17 15:31:48,275 - stpipe.step - INFO - Step extract_1d done
2026-04-17 15:31:48,278 - jwst.pipeline.calwebb_spec2 - INFO - Finished processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg002_nrcalong
2026-04-17 15:31:48,290 - jwst.pipeline.calwebb_spec2 - INFO - Ending calwebb_spec2
2026-04-17 15:31:48,290 - jwst.stpipe.core - INFO - Results used CRDS context: jwst_1535.pmap
2026-04-17 15:31:48,480 - stpipe.step - INFO - Saved model in ./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg002_nrcalong_calints.fits
2026-04-17 15:31:48,481 - stpipe.step - INFO - Step Spec2Pipeline done
2026-04-17 15:31:48,481 - jwst.stpipe.core - INFO - Results used jwst version: 2.0.0
2026-04-17 15:31:48,521 - CRDS - ERROR - Error determining best reference for 'pars-targcentroidstep' = Unknown reference type 'pars-targcentroidstep'
2026-04-17 15:31:48,535 - stpipe.pipeline - INFO - PARS-SPEC2PIPELINE parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-spec2pipeline_0006.asdf
2026-04-17 15:31:48,559 - stpipe.step - INFO - Spec2Pipeline instance created.
2026-04-17 15:31:48,561 - stpipe.step - INFO - AssignWcsStep instance created.
2026-04-17 15:31:48,562 - stpipe.step - INFO - BadpixSelfcalStep instance created.
2026-04-17 15:31:48,563 - stpipe.step - INFO - MSAFlagOpenStep instance created.
2026-04-17 15:31:48,564 - stpipe.step - INFO - CleanFlickerNoiseStep instance created.
2026-04-17 15:31:48,566 - stpipe.step - INFO - BackgroundStep instance created.
2026-04-17 15:31:48,567 - stpipe.step - INFO - ImprintStep instance created.
2026-04-17 15:31:48,568 - stpipe.step - INFO - Extract2dStep instance created.
2026-04-17 15:31:48,573 - stpipe.step - INFO - MasterBackgroundMosStep instance created.
2026-04-17 15:31:48,574 - stpipe.step - INFO - FlatFieldStep instance created.
2026-04-17 15:31:48,575 - stpipe.step - INFO - PathLossStep instance created.
2026-04-17 15:31:48,576 - stpipe.step - INFO - BarShadowStep instance created.
2026-04-17 15:31:48,577 - stpipe.step - INFO - PhotomStep instance created.
2026-04-17 15:31:48,578 - stpipe.step - INFO - PixelReplaceStep instance created.
2026-04-17 15:31:48,580 - stpipe.step - INFO - ResampleSpecStep instance created.
2026-04-17 15:31:48,581 - stpipe.step - INFO - Extract1dStep instance created.
2026-04-17 15:31:48,582 - stpipe.step - INFO - TargCentroidStep instance created.
2026-04-17 15:31:48,583 - stpipe.step - INFO - WavecorrStep instance created.
2026-04-17 15:31:48,584 - stpipe.step - INFO - FlatFieldStep instance created.
2026-04-17 15:31:48,585 - stpipe.step - INFO - SourceTypeStep instance created.
2026-04-17 15:31:48,586 - stpipe.step - INFO - StraylightStep instance created.
2026-04-17 15:31:48,587 - stpipe.step - INFO - FringeStep instance created.
2026-04-17 15:31:48,588 - stpipe.step - INFO - ResidualFringeStep instance created.
2026-04-17 15:31:48,589 - stpipe.step - INFO - PathLossStep instance created.
2026-04-17 15:31:48,590 - stpipe.step - INFO - BarShadowStep instance created.
2026-04-17 15:31:48,591 - stpipe.step - INFO - WfssContamStep instance created.
2026-04-17 15:31:48,592 - stpipe.step - INFO - PhotomStep instance created.
2026-04-17 15:31:48,593 - stpipe.step - INFO - AdaptiveTraceModelStep instance created.
2026-04-17 15:31:48,594 - stpipe.step - INFO - PixelReplaceStep instance created.
2026-04-17 15:31:48,595 - stpipe.step - INFO - ResampleSpecStep instance created.
2026-04-17 15:31:48,597 - stpipe.step - INFO - CubeBuildStep instance created.
2026-04-17 15:31:48,599 - stpipe.step - INFO - Extract1dStep instance created.
2026-04-17 15:31:48,797 - stpipe.step - INFO - Step Spec2Pipeline running with args ('/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg003_nrcalong_rateints.fits',).
2026-04-17 15:31:48,832 - stpipe.step - INFO - Step Spec2Pipeline parameters are:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: ./nrc_tso_demo_data/01366/Obs002/stage2
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: True
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_bsub: False
fail_on_exception: True
save_wfss_esec: False
steps:
assign_wcs:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
sip_approx: True
sip_max_pix_error: 0.01
sip_degree: None
sip_max_inv_pix_error: 0.01
sip_inv_degree: None
sip_npoints: 12
slit_y_low: -0.55
slit_y_high: 0.55
nrs_ifu_slice_wcs: False
badpix_selfcal:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
flagfrac_lower: 0.001
flagfrac_upper: 0.001
kernel_size: 15
force_single: False
save_flagged_bkg: False
msa_flagging:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
clean_flicker_noise:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
autoparam: False
fit_method: median
fit_by_channel: False
background_method: median
background_box_size: None
mask_science_regions: False
apply_flat_field: False
n_sigma: 2.0
fit_histogram: False
single_mask: True
user_mask: None
save_mask: False
save_background: False
save_noise: False
bkg_subtract:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
bkg_list: None
save_combined_background: False
sigma: 3.0
maxiters: None
soss_source_percentile: 35.0
soss_bkg_percentile: None
wfss_mmag_extract: None
wfss_mask: None
wfss_maxiter: 5
wfss_rms_stop: 0.0
wfss_outlier_percent: 1.0
imprint_subtract:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
extract_2d:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
slit_names: None
source_ids: None
source_ra: None
source_dec: None
source_max_sep: 2.0
extract_orders: None
grism_objects: None
tsgrism_extract_height: 64
wfss_extract_half_height: 5
wfss_mmag_extract: None
wfss_nbright: 1000
master_background_mos:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: True
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
sigma_clip: 3.0
median_kernel: 1
force_subtract: False
save_background: False
user_background: None
inverse: False
steps:
flat_field:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_interpolated_flat: False
user_supplied_flat: None
inverse: False
pathloss:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
user_slit_loc: None
barshadow:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
photom:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
apply_time_correction: True
pixel_replace:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: True
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
algorithm: fit_profile
n_adjacent_cols: 3
resample_spec:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
pixfrac: 1.0
kernel: square
fillval: NAN
weight_type: ivm
output_shape: None
pixel_scale_ratio: 1.0
pixel_scale: None
output_wcs: ''
single: False
blendheaders: True
in_memory: True
propagate_dq: False
extract_1d:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
subtract_background: None
apply_apcorr: True
extraction_type: box
use_source_posn: None
position_offset: 0.0
model_nod_pair: True
optimize_psf_location: True
smoothing_length: None
bkg_fit: None
bkg_order: None
log_increment: 50
save_profile: False
save_scene_model: False
save_residual_image: False
center_xy: None
ifu_autocen: False
bkg_sigma_clip: 3.0
ifu_rfcorr: True
ifu_set_srctype: None
ifu_rscale: None
ifu_covar_scale: 1.0
soss_atoca: True
soss_threshold: 0.01
soss_n_os: 2
soss_wave_grid_in: None
soss_wave_grid_out: None
soss_estimate: None
soss_rtol: 0.0001
soss_max_grid_size: 20000
soss_tikfac: None
soss_width: 40.0
soss_bad_pix: masking
soss_modelname: None
soss_order_3: True
targ_centroid:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
ta_file: None
wavecorr:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
flat_field:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_interpolated_flat: False
user_supplied_flat: None
inverse: False
srctype:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
source_type: None
straylight:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
clean_showers: False
shower_plane: 3
shower_x_stddev: 18.0
shower_y_stddev: 5.0
shower_low_reject: 0.1
shower_high_reject: 99.9
save_shower_model: False
fringe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
residual_fringe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: residual_fringe
search_output_file: False
input_dir: ''
save_intermediate_results: False
ignore_region_min: None
ignore_region_max: None
pathloss:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
user_slit_loc: None
barshadow:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
wfss_contam:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
save_simulated_image: False
save_contam_images: False
maximum_cores: '1'
orders: None
magnitude_limit: None
wl_oversample: 2
max_pixels_per_chunk: 5000
photom:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
apply_time_correction: True
adaptive_trace_model:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: True
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
fit_threshold: 10.0
oversample: 1.0
slope_limit: 0.1
psf_optimal: False
save_intermediate_results: False
pixel_replace:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: True
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
algorithm: fit_profile
n_adjacent_cols: 3
resample_spec:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
pixfrac: 1.0
kernel: square
fillval: NAN
weight_type: ivm
output_shape: None
pixel_scale_ratio: 1.0
pixel_scale: None
output_wcs: ''
single: False
blendheaders: True
in_memory: True
propagate_dq: False
cube_build:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: True
output_use_index: True
save_results: False
skip: True
suffix: s3d
search_output_file: False
input_dir: ''
pipeline: 3
channel: all
band: all
grating: all
filter: all
output_type: None
linear_wave: True
scalexy: 0.0
scalew: 0.0
weighting: drizzle
coord_system: skyalign
ra_center: None
dec_center: None
cube_pa: None
nspax_x: None
nspax_y: None
rois: 0.0
roiw: 0.0
weight_power: 2.0
wavemin: None
wavemax: None
single: False
skip_dqflagging: False
offset_file: None
debug_spaxel: -1 -1 -1
extract_1d:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
subtract_background: None
apply_apcorr: True
extraction_type: box
use_source_posn: None
position_offset: 0.0
model_nod_pair: True
optimize_psf_location: True
smoothing_length: None
bkg_fit: None
bkg_order: None
log_increment: 50
save_profile: False
save_scene_model: False
save_residual_image: False
center_xy: None
ifu_autocen: False
bkg_sigma_clip: 3.0
ifu_rfcorr: True
ifu_set_srctype: None
ifu_rscale: None
ifu_covar_scale: 1.0
soss_atoca: True
soss_threshold: 0.01
soss_n_os: 2
soss_wave_grid_in: None
soss_wave_grid_out: None
soss_estimate: None
soss_rtol: 0.0001
soss_max_grid_size: 20000
soss_tikfac: None
soss_width: 40.0
soss_bad_pix: masking
soss_modelname: None
soss_order_3: True
2026-04-17 15:31:48,864 - stpipe.pipeline - INFO - Prefetching reference files for dataset: 'jw01366002001_04103_00001-seg003_nrcalong_rateints.fits' reftypes = ['apcorr', 'camera', 'collimator', 'dflat', 'disperser', 'distortion', 'fflat', 'filteroffset', 'flat', 'fore', 'fpa', 'fringe', 'ifufore', 'ifupost', 'ifuslicer', 'msa', 'ote', 'pastasoss', 'psf', 'regions', 'sflat', 'speckernel', 'specprofile', 'specwcs', 'wavecorr', 'wavelengthrange']
2026-04-17 15:31:48,868 - stpipe.pipeline - INFO - Prefetch for APCORR reference file is 'N/A'.
2026-04-17 15:31:48,868 - stpipe.pipeline - INFO - Prefetch for CAMERA reference file is 'N/A'.
2026-04-17 15:31:48,869 - stpipe.pipeline - INFO - Prefetch for COLLIMATOR reference file is 'N/A'.
2026-04-17 15:31:48,869 - stpipe.pipeline - INFO - Prefetch for DFLAT reference file is 'N/A'.
2026-04-17 15:31:48,870 - stpipe.pipeline - INFO - Prefetch for DISPERSER reference file is 'N/A'.
2026-04-17 15:31:48,870 - stpipe.pipeline - INFO - Prefetch for DISTORTION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_distortion_0259.asdf'.
2026-04-17 15:31:48,871 - stpipe.pipeline - INFO - Override for EXTRACT1D reference file is '/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage2/JWST_NIRCAM_extract1d_F322W2.json'.
2026-04-17 15:31:48,872 - stpipe.pipeline - INFO - Prefetch for FFLAT reference file is 'N/A'.
2026-04-17 15:31:48,873 - stpipe.pipeline - INFO - Prefetch for FILTEROFFSET reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_filteroffset_0007.asdf'.
2026-04-17 15:31:48,873 - stpipe.pipeline - INFO - Prefetch for FLAT reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0756.fits'.
2026-04-17 15:31:48,874 - stpipe.pipeline - INFO - Prefetch for FORE reference file is 'N/A'.
2026-04-17 15:31:48,874 - stpipe.pipeline - INFO - Prefetch for FPA reference file is 'N/A'.
2026-04-17 15:31:48,875 - stpipe.pipeline - INFO - Prefetch for FRINGE reference file is 'N/A'.
2026-04-17 15:31:48,875 - stpipe.pipeline - INFO - Prefetch for IFUFORE reference file is 'N/A'.
2026-04-17 15:31:48,877 - stpipe.pipeline - INFO - Prefetch for IFUPOST reference file is 'N/A'.
2026-04-17 15:31:48,877 - stpipe.pipeline - INFO - Prefetch for IFUSLICER reference file is 'N/A'.
2026-04-17 15:31:48,878 - stpipe.pipeline - INFO - Prefetch for MSA reference file is 'N/A'.
2026-04-17 15:31:48,878 - stpipe.pipeline - INFO - Prefetch for OTE reference file is 'N/A'.
2026-04-17 15:31:48,879 - stpipe.pipeline - INFO - Prefetch for PASTASOSS reference file is 'N/A'.
2026-04-17 15:31:48,879 - stpipe.pipeline - INFO - Prefetch for PSF reference file is 'N/A'.
2026-04-17 15:31:48,880 - stpipe.pipeline - INFO - Prefetch for REGIONS reference file is 'N/A'.
2026-04-17 15:31:48,880 - stpipe.pipeline - INFO - Prefetch for SFLAT reference file is 'N/A'.
2026-04-17 15:31:48,881 - stpipe.pipeline - INFO - Prefetch for SPECKERNEL reference file is 'N/A'.
2026-04-17 15:31:48,882 - stpipe.pipeline - INFO - Prefetch for SPECPROFILE reference file is 'N/A'.
2026-04-17 15:31:48,883 - stpipe.pipeline - INFO - Prefetch for SPECWCS reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_specwcs_0259.asdf'.
2026-04-17 15:31:48,883 - stpipe.pipeline - INFO - Prefetch for WAVECORR reference file is 'N/A'.
2026-04-17 15:31:48,884 - stpipe.pipeline - INFO - Prefetch for WAVELENGTHRANGE reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_wavelengthrange_0004.asdf'.
2026-04-17 15:31:48,884 - jwst.pipeline.calwebb_spec2 - INFO - Starting calwebb_spec2 ...
2026-04-17 15:31:48,885 - jwst.pipeline.calwebb_spec2 - INFO - Processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg003_nrcalong
2026-04-17 15:31:48,886 - jwst.pipeline.calwebb_spec2 - INFO - Working on input /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg003_nrcalong_rateints.fits ...
2026-04-17 15:31:51,133 - stpipe.step - INFO - Step assign_wcs running with args (<CubeModel(91, 256, 2048) from jw01366002001_04103_00001-seg003_nrcalong_rateints.fits>,).
2026-04-17 15:31:51,186 - jwst.assign_wcs.nircam - INFO - Added Barycentric velocity correction: 0.9999043045827591
2026-04-17 15:31:51,293 - jwst.assign_wcs.assign_wcs - INFO - COMPLETED assign_wcs
2026-04-17 15:31:51,298 - stpipe.step - INFO - Step assign_wcs done
2026-04-17 15:31:51,456 - stpipe.step - INFO - Step badpix_selfcal running with args (<CubeModel(91, 256, 2048) from jw01366002001_04103_00001-seg003_nrcalong_rateints.fits>, [], []).
2026-04-17 15:31:51,457 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:51,609 - stpipe.step - INFO - Step msa_flagging running with args (<CubeModel(91, 256, 2048) from jw01366002001_04103_00001-seg003_nrcalong_rateints.fits>,).
2026-04-17 15:31:51,610 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:51,761 - stpipe.step - INFO - Step clean_flicker_noise running with args (<CubeModel(91, 256, 2048) from jw01366002001_04103_00001-seg003_nrcalong_rateints.fits>,).
2026-04-17 15:31:51,762 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:51,913 - stpipe.step - INFO - Step imprint_subtract running with args (<CubeModel(91, 256, 2048) from jw01366002001_04103_00001-seg003_nrcalong_rateints.fits>, []).
2026-04-17 15:31:51,914 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:52,064 - stpipe.step - INFO - Step bkg_subtract running with args (<CubeModel(91, 256, 2048) from jw01366002001_04103_00001-seg003_nrcalong_rateints.fits>, []).
2026-04-17 15:31:52,065 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:52,216 - stpipe.step - INFO - Step flat_field running with args (<CubeModel(91, 256, 2048) from jw01366002001_04103_00001-seg003_nrcalong_rateints.fits>,).
2026-04-17 15:31:52,286 - jwst.flatfield.flat_field_step - INFO - Using FLAT reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0756.fits
2026-04-17 15:31:52,287 - jwst.flatfield.flat_field_step - INFO - No reference found for type FFLAT
2026-04-17 15:31:52,287 - jwst.flatfield.flat_field_step - INFO - No reference found for type SFLAT
2026-04-17 15:31:52,289 - jwst.flatfield.flat_field_step - INFO - No reference found for type DFLAT
2026-04-17 15:31:52,289 - jwst.flatfield.flat_field - INFO - Extracting matching subarray from flat
2026-04-17 15:31:52,875 - stpipe.step - INFO - Step flat_field done
2026-04-17 15:31:53,051 - stpipe.step - INFO - Step extract_2d running with args (<CubeModel(91, 256, 2048) from jw01366002001_04103_00001-seg003_nrcalong_rateints.fits>,).
2026-04-17 15:31:53,055 - jwst.extract_2d.extract_2d - INFO - EXP_TYPE is NRC_TSGRISM
2026-04-17 15:31:53,065 - jwst.extract_2d.grisms - INFO - Using default order extraction from reference file
2026-04-17 15:31:53,066 - jwst.extract_2d.grisms - INFO - Extracting order: [1]
2026-04-17 15:31:53,104 - jwst.extract_2d.grisms - INFO - Setting extraction height to 64
2026-04-17 15:31:53,113 - jwst.extract_2d.grisms - INFO - WCS made explicit for order: 1
2026-04-17 15:31:53,114 - jwst.extract_2d.grisms - INFO - Spectral trace extents: (xmin: 104, ymin: 0), (xmax: 1867, ymax: 63)
2026-04-17 15:31:53,115 - jwst.extract_2d.grisms - INFO - Extraction limits: (xmin: 0, ymin: 0), (xmax: 2047, ymax: 63)
2026-04-17 15:31:53,283 - jwst.extract_2d.grisms - INFO - Finished extraction
2026-04-17 15:31:53,286 - stpipe.step - INFO - Step extract_2d done
2026-04-17 15:31:53,471 - stpipe.step - INFO - Step srctype running with args (<SlitModel(91, 64, 2048) from jw01366002001_04103_00001-seg003_nrcalong_rateints.fits>,).
2026-04-17 15:31:53,472 - jwst.srctype.srctype - INFO - Input EXP_TYPE is NRC_TSGRISM
2026-04-17 15:31:53,473 - jwst.srctype.srctype - INFO - Input SRCTYAPT = UNKNOWN
2026-04-17 15:31:53,473 - jwst.srctype.srctype - INFO - Input is a TSO exposure; setting SRCTYPE = POINT
2026-04-17 15:31:53,476 - stpipe.step - INFO - Step srctype done
2026-04-17 15:31:53,629 - stpipe.step - INFO - Step straylight running with args (<SlitModel(91, 64, 2048) from jw01366002001_04103_00001-seg003_nrcalong_rateints.fits>,).
2026-04-17 15:31:53,630 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:53,783 - stpipe.step - INFO - Step fringe running with args (<SlitModel(91, 64, 2048) from jw01366002001_04103_00001-seg003_nrcalong_rateints.fits>,).
2026-04-17 15:31:53,784 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:53,936 - stpipe.step - INFO - Step pathloss running with args (<SlitModel(91, 64, 2048) from jw01366002001_04103_00001-seg003_nrcalong_rateints.fits>,).
2026-04-17 15:31:53,937 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:54,090 - stpipe.step - INFO - Step barshadow running with args (<SlitModel(91, 64, 2048) from jw01366002001_04103_00001-seg003_nrcalong_rateints.fits>,).
2026-04-17 15:31:54,091 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:54,243 - stpipe.step - INFO - Step wfss_contam running with args (<SlitModel(91, 64, 2048) from jw01366002001_04103_00001-seg003_nrcalong_rateints.fits>,).
2026-04-17 15:31:54,245 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:54,397 - stpipe.step - INFO - Step photom running with args (<SlitModel(91, 64, 2048) from jw01366002001_04103_00001-seg003_nrcalong_rateints.fits>,).
2026-04-17 15:31:54,398 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:54,636 - stpipe.step - INFO - Step pixel_replace running with args (<SlitModel(91, 64, 2048) from ./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg003_nrcalong_calints.fits>,).
2026-04-17 15:31:54,637 - stpipe.step - INFO - Step skipped.
2026-04-17 15:31:54,875 - stpipe.step - INFO - Step extract_1d running with args (<SlitModel(91, 64, 2048) from ./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg003_nrcalong_calints.fits>,).
2026-04-17 15:31:54,876 - jwst.extract_1d.extract_1d_step - INFO - Using EXTRACT1D reference file /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage2/JWST_NIRCAM_extract1d_F322W2.json
2026-04-17 15:31:54,912 - jwst.extract_1d.extract - INFO - Processing spectral order 1
2026-04-17 15:31:54,973 - jwst.extract_1d.extract - WARNING - The photom step has not been run.
2026-04-17 15:31:54,975 - jwst.extract_1d.extract - INFO - Aperture start/stop: 31.00 -> 37.00 (inclusive)
2026-04-17 15:31:54,977 - jwst.extract_1d.extract - INFO - Mean aperture start/stop from bkg_coeff: 21.50 -> 30.50 (inclusive)
2026-04-17 15:31:54,980 - jwst.extract_1d.extract - INFO - Mean aperture start/stop from bkg_coeff: 35.50 -> 44.50 (inclusive)
2026-04-17 15:31:54,986 - jwst.extract_1d.extract - INFO - Beginning loop over 91 integrations ...
2026-04-17 15:31:57,493 - jwst.extract_1d.extract - INFO - ... 50 integrations done
2026-04-17 15:31:59,771 - jwst.extract_1d.extract - INFO - All 91 integrations done
2026-04-17 15:32:00,002 - stpipe.step - INFO - Saved model in ./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg003_nrcalong_x1dints.fits
2026-04-17 15:32:00,002 - stpipe.step - INFO - Step extract_1d done
2026-04-17 15:32:00,006 - jwst.pipeline.calwebb_spec2 - INFO - Finished processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg003_nrcalong
2026-04-17 15:32:00,018 - jwst.pipeline.calwebb_spec2 - INFO - Ending calwebb_spec2
2026-04-17 15:32:00,018 - jwst.stpipe.core - INFO - Results used CRDS context: jwst_1535.pmap
2026-04-17 15:32:00,213 - stpipe.step - INFO - Saved model in ./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg003_nrcalong_calints.fits
2026-04-17 15:32:00,213 - stpipe.step - INFO - Step Spec2Pipeline done
2026-04-17 15:32:00,214 - jwst.stpipe.core - INFO - Results used jwst version: 2.0.0
2026-04-17 15:32:00,251 - CRDS - ERROR - Error determining best reference for 'pars-targcentroidstep' = Unknown reference type 'pars-targcentroidstep'
2026-04-17 15:32:00,265 - stpipe.pipeline - INFO - PARS-SPEC2PIPELINE parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-spec2pipeline_0006.asdf
2026-04-17 15:32:00,289 - stpipe.step - INFO - Spec2Pipeline instance created.
2026-04-17 15:32:00,290 - stpipe.step - INFO - AssignWcsStep instance created.
2026-04-17 15:32:00,292 - stpipe.step - INFO - BadpixSelfcalStep instance created.
2026-04-17 15:32:00,292 - stpipe.step - INFO - MSAFlagOpenStep instance created.
2026-04-17 15:32:00,294 - stpipe.step - INFO - CleanFlickerNoiseStep instance created.
2026-04-17 15:32:00,296 - stpipe.step - INFO - BackgroundStep instance created.
2026-04-17 15:32:00,297 - stpipe.step - INFO - ImprintStep instance created.
2026-04-17 15:32:00,299 - stpipe.step - INFO - Extract2dStep instance created.
2026-04-17 15:32:00,304 - stpipe.step - INFO - MasterBackgroundMosStep instance created.
2026-04-17 15:32:00,305 - stpipe.step - INFO - FlatFieldStep instance created.
2026-04-17 15:32:00,306 - stpipe.step - INFO - PathLossStep instance created.
2026-04-17 15:32:00,307 - stpipe.step - INFO - BarShadowStep instance created.
2026-04-17 15:32:00,308 - stpipe.step - INFO - PhotomStep instance created.
2026-04-17 15:32:00,310 - stpipe.step - INFO - PixelReplaceStep instance created.
2026-04-17 15:32:00,312 - stpipe.step - INFO - ResampleSpecStep instance created.
2026-04-17 15:32:00,314 - stpipe.step - INFO - Extract1dStep instance created.
2026-04-17 15:32:00,315 - stpipe.step - INFO - TargCentroidStep instance created.
2026-04-17 15:32:00,316 - stpipe.step - INFO - WavecorrStep instance created.
2026-04-17 15:32:00,317 - stpipe.step - INFO - FlatFieldStep instance created.
2026-04-17 15:32:00,318 - stpipe.step - INFO - SourceTypeStep instance created.
2026-04-17 15:32:00,320 - stpipe.step - INFO - StraylightStep instance created.
2026-04-17 15:32:00,321 - stpipe.step - INFO - FringeStep instance created.
2026-04-17 15:32:00,322 - stpipe.step - INFO - ResidualFringeStep instance created.
2026-04-17 15:32:00,323 - stpipe.step - INFO - PathLossStep instance created.
2026-04-17 15:32:00,324 - stpipe.step - INFO - BarShadowStep instance created.
2026-04-17 15:32:00,326 - stpipe.step - INFO - WfssContamStep instance created.
2026-04-17 15:32:00,327 - stpipe.step - INFO - PhotomStep instance created.
2026-04-17 15:32:00,328 - stpipe.step - INFO - AdaptiveTraceModelStep instance created.
2026-04-17 15:32:00,329 - stpipe.step - INFO - PixelReplaceStep instance created.
2026-04-17 15:32:00,330 - stpipe.step - INFO - ResampleSpecStep instance created.
2026-04-17 15:32:00,332 - stpipe.step - INFO - CubeBuildStep instance created.
2026-04-17 15:32:00,333 - stpipe.step - INFO - Extract1dStep instance created.
2026-04-17 15:32:00,537 - stpipe.step - INFO - Step Spec2Pipeline running with args ('/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg004_nrcalong_rateints.fits',).
2026-04-17 15:32:00,572 - stpipe.step - INFO - Step Spec2Pipeline parameters are:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: ./nrc_tso_demo_data/01366/Obs002/stage2
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: True
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_bsub: False
fail_on_exception: True
save_wfss_esec: False
steps:
assign_wcs:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
sip_approx: True
sip_max_pix_error: 0.01
sip_degree: None
sip_max_inv_pix_error: 0.01
sip_inv_degree: None
sip_npoints: 12
slit_y_low: -0.55
slit_y_high: 0.55
nrs_ifu_slice_wcs: False
badpix_selfcal:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
flagfrac_lower: 0.001
flagfrac_upper: 0.001
kernel_size: 15
force_single: False
save_flagged_bkg: False
msa_flagging:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
clean_flicker_noise:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
autoparam: False
fit_method: median
fit_by_channel: False
background_method: median
background_box_size: None
mask_science_regions: False
apply_flat_field: False
n_sigma: 2.0
fit_histogram: False
single_mask: True
user_mask: None
save_mask: False
save_background: False
save_noise: False
bkg_subtract:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
bkg_list: None
save_combined_background: False
sigma: 3.0
maxiters: None
soss_source_percentile: 35.0
soss_bkg_percentile: None
wfss_mmag_extract: None
wfss_mask: None
wfss_maxiter: 5
wfss_rms_stop: 0.0
wfss_outlier_percent: 1.0
imprint_subtract:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
extract_2d:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
slit_names: None
source_ids: None
source_ra: None
source_dec: None
source_max_sep: 2.0
extract_orders: None
grism_objects: None
tsgrism_extract_height: 64
wfss_extract_half_height: 5
wfss_mmag_extract: None
wfss_nbright: 1000
master_background_mos:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: True
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
sigma_clip: 3.0
median_kernel: 1
force_subtract: False
save_background: False
user_background: None
inverse: False
steps:
flat_field:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_interpolated_flat: False
user_supplied_flat: None
inverse: False
pathloss:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
user_slit_loc: None
barshadow:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
photom:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
apply_time_correction: True
pixel_replace:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: True
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
algorithm: fit_profile
n_adjacent_cols: 3
resample_spec:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
pixfrac: 1.0
kernel: square
fillval: NAN
weight_type: ivm
output_shape: None
pixel_scale_ratio: 1.0
pixel_scale: None
output_wcs: ''
single: False
blendheaders: True
in_memory: True
propagate_dq: False
extract_1d:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
subtract_background: None
apply_apcorr: True
extraction_type: box
use_source_posn: None
position_offset: 0.0
model_nod_pair: True
optimize_psf_location: True
smoothing_length: None
bkg_fit: None
bkg_order: None
log_increment: 50
save_profile: False
save_scene_model: False
save_residual_image: False
center_xy: None
ifu_autocen: False
bkg_sigma_clip: 3.0
ifu_rfcorr: True
ifu_set_srctype: None
ifu_rscale: None
ifu_covar_scale: 1.0
soss_atoca: True
soss_threshold: 0.01
soss_n_os: 2
soss_wave_grid_in: None
soss_wave_grid_out: None
soss_estimate: None
soss_rtol: 0.0001
soss_max_grid_size: 20000
soss_tikfac: None
soss_width: 40.0
soss_bad_pix: masking
soss_modelname: None
soss_order_3: True
targ_centroid:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
ta_file: None
wavecorr:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
flat_field:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_interpolated_flat: False
user_supplied_flat: None
inverse: False
srctype:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
source_type: None
straylight:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
clean_showers: False
shower_plane: 3
shower_x_stddev: 18.0
shower_y_stddev: 5.0
shower_low_reject: 0.1
shower_high_reject: 99.9
save_shower_model: False
fringe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
residual_fringe:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: residual_fringe
search_output_file: False
input_dir: ''
save_intermediate_results: False
ignore_region_min: None
ignore_region_max: None
pathloss:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
user_slit_loc: None
barshadow:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
wfss_contam:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
save_simulated_image: False
save_contam_images: False
maximum_cores: '1'
orders: None
magnitude_limit: None
wl_oversample: 2
max_pixels_per_chunk: 5000
photom:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
apply_time_correction: True
adaptive_trace_model:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: True
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
fit_threshold: 10.0
oversample: 1.0
slope_limit: 0.1
psf_optimal: False
save_intermediate_results: False
pixel_replace:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: True
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
algorithm: fit_profile
n_adjacent_cols: 3
resample_spec:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
pixfrac: 1.0
kernel: square
fillval: NAN
weight_type: ivm
output_shape: None
pixel_scale_ratio: 1.0
pixel_scale: None
output_wcs: ''
single: False
blendheaders: True
in_memory: True
propagate_dq: False
cube_build:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: True
output_use_index: True
save_results: False
skip: True
suffix: s3d
search_output_file: False
input_dir: ''
pipeline: 3
channel: all
band: all
grating: all
filter: all
output_type: None
linear_wave: True
scalexy: 0.0
scalew: 0.0
weighting: drizzle
coord_system: skyalign
ra_center: None
dec_center: None
cube_pa: None
nspax_x: None
nspax_y: None
rois: 0.0
roiw: 0.0
weight_power: 2.0
wavemin: None
wavemax: None
single: False
skip_dqflagging: False
offset_file: None
debug_spaxel: -1 -1 -1
extract_1d:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
subtract_background: None
apply_apcorr: True
extraction_type: box
use_source_posn: None
position_offset: 0.0
model_nod_pair: True
optimize_psf_location: True
smoothing_length: None
bkg_fit: None
bkg_order: None
log_increment: 50
save_profile: False
save_scene_model: False
save_residual_image: False
center_xy: None
ifu_autocen: False
bkg_sigma_clip: 3.0
ifu_rfcorr: True
ifu_set_srctype: None
ifu_rscale: None
ifu_covar_scale: 1.0
soss_atoca: True
soss_threshold: 0.01
soss_n_os: 2
soss_wave_grid_in: None
soss_wave_grid_out: None
soss_estimate: None
soss_rtol: 0.0001
soss_max_grid_size: 20000
soss_tikfac: None
soss_width: 40.0
soss_bad_pix: masking
soss_modelname: None
soss_order_3: True
2026-04-17 15:32:00,604 - stpipe.pipeline - INFO - Prefetching reference files for dataset: 'jw01366002001_04103_00001-seg004_nrcalong_rateints.fits' reftypes = ['apcorr', 'camera', 'collimator', 'dflat', 'disperser', 'distortion', 'fflat', 'filteroffset', 'flat', 'fore', 'fpa', 'fringe', 'ifufore', 'ifupost', 'ifuslicer', 'msa', 'ote', 'pastasoss', 'psf', 'regions', 'sflat', 'speckernel', 'specprofile', 'specwcs', 'wavecorr', 'wavelengthrange']
2026-04-17 15:32:00,607 - stpipe.pipeline - INFO - Prefetch for APCORR reference file is 'N/A'.
2026-04-17 15:32:00,608 - stpipe.pipeline - INFO - Prefetch for CAMERA reference file is 'N/A'.
2026-04-17 15:32:00,608 - stpipe.pipeline - INFO - Prefetch for COLLIMATOR reference file is 'N/A'.
2026-04-17 15:32:00,609 - stpipe.pipeline - INFO - Prefetch for DFLAT reference file is 'N/A'.
2026-04-17 15:32:00,609 - stpipe.pipeline - INFO - Prefetch for DISPERSER reference file is 'N/A'.
2026-04-17 15:32:00,610 - stpipe.pipeline - INFO - Prefetch for DISTORTION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_distortion_0259.asdf'.
2026-04-17 15:32:00,610 - stpipe.pipeline - INFO - Override for EXTRACT1D reference file is '/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage2/JWST_NIRCAM_extract1d_F322W2.json'.
2026-04-17 15:32:00,611 - stpipe.pipeline - INFO - Prefetch for FFLAT reference file is 'N/A'.
2026-04-17 15:32:00,612 - stpipe.pipeline - INFO - Prefetch for FILTEROFFSET reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_filteroffset_0007.asdf'.
2026-04-17 15:32:00,612 - stpipe.pipeline - INFO - Prefetch for FLAT reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0756.fits'.
2026-04-17 15:32:00,612 - stpipe.pipeline - INFO - Prefetch for FORE reference file is 'N/A'.
2026-04-17 15:32:00,613 - stpipe.pipeline - INFO - Prefetch for FPA reference file is 'N/A'.
2026-04-17 15:32:00,613 - stpipe.pipeline - INFO - Prefetch for FRINGE reference file is 'N/A'.
2026-04-17 15:32:00,614 - stpipe.pipeline - INFO - Prefetch for IFUFORE reference file is 'N/A'.
2026-04-17 15:32:00,614 - stpipe.pipeline - INFO - Prefetch for IFUPOST reference file is 'N/A'.
2026-04-17 15:32:00,615 - stpipe.pipeline - INFO - Prefetch for IFUSLICER reference file is 'N/A'.
2026-04-17 15:32:00,615 - stpipe.pipeline - INFO - Prefetch for MSA reference file is 'N/A'.
2026-04-17 15:32:00,615 - stpipe.pipeline - INFO - Prefetch for OTE reference file is 'N/A'.
2026-04-17 15:32:00,616 - stpipe.pipeline - INFO - Prefetch for PASTASOSS reference file is 'N/A'.
2026-04-17 15:32:00,616 - stpipe.pipeline - INFO - Prefetch for PSF reference file is 'N/A'.
2026-04-17 15:32:00,617 - stpipe.pipeline - INFO - Prefetch for REGIONS reference file is 'N/A'.
2026-04-17 15:32:00,617 - stpipe.pipeline - INFO - Prefetch for SFLAT reference file is 'N/A'.
2026-04-17 15:32:00,618 - stpipe.pipeline - INFO - Prefetch for SPECKERNEL reference file is 'N/A'.
2026-04-17 15:32:00,618 - stpipe.pipeline - INFO - Prefetch for SPECPROFILE reference file is 'N/A'.
2026-04-17 15:32:00,619 - stpipe.pipeline - INFO - Prefetch for SPECWCS reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_specwcs_0259.asdf'.
2026-04-17 15:32:00,619 - stpipe.pipeline - INFO - Prefetch for WAVECORR reference file is 'N/A'.
2026-04-17 15:32:00,620 - stpipe.pipeline - INFO - Prefetch for WAVELENGTHRANGE reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_wavelengthrange_0004.asdf'.
2026-04-17 15:32:00,621 - jwst.pipeline.calwebb_spec2 - INFO - Starting calwebb_spec2 ...
2026-04-17 15:32:00,635 - jwst.pipeline.calwebb_spec2 - INFO - Processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg004_nrcalong
2026-04-17 15:32:00,636 - jwst.pipeline.calwebb_spec2 - INFO - Working on input /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg004_nrcalong_rateints.fits ...
2026-04-17 15:32:01,029 - stpipe.step - INFO - Step assign_wcs running with args (<CubeModel(91, 256, 2048) from jw01366002001_04103_00001-seg004_nrcalong_rateints.fits>,).
2026-04-17 15:32:01,082 - jwst.assign_wcs.nircam - INFO - Added Barycentric velocity correction: 0.9999043045827591
2026-04-17 15:32:01,190 - jwst.assign_wcs.assign_wcs - INFO - COMPLETED assign_wcs
2026-04-17 15:32:01,195 - stpipe.step - INFO - Step assign_wcs done
2026-04-17 15:32:01,354 - stpipe.step - INFO - Step badpix_selfcal running with args (<CubeModel(91, 256, 2048) from jw01366002001_04103_00001-seg004_nrcalong_rateints.fits>, [], []).
2026-04-17 15:32:01,355 - stpipe.step - INFO - Step skipped.
2026-04-17 15:32:01,507 - stpipe.step - INFO - Step msa_flagging running with args (<CubeModel(91, 256, 2048) from jw01366002001_04103_00001-seg004_nrcalong_rateints.fits>,).
2026-04-17 15:32:01,508 - stpipe.step - INFO - Step skipped.
2026-04-17 15:32:01,660 - stpipe.step - INFO - Step clean_flicker_noise running with args (<CubeModel(91, 256, 2048) from jw01366002001_04103_00001-seg004_nrcalong_rateints.fits>,).
2026-04-17 15:32:01,661 - stpipe.step - INFO - Step skipped.
2026-04-17 15:32:01,813 - stpipe.step - INFO - Step imprint_subtract running with args (<CubeModel(91, 256, 2048) from jw01366002001_04103_00001-seg004_nrcalong_rateints.fits>, []).
2026-04-17 15:32:01,814 - stpipe.step - INFO - Step skipped.
2026-04-17 15:32:01,965 - stpipe.step - INFO - Step bkg_subtract running with args (<CubeModel(91, 256, 2048) from jw01366002001_04103_00001-seg004_nrcalong_rateints.fits>, []).
2026-04-17 15:32:01,966 - stpipe.step - INFO - Step skipped.
2026-04-17 15:32:02,133 - stpipe.step - INFO - Step flat_field running with args (<CubeModel(91, 256, 2048) from jw01366002001_04103_00001-seg004_nrcalong_rateints.fits>,).
2026-04-17 15:32:02,205 - jwst.flatfield.flat_field_step - INFO - Using FLAT reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0756.fits
2026-04-17 15:32:02,206 - jwst.flatfield.flat_field_step - INFO - No reference found for type FFLAT
2026-04-17 15:32:02,207 - jwst.flatfield.flat_field_step - INFO - No reference found for type SFLAT
2026-04-17 15:32:02,208 - jwst.flatfield.flat_field_step - INFO - No reference found for type DFLAT
2026-04-17 15:32:02,209 - jwst.flatfield.flat_field - INFO - Extracting matching subarray from flat
2026-04-17 15:32:02,797 - stpipe.step - INFO - Step flat_field done
2026-04-17 15:32:02,970 - stpipe.step - INFO - Step extract_2d running with args (<CubeModel(91, 256, 2048) from jw01366002001_04103_00001-seg004_nrcalong_rateints.fits>,).
2026-04-17 15:32:02,974 - jwst.extract_2d.extract_2d - INFO - EXP_TYPE is NRC_TSGRISM
2026-04-17 15:32:02,984 - jwst.extract_2d.grisms - INFO - Using default order extraction from reference file
2026-04-17 15:32:02,985 - jwst.extract_2d.grisms - INFO - Extracting order: [1]
2026-04-17 15:32:03,024 - jwst.extract_2d.grisms - INFO - Setting extraction height to 64
2026-04-17 15:32:03,033 - jwst.extract_2d.grisms - INFO - WCS made explicit for order: 1
2026-04-17 15:32:03,034 - jwst.extract_2d.grisms - INFO - Spectral trace extents: (xmin: 104, ymin: 0), (xmax: 1867, ymax: 63)
2026-04-17 15:32:03,035 - jwst.extract_2d.grisms - INFO - Extraction limits: (xmin: 0, ymin: 0), (xmax: 2047, ymax: 63)
2026-04-17 15:32:03,204 - jwst.extract_2d.grisms - INFO - Finished extraction
2026-04-17 15:32:03,208 - stpipe.step - INFO - Step extract_2d done
2026-04-17 15:32:03,394 - stpipe.step - INFO - Step srctype running with args (<SlitModel(91, 64, 2048) from jw01366002001_04103_00001-seg004_nrcalong_rateints.fits>,).
2026-04-17 15:32:03,395 - jwst.srctype.srctype - INFO - Input EXP_TYPE is NRC_TSGRISM
2026-04-17 15:32:03,396 - jwst.srctype.srctype - INFO - Input SRCTYAPT = UNKNOWN
2026-04-17 15:32:03,396 - jwst.srctype.srctype - INFO - Input is a TSO exposure; setting SRCTYPE = POINT
2026-04-17 15:32:03,399 - stpipe.step - INFO - Step srctype done
2026-04-17 15:32:03,553 - stpipe.step - INFO - Step straylight running with args (<SlitModel(91, 64, 2048) from jw01366002001_04103_00001-seg004_nrcalong_rateints.fits>,).
2026-04-17 15:32:03,554 - stpipe.step - INFO - Step skipped.
2026-04-17 15:32:03,707 - stpipe.step - INFO - Step fringe running with args (<SlitModel(91, 64, 2048) from jw01366002001_04103_00001-seg004_nrcalong_rateints.fits>,).
2026-04-17 15:32:03,708 - stpipe.step - INFO - Step skipped.
2026-04-17 15:32:03,862 - stpipe.step - INFO - Step pathloss running with args (<SlitModel(91, 64, 2048) from jw01366002001_04103_00001-seg004_nrcalong_rateints.fits>,).
2026-04-17 15:32:03,863 - stpipe.step - INFO - Step skipped.
2026-04-17 15:32:04,015 - stpipe.step - INFO - Step barshadow running with args (<SlitModel(91, 64, 2048) from jw01366002001_04103_00001-seg004_nrcalong_rateints.fits>,).
2026-04-17 15:32:04,016 - stpipe.step - INFO - Step skipped.
2026-04-17 15:32:04,168 - stpipe.step - INFO - Step wfss_contam running with args (<SlitModel(91, 64, 2048) from jw01366002001_04103_00001-seg004_nrcalong_rateints.fits>,).
2026-04-17 15:32:04,169 - stpipe.step - INFO - Step skipped.
2026-04-17 15:32:04,324 - stpipe.step - INFO - Step photom running with args (<SlitModel(91, 64, 2048) from jw01366002001_04103_00001-seg004_nrcalong_rateints.fits>,).
2026-04-17 15:32:04,325 - stpipe.step - INFO - Step skipped.
2026-04-17 15:32:04,559 - stpipe.step - INFO - Step pixel_replace running with args (<SlitModel(91, 64, 2048) from ./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg004_nrcalong_calints.fits>,).
2026-04-17 15:32:04,561 - stpipe.step - INFO - Step skipped.
2026-04-17 15:32:04,792 - stpipe.step - INFO - Step extract_1d running with args (<SlitModel(91, 64, 2048) from ./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg004_nrcalong_calints.fits>,).
2026-04-17 15:32:04,793 - jwst.extract_1d.extract_1d_step - INFO - Using EXTRACT1D reference file /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage2/JWST_NIRCAM_extract1d_F322W2.json
2026-04-17 15:32:04,829 - jwst.extract_1d.extract - INFO - Processing spectral order 1
2026-04-17 15:32:04,890 - jwst.extract_1d.extract - WARNING - The photom step has not been run.
2026-04-17 15:32:04,893 - jwst.extract_1d.extract - INFO - Aperture start/stop: 31.00 -> 37.00 (inclusive)
2026-04-17 15:32:04,895 - jwst.extract_1d.extract - INFO - Mean aperture start/stop from bkg_coeff: 21.50 -> 30.50 (inclusive)
2026-04-17 15:32:04,897 - jwst.extract_1d.extract - INFO - Mean aperture start/stop from bkg_coeff: 35.50 -> 44.50 (inclusive)
2026-04-17 15:32:04,904 - jwst.extract_1d.extract - INFO - Beginning loop over 91 integrations ...
2026-04-17 15:32:07,414 - jwst.extract_1d.extract - INFO - ... 50 integrations done
2026-04-17 15:32:09,654 - jwst.extract_1d.extract - INFO - All 91 integrations done
2026-04-17 15:32:09,887 - stpipe.step - INFO - Saved model in ./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg004_nrcalong_x1dints.fits
2026-04-17 15:32:09,888 - stpipe.step - INFO - Step extract_1d done
2026-04-17 15:32:09,892 - jwst.pipeline.calwebb_spec2 - INFO - Finished processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage1/jw01366002001_04103_00001-seg004_nrcalong
2026-04-17 15:32:09,903 - jwst.pipeline.calwebb_spec2 - INFO - Ending calwebb_spec2
2026-04-17 15:32:09,904 - jwst.stpipe.core - INFO - Results used CRDS context: jwst_1535.pmap
2026-04-17 15:32:10,095 - stpipe.step - INFO - Saved model in ./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg004_nrcalong_calints.fits
2026-04-17 15:32:10,096 - stpipe.step - INFO - Step Spec2Pipeline done
2026-04-17 15:32:10,096 - jwst.stpipe.core - INFO - Results used jwst version: 2.0.0
Run time: 0.7483333333333333 min
# Print out the time benchmark
time1 = time.perf_counter()
print(f"Runtime so far: {(time1 - time0) / 60:0.2f} minutes")
print(f"Runtime for Spec2: {(time1 - time_spec2) / 60:0.2f} minutes")
Runtime so far: 14.49 minutes
Runtime for Spec2: 0.75 minutes
# List the Stage 2 products.
# -----------------------------Science files-----------------------------
sci_cal = sorted(glob.glob(spec2_dir + '/*nrcalong*_calints.fits'))
sci_x1d = sorted(glob.glob(spec2_dir + '/*nrcalong*_x1dints.fits'))
print(f"SCIENCE | Stage 2 CAL Products:\n{'-' * 20}\n" + "\n".join(sci_cal))
print(f"\nSCIENCE | Stage 2 X1D Products:\n{'-' * 20}\n" + "\n".join(sci_x1d))
SCIENCE | Stage 2 CAL Products:
--------------------
./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg001_nrcalong_calints.fits
./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg002_nrcalong_calints.fits
./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg003_nrcalong_calints.fits
./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg004_nrcalong_calints.fits
SCIENCE | Stage 2 X1D Products:
--------------------
./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg001_nrcalong_x1dints.fits
./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg002_nrcalong_x1dints.fits
./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg003_nrcalong_x1dints.fits
./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg004_nrcalong_x1dints.fits
# Look in the header of one of the output files for a record of pipeline steps
# completed and those skipped
if do_spec2:
# Select first file to gather information
cal_f = datamodels.open(sci_cal[0])
# Check which steps were run:
cal_f.meta.cal_step.instance
{'assign_wcs': 'COMPLETE',
'badpix_selfcal': 'SKIPPED',
'barshadow': 'SKIPPED',
'bkg_subtract': 'SKIPPED',
'charge_migration': 'SKIPPED',
'clean_flicker_noise': 'SKIPPED',
'dark_sub': 'COMPLETE',
'dq_init': 'COMPLETE',
'extract_2d': 'COMPLETE',
'flat_field': 'COMPLETE',
'fringe': 'SKIPPED',
'gain_scale': 'SKIPPED',
'group_scale': 'SKIPPED',
'imprint': 'SKIPPED',
'ipc': 'SKIPPED',
'jump': 'COMPLETE',
'linearity': 'COMPLETE',
'msa_flagging': 'SKIPPED',
'pathloss': 'SKIPPED',
'persistence': 'SKIPPED',
'photom': 'SKIPPED',
'picture_frame': 'SKIPPED',
'ramp_fit': 'COMPLETE',
'refpix': 'COMPLETE',
'saturation': 'COMPLETE',
'srctype': 'COMPLETE',
'straylight': 'SKIPPED',
'superbias': 'COMPLETE',
'wfss_contam': 'SKIPPED'}
if do_spec2:
cal_f.meta.ref_file.instance
{'camera': {'name': 'N/A'},
'collimator': {'name': 'N/A'},
'crds': {'context_used': 'jwst_1535.pmap', 'sw_version': '13.1.14'},
'dark': {'name': 'crds://jwst_nircam_dark_0444.fits'},
'dflat': {'name': 'N/A'},
'disperser': {'name': 'N/A'},
'distortion': {'name': 'crds://jwst_nircam_distortion_0259.asdf'},
'fflat': {'name': 'N/A'},
'filteroffset': {'name': 'crds://jwst_nircam_filteroffset_0007.asdf'},
'flat': {'name': 'crds://jwst_nircam_flat_0756.fits'},
'fore': {'name': 'N/A'},
'fpa': {'name': 'N/A'},
'gain': {'name': 'crds://jwst_nircam_gain_0097.fits'},
'ifufore': {'name': 'N/A'},
'ifupost': {'name': 'N/A'},
'ifuslicer': {'name': 'N/A'},
'linearity': {'name': 'crds://jwst_nircam_linearity_0052.fits'},
'mask': {'name': 'crds://jwst_nircam_mask_0076.fits'},
'msa': {'name': 'N/A'},
'ote': {'name': 'N/A'},
'readnoise': {'name': 'crds://jwst_nircam_readnoise_0227.fits'},
'regions': {'name': 'N/A'},
'saturation': {'name': 'crds://jwst_nircam_saturation_0097.fits'},
'sflat': {'name': 'N/A'},
'specwcs': {'name': 'crds://jwst_nircam_specwcs_0259.asdf'},
'superbias': {'name': 'crds://jwst_nircam_superbias_0210.fits'},
'wavelengthrange': {'name': 'crds://jwst_nircam_wavelengthrange_0004.asdf'}}
7. Stage 3: Tso3Pipeline (calwebb_tso3)#
In this section, we process our calibrated spectra (_calints.fits files) from Stage 2 (calwebb_spec2) through the Tso3 (calwebb_tso3) pipeline to create Stage 3 data products.
Input: An Association (ASN) file that lists multiple exposures or exposure segments of a science exposure (
_calints.fits).Output: Calibrated time-series spectra and white light curve.
_x1dints.fits: Extracted 1D spectroscopic data for all integrations contained in the input exposures._whtlt.ecsv: An ASCII catalog inecsvformat containing the wavelength-integrated white light photometry of the source.
The TSO3Pipeline performs additional corrections (e.g., outlier detection) and produces calibrated time-series spectra and white light curve of the source object.
The default pipeline extraction uses an aperture width defined in the CRDS extract 1D reference file for a given filter, centered on the expected location of the spectral trace. Users can alter the location and width of the extraction aperture and modify background subtraction strategies by providing a custom extract1d reference file, as we did in the Configure Spec2Pipeline section above. For details concerning the proper format and available parameter settings for such reference files, consult the extract1d refrence file description.
See the calwebb_tso3 documentation for a detailed overview of the various pipeline steps that comprise TSO3.
7.1 Configure Tso3Pipeline#
The TSO3Pipeline has the following steps available:
outlier_detection: Identification of bad pixels or cosmic-rays that remain in each of the input images.
extract_1d: Extracts a 1D signal from 2D or 3D datasets.
white_light: Sums the spectroscopic flux over all wavelengths in each integration of a multi-integration extracted spectrum product to produce an integrated (“white”) flux as a function of time for the target.
For more information about each step and a full list of step arguments, please refer to the official documentation: JDox • ReadtheDocs
To override specific steps and reference files, use the examples below.
time_tso3 = time.perf_counter()
# Set up a dictionary to define how the Tso3 pipeline should be configured
# Boilerplate dictionary setup
tso3dict = {}
tso3dict['outlier_detection'], tso3dict['extract_1d'], tso3dict['white_light'], tso3dict['pixel_replace'] = {}, {}, {}, {}
# Overrides for whether or not certain steps should be skipped (example)
tso3dict['outlier_detection']['skip'] = True
#tso3dict['outlier_detection']['snr'] = '8.0 8.0' # Signal-to-noise thresholds for bad pixel identification
#tso3dict['extract_1d']['apply_apcorr'] = False
tso3dict['extract_1d']['subtract_background'] = True
#tso3dict['white_light']['min_wavelength'] = 2.420 # minimum wavelength from which to sum the flux array
#tso3dict['white_light']['max_wavelength'] = 4.025 # maximum wavelength from which to sum the flux array
#tso3dict['white_light']['skip'] = True
# Overrides for various reference files
# Files should be in the base local directory or provide full path
# Use the same custom extract1d reference file we created in the Spec2 section
tso3dict['extract_1d']['override_extract1d'] = custom_extract1d_reffile # Spectral extraction parameters (ASDF file)
#tso3dict['extract_1d']['override_apcorr'] = 'myfile.asdf' # Aperture correction parameters (ASDF file)
# Adjust moving median outlier detection parameters
#tso3dict['outlier_detection']['rolling_window_width'] = 25 # Default is 25
7.2 Create Tso3Pipeline ASN Files#
Association (ASN) files define the relationships between multiple exposures, allowing them to be processed as a set rather than individually.
Here we create a Stage 3 ASN file that will process the time series together.
def writel3asn(cal_files):
"""
Create a Level 3 association file.
Parameters
----------
scifiles : numpy.ndarray
NumPy array of all science exposure files absolute paths.
Returns
-------
asnfile: str
Name of Level 3 association file.
"""
hdr = fits.getheader(cal_files[0])
program = hdr['PROGRAM']
name = hdr['TARGNAME'].replace(' ', '-')
print(name)
# Create and save the asn file to the TSO3 directory
asnfile = os.path.join(tso3_dir, f'level3_{program}_asn.json')
# Define the basic association of science files
asn = afl.asn_from_list(cal_files, rule=DMS_Level3_Base, product_name=name)
asn.data['asn_type'] = 'tso3'
asn.data['program'] = program
asn.data['target'] = name
with open(asnfile, 'w') as f:
f.write(asn.dump()[1])
if os.path.exists(asnfile):
print(rf"Level 3 association successfully created and saved to: {asnfile}")
return asnfile
# Get calints files from the Spec2 output folder
calints_files = sorted(glob.glob(os.path.join(spec2_dir, '*nrcalong*calints.fits')))
# Use the absolute file paths
calints_files = [os.path.abspath(file) for file in calints_files]
calints_files = np.array(calints_files)
# Create an association file containing the calints files
asn_file = writel3asn(calints_files)
WASP-39
Level 3 association successfully created and saved to: ./nrc_tso_demo_data/01366/Obs002/stage3/level3_01366_asn.json
/home/runner/micromamba/envs/ci-env/lib/python3.11/site-packages/jwst/associations/association.py:232: UserWarning: Input association file contains path information; note that this can complicate usage and/or sharing of such files.
warnings.warn(err_str, UserWarning, stacklevel=1)
# Open an ASN file as an example.
# Check that file paths have been correctly updated.
with open(asn_file, 'r') as f_obj:
asnfile_data = json.load(f_obj)
JSON(asnfile_data, expanded=True)
<IPython.core.display.JSON object>
7.3 Run Tso3Pipeline#
Run the science files through the calwebb_tso3 pipeline using the .call() method.
start = time.time()
if do_tso3:
tso3_result = Tso3Pipeline.call(asn_file, steps=tso3dict, save_results=True, output_dir=tso3_dir)
else:
print('Skipping TSO3 processing...')
end = time.time()
print("Run time: ", round(end-start, 1)/60.0, " min")
2026-04-17 15:32:10,470 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-outlierdetectionstep_0001.asdf 1.1 K bytes (1 / 1 files) (0 / 1.1 K bytes)
2026-04-17 15:32:10,556 - stpipe.step - INFO - PARS-OUTLIERDETECTIONSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-outlierdetectionstep_0001.asdf
2026-04-17 15:32:10,570 - CRDS - INFO - Fetching /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-whitelightstep_0005.asdf 1.5 K bytes (1 / 1 files) (0 / 1.5 K bytes)
2026-04-17 15:32:10,658 - stpipe.step - INFO - PARS-WHITELIGHTSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-whitelightstep_0005.asdf
2026-04-17 15:32:10,669 - stpipe.step - INFO - Tso3Pipeline instance created.
2026-04-17 15:32:10,671 - stpipe.step - INFO - OutlierDetectionStep instance created.
2026-04-17 15:32:10,672 - stpipe.step - INFO - TSOPhotometryStep instance created.
2026-04-17 15:32:10,674 - stpipe.step - INFO - PixelReplaceStep instance created.
2026-04-17 15:32:10,675 - stpipe.step - INFO - Extract1dStep instance created.
2026-04-17 15:32:10,676 - stpipe.step - INFO - PhotomStep instance created.
2026-04-17 15:32:10,677 - stpipe.step - INFO - WhiteLightStep instance created.
2026-04-17 15:32:10,892 - stpipe.step - INFO - Step Tso3Pipeline running with args ('./nrc_tso_demo_data/01366/Obs002/stage3/level3_01366_asn.json',).
2026-04-17 15:32:10,902 - stpipe.step - INFO - Step Tso3Pipeline parameters are:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: ./nrc_tso_demo_data/01366/Obs002/stage3
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: True
skip: False
suffix: None
search_output_file: True
input_dir: ''
steps:
outlier_detection:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: False
input_dir: ''
weight_type: ivm
pixfrac: 1.0
kernel: square
fillval: NAN
maskpt: 0.7
snr: 5.0 4.0
scale: 1.2 0.7
backg: 0.0
kernel_size: 7 7
threshold_percent: 99.8
rolling_window_width: 25
ifu_second_check: False
save_intermediate_results: False
resample_data: False
good_bits: ~DO_NOT_USE
in_memory: True
pixmap_stepsize: 1.0
pixmap_order: 1
tso_photometry:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
save_catalog: False
radius: 3.0
radius_inner: 4.0
radius_outer: 5.0
centroid_source: True
search_box_width: 41
fit_box_width: 11
moving_centroid: False
pixel_replace:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: True
output_use_index: True
save_results: False
skip: True
suffix: None
search_output_file: True
input_dir: ''
algorithm: fit_profile
n_adjacent_cols: 3
extract_1d:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
subtract_background: True
apply_apcorr: True
extraction_type: box
use_source_posn: None
position_offset: 0.0
model_nod_pair: True
optimize_psf_location: True
smoothing_length: None
bkg_fit: None
bkg_order: None
log_increment: 50
save_profile: False
save_scene_model: False
save_residual_image: False
center_xy: None
ifu_autocen: False
bkg_sigma_clip: 3.0
ifu_rfcorr: True
ifu_set_srctype: None
ifu_rscale: None
ifu_covar_scale: 1.0
soss_atoca: True
soss_threshold: 0.01
soss_n_os: 2
soss_wave_grid_in: None
soss_wave_grid_out: None
soss_estimate: None
soss_rtol: 0.0001
soss_max_grid_size: 20000
soss_tikfac: None
soss_width: 40.0
soss_bad_pix: masking
soss_modelname: None
soss_order_3: True
photom:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .fits
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: None
search_output_file: True
input_dir: ''
inverse: False
source_type: None
apply_time_correction: True
white_light:
pre_hooks: []
post_hooks: []
output_file: None
output_dir: None
output_ext: .ecsv
output_use_model: False
output_use_index: True
save_results: False
skip: False
suffix: whtlt
search_output_file: True
input_dir: ''
min_wavelength: 2.47
max_wavelength: 3.94
2026-04-17 15:32:10,973 - stpipe.pipeline - INFO - Prefetching reference files for dataset: 'level3_01366_asn.json' reftypes = ['gain', 'readnoise']
2026-04-17 15:32:10,975 - stpipe.pipeline - INFO - Prefetch for GAIN reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits'.
2026-04-17 15:32:10,976 - stpipe.pipeline - INFO - Prefetch for READNOISE reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0227.fits'.
2026-04-17 15:32:10,977 - jwst.pipeline.calwebb_tso3 - INFO - Starting calwebb_tso3...
2026-04-17 15:32:12,141 - jwst.pipeline.calwebb_tso3 - INFO - Performing outlier detection on input images ...
2026-04-17 15:32:12,326 - stpipe.step - INFO - Step outlier_detection running with args (<SlitModel(92, 64, 2048) from jw01366002001_04103_00001-seg001_nrcalong_calints.fits>,).
2026-04-17 15:32:12,327 - stpipe.step - INFO - Step skipped.
2026-04-17 15:32:12,330 - jwst.pipeline.calwebb_tso3 - INFO - Performing outlier detection on input images ...
2026-04-17 15:32:12,492 - stpipe.step - INFO - Step outlier_detection running with args (<SlitModel(92, 64, 2048) from jw01366002001_04103_00001-seg002_nrcalong_calints.fits>,).
2026-04-17 15:32:12,494 - stpipe.step - INFO - Step skipped.
2026-04-17 15:32:12,497 - jwst.pipeline.calwebb_tso3 - INFO - Performing outlier detection on input images ...
2026-04-17 15:32:12,657 - stpipe.step - INFO - Step outlier_detection running with args (<SlitModel(91, 64, 2048) from jw01366002001_04103_00001-seg003_nrcalong_calints.fits>,).
2026-04-17 15:32:12,658 - stpipe.step - INFO - Step skipped.
2026-04-17 15:32:12,662 - jwst.pipeline.calwebb_tso3 - INFO - Performing outlier detection on input images ...
2026-04-17 15:32:12,821 - stpipe.step - INFO - Step outlier_detection running with args (<SlitModel(91, 64, 2048) from jw01366002001_04103_00001-seg004_nrcalong_calints.fits>,).
2026-04-17 15:32:12,823 - stpipe.step - INFO - Step skipped.
2026-04-17 15:32:13,013 - stpipe.step - INFO - Step pixel_replace running with args (<SlitModel(92, 64, 2048) from jw01366002001_04103_00001-seg001_nrcalong_calints.fits>,).
2026-04-17 15:32:13,014 - stpipe.step - INFO - Step skipped.
2026-04-17 15:32:13,017 - jwst.pipeline.calwebb_tso3 - INFO - Extracting 1-D spectra ...
2026-04-17 15:32:13,185 - stpipe.step - INFO - Step extract_1d running with args (<SlitModel(92, 64, 2048) from jw01366002001_04103_00001-seg001_nrcalong_calints.fits>,).
2026-04-17 15:32:13,187 - jwst.extract_1d.extract_1d_step - INFO - Using EXTRACT1D reference file /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage2/JWST_NIRCAM_extract1d_F322W2.json
2026-04-17 15:32:13,240 - jwst.extract_1d.extract - INFO - Processing spectral order 1
2026-04-17 15:32:13,304 - jwst.extract_1d.extract - WARNING - The photom step has not been run.
2026-04-17 15:32:13,307 - jwst.extract_1d.extract - INFO - Aperture start/stop: 31.00 -> 37.00 (inclusive)
2026-04-17 15:32:13,310 - jwst.extract_1d.extract - INFO - Mean aperture start/stop from bkg_coeff: 21.50 -> 30.50 (inclusive)
2026-04-17 15:32:13,312 - jwst.extract_1d.extract - INFO - Mean aperture start/stop from bkg_coeff: 35.50 -> 44.50 (inclusive)
2026-04-17 15:32:13,319 - jwst.extract_1d.extract - INFO - Beginning loop over 92 integrations ...
2026-04-17 15:32:15,845 - jwst.extract_1d.extract - INFO - ... 50 integrations done
2026-04-17 15:32:18,188 - jwst.extract_1d.extract - INFO - All 92 integrations done
2026-04-17 15:32:18,342 - stpipe.step - INFO - Step extract_1d done
2026-04-17 15:32:18,583 - stpipe.step - INFO - Step pixel_replace running with args (<SlitModel(92, 64, 2048) from jw01366002001_04103_00001-seg002_nrcalong_calints.fits>,).
2026-04-17 15:32:18,584 - stpipe.step - INFO - Step skipped.
2026-04-17 15:32:18,587 - jwst.pipeline.calwebb_tso3 - INFO - Extracting 1-D spectra ...
2026-04-17 15:32:18,758 - stpipe.step - INFO - Step extract_1d running with args (<SlitModel(92, 64, 2048) from jw01366002001_04103_00001-seg002_nrcalong_calints.fits>,).
2026-04-17 15:32:18,759 - jwst.extract_1d.extract_1d_step - INFO - Using EXTRACT1D reference file /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage2/JWST_NIRCAM_extract1d_F322W2.json
2026-04-17 15:32:18,803 - jwst.extract_1d.extract - INFO - Processing spectral order 1
2026-04-17 15:32:18,868 - jwst.extract_1d.extract - WARNING - The photom step has not been run.
2026-04-17 15:32:18,870 - jwst.extract_1d.extract - INFO - Aperture start/stop: 31.00 -> 37.00 (inclusive)
2026-04-17 15:32:18,873 - jwst.extract_1d.extract - INFO - Mean aperture start/stop from bkg_coeff: 21.50 -> 30.50 (inclusive)
2026-04-17 15:32:18,875 - jwst.extract_1d.extract - INFO - Mean aperture start/stop from bkg_coeff: 35.50 -> 44.50 (inclusive)
2026-04-17 15:32:18,882 - jwst.extract_1d.extract - INFO - Beginning loop over 92 integrations ...
2026-04-17 15:32:21,419 - jwst.extract_1d.extract - INFO - ... 50 integrations done
2026-04-17 15:32:23,757 - jwst.extract_1d.extract - INFO - All 92 integrations done
2026-04-17 15:32:23,910 - stpipe.step - INFO - Step extract_1d done
2026-04-17 15:32:24,152 - stpipe.step - INFO - Step pixel_replace running with args (<SlitModel(91, 64, 2048) from jw01366002001_04103_00001-seg003_nrcalong_calints.fits>,).
2026-04-17 15:32:24,153 - stpipe.step - INFO - Step skipped.
2026-04-17 15:32:24,157 - jwst.pipeline.calwebb_tso3 - INFO - Extracting 1-D spectra ...
2026-04-17 15:32:24,338 - stpipe.step - INFO - Step extract_1d running with args (<SlitModel(91, 64, 2048) from jw01366002001_04103_00001-seg003_nrcalong_calints.fits>,).
2026-04-17 15:32:24,340 - jwst.extract_1d.extract_1d_step - INFO - Using EXTRACT1D reference file /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage2/JWST_NIRCAM_extract1d_F322W2.json
2026-04-17 15:32:24,385 - jwst.extract_1d.extract - INFO - Processing spectral order 1
2026-04-17 15:32:24,449 - jwst.extract_1d.extract - WARNING - The photom step has not been run.
2026-04-17 15:32:24,451 - jwst.extract_1d.extract - INFO - Aperture start/stop: 31.00 -> 37.00 (inclusive)
2026-04-17 15:32:24,454 - jwst.extract_1d.extract - INFO - Mean aperture start/stop from bkg_coeff: 21.50 -> 30.50 (inclusive)
2026-04-17 15:32:24,456 - jwst.extract_1d.extract - INFO - Mean aperture start/stop from bkg_coeff: 35.50 -> 44.50 (inclusive)
2026-04-17 15:32:24,463 - jwst.extract_1d.extract - INFO - Beginning loop over 91 integrations ...
2026-04-17 15:32:26,999 - jwst.extract_1d.extract - INFO - ... 50 integrations done
2026-04-17 15:32:29,294 - jwst.extract_1d.extract - INFO - All 91 integrations done
2026-04-17 15:32:29,451 - stpipe.step - INFO - Step extract_1d done
2026-04-17 15:32:29,693 - stpipe.step - INFO - Step pixel_replace running with args (<SlitModel(91, 64, 2048) from jw01366002001_04103_00001-seg004_nrcalong_calints.fits>,).
2026-04-17 15:32:29,694 - stpipe.step - INFO - Step skipped.
2026-04-17 15:32:29,698 - jwst.pipeline.calwebb_tso3 - INFO - Extracting 1-D spectra ...
2026-04-17 15:32:29,872 - stpipe.step - INFO - Step extract_1d running with args (<SlitModel(91, 64, 2048) from jw01366002001_04103_00001-seg004_nrcalong_calints.fits>,).
2026-04-17 15:32:29,874 - jwst.extract_1d.extract_1d_step - INFO - Using EXTRACT1D reference file /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/LW_Grism_Time_Series/nrc_tso_demo_data/01366/Obs002/stage2/JWST_NIRCAM_extract1d_F322W2.json
2026-04-17 15:32:29,919 - jwst.extract_1d.extract - INFO - Processing spectral order 1
2026-04-17 15:32:29,982 - jwst.extract_1d.extract - WARNING - The photom step has not been run.
2026-04-17 15:32:29,985 - jwst.extract_1d.extract - INFO - Aperture start/stop: 31.00 -> 37.00 (inclusive)
2026-04-17 15:32:29,987 - jwst.extract_1d.extract - INFO - Mean aperture start/stop from bkg_coeff: 21.50 -> 30.50 (inclusive)
2026-04-17 15:32:29,989 - jwst.extract_1d.extract - INFO - Mean aperture start/stop from bkg_coeff: 35.50 -> 44.50 (inclusive)
2026-04-17 15:32:29,996 - jwst.extract_1d.extract - INFO - Beginning loop over 91 integrations ...
2026-04-17 15:32:32,539 - jwst.extract_1d.extract - INFO - ... 50 integrations done
2026-04-17 15:32:34,824 - jwst.extract_1d.extract - INFO - All 91 integrations done
2026-04-17 15:32:34,977 - stpipe.step - INFO - Step extract_1d done
2026-04-17 15:32:35,015 - jwst.pipeline.calwebb_tso3 - INFO - Performing white-light photometry ...
2026-04-17 15:32:35,222 - stpipe.step - INFO - Step white_light running with args (<TSOMultiSpecModel from jw01366002001_04103_00001-seg001_nrcalong_calints.fits>,).
2026-04-17 15:32:35,227 - stpipe.step - INFO - Step white_light done
2026-04-17 15:32:35,398 - stpipe.step - INFO - Saved model in ./nrc_tso_demo_data/01366/Obs002/stage3/WASP-39_x1dints.fits
2026-04-17 15:32:35,401 - jwst.pipeline.calwebb_tso3 - INFO - Writing Level 3 photometry catalog ./nrc_tso_demo_data/01366/Obs002/stage3/WASP-39_whtlt.ecsv
2026-04-17 15:32:35,423 - stpipe.step - INFO - Step Tso3Pipeline done
2026-04-17 15:32:35,423 - jwst.stpipe.core - INFO - Results used jwst version: 2.0.0
Run time: 0.4166666666666667 min
# Print out the time benchmark
time4 = time.perf_counter()
print(f"Runtime so far: {(time4 - time0) / 60:0.2f} minutes")
print(f"Runtime for Tso3: {(time4 - time_tso3) / 60:0.2f} minutes")
Runtime so far: 14.91 minutes
Runtime for Tso3: 0.42 minutes
# List the Stage 3 products.
stage3_whtlt = sorted(glob.glob(tso3_dir + '/*_whtlt.ecsv'))
stage3_x1d = sorted(glob.glob(tso3_dir + '/*_x1dints.fits'))
print(f"Stage 3 White Light Products:\n{'-' * 20}\n" + "\n".join(stage3_whtlt))
print(f"\nStage 3 X1D Products:\n{'-' * 20}\n" + "\n".join(stage3_x1d))
Stage 3 White Light Products:
--------------------
./nrc_tso_demo_data/01366/Obs002/stage3/WASP-39_whtlt.ecsv
Stage 3 X1D Products:
--------------------
./nrc_tso_demo_data/01366/Obs002/stage3/WASP-39_x1dints.fits
8. Visualize the Data#
8.1 Display Detector1Pipeline Products#
Inspect the Stage 1 slope products.
if do_viz:
# Load data
ramp_HDUL = datamodels.open(rateints_files_lw[0])
ramp_sci = ramp_HDUL.data
detector = ramp_HDUL.meta.instrument.detector.upper()
filtname = ramp_HDUL.meta.instrument.filter.upper()
# Select an integration
int_id = -1
# Get min and max for scaling and normalize
vmin, vmax = np.nanpercentile(ramp_sci[int_id, :, :], [5, 95])
norm = ImageNormalize(ramp_sci[int_id, :, :],
vmin=vmin,
vmax=vmax,
stretch=AsinhStretch(a=0.05)
)
# Plot
fig, ax = plt.subplots(1, 1, figsize=(12, 5))
im = ax.imshow(ramp_sci[int_id, :, :],
interpolation="None",
aspect="auto",
cmap="inferno",
origin="lower",
norm=norm
)
_ = ax.set_title(f"2D spectrum {detector} {filtname}", fontsize=15)
_ = ax.set_xlabel("Pixel Column", fontsize=15)
_ = ax.set_ylabel("Pixel Row", fontsize=15)
plt.savefig(os.path.join(det1_dir, "nrc_gts_rateints.png"), dpi=150)
plt.show()
The source spectrum is the bright horizontal trace centered at about y = 34. There are some faint horizontal stripes visible in other locations within the image. These could potentially originate from neighboring sources. To avoid contamination while extracting the source spectrum, users can adjust the extract 1D reference file as needed, and as was done in the sections above.
8.2 Display Spec2Pipeline Products#
Inspect Stage 2 calibrated spectra by displaying the 2D extracted spectrum, and then plotting three one-dimensional spectra from the spectral time series with a constant offset for clarity.
First, load one integration from one of the calints files and scale it such that we get a good view of the center of the trace. This is the 2D array on which the extract1D step of the pipeline operates. The coordinates in the extract1D reference file described in Section 6.1 are in the coordinate system of this 2D array. Noting the x and y locations of the trace, as well as areas suitable for calculating the background, users can tailor their extract1D reference file if the default reference file does not give optimal results.
if do_viz:
# Load data
cal_HDUL = datamodels.open(calints_files[0])
calints = cal_HDUL.data
detector = cal_HDUL.meta.instrument.detector.upper()
filtname = cal_HDUL.meta.instrument.filter.upper()
# Select an integration
int_id = -1
# Get min and max for scaling and normalize
vmin, vmax = np.nanpercentile(calints[int_id, :, :], [3, 97])
norm = ImageNormalize(calints[int_id, :, :],
vmin=vmin,
vmax=vmax,
stretch=AsinhStretch(a=0.05)
)
# Plot
fig, ax = plt.subplots(1, 1, figsize=(12, 5))
im = ax.imshow(calints[int_id, :, :],
interpolation="None",
aspect="auto",
cmap="inferno",
origin="lower",
norm=norm
)
_ = ax.set_title(f"Extracted 2D spectrum {detector} {filtname}", fontsize=15)
_ = ax.set_xlabel("Pixel Column", fontsize=15)
_ = ax.set_ylabel("Pixel Row", fontsize=15)
plt.savefig(os.path.join(det1_dir, "nrc_gts_calints.png"), dpi=150)
plt.show()
Next, look at the 1D extracted spectra from the Spec2 pipeline. To make that easier, first consolidate all extracted spectra (from each segment) and their corresponding timestamps into single large arrays.
x1d = datamodels.open(sci_x1d[0])
#x1d.info(max_rows=None) # prints the full contents
x1d.info(max_rows=10)
#print(x1d)
# Obtain the wavelength and flux from 1d spectral products
x1d_wave = x1d.spec[0].spec_table.WAVELENGTH
x1d_flux = x1d.spec[0].spec_table.FLUX
# Get flux unit from FITS header of _x1dints.fits
flux_unit = x1d.spec[0].spec_table.columns['FLUX'].unit
wave_um = x1d_wave
all_times = x1d.int_times.int_mid_BJD_TDB
n_spec = len(x1d.spec)
n_pix = len(x1d.spec[0].spec_table.FLUX)
root (AsdfObject)
├─asdf_library (Software) ...
├─history (dict)
│ └─extensions (list) ...
├─_fits_hash (str): 0f4c76e40cd3b1d308b6e47e3d3e109f191905da89130f9ed7d45ec478b3f626
├─cal_logs (dict) ...
├─int_times (FITS_rec)
├─meta (dict) ...
└─spec (list) ...
Some nodes not shown.
# Obtain all spectra for the list of segments
for i in range(len(sci_x1d)):
print("Processing segment: ", i+1, ' out of ', len(sci_x1d), ': ', sci_x1d[i])
x1d = datamodels.open(sci_x1d[i])
seg_spec_1D = np.vstack([spec.spec_table['FLUX'] for spec in x1d.spec])
if i == 0:
all_spec_1D = seg_spec_1D
all_times = x1d.int_times.int_mid_BJD_TDB
wave_um = x1d.spec[0].spec_table['WAVELENGTH'][0]
else:
all_spec_1D = np.concatenate((all_spec_1D, seg_spec_1D), axis=0)
all_times = np.concatenate((all_times, x1d.int_times.int_mid_BJD_TDB), axis=0)
x1d.close()
print(' ')
print('Total number of time stamps: ', len(all_times))
print('Total number of 1D spectra: ', all_spec_1D.shape[0])
print(' ')
print('Total number of columns: ', all_spec_1D.shape[1])
print('Total length of wavemap: ', len(wave_um))
Processing segment: 1 out of 4 : ./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg001_nrcalong_x1dints.fits
Processing segment: 2 out of 4 : ./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg002_nrcalong_x1dints.fits
Processing segment: 3 out of 4 : ./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg003_nrcalong_x1dints.fits
Processing segment: 4 out of 4 : ./nrc_tso_demo_data/01366/Obs002/stage2/jw01366002001_04103_00001-seg004_nrcalong_x1dints.fits
Total number of time stamps: 366
Total number of 1D spectra: 366
Total number of columns: 1701
Total length of wavemap: 1701
Now plot some examples of extracted 1D spectra, from the x1dints files generated by the Spec2 pipeline. Plot the extracted spectra from integrations 0, 10, and 100, with vertical offsets between them for clarity.
# Plot three spectra from the time series with a constant offset for clarity
if do_viz:
fig, ax = plt.subplots(figsize=(14, 8))
_ = ax.plot(wave_um, all_spec_1D[0, :]-0.0, label='Integ 0')
_ = ax.plot(wave_um, all_spec_1D[10, :]-70.0, label='Integ 10')
_ = ax.plot(wave_um, all_spec_1D[100, :]-140.0, label='Integ 100')
_ = ax.set_title("Extracted Spectra", fontsize=15)
_ = ax.set_xlabel("Wavelength (μm)", fontsize=15)
_ = ax.set_ylabel(f"Flux [{flux_unit}] + Constant Offsets", fontsize=15)
ax.grid(True, alpha=0.3)
_ = ax.legend(loc='upper left')
plt.tight_layout()
plt.savefig(os.path.join(spec2_dir, "nrc_gts_extracted_spectra.png"), dpi=150)
plt.show()
Let’s create a white light curve from the 1D spectra output by the Spec2 pipeline. Note that a final white light curve is created by the TSO3 pipeline, as shown in the Display Tso3Pipeline Products section below. This version, created from the Spec2 x1dints files, can be used as a quick quality check.
Create and plot the extracted white light photometric light curve by summing the flux from the wavelength range of each extracted one-dimensional spectrum. Normalize the light curve by dividing the light curve flux to the median flux of the first twenty data points. Calculate and report the scatter using the first ~100 data points.
if do_viz:
n_spec = len(all_times)
# Obtain white light curve
wlc_flux = np.zeros(n_spec)
# Sum of flux to calculate the white light curve
total_flux_cols = (20, -30)
col_start, col_end = total_flux_cols
for i in range(n_spec):
wlc_flux[i] = np.nansum(all_spec_1D[i, col_start:col_end])
# Normalize by the median flux of the first twenty points
wlc_flux /= np.median(wlc_flux[0:20])
# Calculate light curve scatter from first ~100 points
wlc_flux_s = sigma_clip(wlc_flux, sigma=1, maxiters=2, masked=False)
sigma_wlc = np.sqrt(np.nanvar(wlc_flux_s[2:100]))
sigma_wlc_ppm = round(sigma_wlc * 1e6, 0)
time_axis = (all_times - np.nanmean(all_times)) * 24.0
wavestart = round(wave_um[col_start], 4)
waveend = round(wave_um[col_end], 4)
fig, ax = plt.subplots(figsize=(12, 5), dpi=150)
_ = ax.plot(
time_axis,
wlc_flux,
color='blue',
marker="o",
markersize=2,
label=f"White light curve (r.m.s.={sigma_wlc_ppm} ppm)"
)
_ = ax.set_title(f"White Light Curve for {detector} λ = {wavestart} - {waveend} μm", fontsize=15)
_ = ax.set_xlabel("Time since mid-exposure (hr)", fontsize=15)
_ = ax.set_ylabel("Normalized flux", fontsize=15)
_ = ax.legend(loc='lower right', fontsize=12)
ax.grid(True, alpha=0.3)
plt.tight_layout()
plt.savefig(os.path.join(spec2_dir, "nrc_gts_whitelight.png"), dpi=150)
plt.show()
8.3 Display Tso3Pipeline Products#
Read in and plot the white light curve output by the Tso3Pipeline. This can be compared to the manually-created white light plot above.
if do_viz:
data = pd.read_csv(stage3_whtlt[0], comment="#", delimiter=" ")
time_BJD = data['BJD_TDB']
wht = data['whitelight_flux']
wht = wht/np.median(wht[0:20])
print('sigma in ppm: ', np.std(wht[2:100])*1e6)
# Calculate light curve scatter from first ~100 points.
sigma_wlc = np.sqrt(np.nanvar(wht[2:100]))
sigma_wlc_ppm = round(sigma_wlc * 1e6, 0)
# Plot light curve
fig, ax = plt.subplots(figsize=(12, 5), dpi=150)
_ = ax.plot((time_BJD-np.nanmean(time_BJD))*24., wht, color='blue', marker="o", markersize=2,
label=f'White light curve, (r.m.s.={sigma_wlc_ppm} ppm)')
_ = ax.set_title(f'White Light Curve for {detector} {filtname}', fontsize=15)
_ = ax.set_xlabel('Time since mid-exposure (hr)', fontsize=15)
_ = ax.set_ylabel('Normalized flux', fontsize=15)
_ = ax.legend(loc='lower right', fontsize=12)
ax.grid(True, alpha=0.3)
plt.tight_layout()
plt.savefig(os.path.join(tso3_dir, 'nrc_gts_tso3_wlc.png'), dpi=150)
plt.show()
sigma in ppm: 199.36866961626987
Finally, plot the spectroscopic light curves contained in the *x1dints.fits file.
The 2D map below shows all extracted, normalized 1D spectra spanning the full observation. The horizontal axis is the spectral direction in wavelength ($\mu m$), and the vertical axis represents each integration in time (bottom = first integration, top = last). Each column is normalized by the median flux of the first {n_baseline} out-of-transit integrations, so values near 1.0 (darker tones in the colorbar) indicate flux consistent with stellar baseline, while values below 1.0 (lighter tones) indicate flux suppression due to the planetary transit. The transit is visible as a horizontal band of decreased normalized flux spanning the in-transit integrations across all wavelengths. Ingress and egress appear as gradual transitions at the upper and lower boundaries of this band. Reduced sensitivity near the edges of the bandpass is visible as increased scatter at the spectral extremes, consistent with lower instrument throughput of the filter at those wavelengths.
lc_map = np.copy(all_spec_1D)
spec_xlen = len(lc_map[0, :])
# Define out-of-transit baseline
n_baseline = 20
for j in range(spec_xlen):
# Normalize each spectrum by the mean
lc_map[:, j] /= np.nanmean(lc_map[0:n_baseline, j])
slc_sigma = 3
lc_std = np.nanstd(lc_map[0:n_baseline, :])
vmin = 1.0 - slc_sigma * lc_std
vmax = 1.0 + 0.5 * lc_std
fig, ax = plt.subplots(figsize=(12, 10))
_ = ax.set_title(
f"Spectroscopic light curves for {detector} {filtname}",
fontsize=15
)
im1 = ax.imshow(
lc_map,
interpolation="bilinear",
aspect="auto",
cmap="viridis_r",
origin="lower",
clim=(vmin, vmax),
extent=[wavestart, waveend, 0, lc_map.shape[0]]
)
_ = ax.set_xlabel(r"Wavelength ($\mu$m)", fontsize=15)
_ = ax.set_ylabel("Integration", fontsize=15)
cb1 = fig.colorbar(im1, ax=ax, label=r"Normalized Flux")
plt.tight_layout()
plt.savefig(os.path.join(tso3_dir, 'nrc_gts_speclc_map.png'), dpi=150, bbox_inches='tight')
plt.show()
