Correcting for Helium Line Emission Background in IR Exposures using the “Flatten-Ramp” Technique#
Learning Goals#
This notebook shows how to identify IR exposures with time-variable Helium (1.083 micron) line emission background, and how to correct for it using the “flatten-ramp” technique. This method can be used to correct images affected by a sky background that does not vary across the field of view. For recommendations on correcting for spatially variable background due to scattered light, see the IR IMA Visualization Notebook (O’Connor 2023).
By the end of this tutorial, you will be able to:
Identify exposures with time-variable background.
Compute and analyze statistics for each read of an IMA file.
Reprocess a single exposure without cosmic ray corrections through
calwf3.Correct for time-variable background by subtracting the median background per read and rerunning the ramp-fitting step with
calwf3turned back on.
Table of Contents#
Introduction
1. Imports
2. Download the Data
3. Identifying Exposures with Time Variable Background
3.1 Displaying the Images
3.2 Plotting a Histogram of the Sky Background
3.3 Inspecting the IR Ramps
4. Reprocessing a Single Exposure for Time Variable Background
4.1 Query CRDS for best reference files
4.2 Recalibrating affected data with ‘ramp fitting’ step turned off in
calwf34.3 Remove Median Background per read from resulting IMA
4.4 Rerun
calwf3on the corrected IMA with ‘ramp fitting’ turned back on
5. Comparing Original and Corrected FLT Products
5.1 Displaying the Images
5.2 Plotting a Histogram of the Sky Background
5.3 Inspecting the IR Ramps
5.4 Comparing the Background Noise
5.5 Morphology
6. Conclusion
Additional Resources
About this Notebook
Citations
Introduction #
Helium I emission from the Earth’s atmosphere may affect exposures taken in the F105W and F110W filters. Affected reads are typically taken closest in time to when the spacecraft leaves the Earth’s shadow and enters the illuminated atmosphere. The 1.083 micron emission produces a flat background signal which is added to the total background in affected reads. The strength of the emission background signal depends on the observed path-length through the illuminated atmosphere, and in some cases is strong enough to compromise the ramp fitting calibration performed by calwf3, which is initially designed to flag and remove cosmic rays and saturated reads as they come through. This results in the FLT image to have much larger noise and a non-gaussian sky background.
This notebook demonstrates multiple ways on how to identify exposures with time variable background, and how to correct for these additional noise through the usage of the “flatten-ramp” technique described in WFC3 ISR 2016-16. This technique turns off ramp fitting in calwf3 and treats the IR detector like a CCD detector to avoid flagging and removing cosmic rays. We then remove the median background per read and add back in the full exposure countrate to preserve pixel statistics and then rerun calwf3 on the resulting IMA file with ramp fitting turned back on. This results in a cosmic-ray and time variable background corrected FLT file.
An alternate approach is the ‘last-minus-first’ technique described in the following CALWF3 notebook. This turns off the ramp fitting step in calwf3 (CRCORR=omit), treating the IR detector like a CCD accumulating charge and read out only at the end of the exposure. In this approach, the observed count rate is determined by simply subtracting the first read from the last of the detector and dividing by the time elapsed between the two reads. A disadvantage of this method is that this leaves cosmic-rays in the calibrated FLT frames which must be rejected with other software, such as AstroDrizzle. However, this approach may be used for extended targets and crowded stellar fields where it may be difficult to detangle the sky background from the light from astronomical sources using simple median statistics.
Time variable background also affects IR grisms. However, the method in this notebook should not be used to correct G102 and G141 observations, as they are affected by a combination of Helium I and Zodiacal background. Details on correcting grism data is provided in WFC3 ISR 2017-05: Variable He I emission in grism data.
1. Imports #
This notebook assumes you have created the virtual environment in WFC3 Library’s installation instructions.
Imports:
os for setting environment variables
glob for filtering out list of files
shutil for managing directories
numpy for array functionality
pandas for organizing data into a dataframe
matplotlib.pyplot for plotting data and images
astropy.io fits for opening FITS files
astroquery.mast Observations for downloading data from MAST
ccdproc ImageFileCollection for building the association.
wfc3tools for computing statistics on IMA files.
import os
import glob
import shutil
from astropy.io import fits
from astroquery.mast import Observations
from ccdproc import ImageFileCollection
import wfc3tools
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
2. Download the Data #
First, we query MAST for the necessary data and download them to the current directory.
We obtain WFC3/IR observations from Visit BF of CANDELS program 12242. ASN, RAW, IMA, and FLT files are requested.
Warning: This cell may take a few minutes to complete depending on computer speed and storage.
data_list = Observations.query_criteria(obs_id='IBOHBF*')
Observations.download_products(data_list['obsid'], project='CALWF3', mrp_only=False, download_dir='./data',
productSubGroupDescription=['ASN', 'RAW', 'IMA', 'FLT'])
science_files = glob.glob("data/mastDownload/HST/*/*fits")
for im in science_files:
root = os.path.basename(im)
new_path = os.path.join("./", root)
os.rename(im, new_path)
shutil.rmtree("data/")
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/ibohbf040_asn.fits to ./data/mastDownload/HST/ibohbf040/ibohbf040_asn.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/ibohbfb7q_flt.fits to ./data/mastDownload/HST/ibohbfb7q/ibohbfb7q_flt.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/ibohbfb7q_ima.fits to ./data/mastDownload/HST/ibohbfb7q/ibohbfb7q_ima.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/ibohbfb7q_raw.fits to ./data/mastDownload/HST/ibohbfb7q/ibohbfb7q_raw.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/ibohbfb9q_flt.fits to ./data/mastDownload/HST/ibohbfb9q/ibohbfb9q_flt.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/ibohbfb9q_ima.fits to ./data/mastDownload/HST/ibohbfb9q/ibohbfb9q_ima.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/ibohbfb9q_raw.fits to ./data/mastDownload/HST/ibohbfb9q/ibohbfb9q_raw.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/ibohbfbdq_flt.fits to ./data/mastDownload/HST/ibohbfbdq/ibohbfbdq_flt.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/ibohbfbdq_ima.fits to ./data/mastDownload/HST/ibohbfbdq/ibohbfbdq_ima.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/ibohbfbdq_raw.fits to ./data/mastDownload/HST/ibohbfbdq/ibohbfbdq_raw.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/ibohbfbgq_flt.fits to ./data/mastDownload/HST/ibohbfbgq/ibohbfbgq_flt.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/ibohbfbgq_ima.fits to ./data/mastDownload/HST/ibohbfbgq/ibohbfbgq_ima.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/ibohbfbgq_raw.fits to ./data/mastDownload/HST/ibohbfbgq/ibohbfbgq_raw.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/ibohbfbkq_flt.fits to ./data/mastDownload/HST/ibohbfbkq/ibohbfbkq_flt.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/ibohbfbkq_ima.fits to ./data/mastDownload/HST/ibohbfbkq/ibohbfbkq_ima.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/ibohbfbkq_raw.fits to ./data/mastDownload/HST/ibohbfbkq/ibohbfbkq_raw.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/ibohbfbpq_flt.fits to ./data/mastDownload/HST/ibohbfbpq/ibohbfbpq_flt.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/ibohbfbpq_ima.fits to ./data/mastDownload/HST/ibohbfbpq/ibohbfbpq_ima.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/ibohbfbpq_raw.fits to ./data/mastDownload/HST/ibohbfbpq/ibohbfbpq_raw.fits ...
[Done]
Visit BF association file ibohbf040_asn.fits contains a single visit over 3 orbits with 6 consecutive exposures in F105W dithered by a small fraction of the field of view. Each orbit consists of 2 back-to-back 1600 second exposure followed by the Earth occulation.
# printing observation conditions for visits used
collec = ImageFileCollection('./',
keywords=["asn_id", "targname", "filter",
"samp_seq", "nsamp", "exptime",
"postarg1", "postarg2", "date-obs",
"time-obs"],
glob_include="*flt.fits", ext=0)
out_table = collec.summary
out_table
| file | asn_id | targname | filter | samp_seq | nsamp | exptime | postarg1 | postarg2 | date-obs | time-obs |
|---|---|---|---|---|---|---|---|---|---|---|
| str18 | str9 | str12 | str5 | str7 | int64 | float64 | float64 | float64 | str10 | str8 |
| ibohbfb7q_flt.fits | IBOHBF040 | GOODN-N1-VBF | F105W | STEP200 | 16 | 1599.23291 | -0.237 | -0.212 | 2013-01-06 | 18:23:01 |
| ibohbfb9q_flt.fits | IBOHBF040 | GOODN-N1-VBF | F105W | STEP200 | 16 | 1599.23291 | 0.305 | -0.03 | 2013-01-06 | 18:50:42 |
| ibohbfbdq_flt.fits | IBOHBF040 | GOODN-N1-VBF | F105W | STEP200 | 16 | 1599.23291 | 0.102 | 0.273 | 2013-01-06 | 19:18:23 |
| ibohbfbgq_flt.fits | IBOHBF040 | GOODN-N1-VBF | F105W | STEP200 | 16 | 1599.23291 | -0.44 | 0.091 | 2013-01-06 | 19:48:08 |
| ibohbfbkq_flt.fits | IBOHBF040 | GOODN-N1-VBF | F105W | STEP200 | 16 | 1599.23291 | 0.237 | 0.212 | 2013-01-06 | 20:15:49 |
| ibohbfbpq_flt.fits | IBOHBF040 | GOODN-N1-VBF | F105W | STEP200 | 16 | 1599.23291 | 0.779 | 0.394 | 2013-01-06 | 20:43:30 |
3. Identifying Exposures with Time Variable Background #
There are three ways to identify exposures with poor ramp fitting due to time variable background:
Display the FLT images and look for unusually large noise.
Plot a histogram of the FLT sky background and look for a non-gaussian distribution.
Look for non-linearity in the median background of each ramp in the IMA file.
There are several exposures in this visit that are impacted by time variable background. For conciseness, this notebook will focus only on the first two exposures in the visit. Both exposures make up a single orbit. The first exposure ibohbfb7q is not impacted, while the second exposure ibohbfb9q suffers from additional time variable background.
3.1 Displaying the Images #
An FLT exposure with a poor ramp fit will have a noisier background than a FLT exposure with a good ramp fitting.
flt1 = fits.getdata('ibohbfb7q_flt.fits', ext=1)
flt2 = fits.getdata('ibohbfb9q_flt.fits', ext=1)
fig = plt.figure(figsize=(20, 8))
ax1 = fig.add_subplot(1, 2, 1)
ax2 = fig.add_subplot(1, 2, 2)
A = ax1.imshow(flt1, vmin=0.25, vmax=1.25, cmap='Greys_r', origin='lower')
B = ax2.imshow(flt2, vmin=1.25, vmax=2.25, cmap='Greys_r', origin='lower')
ax1.set_title('ibohbfb7q (Linear Bkg)', fontsize=20)
ax2.set_title('ibohbfb9q (Time Variable Bkg)', fontsize=20)
fig.colorbar(A, ax=ax1)
fig.colorbar(B, ax=ax2)
<matplotlib.colorbar.Colorbar at 0x7f2ec9f80d90>
3.2 Plotting a Histogram of the Background #
Impacted FLT exposures will typically have a non-gaussian background, often consisting of a double peak and a large sigma.
flt1f = flt1.flatten()
flt2f = flt2.flatten()
fig = plt.figure(figsize=(20, 5))
ax1 = fig.add_subplot(1, 2, 1)
ax2 = fig.add_subplot(1, 2, 2)
n, bins, patches = ax1.hist(flt1f, bins=200, range=(0, 1))
n, bins, patches = ax2.hist(flt2f, bins=200, range=(1, 2))
ax1.set_title('ibohbfb7q (Linear Bkg)', fontsize=15)
ax2.set_title('ibohbfb9q (Time Variable Bkg)', fontsize=15)
Text(0.5, 1.0, 'ibohbfb9q (Time Variable Bkg)')
3.3 Inspecting the IR Ramps #
We use the pstat task in wfc3tools to compute statistics up the stack of each IMA image. We use the midpoint of each ramp as a good representation of the total background, due to the target being mostly blank sky.
Plotting Count vs. Time#
The median signal in the first image accumulates linearly, while the affected image’s median signal accumulates non-linearly over the exposure.
imafiles = ('ibohbfb7q_ima.fits', 'ibohbfb9q_ima.fits')
fig, axarr = plt.subplots(1, 2, figsize=(15, 5))
fig.set_dpi(100)
for i, ima in enumerate(imafiles):
time, counts = wfc3tools.pstat(ima, stat='midpt', units='counts', plot=False)
axarr[i].plot(time, counts, '+', markersize=8)
axarr[i].set_title(ima)
axarr[i].set_xlabel('Exposure Time (s)')
axarr[i].set_ylabel('Counts (e-)')
Plotting Count Rate versus Time#
The median count rate of the first image is relatively flat at ~0.5 electrons/sec, which is primarily zodiacal light. There is a small excess signal seen in the first few ramps, but this is unlikely to impact calwf3’s ramp fitting step. The median count rate of the affected image varies from ~0.5 electrons/sec at the beginning of the exposure to ~1.5 electrons/sec at the end of the exposure, nearly a change by a factor of ~3. Inspecting the ramp of ibohbfb9q_ima reveals that the background signal due to the helium line emission increased rapidly in the middle of the exposure then flattens at the end.
imafiles = ('ibohbfb7q_ima.fits', 'ibohbfb9q_ima.fits')
ylim = [(0.3, 0.7), (0.3, 1.8)]
fig, axarr = plt.subplots(1, 2, figsize=(15, 5))
fig.set_dpi(100)
for i, ima in enumerate(imafiles):
time, counts = wfc3tools.pstat(ima, stat='midpt', units='rate', plot=False)
axarr[i].plot(time, counts, '+', markersize=8)
axarr[i].set_title(ima)
axarr[i].set_xlabel('Exposure Time (s)')
axarr[i].set_ylabel('Count Rate (e-/s)')
axarr[i].set_ylim(ylim[i])
4. Reprocessing a Single Exposure for Time Variable Background #
Now that we have identified signs of time variable background, the next step is to reprocess the data to improve the quality of the FLT data product. In brief, the steps are as follows:
Query CRDS for best reference files.
Recalibrate affected raw data with ‘ramp fitting’ step turned off using
calwf3.Remove the median background per read from the resulting IMA file.
Rerun
calwf3on the corrected IMA, with ‘ramp fitting’ turned back on.
4.1 Query CRDS for Best Reference Files #
We first set environment variables for several calibration tasks.
We point to subdirectory crds_cache/ using the IREF environment variable. The IREF variable is used for WFC3 reference files.
os.environ['CRDS_SERVER_URL'] = 'https://hst-crds.stsci.edu'
os.environ['CRDS_SERVER'] = 'https://hst-crds.stsci.edu'
os.environ['CRDS_PATH'] = './crds_cache'
os.environ['iref'] = './crds_cache/references/hst/wfc3/'
We now download the “bestrefs” for these RAW files.
The code block belows queries CRDS for the current best reference files for the dataset and updates the header keyword to point to these new files. The python package os is used to run terminal commands.
Warning: This cell may take a few minutes to complete depending on computer speed and storage.
raw_files = glob.glob('*_raw.fits')
for file in raw_files:
!crds bestrefs --files $file --sync-references=1 --update-bestrefs;
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfpc2_wf4tfile_0250.rmap 678 bytes (1 / 142 files) (0 / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfpc2_shadfile_0250.rmap 977 bytes (2 / 142 files) (678 / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfpc2_offtab_0250.rmap 642 bytes (3 / 142 files) (1.7 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfpc2_maskfile_0250.rmap 685 bytes (4 / 142 files) (2.3 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfpc2_idctab_0250.rmap 696 bytes (5 / 142 files) (3.0 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfpc2_flatfile_0250.rmap 30.0 K bytes (6 / 142 files) (3.7 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfpc2_dgeofile_0250.rmap 801 bytes (7 / 142 files) (33.7 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfpc2_darkfile_0250.rmap 178.4 K bytes (8 / 142 files) (34.5 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfpc2_biasfile_0250.rmap 3.3 K bytes (9 / 142 files) (212.8 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfpc2_atodfile_0250.rmap 874 bytes (10 / 142 files) (216.1 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfpc2_0250.imap 782 bytes (11 / 142 files) (217.0 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfc3_snkcfile_0003.rmap 681 bytes (12 / 142 files) (217.8 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfc3_satufile_0002.rmap 1.0 K bytes (13 / 142 files) (218.5 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfc3_pfltfile_0253.rmap 34.2 K bytes (14 / 142 files) (219.5 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfc3_pctetab_0004.rmap 698 bytes (15 / 142 files) (253.7 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfc3_oscntab_0250.rmap 747 bytes (16 / 142 files) (254.4 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfc3_npolfile_0254.rmap 4.0 K bytes (17 / 142 files) (255.1 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfc3_nlinfile_0250.rmap 726 bytes (18 / 142 files) (259.2 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfc3_mdriztab_0254.rmap 845 bytes (19 / 142 files) (259.9 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfc3_imphttab_0256.rmap 683 bytes (20 / 142 files) (260.7 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfc3_idctab_0254.rmap 661 bytes (21 / 142 files) (261.4 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfc3_flshfile_0256.rmap 5.8 K bytes (22 / 142 files) (262.1 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfc3_drkcfile_0194.rmap 234.8 K bytes (23 / 142 files) (267.9 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfc3_dfltfile_0002.rmap 17.1 K bytes (24 / 142 files) (502.7 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfc3_darkfile_0492.rmap 276.0 K bytes (25 / 142 files) (519.8 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfc3_d2imfile_0251.rmap 605 bytes (26 / 142 files) (795.8 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfc3_crrejtab_0250.rmap 803 bytes (27 / 142 files) (796.4 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfc3_ccdtab_0250.rmap 799 bytes (28 / 142 files) (797.2 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfc3_bpixtab_0310.rmap 11.7 K bytes (29 / 142 files) (798.0 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfc3_biasfile_0267.rmap 23.4 K bytes (30 / 142 files) (809.7 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfc3_biacfile_0003.rmap 692 bytes (31 / 142 files) (833.1 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfc3_atodtab_0250.rmap 651 bytes (32 / 142 files) (833.8 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_wfc3_0595.imap 1.3 K bytes (33 / 142 files) (834.4 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_synphot_tmttab_0002.rmap 745 bytes (34 / 142 files) (835.7 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_synphot_tmgtab_0012.rmap 767 bytes (35 / 142 files) (836.4 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_synphot_tmctab_0052.rmap 743 bytes (36 / 142 files) (837.2 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_synphot_thruput_0056.rmap 329.6 K bytes (37 / 142 files) (837.9 K / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_synphot_thermal_0003.rmap 20.4 K bytes (38 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_synphot_obsmodes_0004.rmap 743 bytes (39 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_synphot_0067.imap 579 bytes (40 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_xtractab_0250.rmap 815 bytes (41 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_wcptab_0251.rmap 578 bytes (42 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_teltab_0250.rmap 745 bytes (43 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_tdstab_0254.rmap 921 bytes (44 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_tdctab_0252.rmap 650 bytes (45 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_srwtab_0250.rmap 745 bytes (46 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_sptrctab_0251.rmap 895 bytes (47 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_sdctab_0251.rmap 889 bytes (48 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_riptab_0254.rmap 877 bytes (49 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_phottab_0258.rmap 1.6 K bytes (50 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_pfltfile_0250.rmap 23.7 K bytes (51 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_pctab_0250.rmap 3.1 K bytes (52 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_mofftab_0250.rmap 747 bytes (53 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_mlintab_0250.rmap 601 bytes (54 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_lfltfile_0250.rmap 11.8 K bytes (55 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_lamptab_0250.rmap 610 bytes (56 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_inangtab_0250.rmap 815 bytes (57 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_imphttab_0252.rmap 616 bytes (58 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_idctab_0251.rmap 775 bytes (59 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_halotab_0250.rmap 747 bytes (60 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_gactab_0250.rmap 651 bytes (61 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_exstab_0250.rmap 745 bytes (62 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_echsctab_0250.rmap 749 bytes (63 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_disptab_0250.rmap 813 bytes (64 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_darkfile_0353.rmap 59.7 K bytes (65 / 142 files) (1.2 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_crrejtab_0250.rmap 711 bytes (66 / 142 files) (1.3 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_cdstab_0250.rmap 745 bytes (67 / 142 files) (1.3 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_ccdtab_0252.rmap 893 bytes (68 / 142 files) (1.3 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_bpixtab_0250.rmap 845 bytes (69 / 142 files) (1.3 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_biasfile_0355.rmap 119.5 K bytes (70 / 142 files) (1.3 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_apertab_0250.rmap 588 bytes (71 / 142 files) (1.4 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_apdestab_0252.rmap 636 bytes (72 / 142 files) (1.4 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_stis_0369.imap 1.7 K bytes (73 / 142 files) (1.4 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_nicmos_zprattab_0250.rmap 646 bytes (74 / 142 files) (1.4 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_nicmos_tempfile_0250.rmap 1.1 K bytes (75 / 142 files) (1.4 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_nicmos_tdffile_0250.rmap 8.9 K bytes (76 / 142 files) (1.4 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_nicmos_saadfile_0250.rmap 771 bytes (77 / 142 files) (1.4 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_nicmos_saacntab_0250.rmap 594 bytes (78 / 142 files) (1.4 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_nicmos_rnlcortb_0250.rmap 771 bytes (79 / 142 files) (1.4 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_nicmos_pmskfile_0250.rmap 603 bytes (80 / 142 files) (1.4 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_nicmos_pmodfile_0250.rmap 603 bytes (81 / 142 files) (1.4 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_nicmos_phottab_0250.rmap 862 bytes (82 / 142 files) (1.4 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_nicmos_pedsbtab_0250.rmap 594 bytes (83 / 142 files) (1.4 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_nicmos_noisfile_0250.rmap 2.6 K bytes (84 / 142 files) (1.4 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_nicmos_nlinfile_0250.rmap 1.7 K bytes (85 / 142 files) (1.4 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_nicmos_maskfile_0250.rmap 1.2 K bytes (86 / 142 files) (1.4 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_nicmos_illmfile_0250.rmap 5.8 K bytes (87 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_nicmos_idctab_0250.rmap 767 bytes (88 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_nicmos_flatfile_0250.rmap 11.0 K bytes (89 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_nicmos_darkfile_0250.rmap 14.9 K bytes (90 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_nicmos_0250.imap 1.1 K bytes (91 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_ywlkfile_0003.rmap 922 bytes (92 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_xwlkfile_0002.rmap 922 bytes (93 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_xtractab_0269.rmap 1.6 K bytes (94 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_wcptab_0257.rmap 1.3 K bytes (95 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_twozxtab_0277.rmap 990 bytes (96 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_tracetab_0276.rmap 998 bytes (97 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_tdstab_0271.rmap 803 bytes (98 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_spwcstab_0255.rmap 1.1 K bytes (99 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_spottab_0006.rmap 766 bytes (100 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_proftab_0276.rmap 1.0 K bytes (101 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_phatab_0250.rmap 668 bytes (102 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_lamptab_0264.rmap 1.4 K bytes (103 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_hvtab_0259.rmap 567 bytes (104 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_hvdstab_0002.rmap 1.0 K bytes (105 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_gsagtab_0259.rmap 712 bytes (106 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_geofile_0250.rmap 670 bytes (107 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_fluxtab_0282.rmap 1.7 K bytes (108 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_flatfile_0264.rmap 1.8 K bytes (109 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_disptab_0276.rmap 1.7 K bytes (110 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_dgeofile_0002.rmap 909 bytes (111 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_deadtab_0250.rmap 711 bytes (112 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_brsttab_0250.rmap 696 bytes (113 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_brftab_0250.rmap 614 bytes (114 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_bpixtab_0260.rmap 773 bytes (115 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_badttab_0252.rmap 643 bytes (116 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_cos_0357.imap 1.4 K bytes (117 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_acs_spottab_0251.rmap 641 bytes (118 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_acs_snkcfile_0097.rmap 7.3 K bytes (119 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_acs_shadfile_0251.rmap 531 bytes (120 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_acs_satufile_0002.rmap 1.2 K bytes (121 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_acs_pfltfile_0253.rmap 69.2 K bytes (122 / 142 files) (1.5 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_acs_pctetab_0253.rmap 615 bytes (123 / 142 files) (1.6 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_acs_oscntab_0251.rmap 781 bytes (124 / 142 files) (1.6 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_acs_npolfile_0253.rmap 3.2 K bytes (125 / 142 files) (1.6 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_acs_mlintab_0250.rmap 646 bytes (126 / 142 files) (1.6 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_acs_mdriztab_0253.rmap 769 bytes (127 / 142 files) (1.6 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_acs_imphttab_0260.rmap 769 bytes (128 / 142 files) (1.6 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_acs_idctab_0256.rmap 1.5 K bytes (129 / 142 files) (1.6 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_acs_flshfile_0268.rmap 3.4 K bytes (130 / 142 files) (1.6 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_acs_drkcfile_0448.rmap 14.8 K bytes (131 / 142 files) (1.6 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_acs_dgeofile_0250.rmap 3.2 K bytes (132 / 142 files) (1.6 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_acs_darkfile_0439.rmap 86.8 K bytes (133 / 142 files) (1.6 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_acs_d2imfile_0253.rmap 601 bytes (134 / 142 files) (1.7 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_acs_crrejtab_0251.rmap 945 bytes (135 / 142 files) (1.7 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_acs_cfltfile_0250.rmap 1.2 K bytes (136 / 142 files) (1.7 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_acs_ccdtab_0256.rmap 1.4 K bytes (137 / 142 files) (1.7 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_acs_bpixtab_0252.rmap 1.0 K bytes (138 / 142 files) (1.7 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_acs_biasfile_0437.rmap 56.9 K bytes (139 / 142 files) (1.7 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_acs_atodtab_0251.rmap 528 bytes (140 / 142 files) (1.8 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_acs_0538.imap 1.3 K bytes (141 / 142 files) (1.8 M / 1.8 M bytes)
CRDS - INFO - Fetching ./crds_cache/mappings/hst/hst_1177.pmap 495 bytes (142 / 142 files) (1.8 M / 1.8 M bytes)
CRDS - INFO - No comparison context or source comparison requested.
CRDS - INFO - ===> Processing ibohbfb9q_raw.fits
CRDS - INFO - Fetching ./crds_cache/references/hst/wfc3/3562021pi_mdz.fits 40.3 K bytes (1 / 11 files) (0 / 404.6 M bytes)
CRDS - INFO - Fetching ./crds_cache/references/hst/wfc3/35620245i_drk.fits 302.6 M bytes (2 / 11 files) (40.3 K / 404.6 M bytes)
CRDS - INFO - Fetching ./crds_cache/references/hst/wfc3/3562033hi_bpx.fits 838.1 K bytes (3 / 11 files) (302.6 M / 404.6 M bytes)
CRDS - INFO - Fetching ./crds_cache/references/hst/wfc3/4ac18263i_dfl.fits 10.5 M bytes (4 / 11 files) (303.5 M / 404.6 M bytes)
CRDS - INFO - Fetching ./crds_cache/references/hst/wfc3/4ac19225i_pfl.fits 10.5 M bytes (5 / 11 files) (314.0 M / 404.6 M bytes)
CRDS - INFO - Fetching ./crds_cache/references/hst/wfc3/4af1533ai_imp.fits 40.3 K bytes (6 / 11 files) (324.6 M / 404.6 M bytes)
CRDS - INFO - Fetching ./crds_cache/references/hst/wfc3/q911321mi_osc.fits 20.2 K bytes (7 / 11 files) (324.6 M / 404.6 M bytes)
CRDS - INFO - Fetching ./crds_cache/references/hst/wfc3/t2c16200i_ccd.fits 17.3 K bytes (8 / 11 files) (324.6 M / 404.6 M bytes)
CRDS - INFO - Fetching ./crds_cache/references/hst/wfc3/u1k1727mi_lin.fits 79.9 M bytes (9 / 11 files) (324.6 M / 404.6 M bytes)
CRDS - INFO - Fetching ./crds_cache/references/hst/wfc3/u6a1748ri_crr.fits 14.4 K bytes (10 / 11 files) (404.5 M / 404.6 M bytes)
CRDS - INFO - Fetching ./crds_cache/references/hst/wfc3/w3m18525i_idc.fits 20.2 K bytes (11 / 11 files) (404.6 M / 404.6 M bytes)
CRDS - INFO - 0 errors
CRDS - INFO - 0 warnings
CRDS - INFO - 155 infos
CRDS - INFO - No comparison context or source comparison requested.
CRDS - INFO - ===> Processing ibohbfbgq_raw.fits
CRDS - INFO - 0 errors
CRDS - INFO - 0 warnings
CRDS - INFO - 2 infos
CRDS - INFO - No comparison context or source comparison requested.
CRDS - INFO - ===> Processing ibohbfbkq_raw.fits
CRDS - INFO - 0 errors
CRDS - INFO - 0 warnings
CRDS - INFO - 2 infos
CRDS - INFO - No comparison context or source comparison requested.
CRDS - INFO - ===> Processing ibohbfb7q_raw.fits
CRDS - INFO - 0 errors
CRDS - INFO - 0 warnings
CRDS - INFO - 2 infos
CRDS - INFO - No comparison context or source comparison requested.
CRDS - INFO - ===> Processing ibohbfbdq_raw.fits
CRDS - INFO - 0 errors
CRDS - INFO - 0 warnings
CRDS - INFO - 2 infos
CRDS - INFO - No comparison context or source comparison requested.
CRDS - INFO - ===> Processing ibohbfbpq_raw.fits
CRDS - INFO - 0 errors
CRDS - INFO - 0 warnings
CRDS - INFO - 2 infos
4.2 Recalibrating affected data with ‘ramp fitting’ step turned off in calwf3 #
Next, we run calwf3 on the affected file ibohbfb9q with CRCORR=OMIT to perform all steps except the ramp fitting.
# Renaming files
os.rename('ibohbfb9q_ima.fits', 'ibohbfb9q_ima_orig.fits')
os.rename('ibohbfb9q_flt.fits', 'ibohbfb9q_flt_orig.fits')
# Updating the raw file
raw_file = 'ibohbfb9q_raw.fits'
with fits.open(raw_file, mode='update') as raw:
raw[0].header['CRCORR'] = 'OMIT'
# Running wfc3tools.calwf3
wfc3tools.calwf3(raw_file)
# Renaming the processed flt file
os.rename('ibohbfb9q_flt.fits', 'ibohbfb9q_flt_norampfit.fits')
git tag: e0988181-dirty
git branch: HEAD
HEAD @: e098818105734475101120a544dfe468207bc690
CALBEG*** CALWF3 -- Version 3.7.1 (Oct-18-2023) ***
Begin 04-Sep-2024 17:51:43 UTC
Input ibohbfb9q_raw.fits
loading asn
LoadAsn: Processing SINGLE exposure
Trying to open ibohbfb9q_raw.fits...
Read in Primary header from ibohbfb9q_raw.fits...
Creating new trailer file `ibohbfb9q.tra'.
CALBEG*** WF3IR -- Version 3.7.1 (Oct-18-2023) ***
Begin 04-Sep-2024 17:51:43 UTC
Input ibohbfb9q_raw.fits
Output ibohbfb9q_flt.fits
Trying to open ibohbfb9q_raw.fits...
Read in Primary header from ibohbfb9q_raw.fits...
APERTURE IR-FIX
FILTER F105W
DETECTOR IR
Reading data from ibohbfb9q_raw.fits ...
CCDTAB iref$t2c16200i_ccd.fits
CCDTAB PEDIGREE=Ground
CCDTAB DESCRIP =Reference data based on Thermal-Vac #3, gain=2.5 results for IR-4
CCDTAB DESCRIP =Readnoise,gain,saturation from TV3,MEB2 values. ISRs 2008-25,39,50
readnoise =20.2,19.8,19.9,20.1
gain =2.34,2.37,2.31,2.38
DQICORR PERFORM
DQITAB iref$3562033hi_bpx.fits
DQITAB PEDIGREE=INFLIGHT 01/11/2012 12/10/2013
DQITAB DESCRIP =Bad Pixel Table generated using Cycle 20 Flats and Darks-----------
DQICORR COMPLETE
ZSIGCORR PERFORM
ZSIGCORR detected 1285 saturated pixels in 0th read
ZSIGCORR detected 1349 saturated pixels in 1st read
ZSIGCORR COMPLETE
BLEVCORR PERFORM
OSCNTAB iref$q911321mi_osc.fits
OSCNTAB PEDIGREE=GROUND
OSCNTAB DESCRIP =Initial values for ground test data processing
BLEVCORR COMPLETE
ZOFFCORR PERFORM
ZOFFCORR COMPLETE
NOISCORR PERFORM
Uncertainty array initialized.
NOISCORR COMPLETE
NLINCORR PERFORM
NLINFILE iref$u1k1727mi_lin.fits
NLINFILE PEDIGREE=INFLIGHT 09/02/2009 12/07/2009
NLINFILE DESCRIP =Non-linearity correction from WFC3 MEB1 TV3 and on-orbit frames----
NLINCORR detected 1285 saturated pixels in imset 16
NLINCORR detected 1354 saturated pixels in imset 15
NLINCORR detected 1381 saturated pixels in imset 14
NLINCORR detected 1388 saturated pixels in imset 13
NLINCORR detected 1395 saturated pixels in imset 12
NLINCORR detected 1413 saturated pixels in imset 11
NLINCORR detected 1433 saturated pixels in imset 10
NLINCORR detected 1466 saturated pixels in imset 9
NLINCORR detected 1502 saturated pixels in imset 8
NLINCORR detected 1561 saturated pixels in imset 7
NLINCORR detected 1671 saturated pixels in imset 6
NLINCORR detected 2295 saturated pixels in imset 5
NLINCORR detected 2916 saturated pixels in imset 4
NLINCORR detected 3206 saturated pixels in imset 3
NLINCORR detected 3328 saturated pixels in imset 2
NLINCORR detected 3400 saturated pixels in imset 1
NLINCORR COMPLETE
DARKCORR PERFORM
DARKFILE iref$35620245i_drk.fits
DARKFILE PEDIGREE=INFLIGHT 20/09/2009 15/11/2016
DARKFILE DESCRIP =Dark Created from 154 frames spanning cycles 17 to 24--------------
DARKCORR using dark imset 16 for imset 16 with exptime= 0
DARKCORR using dark imset 15 for imset 15 with exptime= 2.93229
DARKCORR using dark imset 14 for imset 14 with exptime= 5.86458
DARKCORR using dark imset 13 for imset 13 with exptime= 8.79687
DARKCORR using dark imset 12 for imset 12 with exptime= 11.7292
DARKCORR using dark imset 11 for imset 11 with exptime= 24.2297
DARKCORR using dark imset 10 for imset 10 with exptime= 49.2302
DARKCORR using dark imset 9 for imset 9 with exptime= 99.2306
DARKCORR using dark imset 8 for imset 8 with exptime= 199.231
DARKCORR using dark imset 7 for imset 7 with exptime= 399.231
DARKCORR using dark imset 6 for imset 6 with exptime= 599.232
DARKCORR using dark imset 5 for imset 5 with exptime= 799.232
DARKCORR using dark imset 4 for imset 4 with exptime= 999.232
DARKCORR using dark imset 3 for imset 3 with exptime= 1199.23
DARKCORR using dark imset 2 for imset 2 with exptime= 1399.23
DARKCORR using dark imset 1 for imset 1 with exptime= 1599.23
DARKCORR COMPLETE
PHOTCORR PERFORM
IMPHTTAB iref$4af1533ai_imp.fits
IMPHTTAB PEDIGREE=INFLIGHT 08/05/2009 06/04/2012
IMPHTTAB DESCRIP =photometry keywords reference file---------------------------------
NUMPAR=0, N=1
Allocated 1 parnames
==> Value of PHOTFLAM = 3.0507e-20
==> Value of PHOTPLAM = 10551.047
==> Value of PHOTBW = 845.61804
PHOTCORR COMPLETE
UNITCORR PERFORM
UNITCORR COMPLETE
CRCORR OMIT
FLATCORR PERFORM
PFLTFILE iref$4ac19225i_pfl.fits
PFLTFILE PEDIGREE=INFLIGHT 31/07/2009 05/12/2019
PFLTFILE DESCRIP =Sky Flat from Combined In-flight observations between 2009 and 2019
DFLTFILE iref$4ac18263i_dfl.fits
DFLTFILE PEDIGREE=INFLIGHT 31/07/2009 05/12/2019
DFLTFILE DESCRIP =Delta-Flat for IR Blobs by Date of Appearance from Sky Flat Dataset
FLATCORR COMPLETE
Writing calibrated readouts to ibohbfb9q_ima.fits
Writing final image to ibohbfb9q_flt.fits
with trimx = 5,5, trimy = 5,5
End 04-Sep-2024 17:51:47 UTC
*** WF3IR complete ***
End 04-Sep-2024 17:51:47 UTC
*** CALWF3 complete ***
CALWF3 completion for ibohbfb9q_raw.fits
4.3 Remove Median Background per read from resulting IMA #
Next, the median background level from each read of the resulting IMA is subtracted from the read, and is later added back to the full exposure to preserve pixel statistics.
You can define a specific subregion for stats, as well as add additional lower or upper thresholds to fine-tune the calculation of the sky median value. Here, we use the median value of the entire image minus the 5 pixel wide overscan regions with no sigma clipping.
This technique works best for relatively sparse fields, where the median value of the ramp is equivalent to the sky level.
with fits.open(raw_file.replace('raw', 'ima'), mode='update') as ima:
# You can define a subregion for stats. Here we use the whole image, minus the 5 pixel wide overscan regions
stats_region = [[5, 1014], [5, 1014]]
slx = slice(stats_region[0][0], stats_region[0][1])
sly = slice(stats_region[1][0], stats_region[1][1])
# Subtract the median countrate from each read and add back the full exposure countrate to preserve pixel statistics
total_countrate = np.median(ima['SCI', 1].data[sly, slx])
median = []
sci_num = []
for i in range(ima[0].header['NSAMP'] - 1):
med = np.median(ima['SCI', i + 1].data[sly, slx])
ima['SCI', i + 1].data += total_countrate - med
print(f'{raw_file}, [SCI, {i + 1}], median_bkg: {med:.2f}')
sci_num.append(i + 1)
median.append(med)
plt.figure(figsize=(8, 5))
plt.plot(sci_num, median)
plt.gca().invert_xaxis()
plt.title("Median Background vs. SCI Exposure Number")
plt.xlabel('SCI Exposure Number')
plt.ylabel('Median Background Value (e-)')
# Turn back on the ramp fitting for running calwf3 in the next step
ima[0].header['CRCORR'] = 'PERFORM'
ibohbfb9q_raw.fits, [SCI, 1], median_bkg: 1.50
ibohbfb9q_raw.fits, [SCI, 2], median_bkg: 1.46
ibohbfb9q_raw.fits, [SCI, 3], median_bkg: 1.44
ibohbfb9q_raw.fits, [SCI, 4], median_bkg: 1.40
ibohbfb9q_raw.fits, [SCI, 5], median_bkg: 1.26
ibohbfb9q_raw.fits, [SCI, 6], median_bkg: 0.97
ibohbfb9q_raw.fits, [SCI, 7], median_bkg: 0.67
ibohbfb9q_raw.fits, [SCI, 8], median_bkg: 0.49
ibohbfb9q_raw.fits, [SCI, 9], median_bkg: 0.47
ibohbfb9q_raw.fits, [SCI, 10], median_bkg: 0.47
ibohbfb9q_raw.fits, [SCI, 11], median_bkg: 0.46
ibohbfb9q_raw.fits, [SCI, 12], median_bkg: 0.49
ibohbfb9q_raw.fits, [SCI, 13], median_bkg: 0.47
ibohbfb9q_raw.fits, [SCI, 14], median_bkg: 0.48
ibohbfb9q_raw.fits, [SCI, 15], median_bkg: 0.42
Note: [SCI,15] is the first read and [SCI,1] is the last read of the IR image.
From the printed median background values, we can see that the increased background signal occured in the middle and the end of the exposure, just prior to Earth occulation.
4.4 Rerun calwf3 on the corrected IMA, with ‘ramp fitting’ turned back on #
We resume calwf3 on the cleaned IMA with the CRCORR step turned on. This will produce a corrected IMA and FLT image.
wfc3tools.calwf3(raw_file.replace('raw', 'ima'))
# Clean up and rename files
os.remove('ibohbfb9q_ima.fits')
os.rename('ibohbfb9q_ima_ima.fits', 'ibohbfb9q_ima.fits')
os.rename('ibohbfb9q_ima_flt.fits', 'ibohbfb9q_flt.fits')
git tag: e0988181-dirty
git branch: HEAD
HEAD @: e098818105734475101120a544dfe468207bc690
CALBEG*** CALWF3 -- Version 3.7.1 (Oct-18-2023) ***
Begin 04-Sep-2024 17:51:48 UTC
Input ibohbfb9q_ima.fits
loading asn
LoadAsn: Processing SINGLE exposure
Trying to open ibohbf040_asn.fits...
Read in Primary header from ibohbf040_asn.fits...
Creating new trailer file `ibohbfb9q_ima.tra'.
CALBEG*** WF3IR -- Version 3.7.1 (Oct-18-2023) ***
Begin 04-Sep-2024 17:51:48 UTC
Input ibohbfb9q_ima.fits
Output ibohbfb9q_ima_flt.fits
Trying to open ibohbfb9q_ima.fits...
Read in Primary header from ibohbfb9q_ima.fits...
APERTURE IR-FIX
FILTER F105W
DETECTOR IR
Reading data from ibohbfb9q_ima.fits ...
CCDTAB iref$t2c16200i_ccd.fits
CCDTAB PEDIGREE=Ground
CCDTAB DESCRIP =Reference data based on Thermal-Vac #3, gain=2.5 results for IR-4
CCDTAB DESCRIP =Readnoise,gain,saturation from TV3,MEB2 values. ISRs 2008-25,39,50
readnoise =20.2,19.8,19.9,20.1
gain =2.34,2.37,2.31,2.38
DQICORR COMPLETE
ZSIGCORR COMPLETE
BLEVCORR COMPLETE
ZOFFCORR COMPLETE
NOISCORR PERFORM
NOISCORR SKIPPED
NLINCORR COMPLETE
DARKCORR COMPLETE
PHOTCORR COMPLETE
UNITCORR COMPLETE
CRCORR PERFORM
CRREJTAB iref$u6a1748ri_crr.fits
CRIDCALC using 4 sigma rejection threshold
256 bad DQ mask
4 max CRs for UNSTABLE
55 pixels detected as unstable
CRCORR COMPLETE
FLATCORR COMPLETE
Writing calibrated readouts to ibohbfb9q_ima_ima.fits
Writing final image to ibohbfb9q_ima_flt.fits
with trimx = 5,5, trimy = 5,5
End 04-Sep-2024 17:51:50 UTC
*** WF3IR complete ***
End 04-Sep-2024 17:51:50 UTC
*** CALWF3 complete ***
CALWF3 completion for ibohbfb9q_ima.fits
5. Comparing Original and Corrected FLT products #
Here we compare the original and corrected FLT products of ibogbfb9q and the unaffected ibohbfb7q.
5.1 Displaying the Images #
Here we see that the corrected FLT image has reduced in visible noise.
# Display the images
flt1 = fits.getdata('ibohbfb7q_flt.fits', ext=1)
flt2 = fits.getdata('ibohbfb9q_flt_orig.fits', ext=1)
flt2corr = fits.getdata('ibohbfb9q_flt.fits', ext=1)
fig = plt.figure(figsize=(20, 10))
ax1 = fig.add_subplot(1, 3, 1)
ax2 = fig.add_subplot(1, 3, 2)
ax3 = fig.add_subplot(1, 3, 3)
A = ax1.imshow(flt1, vmin=0.25, vmax=1.25, cmap='Greys_r', origin='lower')
B = ax2.imshow(flt2, vmin=1.30, vmax=2.30, cmap='Greys_r', origin='lower')
C = ax3.imshow(flt2corr, vmin=1.30, vmax=2.30, cmap='Greys_r', origin='lower')
ax1.set_title('ibohbfb7q (Linear Bkg)', fontsize=20)
ax2.set_title('ibohbfb9q (Time Variable Bkg)', fontsize=20)
ax3.set_title('ibohbfb9q (Corrected Bkg)', fontsize=20)
fig.colorbar(A, ax=ax1, shrink=.5)
fig.colorbar(B, ax=ax2, shrink=.49)
fig.colorbar(C, ax=ax3, shrink=.5)
<matplotlib.colorbar.Colorbar at 0x7f2ef26f7ad0>
5.2 Plotting a Histogram of the Sky Background #
The corrected FLT product of the affected image now has a gaussian distribution that is expected from the sky background.
# Plot the histograms
flt1f = flt1.flatten()
flt2f = flt2.flatten()
flt2fcorr = flt2corr.flatten()
fig = plt.figure(figsize=(20, 5))
ax1 = fig.add_subplot(1, 3, 1)
ax2 = fig.add_subplot(1, 3, 2)
ax3 = fig.add_subplot(1, 3, 3)
n, bins, patches = ax1.hist(flt1f, bins=200, range=(0, 1))
n, bins, patches = ax2.hist(flt2f, bins=200, range=(1, 2))
n, bins, patches = ax3.hist(flt2fcorr, bins=200, range=(1, 2))
ax1.set_title('ibohbfb7q (Linear Bkg)', fontsize=20)
ax2.set_title('ibohbfb9q (Time Variable Bkg)', fontsize=20)
ax3.set_title('ibohbfb9q (Corrected)', fontsize=20)
Text(0.5, 1.0, 'ibohbfb9q (Corrected)')
5.3 Inspecting the IR Ramps #
Below we plot the counts versus time for the original and corrected ramp of the affected exposure ibohbfb9q. The median background rate is now linear across all of the ramps.
imafiles = ('ibohbfb9q_ima_orig.fits', 'ibohbfb9q_ima.fits')
ylim = [(0.3, 0.7), (0.3, 1.8)]
fig, axarr = plt.subplots(1, 2, figsize=(15, 5))
fig.set_dpi(100)
for i, ima in enumerate(imafiles):
time, counts = wfc3tools.pstat(ima, stat='midpt', units='counts', plot=False)
axarr[i].plot(time, counts, '+', markersize=8)
axarr[i].set_title(ima)
axarr[i].set_xlabel('Exposure Time (s)')
axarr[i].set_ylabel('counts (e-)')
Here we plot the median count rate versus time for the original and corrected ramp. We see the corrected image now has a flat median count rate.
imafiles = ('ibohbfb9q_ima_orig.fits', 'ibohbfb9q_ima.fits')
ylim = [(0.3, 1.8), (0.3, 1.8)]
fig, axarr = plt.subplots(1, 2, figsize=(15, 5))
fig.set_dpi(100)
for i, ima in enumerate(imafiles):
time, counts = wfc3tools.pstat(ima, stat='midpt', units='rate', plot=False)
axarr[i].plot(time, counts, '+', markersize=8)
axarr[i].set_title(ima)
axarr[i].set_xlabel('Exposure Time (s)')
axarr[i].set_ylabel('Count Rate (e-/s)')
axarr[i].set_ylim(ylim[i])
5.4 Comparing the Background Noise #
Below we compute the mean, median, and standard deviation (electrons/sec) of the different FLT data products.
w = np.where((0. < flt1f) & (flt1f < 1.))
flt1_mean = np.mean(flt1f[w])
flt1_med = np.median(flt1f[w])
flt1_std = np.std(flt1f[w])
w = np.where((1. < flt2f) & (flt2f < 2.))
flt2_mean = np.mean(flt2f[w])
flt2_med = np.median(flt2f[w])
flt2_std = np.std(flt2f[w])
w = np.where((1. < flt2fcorr) & (flt2fcorr < 2.))
flt2f_mean = np.mean(flt2fcorr[w])
flt2f_med = np.median(flt2fcorr[w])
flt2f_std = np.std(flt2fcorr[w])
ramp = ['Linear Bkg', 'Time Variable Bkg', 'Corrected Bkg']
filename = ['ibohbfb7q_flt', 'ibohbfb9q_flt_orig.fits', 'ibohbfb9q_flt.fits']
mean = [flt1_mean, flt2_mean, flt2f_mean]
med = [flt1_med, flt2_med, flt2f_med]
std = [flt1_std, flt2_std, flt2f_std]
Table = {
"RAMP": ramp,
"Filename": filename,
"Mean": mean,
"Median": med,
"Stdev": std,
}
df = pd.DataFrame(Table)
df
| RAMP | Filename | Mean | Median | Stdev | |
|---|---|---|---|---|---|
| 0 | Linear Bkg | ibohbfb7q_flt | 0.469745 | 0.466199 | 0.037908 |
| 1 | Time Variable Bkg | ibohbfb9q_flt_orig.fits | 1.538454 | 1.623123 | 0.169355 |
| 2 | Corrected Bkg | ibohbfb9q_flt.fits | 1.503880 | 1.501057 | 0.045676 |
Here we can see that the corrected background of the affected FLT file has a standard deviation that improved by a factor of ~4. This corrected standard deviation is also more agreeable with the standard deviation of the linear background of the unaffected image ibohbfb7q.
5.5 Morphology #
The improved FLT products can be useful for studies on faint, extended sources. Here, we plot 25”x25” cutouts of the original and corrected FLT data product. The corrected FLT product no longer shows a bimodal distribution in the pixel values (due to poor ramp fit) and therefore appears less noisy. The reprocessed image will now produce more accurate photometry and have reduced systematic errors.
# Display the images
flt2 = fits.getdata('ibohbfb9q_flt_orig.fits', ext=1)
flt2corr = fits.getdata('ibohbfb9q_flt.fits', ext=1)
fig = plt.figure(figsize=(20, 8))
ax1 = fig.add_subplot(1, 2, 1)
ax2 = fig.add_subplot(1, 2, 2)
ax1.imshow(flt2[520:720, 750:970], vmin=1.3, vmax=2.5, cmap='Greys_r', origin='lower')
ax2.imshow(flt2corr[520:720, 750:970], vmin=1.3, vmax=2.5, cmap='Greys_r', origin='lower')
ax1.set_title('ibohbfb9q (Time Variable Bkg)', fontsize=20)
ax2.set_title('ibohbfb9q (Corrected Bkg)', fontsize=20)
Text(0.5, 1.0, 'ibohbfb9q (Corrected Bkg)')
6. Conclusion #
Thank you for walking through this notebook. When working with WFC3/IR data, you should be more familiar with:
Identifying IR exposures with time variable background.
Compute and analyze statistics on IMA files.
Reprocess a single exposure without cosmic ray corrections through
calwfc3.Correct for spatially uniform time variable background by flattening IR ramps.
Congratulations! You have completed the notebook!#
Additional Resources #
Below are some additional resources that may be helpful. Please send any questions through the HST Help Desk.
-
see section 7.10 for more information on time-variable background.
About this Notebook #
Author: Ky Huynh, Jennifer Mack, WFC3 Instrument Team
Published On: 2023-05-25
Updated On: 2023-11-21
Citations #
If you use numpy, astropy, astroquery, or wfc3tools for published research, please cite the
authors. Follow these links for more information about citing the libraries below:
Top of Page
