ACS Image Reduction for Subarray Data#
Introduction#
Subarray data requires different considerations than working with full frame images. This notebook will guide you through working with standard post-SM4 subarray data and custom subarray data.
After Servicing Mission 4 (SM4; May 2009), the installation of the ASIC during the repair of ACS introduced \(1/f\) noise in all ACS images. In the calacs pipeline, only full-frame data have this striping removed. To correct subarray data, the alternative acs_destripe_plus pipeline must be used, which will apply all of calibration steps normally performed by calacs in addition to de-striping for subarray data. De-striping is only possible for 2K subarrays after SM4 until Cycle 24, after which a change to the flight software makes all subarrays eligible for de-striping.
This tutorial will show you how to handle…#
1. Post-SM4 Subarray Data#
Update header keywords.
Clean Subarray images with the option to correct CTE losses.
Update WCS solution.
2. Custom Subarray Data#
Use
AstroDrizzlewith ASN files to combine images.
Imports#
Here we list the Python packages used in this notebook. Links to the documentation for each module is provided for convenience.
Package Name |
module |
docs |
used for |
|---|---|---|---|
|
|
command line input |
|
|
|
setting environments |
|
|
|
remove directory tree |
|
|
|
search for files based on Unix shell rules |
|
|
|
download data from MAST |
|
|
|
access and update fits files |
|
|
|
constructing and editing in a tabular format |
|
|
|
update wcs solution |
|
|
|
destripe acs images and optionally CTE correct |
|
|
|
check for settings in oscntab |
import os
import shutil
from astroquery.mast import Observations
from astropy.io import fits
from astropy.table import Table
from stwcs import updatewcs
from acstools import (acs_destripe_plus, utils_calib)
from IPython.display import clear_output
WARNING: matplotlib not found, plotting is disabled [acstools.satdet]
WARNING:astroquery:matplotlib not found, plotting is disabled
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/acstools/utils_findsat_mrt.py:24: UserWarning: skimage not installed. MRT calculation will not work: ModuleNotFoundError("No module named 'skimage'")
warnings.warn(f'skimage not installed. MRT calculation will not work: {repr(e)}') # noqa
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/acstools/utils_findsat_mrt.py:30: UserWarning: scipy not installed. Kernel generation will not work
warnings.warn('scipy not installed. Kernel generation will not work')
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/acstools/utils_findsat_mrt.py:37: UserWarning: matplotlib not found, plotting is disabled
warnings.warn('matplotlib not found, plotting is disabled')
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/acstools/findsat_mrt.py:117: UserWarning: matplotlib not found, plotting is disabled
warnings.warn('matplotlib not found, plotting is disabled')
Here we set environment variables for later use with the Calibration Reference Data System (CRDS).
os.environ['CRDS_SERVER_URL'] = 'https://hst-crds.stsci.edu'
os.environ['CRDS_SERVER'] = 'https://hst-crds.stsci.edu'
os.environ['CRDS_PATH'] = './crds_cache'
os.environ['jref'] = './crds_cache/references/hst/acs/'
Download the Data#
Here we download all of the data required for this notebook. This is an important step! Some of the image processing steps require all relevant files to be in the working directory. We recommend working with a brand new directory for every new set of data.
GO Proposal 14511: “ACS CCD Stability Monitor”#
Our data for the first example comes from a routine calibration program of two images using the post-Cycle 24 WFC1A-1K 1024 x 2072 subarray. We will only use the data associated with the observation id JD5702JWQ.
Using the python package astroquery, we can download files from the MAST archive.
obs_table = Observations.query_criteria(proposal_id=14511, obs_id='JD5702JWQ')
dl_table = Observations.download_products(obs_table['obsid'],
productSubGroupDescription=['RAW'],
mrp_only=False)
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jd5702jwq_raw.fits to ./mastDownload/HST/jd5702jwq/jd5702jwq_raw.fits ...
[Done]
We’ll use the package os to put all of these files in our working directory for convenience.
for row in dl_table:
oldfname = row['Local Path']
newfname = os.path.basename(oldfname)
os.rename(oldfname, newfname)
GO Proposal 10206: “What drives the outflows in powerful radio galaxies?”#
For the second example, we will use a ramp filter observation of the galaxy PLS1345+12 (HST proposal 10206). The association name is J92SA0010, and we will only use one image in the association: J92SA0W6Q.
Again, we use astroquery, to download files from the MAST archive. Along with the raw images, we will also need the spt (telemetry) files to reduce our custom subarray data.
obs_table = Observations.query_criteria(proposal_id=10206, obs_id='J92SA0010')
dl_table = Observations.download_products(obs_table['obsid'],
productSubGroupDescription=['RAW', 'SPT'],
mrp_only=False)
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/j92sa0010_spt.fits to ./mastDownload/HST/j92sa0010/j92sa0010_spt.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/j92sa0w6q_raw.fits to ./mastDownload/HST/j92sa0w6q/j92sa0w6q_raw.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/j92sa0w6q_spt.fits to ./mastDownload/HST/j92sa0w6q/j92sa0w6q_spt.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/j92sa0w7q_raw.fits to ./mastDownload/HST/j92sa0w7q/j92sa0w7q_raw.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/j92sa0w7q_spt.fits to ./mastDownload/HST/j92sa0w7q/j92sa0w7q_spt.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/j92sa0w8q_raw.fits to ./mastDownload/HST/j92sa0w8q/j92sa0w8q_raw.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/j92sa0w8q_spt.fits to ./mastDownload/HST/j92sa0w8q/j92sa0w8q_spt.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/j92sa0w9q_raw.fits to ./mastDownload/HST/j92sa0w9q/j92sa0w9q_raw.fits ...
[Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/j92sa0w9q_spt.fits to ./mastDownload/HST/j92sa0w9q/j92sa0w9q_spt.fits ...
[Done]
Again, we’ll use the package os to put all of these files in our working directory for convenience.
for row in dl_table:
oldfname = row['Local Path']
newfname = os.path.basename(oldfname)
os.rename(oldfname, newfname)
Now that all of our files are in the current working directory, we delete the leftover MAST file structure.
shutil.rmtree('mastDownload')
File Information#
The structure of the fits files from ACS may be different depending on what kind of observation was made. For more information, refer to Section 2.2 of the ACS Data Handbook.
Raw Files#
A standard raw image file from a subarray has the same structure as you’d expect from full frame observation from ACS/WCS.
Ext |
Name |
Type |
Contains |
|---|---|---|---|
0 |
PRIMARY |
(PrimaryHDU) |
Meta-data related to the entire file. |
1 |
SCI (Image) |
(ImageHDU) |
Raw image data. |
2 |
ERR (Error) |
(ImageHDU) |
Error array. |
3 |
DQ (Data Quality) |
(ImageHDU) |
Data quality array. |
SPT Files#
SPT files contain telemetry and engineering data from the telescope.
Ext |
Name |
Type |
Contains |
|---|---|---|---|
0 |
PRIMARY |
(PrimaryHDU) |
Meta-data related to the entire file. |
1 |
UDL (Image) |
(ImageHDU) |
Raw image data. |
You can always use .info() on an HDUlist for an overview of the structure
with fits.open('j92sa0w6q_raw.fits') as hdulist:
hdulist.info()
with fits.open('j92sa0w6q_spt.fits') as hdulist:
hdulist.info()
Filename: j92sa0w6q_raw.fits
No. Name Ver Type Cards Dimensions Format
0 PRIMARY 1 PrimaryHDU 216 ()
1 SCI 1 ImageHDU 97 (1180, 1200) int16 (rescales to uint16)
2 ERR 1 ImageHDU 49 ()
3 DQ 1 ImageHDU 43 ()
Filename: j92sa0w6q_spt.fits
No. Name Ver Type Cards Dimensions Format
0 PRIMARY 1 PrimaryHDU 341 ()
1 UDL 1 ImageHDU 305 (965,) int16
De-striping and CTE Correction of Post-SM4 Subarray Observations#
Download Calibration Files: Update Header Keywords#
We can call the Calibration Reference Data System (CRDS) to get the associated calibration files for this image.
First we will need to turn on the CTE correction switch in the primary header. Turning on this switch will notify the CRDS bestrefs tool to add the names of the CTE correction parameters table ‘PCTETAB’ and CTE-corrected dark current image ‘DRKCFILE’ reference files to the header.
sbc_fits = 'jd5702jwq_raw.fits'
with fits.open(sbc_fits, mode='update') as hdulist:
hdulist[0].header['PCTECORR'] = 'PERFORM'
Download Calibration Files#
The CRDS program can be run from the terminal to download the files specific to your observation. This program is packaged with astroconda.
The command line input to download our files for jd5702jwq_raw.fits is as follows:
crds bestrefs --files jd5702jwq_raw.fits --sync-references=1 --update-bestrefs
Here, we use the python package os to run this command.
!crds bestrefs --files {sbc_fits} --sync-references=1 --update-bestrefs
clear_output()
Clean Subarray Images#
Next we will run the acs_destripe_plus code on our image. This will execute all of the calibration steps that are set to ‘PERFORM’ in the primary header of the FITS file.
The acs_destripe_plus code will produce the FLT file is the calibrated output product from CALACS, jd5702jmq_flt.fits. With the CTE correction turned on, an FLC file will also be produced, which is the same as the FLT file but with the CTE correction applied.
acs_destripe_plus.destripe_plus(sbc_fits, cte_correct=True)
git tag: e0988181-dirty
git branch: HEAD
HEAD @: e098818105734475101120a544dfe468207bc690
Trying to open jd5702jwq_raw.fits...
Read in Primary header from jd5702jwq_raw.fits...
CALBEG*** ACSCCD -- Version 10.3.5 (08-Feb-2022) ***
Begin 18-Sep-2024 12:45:11 UTC
Input jd5702jwq_raw.fits
Output jd5702jwq_blv_tmp.fits
Trying to open jd5702jwq_raw.fits...
Read in Primary header from jd5702jwq_raw.fits...
APERTURE WFC1A-2K
FILTER1 F606W
FILTER2 CLEAR2L
DETECTOR WFC
CCDTAB jref$72m1821dj_ccd.fits
CCDTAB PEDIGREE=inflight
CCDTAB DESCRIP =CCD table with updated readnoise values for CCDGAIN=2.-------------
CCDTAB DESCRIP =July 2009
DQICORR PERFORM
DQITAB jref$25g1256nj_bpx.fits
DQITAB PEDIGREE=INFLIGHT 30/03/2002 30/05/2002
DQITAB DESCRIP =FITS reference table for bad pixel locations on ACS WFC
DQICORR COMPLETE
BIASCORR PERFORM
BIASFILE jref$6641832dj_bia.fits
BIASFILE PEDIGREE=INFLIGHT 08/02/2017 08/03/2017
BIASFILE DESCRIP =Standard full-frame bias for data taken after Feb 07 2017 11:26:21-
BIASCORR COMPLETE
BLEVCORR PERFORM
OSCNTAB jref$17717071j_osc.fits
OSCNTAB PEDIGREE=GROUND
OSCNTAB DESCRIP =New OSCNTAB which includes entries for all subarrays.--------------
This subarray data is supported for the bias shift correction.
BLEVCORR PERFORM
Performing bias-shift correction for subarray data.
BLEVCORR PERFORM
Bias shift correcting for bias level in Amp A of 4197.2216 electrons.
BLEVCORR COMPLETE
Supported Subarray adjusted Darktime: 43.094452 for aperture: WFC1A-2K
DARKTIME from SCI header: 43.094452 Offset from CCDTAB: 0.000000 Final DARKTIME: 43.094452
Full-well saturation flagging being performed for imset 1.
Subarray full-well saturation image flagging step being performed.
Subarray full-well saturation image flagging step done.
Uncertainty array initialized,
readnoise =4.35,
gain =2.02,
default bias levels =4242,
SINKCORR PERFORM
SNKCFILE jref$25e1542jj_snk.fits
SNKCFILE PEDIGREE=INFLIGHT 08/02/2017 08/03/2017
SNKCFILE DESCRIP =Sink pixel reference file ----------------------------
SINKCORR COMPLETE
INFO:acs_destripe:Processing jd5702jwq_blv_tmp.fits
End 18-Sep-2024 12:45:12 UTC
*** ACSCCD complete ***
INFO:acs_destripe:clean_streak - Performing image bias de-stripe:
INFO:acs_destripe:clean_streak - Image bias de-stripe: Done.
WARNING:acs_destripe:clean_streak - Maximum number of clipping iterations specified by the user (15) has been reached.
INFO:acs_destripe:perform_correction - ===== Overall statistics for de-stripe corrections: =====
INFO:acs_destripe:perform_correction - STDDEV of applied de-stripe corrections 0.952.
INFO:acs_destripe:perform_correction - Estimated background: 4.6138.
INFO:acs_destripe:perform_correction - Maximum applied correction: 3.51.
INFO:acs_destripe:perform_correction - Effective number of clipping iterations: 15.
INFO:acs_destripe:perform_correction - Effective number of additional (repeated) cleanings: 0.
INFO:acs_destripe:perform_correction - Total number of corrected rows: 2048.
INFO:acs_destripe:jd5702jwq_blv_tmp_strp.fits created
git tag: e0988181-dirty
git branch: HEAD
HEAD @: e098818105734475101120a544dfe468207bc690
Setting max threads to 4 out of 4 available
Trying to open jd5702jwq_blv_tmp.fits...
Read in Primary header from jd5702jwq_blv_tmp.fits...
CALBEG*** ACSCTE -- Version 10.3.5 (08-Feb-2022) ***
Begin 18-Sep-2024 12:45:13 UTC
Input jd5702jwq_blv_tmp.fits
Output jd5702jwq_blc_tmp.fits
Trying to open jd5702jwq_blv_tmp.fits...
Read in Primary header from jd5702jwq_blv_tmp.fits...
APERTURE WFC1A-2K
FILTER1 F606W
FILTER2 CLEAR2L
DETECTOR WFC
CCDTAB jref$72m1821dj_ccd.fits
CCDTAB PEDIGREE=inflight
CCDTAB DESCRIP =CCD table with updated readnoise values for CCDGAIN=2.-------------
CCDTAB DESCRIP =July 2009
PCTECORR PERFORM
PCTEFILE jref$19i16323j_cte.fits
PCTEFILE PEDIGREE=INFLIGHT 01/03/2002 22/07/2010
PCTEFILE DESCRIP =Parameters needed for gen2 pixel-based CTE correction -------------
Trying to open jref$19i16323j_cte.fits...
Read in Primary header from jref$19i16323j_cte.fits...
(pctecorr) Generation 2 PCTETAB file auto-detected.
(pctecorr) Reading CTE parameters from PCTETAB file: 'jref$19i16323j_cte.fits'...
Trying to open jref$19i16323j_cte.fits...
Read in Primary header from jref$19i16323j_cte.fits...
CTE_NAME: PixelCTE 2017
CTE_VER: 2.0
CTEDATE0: 52334.9
CTEDATE1: 57710.4
PCTETLEN: 60
PCTERNOI: 4.300000
PCTERNFOR: 5
PCTERNPAR: 7
PCTENSMD: 0
PCTETRSH: -10
FIXROCR: 1
Reading in image from extension 3
Reading in image from extension 4
ERROR: Keyword = `PCTENSMD'.
ERROR: Keyword = `PCTETLEN'.
ERROR: Keyword = `PCTERNOI'.
ERROR: Keyword = `PCTENFOR'.
ERROR: Keyword = `PCTENPAR'.
ERROR: Keyword = `FIXROCR'.
Warning (pctecorr) IGNORING read noise level PCTERNOI from PCTETAB: 4.300000. Using amp dependent values from CCDTAB instead
(pctecorr) Readout simulation forward modeling iterations PCTENFOR: 5
(pctecorr) Number of iterations used in the parallel transfer PCTENPAR: 7
(pctecorr) CTE_FRAC: 1.020340
(pctecorr) PCTETAB read
(pctecorr) Using parallel processing provided by OpenMP inside CTE routine
(pctecorr) Performing CTE correction for amp A
(pctecorr) Read noise level from CCDTAB: 4.350000.
(pctecorr) Calculating smooth readnoise image...
(pctecorr) Time taken to smooth image: 2.07(s) with 4 threads
(pctecorr) ...complete.
(pctecorr) Creating charge trap image...
(pctecorr) Time taken to populate pixel trap map image: 0.01(s) with 4 threads
(pctecorr) ...complete.
(pctecorr) Running correction algorithm...
(pctecorr) ...complete.
(pctecorr) Total count difference (corrected-raw) incurred from correction: 530760.750000 (0.575928%)
(pctecorr) CTE run time for current chip: 398.87(s) with 4 procs/threads
PCTECORR COMPLETE
End 18-Sep-2024 12:51:54 UTC
*** ACSCTE complete ***
git tag: e0988181-dirty
git branch: HEAD
HEAD @: e098818105734475101120a544dfe468207bc690
Trying to open jd5702jwq_blv_tmp.fits...
Read in Primary header from jd5702jwq_blv_tmp.fits...
CALBEG*** ACS2D -- Version 10.3.5 (08-Feb-2022) ***
Begin 18-Sep-2024 12:51:54 UTC
Input jd5702jwq_blv_tmp.fits
Output jd5702jwq_flt.fits
Trying to open jd5702jwq_blv_tmp.fits...
Read in Primary header from jd5702jwq_blv_tmp.fits...
APERTURE WFC1A-2K
FILTER1 F606W
FILTER2 CLEAR2L
DETECTOR WFC
Imset 1 Begin 12:51:54 UTC
CCDTAB jref$72m1821dj_ccd.fits
CCDTAB PEDIGREE=inflight
CCDTAB DESCRIP =CCD table with updated readnoise values for CCDGAIN=2.-------------
CCDTAB DESCRIP =July 2009
DQICORR OMIT
DARKCORR PERFORM
DARKFILE jref$78f1759ej_drk.fits
DARKFILE PEDIGREE=INFLIGHT 08/02/2017 08/03/2017
DARKFILE DESCRIP =Standard full-frame dark for data taken after Feb 07 2017 11:26:21-
Darktime from header 43.094452
Mean of dark image (MEANDARK) = 0.503474
DARKCORR COMPLETE
FLSHCORR OMIT
FLATCORR PERFORM
PFLTFILE jref$qb12257sj_pfl.fits
PFLTFILE PEDIGREE=INFLIGHT 18/04/2002 - 04/07/2006
PFLTFILE DESCRIP =F606W step +1 flat w/ mote shifted to -1 step
FLATCORR COMPLETE
SHADCORR OMIT
PHOTCORR PERFORM
Found parameterized variable 1.
NUMPAR=1, N=1
Allocated 1 parnames
Adding parameter mjd#57819.7836 as parnames[0]
==> Value of PHOTFLAM = 7.8819263e-20
==> Value of PHOTPLAM = 5921.892
==> Value of PHOTBW = 672.48329
IMPHTTAB jref$4af1559ij_imp.fits
IMPHTTAB PEDIGREE=INFLIGHT 18/04/2000 29/09/2020
IMPHTTAB DESCRIP =photometry keywords reference file---------------------------------
PHOTCORR COMPLETE
Imset 1 End 12:51:54 UTC
End 18-Sep-2024 12:51:54 UTC
*** ACS2D complete ***
git tag: e0988181-dirty
git branch: HEAD
HEAD @: e098818105734475101120a544dfe468207bc690
Trying to open jd5702jwq_blc_tmp.fits...
Read in Primary header from jd5702jwq_blc_tmp.fits...
CALBEG*** ACS2D -- Version 10.3.5 (08-Feb-2022) ***
Begin 18-Sep-2024 12:51:54 UTC
Input jd5702jwq_blc_tmp.fits
Output jd5702jwq_flc.fits
Trying to open jd5702jwq_blc_tmp.fits...
Read in Primary header from jd5702jwq_blc_tmp.fits...
APERTURE WFC1A-2K
FILTER1 F606W
FILTER2 CLEAR2L
DETECTOR WFC
Imset 1 Begin 12:51:54 UTC
CCDTAB jref$72m1821dj_ccd.fits
CCDTAB PEDIGREE=inflight
CCDTAB DESCRIP =CCD table with updated readnoise values for CCDGAIN=2.-------------
CCDTAB DESCRIP =July 2009
DQICORR OMIT
DARKCORR PERFORM
DARKFILE jref$78g0231bj_dkc.fits
DARKFILE PEDIGREE=INFLIGHT 08/02/2017 08/03/2017
DARKFILE DESCRIP =CTE corrected dark for WFC data taken after Feb 07 2017 11:26:21---
Darktime from header 43.094452
Mean of dark image (MEANDARK) = 0.411565
DARKCORR COMPLETE
FLSHCORR OMIT
FLATCORR PERFORM
PFLTFILE jref$qb12257sj_pfl.fits
PFLTFILE PEDIGREE=INFLIGHT 18/04/2002 - 04/07/2006
PFLTFILE DESCRIP =F606W step +1 flat w/ mote shifted to -1 step
INFO:acs_destripe_plus:Done.
FLT: jd5702jwq_flt.fits
FLC: jd5702jwq_flc.fits
FLATCORR COMPLETE
SHADCORR OMIT
PHOTCORR PERFORM
Found parameterized variable 1.
NUMPAR=1, N=1
Allocated 1 parnames
Adding parameter mjd#57819.7836 as parnames[0]
==> Value of PHOTFLAM = 7.8819263e-20
==> Value of PHOTPLAM = 5921.892
==> Value of PHOTBW = 672.48329
IMPHTTAB jref$4af1559ij_imp.fits
IMPHTTAB PEDIGREE=INFLIGHT 18/04/2000 29/09/2020
IMPHTTAB DESCRIP =photometry keywords reference file---------------------------------
PHOTCORR COMPLETE
Imset 1 End 12:51:54 UTC
End 18-Sep-2024 12:51:54 UTC
*** ACS2D complete ***
Correct the WCS#
The subarray products produced by this process do not have the proper WCS information in the header. The WCS is normally updated by the pipeline via an additional call to AstroDrizzle. Here, we can manually update the WCS of our FLC product using stwcs.updatewcs.
updatewcs.updatewcs('jd5702jwq_flc.fits', use_db=False)
Using 2015-calibrated VAFACTOR-corrected TDD correction...
- IDCTAB: Distortion model from row 443 for chip 1 : F606W
Using 2015-calibrated VAFACTOR-corrected TDD correction...
- IDCTAB: Distortion model from row 443 for chip 1 : F606W
['jd5702jwq_flc.fits']
Reducing Custom Subarray Data#
Download Calibration Files#
Like before, we can use CRDS to get the associated calibration files for this image.
sbc_fits = 'j92sa0w6q_raw.fits'
!crds bestrefs --files {sbc_fits} --sync-references=1 --update-bestrefs
clear_output()
Access OSCN Table#
The name in the header is of the format ‘ref\\(oscn_name.fits</font>', therefore we need to split the string on the '\)’ character.
prihdr = fits.getheader(sbc_fits)
scihdr = fits.getheader(sbc_fits, ext=1)
We also want to retrieve the OSCNTAB reference file from the JREF directory. We can get the name of the file from the primary header of the image.
oscn_name = prihdr['OSCNTAB'].split('$')[-1]
path_to_oscn = os.path.join(os.environ['jref'], oscn_name)
print(path_to_oscn)
./crds_cache/references/hst/acs/17717071j_osc.fits
The utils_calib.check_oscntab from acstools checks the OSCNTAB file if any entry matches the combination of readout amplifier, image size, and number of bias prescan columns for a given subarray observation. We will need to use several header keyword values to test if a subarray is in OSCNTAB.
oscnrec = fits.getdata(path_to_oscn)
oscnhdr = fits.getheader(path_to_oscn)
oscntable = Table(oscnrec)
The raw image has 1180 columns and 1200 rows, which does not correspond to any entry in the OSCNTAB file, but a visual examination of the image shows that it contains bias prescan columns.
Get the Bias Prescan Information#
From the science image PRI and SCI extension headers that we opened earlier, we can get the information about the readout amplifier and dimensions of the image.
amp = prihdr['CCDAMP']
xsize = scihdr['NAXIS1']
ysize = scihdr['NAXIS2']
print(xsize, ysize)
1180 1200
To get information on the number of prescan columns (if any), we need to access the SPT first extension header.
spthdr = fits.getheader('j92sa0w6q_spt.fits', ext=1)
leading = spthdr['OVERSCNA']
trailing = spthdr['OVERSCNB']
Finally, we check if this subarray definition is in the OSCNTAB file. The code returns a boolean result, which we have saved as the variable supported, to describe this.
supported = utils_calib.check_oscntab(path_to_oscn, amp, xsize, ysize, leading, trailing)
print(supported)
WARNING: UnitsWarning: 'pixels' did not parse as fits unit: At col 0, Unit 'pixels' not supported by the FITS standard. Did you mean pixel? If this is meant to be a custom unit, define it with 'u.def_unit'. To have it recognized inside a file reader or other code, enable it with 'u.add_enabled_units'. For details, see https://docs.astropy.org/en/latest/units/combining_and_defining.html [astropy.units.core]
WARNING:astroquery:UnitsWarning: 'pixels' did not parse as fits unit: At col 0, Unit 'pixels' not supported by the FITS standard. Did you mean pixel? If this is meant to be a custom unit, define it with 'u.def_unit'. To have it recognized inside a file reader or other code, enable it with 'u.add_enabled_units'. For details, see https://docs.astropy.org/en/latest/units/combining_and_defining.html
False
Update OSCNTAB#
Now that we have confirmed that the OSCNTAB file does not contain information about our subarray data, we need to add a new row to the table with our definitions. Let’s first view the first few rows of OSCNTAB to see what our new entry needs to look like.
print(oscntable)
CCDAMP CCDCHIP BINX BINY NX ... VX1 VX2 VY1 VY2
------ ------- ---- ---- ---- ... --- --- --- ---
C 2 1 1 4144 ... 0 0 0 0
C 2 1 1 2072 ... 0 0 0 0
C 2 1 1 2072 ... 0 0 0 0
C 2 1 1 2072 ... 0 0 0 0
C 2 1 1 2070 ... 0 0 0 0
C 2 1 1 534 ... 0 0 0 0
D 2 1 1 4144 ... 0 0 0 0
D 2 1 1 2072 ... 0 0 0 0
D 2 1 1 2072 ... 0 0 0 0
D 2 1 1 2072 ... 0 0 0 0
... ... ... ... ... ... ... ... ... ...
B 1 1 1 2072 ... 0 0 0 0
B 1 1 1 2072 ... 0 0 0 0
B 1 1 1 2070 ... 0 0 0 0
B 1 1 1 1046 ... 0 0 0 0
B 1 1 1 534 ... 0 0 0 0
AC 1 1 1 4144 ... 0 0 0 0
AD 1 1 1 4144 ... 0 0 0 0
BC 1 1 1 4144 ... 0 0 0 0
BD 1 1 1 4144 ... 0 0 0 0
ABCD 1 1 1 4144 ... 0 0 0 0
Length = 35 rows
We can also choose to just view all of the columns names
print(oscntable.colnames)
['CCDAMP', 'CCDCHIP', 'BINX', 'BINY', 'NX', 'NY', 'TRIMX1', 'TRIMX2', 'TRIMY1', 'TRIMY2', 'BIASSECTA1', 'BIASSECTA2', 'BIASSECTB1', 'BIASSECTB2', 'DESCRIPTION', 'VX1', 'VX2', 'VY1', 'VY2']
Several column names are obvious, but here we define the less obvious ones.
Column Name |
Description |
|---|---|
BINX, BINY |
Binning in X and Y. ACS data are never binned, so these will always be 1. |
TRIMXn |
Number of prescan columns on the left (1) and right (2) sides of the image to remove. |
TRIMYn |
Number of virtual rows on the bottom (1) and top (2) sides of the image to remove. For subarray data, these are always 0. |
BIASSECTAn |
Start and end columns to use for the bias level estimation on the left (A) side of the image. |
BIASSECTBn |
Start and end columns to use for the bias level estimation on the right (B) side of the image. |
VXn, VYn |
The coordinates of the bottom-left (VX1, VY1) and top-right (VX2, VY2) corners of the virtual overscan region |
The following line sets chip to 1 if the subarray is on WFC1, and 2 if the subarray is on WFC2.
chip = 1 if amp in ['A', 'B'] else 2
For the BIASSECTAn and BIASSECTBn values, we want to use the six columns of prescan nearest the exposed area of the CCD.
bias_a = [0, 0]
if leading > 0:
bias_a[1] = leading
bias_a[0] = leading-5 if leading > 5 else 0
bias_b = [0, 0]
if trailing > 0:
bias_b[0] = xsize+1
bias_b[1] = xsize+5 if trailing > 5 else xsize+trailing
Now we can define a new row for our settings. For subarray data, as there is no virtual overscan, the VXn and VYn values will always be 0. Here we use a dictionary to explicitly define the new values.
new_row = {'CCDAMP': amp,
'CCDCHIP': chip,
'BINX': 1,
'BINY': 1,
'NX': xsize,
'NY': ysize,
'TRIMX1': leading,
'TRIMX2': trailing,
'TRIMY1': 0,
'TRIMY2': 0,
'BIASSECTA1': bias_a[0],
'BIASSECTA2': bias_a[1],
'BIASSECTB1': bias_b[0],
'BIASSECTB2': bias_b[1],
'DESCRIPTION': 'Custom OSCN',
'VX1': 0,
'VX2': 0,
'VY1': 0,
'VY2': 0}
Now we need to open the custom OSCTNAB file and update the table to have our new definition. We will also need to update the raw FITS file to point to the new custom OSCNTAB reference file.
oscntable.add_row(new_row)
print(oscntable)
CCDAMP CCDCHIP BINX BINY NX ... VX1 VX2 VY1 VY2
------ ------- ---- ---- ---- ... --- --- --- ---
C 2 1 1 4144 ... 0 0 0 0
C 2 1 1 2072 ... 0 0 0 0
C 2 1 1 2072 ... 0 0 0 0
C 2 1 1 2072 ... 0 0 0 0
C 2 1 1 2070 ... 0 0 0 0
C 2 1 1 534 ... 0 0 0 0
D 2 1 1 4144 ... 0 0 0 0
D 2 1 1 2072 ... 0 0 0 0
D 2 1 1 2072 ... 0 0 0 0
D 2 1 1 2072 ... 0 0 0 0
... ... ... ... ... ... ... ... ... ...
B 1 1 1 2072 ... 0 0 0 0
B 1 1 1 2070 ... 0 0 0 0
B 1 1 1 1046 ... 0 0 0 0
B 1 1 1 534 ... 0 0 0 0
AC 1 1 1 4144 ... 0 0 0 0
AD 1 1 1 4144 ... 0 0 0 0
BC 1 1 1 4144 ... 0 0 0 0
BD 1 1 1 4144 ... 0 0 0 0
ABCD 1 1 1 4144 ... 0 0 0 0
A 1 1 1 1180 ... 0 0 0 0
Length = 36 rows
Now we have an idential FITS rec table, but with an additional row for our new information.
oscnrec_new = fits.FITS_rec.from_columns(oscnrec.columns,
nrows=len(oscntable))
print(Table(oscnrec_new))
CCDAMP CCDCHIP BINX BINY NX ... VX1 VX2 VY1 VY2
------ ------- ---- ---- ---- ... --- --- --- ---
C 2 1 1 4144 ... 0 0 0 0
C 2 1 1 2072 ... 0 0 0 0
C 2 1 1 2072 ... 0 0 0 0
C 2 1 1 2072 ... 0 0 0 0
C 2 1 1 2070 ... 0 0 0 0
C 2 1 1 534 ... 0 0 0 0
D 2 1 1 4144 ... 0 0 0 0
D 2 1 1 2072 ... 0 0 0 0
D 2 1 1 2072 ... 0 0 0 0
D 2 1 1 2072 ... 0 0 0 0
... ... ... ... ... ... ... ... ... ...
B 1 1 1 2072 ... 0 0 0 0
B 1 1 1 2070 ... 0 0 0 0
B 1 1 1 1046 ... 0 0 0 0
B 1 1 1 534 ... 0 0 0 0
AC 1 1 1 4144 ... 0 0 0 0
AD 1 1 1 4144 ... 0 0 0 0
BC 1 1 1 4144 ... 0 0 0 0
BD 1 1 1 4144 ... 0 0 0 0
ABCD 1 1 1 4144 ... 0 0 0 0
0 0 0 0 ... 0 0 0 0
Length = 36 rows
Let’s populate that last row with our new data!
oscnrec_new[-1] = tuple(oscntable[-1])
Reprint the table and check that we have entered it correctly.
print(Table(oscnrec_new))
CCDAMP CCDCHIP BINX BINY NX ... VX1 VX2 VY1 VY2
------ ------- ---- ---- ---- ... --- --- --- ---
C 2 1 1 4144 ... 0 0 0 0
C 2 1 1 2072 ... 0 0 0 0
C 2 1 1 2072 ... 0 0 0 0
C 2 1 1 2072 ... 0 0 0 0
C 2 1 1 2070 ... 0 0 0 0
C 2 1 1 534 ... 0 0 0 0
D 2 1 1 4144 ... 0 0 0 0
D 2 1 1 2072 ... 0 0 0 0
D 2 1 1 2072 ... 0 0 0 0
D 2 1 1 2072 ... 0 0 0 0
... ... ... ... ... ... ... ... ... ...
B 1 1 1 2072 ... 0 0 0 0
B 1 1 1 2070 ... 0 0 0 0
B 1 1 1 1046 ... 0 0 0 0
B 1 1 1 534 ... 0 0 0 0
AC 1 1 1 4144 ... 0 0 0 0
AD 1 1 1 4144 ... 0 0 0 0
BC 1 1 1 4144 ... 0 0 0 0
BD 1 1 1 4144 ... 0 0 0 0
ABCD 1 1 1 4144 ... 0 0 0 0
A 1 1 1 1180 ... 0 0 0 0
Length = 36 rows
Now we can open up our oscn table and replace the old fits rec with our new one
with fits.open(path_to_oscn, mode='update') as hdu:
hdu[1].data = oscnrec_new
Using the same check we did earlier, we can use check_oscntab to see if our settings are defined. If everything was done correctly, the following line should print “True”!
supported = utils_calib.check_oscntab(path_to_oscn, amp, xsize, ysize, leading, trailing)
print('Defined in OSCNTAB? {}'.format(supported))
Defined in OSCNTAB? True
For more help:#
More details may be found on the ACS website and in the ACS Instrument and Data Handbooks.
Please visit the HST Help Desk. Through the help desk portal, you can explore the HST Knowledge Base and request additional help from experts.
Top of Page
