REQUIREMENT: Before proceeding, install or update your stenv distribution. stenv is the replacement for AstroConda, which is unsupported as of February 2023.

SBC Dark Analysis#

Introduction#


This notebook has been prepared as a demo on how to perform aperture photometry in SBC images that contain an elevated dark rate. This problem arises when the detector temperature goes above ~25 ºC.

More information on the dark rate can be found in ISR ACS 2017-04 (Avila 2017).

This tutorial will show you how to…#

1. Identify Images with Significant Dark Current#

  • Open files and extract information

  • Organize information in a table

  • Sort table by temperature

2. Combine Science Images#

  • Use AstroDrizzle with ASN files to combine images.

3. Combined Dark Images#

  • Identify which dark images to use for your data.

  • Use AstroDrizzle to combine dark images.

4. Perform Photometry#

  • Subtract dark current from science images using aperture photometry.

Imports#


Here we list the Python packages used in this notebook. Links to the documentation for each module is provided for convenience.

Package Name

module

docs

used for

os

system

link

command line input

os

environ

link

setting environments

shutil

rmtree

link

remove directory tree

glob

glob

link

search for files based on Unix shell rules

matplotlib

pyplot

link

plotting

matplotlib

colors.LogNorm

link

data normalization used for contrast plotting

matplotlib

patches.Rectangle

link

add rectangle patch to plot

numpy

_s

link

construct array slice object

astroquery.mast

Observations

link

download data from MAST

drizzlepac

astrodrizzle

link

drizzle combine images

astropy.io

fits

link

access and update fits files

astropy.table

Table

link

constructing and editing in a tabular format

astropy.wcs

WCS

link

extract WCS information from header

photutils

EllipticalAperture

link

construct aperture object for plotting

photutils

aperture_photometry

link

extract counts from aperture

import os
import shutil
import glob

import matplotlib.pyplot as plt
import numpy as np

from astroquery.mast import Observations
from drizzlepac.astrodrizzle import AstroDrizzle as adriz

from astropy.io import fits
from astropy.table import Table


from matplotlib.colors import LogNorm
from matplotlib.patches import Rectangle

from photutils import EllipticalAperture
from photutils import aperture_photometry

The following task in the stsci.skypac package can be run with TEAL:
                                    skymatch                                    
The following tasks in the drizzlepac package can be run with TEAL:
    astrodrizzle       config_testbed      imagefindpars           mapreg       
       photeq            pixreplace           pixtopix            pixtosky      
  refimagefindpars       resetbits          runastrodriz          skytopix      
     tweakback            tweakreg           updatenpol

Download the Data#


Here we download all of the data required for this notebook. This is an important step! Some of the image processing steps require all relevant files to be in the working directory. We recommend working with a brand new directory for every new set of data.

Using the python package astroquery, we can retreive files from the MAST archive.

GO Proposal 13655: “How Lyman alpha bites/beats the dust”#

First, we will grab the FLT and ASN files from program 13655. For this example, we only want to retreive the files from visit 11 of this program. We will specify program ID ‘JCMC’ along with observation set ID ‘11’.

MAY CHANGE: The argument "mrp_only" stands for "minimum recommended products only". It currently needs to be set to False, although in the future, False is intended to be set as the default and can be left out.
science_list = Observations.query_criteria(proposal_id='13655', obs_id='JCMC11*')

sci_dl_table = Observations.download_products(science_list['obsid'], 
                                              productSubGroupDescription=['ASN', 'FLT'],
                                              mrp_only=False)
INFO: 15 of 184 products were duplicates. Only downloading 169 unique product(s). [astroquery.mast.observations]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/hst_13655_11_acs_sbc_f125lp_jcmc11cw_flt.fits to ./mastDownload/HST/hst_13655_11_acs_sbc_f125lp_jcmc11cw/hst_13655_11_acs_sbc_f125lp_jcmc11cw_flt.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/hst_13655_11_acs_sbc_f125lp_jcmc11dt_flt.fits to ./mastDownload/HST/hst_13655_11_acs_sbc_f125lp_jcmc11dt/hst_13655_11_acs_sbc_f125lp_jcmc11dt_flt.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/hst_13655_11_acs_sbc_f140lp_jcmc11cx_flt.fits to ./mastDownload/HST/hst_13655_11_acs_sbc_f140lp_jcmc11cx/hst_13655_11_acs_sbc_f140lp_jcmc11cx_flt.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/hst_13655_11_acs_sbc_f140lp_jcmc11ds_flt.fits to ./mastDownload/HST/hst_13655_11_acs_sbc_f140lp_jcmc11ds/hst_13655_11_acs_sbc_f140lp_jcmc11ds_flt.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/hst_13655_11_acs_sbc_f150lp_jcmc11de_flt.fits to ./mastDownload/HST/hst_13655_11_acs_sbc_f150lp_jcmc11de/hst_13655_11_acs_sbc_f150lp_jcmc11de_flt.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/hst_13655_11_acs_sbc_f150lp_jcmc11dh_flt.fits to ./mastDownload/HST/hst_13655_11_acs_sbc_f150lp_jcmc11dh/hst_13655_11_acs_sbc_f150lp_jcmc11dh_flt.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/hst_13655_11_acs_sbc_f165lp_jcmc11ct_flt.fits to ./mastDownload/HST/hst_13655_11_acs_sbc_f165lp_jcmc11ct/hst_13655_11_acs_sbc_f165lp_jcmc11ct_flt.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/hst_13655_11_acs_sbc_f165lp_jcmc11e6_flt.fits to ./mastDownload/HST/hst_13655_11_acs_sbc_f165lp_jcmc11e6/hst_13655_11_acs_sbc_f165lp_jcmc11e6_flt.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcmc11010_asn.fits to ./mastDownload/HST/jcmc11010/jcmc11010_asn.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcmc11020_asn.fits to ./mastDownload/HST/jcmc11020/jcmc11020_asn.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcmc11030_asn.fits to ./mastDownload/HST/jcmc11030/jcmc11030_asn.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcmc11040_asn.fits to ./mastDownload/HST/jcmc11040/jcmc11040_asn.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcmc11ctq_flt.fits to ./mastDownload/HST/jcmc11ctq/jcmc11ctq_flt.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcmc11cwq_flt.fits to ./mastDownload/HST/jcmc11cwq/jcmc11cwq_flt.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcmc11cxq_flt.fits to ./mastDownload/HST/jcmc11cxq/jcmc11cxq_flt.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcmc11deq_flt.fits to ./mastDownload/HST/jcmc11deq/jcmc11deq_flt.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcmc11dhq_flt.fits to ./mastDownload/HST/jcmc11dhq/jcmc11dhq_flt.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcmc11dsq_flt.fits to ./mastDownload/HST/jcmc11dsq/jcmc11dsq_flt.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcmc11dtq_flt.fits to ./mastDownload/HST/jcmc11dtq/jcmc11dtq_flt.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcmc11e6q_flt.fits to ./mastDownload/HST/jcmc11e6q/jcmc11e6q_flt.fits ...
 [Done]

GO Proposal 13961: “SBC Dark Current Measurement”#

Now we need a set of dark calibration images. You can use any calibration set as long as the dark rate in the image matches that of your science image (discussed later in this notebook). For convenience, here we download the RAW dark frames from one calibration program: GO Proposal 13961.

darks_list = Observations.query_criteria(proposal_id='13961', obstype='cal')

drk_dl_table = Observations.download_products(darks_list['obsid'],
                                              productSubGroupDescription=['RAW'],
                                              mrp_only=False)
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcrx01iiq_raw.fits to ./mastDownload/HST/jcrx01iiq/jcrx01iiq_raw.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcrx01ijq_raw.fits to ./mastDownload/HST/jcrx01ijq/jcrx01ijq_raw.fits ...
 [Done]

Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcrx01ikq_raw.fits to ./mastDownload/HST/jcrx01ikq/jcrx01ikq_raw.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcrx01ilq_raw.fits to ./mastDownload/HST/jcrx01ilq/jcrx01ilq_raw.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcrx01imq_raw.fits to ./mastDownload/HST/jcrx01imq/jcrx01imq_raw.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcrx01inq_raw.fits to ./mastDownload/HST/jcrx01inq/jcrx01inq_raw.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcrx01ioq_raw.fits to ./mastDownload/HST/jcrx01ioq/jcrx01ioq_raw.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcrx01ipq_raw.fits to ./mastDownload/HST/jcrx01ipq/jcrx01ipq_raw.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcrx01iqq_raw.fits to ./mastDownload/HST/jcrx01iqq/jcrx01iqq_raw.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcrx01irq_raw.fits to ./mastDownload/HST/jcrx01irq/jcrx01irq_raw.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcrx01isq_raw.fits to ./mastDownload/HST/jcrx01isq/jcrx01isq_raw.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcrx01itq_raw.fits to ./mastDownload/HST/jcrx01itq/jcrx01itq_raw.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcrx01iuq_raw.fits to ./mastDownload/HST/jcrx01iuq/jcrx01iuq_raw.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcrx01ivq_raw.fits to ./mastDownload/HST/jcrx01ivq/jcrx01ivq_raw.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcrx01iwq_raw.fits to ./mastDownload/HST/jcrx01iwq/jcrx01iwq_raw.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcrx01ixq_raw.fits to ./mastDownload/HST/jcrx01ixq/jcrx01ixq_raw.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcrx01iyq_raw.fits to ./mastDownload/HST/jcrx01iyq/jcrx01iyq_raw.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcrx01j0q_raw.fits to ./mastDownload/HST/jcrx01j0q/jcrx01j0q_raw.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcrx01j1q_raw.fits to ./mastDownload/HST/jcrx01j1q/jcrx01j1q_raw.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/jcrx01j2q_raw.fits to ./mastDownload/HST/jcrx01j2q/jcrx01j2q_raw.fits ...
 [Done]

We’ll use the packages os and shutil to put all of these files in our working directory for convenience and do a little housekeeping. Now let’s place those images in the same directory as this notebook…

for dl_table in [sci_dl_table, drk_dl_table]:
    
    for row in dl_table:
        oldfname = row['Local Path']
        newfname = os.path.basename(oldfname)
        os.rename(oldfname, newfname)

shutil.rmtree('mastDownload/')

Below we define our filenames with variables for convenience using glob.glob.

asn_list = glob.glob('*_asn.fits')
flt_list = glob.glob('*_flt.fits')
drk_list = glob.glob('*_raw.fits')

File Information#


The structure of the fits files from ACS may be different depending on what kind of observation was made. For more information, refer to Section 2.2 of the ACS Data Handbook.

Association Files#

Association files only contain one extension which lists associated files and their types.

Ext

Name

Type

Contains

0

PRIMARY

(PrimaryHDU)

Meta-data related to the entire file.

1

ASN (Association)

(BinTableHDU)

Table of files associated with this group.

Raw Files#

A standard raw image file from the SBC has the same structure as you’d expect from full frame observation from ACS.

Ext

Name

Type

Contains

0

PRIMARY

(PrimaryHDU)

Meta-data related to the entire file.

1

SCI (Image)

(ImageHDU)

Raw image data.

2

ERR (Error)

(ImageHDU)

Error array.

3

DQ (Data Quality)

(ImageHDU)

Data quality array.

FLT Files#

SBC flat-fielded files have the same structure as the raw files, with additional HDUs for WCS corrections.

Ext

Name

Type

Contains

0

PRIMARY

(PrimaryHDU)

Meta-data related to the entire file.

1

SCI (Image)

(ImageHDU)

Raw image data.

2

ERR (Error)

(ImageHDU)

Error array.

3

DQ (Data Quality)

(ImageHDU)

Data quality array.

4-5

WCSDVARR (WCS)

(ImageHDU)

Filter-dependent non-polynomial distortion corrections.

6

WCSCORR (WCS)

(BinTableHDU)

History of changes to the WCS solution.

You can always use .info() on an HDUlist for an overview of the structure.

with fits.open(drk_list[0]) as hdulist:
    hdulist.info()
Filename: jcrx01itq_raw.fits
No.    Name      Ver    Type      Cards   Dimensions   Format
  0  PRIMARY       1 PrimaryHDU     173   ()      
  1  SCI           1 ImageHDU        85   (1024, 1024)   int16   
  2  ERR           1 ImageHDU        49   ()      
  3  DQ            1 ImageHDU        43   ()      

Identify Affected Observations #


Let’s take a look at some information from our science images. We want to find observations with an average temperature greater than 25\(^o\)C. We can organize the information in a Table object from astropy.table for convenience. Here, we define a table with column names and respective data types.

Note: The FITS header keywords mdecodt1 and mdecodt2 refer to the temperature at the beginning and end of the exposure respectively.

flt_table = Table(names=('file', 'start', 'filter1', 'mdecodt1', 'mdecodt2', 'avgtemp'),
                  dtype=('S64', 'S19', 'S6', 'f8', 'f8', 'f8'))

Now we need to obtain information from the headers. The temperatures are stored in the science extensions, while observation information is found in the primary header.

Pro-Tip: Adding rows to a table is a slow way to construct a table. For larger sets of data, consider constructing a table of known size.
for file in flt_list:
    filt = fits.getval(file, 'FILTER1', ext=0)
    date = fits.getval(file, 'DATE-OBS', ext=0)
    time = fits.getval(file, 'TIME-OBS', ext=0)
    
    t1 = fits.getval(file, 'MDECODT1', ext=1)
    t2 = fits.getval(file, 'MDECODT2', ext=1)

    starttime = date + 'T' + time
    avgtemp = (t1+t2) / 2
        
    flt_table.add_row((file, starttime, filt, t1, t2, avgtemp))

print(flt_table)
                     file                     ...      avgtemp      
--------------------------------------------- ... ------------------
                           jcmc11ctq_flt.fits ...            17.3262
                           jcmc11cxq_flt.fits ...             19.524
hst_13655_11_acs_sbc_f165lp_jcmc11e6_flt.fits ...            25.9953
hst_13655_11_acs_sbc_f165lp_jcmc11ct_flt.fits ...            17.3262
                           jcmc11cwq_flt.fits ...            18.6693
                           jcmc11e6q_flt.fits ...            25.9953
hst_13655_11_acs_sbc_f125lp_jcmc11cw_flt.fits ...            18.6693
hst_13655_11_acs_sbc_f140lp_jcmc11cx_flt.fits ...             19.524
                           jcmc11dhq_flt.fits ...            23.3091
hst_13655_11_acs_sbc_f125lp_jcmc11dt_flt.fits ... 24.163800000000002
hst_13655_11_acs_sbc_f150lp_jcmc11dh_flt.fits ...            23.3091
                           jcmc11dsq_flt.fits ...            23.7975
                           jcmc11deq_flt.fits ... 22.820700000000002
hst_13655_11_acs_sbc_f150lp_jcmc11de_flt.fits ... 22.820700000000002
hst_13655_11_acs_sbc_f140lp_jcmc11ds_flt.fits ...            23.7975
                           jcmc11dtq_flt.fits ... 24.163800000000002

We can sort the table by column value for analysis. Since we are interested in temperature, we’ll sort this table by the column ‘avgtemp’

flt_table.sort('avgtemp')
print(flt_table)
                     file                     ...      avgtemp      
--------------------------------------------- ... ------------------
                           jcmc11ctq_flt.fits ...            17.3262
hst_13655_11_acs_sbc_f165lp_jcmc11ct_flt.fits ...            17.3262
                           jcmc11cwq_flt.fits ...            18.6693
hst_13655_11_acs_sbc_f125lp_jcmc11cw_flt.fits ...            18.6693
                           jcmc11cxq_flt.fits ...             19.524
hst_13655_11_acs_sbc_f140lp_jcmc11cx_flt.fits ...             19.524
                           jcmc11deq_flt.fits ... 22.820700000000002
hst_13655_11_acs_sbc_f150lp_jcmc11de_flt.fits ... 22.820700000000002
                           jcmc11dhq_flt.fits ...            23.3091
hst_13655_11_acs_sbc_f150lp_jcmc11dh_flt.fits ...            23.3091
                           jcmc11dsq_flt.fits ...            23.7975
hst_13655_11_acs_sbc_f140lp_jcmc11ds_flt.fits ...            23.7975
hst_13655_11_acs_sbc_f125lp_jcmc11dt_flt.fits ... 24.163800000000002
                           jcmc11dtq_flt.fits ... 24.163800000000002
hst_13655_11_acs_sbc_f165lp_jcmc11e6_flt.fits ...            25.9953
                           jcmc11e6q_flt.fits ...            25.9953

Sorting the table by average temperature gives us a sense of how temperature of the SBC behaves over time. Only the last image was affected by a temperature of over 25\(^o\)C, and therefore the only image to be affected by elevated dark current.

The table shows us that this image was taken with the filter F165LP- which is also same filter that the first image was taken with. This is not a coincidence! Take a moment to consider the time-based symmetry of the images, and what that means for the dark current of the combined images.

Combine science images#


Let’s make drizzled products for each filter. We do this by using the ASN files for each filter. The ASN files will tell AstroDrizzle which flat images to combine for a given filter. Steps 1-6 of the drizzling procedure have been turned off since their purpose is to identify and mask cosmic rays, which do not affect SBC images.

The drizzle keyword parameters below are the appropriate ones for SBC data. For “final_scale” we use the pixel scale of SBC, 0.025 arcseconds.

driz_kwargs = {'runfile': '',
               'context': False,
               'build': False,
               'preserve': False,
               'clean': True,
               'static': False,
               'skysub': False,
               'driz_separate': False,
               'median': False,
               'blot': False,
               'driz_cr': False,
               'driz_combine': True,
               'final_wcs': True,
               'final_scale': 0.025}

Now we’ll run AstroDrizzle on our list of association files.

for file in asn_list:
    output_name = fits.getheader(file)['asn_id']
    adriz(file, output=output_name, **driz_kwargs)
No trailer file created...
AstroDrizzle log file: jcmc11030.tra
AstroDrizzle Version 3.7.1 started at: 12:44:59.111 (18/09/2024)

==== Processing Step  Initialization  started at  12:44:59.11 (18/09/2024)
WARNING: No cte correction will be made for this SBC data.
WARNING: No cte correction will be made for this SBC data.
WCS Keywords

Number of WCS axes: 2
CTYPE : 'RA---TAN' 'DEC--TAN' 
CRVAL : 165.55871142745008 38.043205302369955 
CRPIX : 707.5000105276703 686.0000102072953 
CD1_1 CD1_2  : -5.080790365666934e-06 -4.73401287515374e-06 
CD2_1 CD2_2  : -4.73401287515374e-06 5.080790365666934e-06 
NAXIS : 1416  1373
********************************************************************************
*
*  Estimated memory usage:  up to 30 Mb.
*  Output image size:       1416 X 1373 pixels. 
*  Output image file:       ~ 22 Mb. 
*  Cores available:         1
*
********************************************************************************
==== Processing Step Initialization finished at 12:44:59.34 (18/09/2024)
==== Processing Step  Static Mask  started at  12:44:59.341 (18/09/2024)
==== Processing Step Static Mask finished at 12:44:59.342 (18/09/2024)
==== Processing Step  Subtract Sky  started at  12:44:59.342 (18/09/2024)
==== Processing Step Subtract Sky finished at 12:44:59.392 (18/09/2024)
==== Processing Step  Separate Drizzle  started at  12:44:59.392 (18/09/2024)
==== Processing Step Separate Drizzle finished at 12:44:59.39 (18/09/2024)
==== Processing Step  Create Median  started at  12:44:59.393 (18/09/2024)
==== Processing Step  Blot  started at  12:44:59.394 (18/09/2024)
==== Processing Step Blot finished at 12:44:59.395 (18/09/2024)
==== Processing Step  Driz_CR  started at  12:44:59.395 (18/09/2024)
==== Processing Step  Final Drizzle  started at  12:44:59.39 (18/09/2024)
WCS Keywords

Number of WCS axes: 2
CTYPE : 'RA---TAN' 'DEC--TAN' 
CRVAL : 165.55871142745008 38.043205302369955 
CRPIX : 707.5000105276703 686.0000102072953 
CD1_1 CD1_2  : -5.080790365666934e-06 -4.73401287515374e-06 
CD2_1 CD2_2  : -4.73401287515374e-06 5.080790365666934e-06 
NAXIS : 1416  1373
-Generating simple FITS output: jcmc11030_drz_sci.fits
Deleted all instances of WCS with key A in extensions [0]
Writing out image to disk: jcmc11030_drz_sci.fits
Writing out image to disk: jcmc11030_drz_wht.fits
==== Processing Step Final Drizzle finished at 12:45:00.554 (18/09/2024)
AstroDrizzle Version 3.7.1 is finished processing at 12:45:00.55 (18/09/2024).
   --------------------          --------------------
                   Step          Elapsed time
   --------------------          --------------------

         Initialization          0.2289 sec.
            Static Mask          0.0005 sec.
           Subtract Sky          0.0495 sec.
       Separate Drizzle          0.0005 sec.
          Create Median          0.0000 sec.
                   Blot          0.0011 sec.
                Driz_CR          0.0000 sec.
          Final Drizzle          1.1580 sec.
   ====================          ====================
                  Total          1.4386 sec.
No trailer file saved...
No trailer file created...
AstroDrizzle log file: jcmc11020.tra
AstroDrizzle Version 3.7.1 started at: 12:45:00.582 (18/09/2024)

==== Processing Step  Initialization  started at  12:45:00.58 (18/09/2024)
WARNING: No cte correction will be made for this SBC data.
WARNING: No cte correction will be made for this SBC data.
WCS Keywords

Number of WCS axes: 2
CTYPE : 'RA---TAN' 'DEC--TAN' 
CRVAL : 165.55871129084667 38.04320474124267 
CRPIX : 707.5000105276703 686.0000102072953 
CD1_1 CD1_2  : -5.080790365666934e-06 -4.73401287515374e-06 
CD2_1 CD2_2  : -4.73401287515374e-06 5.080790365666934e-06 
NAXIS : 1416  1373
********************************************************************************
*
*  Estimated memory usage:  up to 30 Mb.
*  Output image size:       1416 X 1373 pixels. 
*  Output image file:       ~ 22 Mb. 
*  Cores available:         1
*
********************************************************************************
==== Processing Step Initialization finished at 12:45:00.812 (18/09/2024)
==== Processing Step  Static Mask  started at  12:45:00.8 (18/09/2024)
==== Processing Step Static Mask finished at 12:45:00.814 (18/09/2024)
==== Processing Step  Subtract Sky  started at  12:45:00.814 (18/09/2024)
==== Processing Step Subtract Sky finished at 12:45:00.864 (18/09/2024)
==== Processing Step  Separate Drizzle  started at  12:45:00.86 (18/09/2024)
==== Processing Step Separate Drizzle finished at 12:45:00.865 (18/09/2024)
==== Processing Step  Create Median  started at  12:45:00.866 (18/09/2024)
==== Processing Step  Blot  started at  12:45:00.866 (18/09/2024)
==== Processing Step Blot finished at 12:45:00.867 (18/09/2024)
==== Processing Step  Driz_CR  started at  12:45:00.867 (18/09/2024)
==== Processing Step  Final Drizzle  started at  12:45:00.868 (18/09/2024)
WCS Keywords

Number of WCS axes: 2
CTYPE : 'RA---TAN' 'DEC--TAN' 
CRVAL : 165.55871129084667 38.04320474124267 
CRPIX : 707.5000105276703 686.0000102072953 
CD1_1 CD1_2  : -5.080790365666934e-06 -4.73401287515374e-06 
CD2_1 CD2_2  : -4.73401287515374e-06 5.080790365666934e-06 
NAXIS : 1416  1373
-Generating simple FITS output: jcmc11020_drz_sci.fits
Deleted all instances of WCS with key A in extensions [0]
Writing out image to disk: jcmc11020_drz_sci.fits
Writing out image to disk: jcmc11020_drz_wht.fits
==== Processing Step Final Drizzle finished at 12:45:01.897 (18/09/2024)
AstroDrizzle Version 3.7.1 is finished processing at 12:45:01.897 (18/09/2024).
   --------------------          --------------------
                   Step          Elapsed time
   --------------------          --------------------

         Initialization          0.2306 sec.
            Static Mask          0.0005 sec.
           Subtract Sky          0.0499 sec.
       Separate Drizzle          0.0006 sec.
          Create Median          0.0000 sec.
                   Blot          0.0009 sec.
                Driz_CR          0.0000 sec.
          Final Drizzle          1.0287 sec.
   ====================          ====================
                  Total          1.3112 sec.
No trailer file saved...
No trailer file created...
AstroDrizzle log file: jcmc11010.tra
AstroDrizzle Version 3.7.1 started at: 12:45:01.92 (18/09/2024)

==== Processing Step  Initialization  started at  12:45:01.926 (18/09/2024)
WARNING: No cte correction will be made for this SBC data.
WARNING: No cte correction will be made for this SBC data.
WCS Keywords

Number of WCS axes: 2
CTYPE : 'RA---TAN' 'DEC--TAN' 
CRVAL : 165.5587124859804 38.04320516616883 
CRPIX : 707.5000105276704 686.0000102072954 
CD1_1 CD1_2  : -5.080803516492815e-06 -4.733998760959866e-06 
CD2_1 CD2_2  : -4.733998760959866e-06 5.080803516492815e-06 
NAXIS : 1416  1373
********************************************************************************
*
*  Estimated memory usage:  up to 30 Mb.
*  Output image size:       1416 X 1373 pixels. 
*  Output image file:       ~ 22 Mb. 
*  Cores available:         1
*
********************************************************************************
==== Processing Step Initialization finished at 12:45:02.161 (18/09/2024)
==== Processing Step  Static Mask  started at  12:45:02.162 (18/09/2024)
==== Processing Step Static Mask finished at 12:45:02.162 (18/09/2024)
==== Processing Step  Subtract Sky  started at  12:45:02.163 (18/09/2024)
==== Processing Step Subtract Sky finished at 12:45:02.215 (18/09/2024)
==== Processing Step  Separate Drizzle  started at  12:45:02.216 (18/09/2024)
==== Processing Step Separate Drizzle finished at 12:45:02.216 (18/09/2024)
==== Processing Step  Create Median  started at  12:45:02.216 (18/09/2024)
==== Processing Step  Blot  started at  12:45:02.217 (18/09/2024)
==== Processing Step Blot finished at 12:45:02.218 (18/09/2024)
==== Processing Step  Driz_CR  started at  12:45:02.218 (18/09/2024)
==== Processing Step  Final Drizzle  started at  12:45:02.219 (18/09/2024)
WCS Keywords

Number of WCS axes: 2
CTYPE : 'RA---TAN' 'DEC--TAN' 
CRVAL : 165.5587124859804 38.04320516616883 
CRPIX : 707.5000105276704 686.0000102072954 
CD1_1 CD1_2  : -5.080803516492815e-06 -4.733998760959866e-06 
CD2_1 CD2_2  : -4.733998760959866e-06 5.080803516492815e-06 
NAXIS : 1416  1373
-Generating simple FITS output: jcmc11010_drz_sci.fits
Deleted all instances of WCS with key A in extensions [0]
Writing out image to disk: jcmc11010_drz_sci.fits
Writing out image to disk: jcmc11010_drz_wht.fits
==== Processing Step Final Drizzle finished at 12:45:03.382 (18/09/2024)
AstroDrizzle Version 3.7.1 is finished processing at 12:45:03.382 (18/09/2024).
   --------------------          --------------------
                   Step          Elapsed time
   --------------------          --------------------

         Initialization          0.2349 sec.
            Static Mask          0.0005 sec.
           Subtract Sky          0.0514 sec.
       Separate Drizzle          0.0005 sec.
          Create Median          0.0000 sec.
                   Blot          0.0010 sec.
                Driz_CR          0.0000 sec.
          Final Drizzle          1.1631 sec.
   ====================          ====================
                  Total          1.4515 sec.
No trailer file saved...
No trailer file created...
AstroDrizzle log file: jcmc11040.tra
AstroDrizzle Version 3.7.1 started at: 12:45:03.411 (18/09/2024)

==== Processing Step  Initialization  started at  12:45:03.412 (18/09/2024)
WARNING: No cte correction will be made for this SBC data.
WARNING: No cte correction will be made for this SBC data.
WCS Keywords

Number of WCS axes: 2
CTYPE : 'RA---TAN' 'DEC--TAN' 
CRVAL : 165.55870940865276 38.04320257724112 
CRPIX : 708.000010535121 686.0000102072954 
CD1_1 CD1_2  : -5.080803516492815e-06 -4.733998760959866e-06 
CD2_1 CD2_2  : -4.733998760959866e-06 5.080803516492815e-06 
NAXIS : 1417  1373
********************************************************************************
*
*  Estimated memory usage:  up to 30 Mb.
*  Output image size:       1417 X 1373 pixels. 
*  Output image file:       ~ 22 Mb. 
*  Cores available:         1
*
********************************************************************************
==== Processing Step Initialization finished at 12:45:03.643 (18/09/2024)
==== Processing Step  Static Mask  started at  12:45:03.643 (18/09/2024)
==== Processing Step Static Mask finished at 12:45:03.644 (18/09/2024)
==== Processing Step  Subtract Sky  started at  12:45:03.645 (18/09/2024)
==== Processing Step Subtract Sky finished at 12:45:03.695 (18/09/2024)
==== Processing Step  Separate Drizzle  started at  12:45:03.696 (18/09/2024)
==== Processing Step Separate Drizzle finished at 12:45:03.697 (18/09/2024)
==== Processing Step  Create Median  started at  12:45:03.6 (18/09/2024)
==== Processing Step  Blot  started at  12:45:03.698 (18/09/2024)
==== Processing Step Blot finished at 12:45:03.698 (18/09/2024)
==== Processing Step  Driz_CR  started at  12:45:03.69 (18/09/2024)
==== Processing Step  Final Drizzle  started at  12:45:03.699 (18/09/2024)
WCS Keywords

Number of WCS axes: 2
CTYPE : 'RA---TAN' 'DEC--TAN' 
CRVAL : 165.55870940865276 38.04320257724112 
CRPIX : 708.000010535121 686.0000102072954 
CD1_1 CD1_2  : -5.080803516492815e-06 -4.733998760959866e-06 
CD2_1 CD2_2  : -4.733998760959866e-06 5.080803516492815e-06 
NAXIS : 1417  1373
-Generating simple FITS output: jcmc11040_drz_sci.fits
Deleted all instances of WCS with key A in extensions [0]
Writing out image to disk: jcmc11040_drz_sci.fits
Writing out image to disk: jcmc11040_drz_wht.fits
==== Processing Step Final Drizzle finished at 12:45:04.750 (18/09/2024)
AstroDrizzle Version 3.7.1 is finished processing at 12:45:04.750 (18/09/2024).
   --------------------          --------------------
                   Step          Elapsed time
   --------------------          --------------------

         Initialization          0.2313 sec.
            Static Mask          0.0004 sec.
           Subtract Sky          0.0507 sec.
       Separate Drizzle          0.0005 sec.
          Create Median          0.0000 sec.
                   Blot          0.0005 sec.
                Driz_CR          0.0000 sec.
          Final Drizzle          1.0505 sec.
   ====================          ====================
                  Total          1.3340 sec.
No trailer file saved...

Create dark images #


We want to use dark frames to make a drizzled product that will be used to approximate the dark rate to be subtracted from the science product. The dark rate above 25C varies. We need to find the dark frame that contains a dark rate similar to your affected image.

Below we open the two F165LP science frames, one being a high temperature image and the other being a lower temperature image with negligible dark current.

To measure the dark rate, we will take the sum of the pixels within a 200x200 box. The box will be placed off-center of our SBC image where dark rate is low and consistent, at (y, x) = (750, 680). We will measure this sum for our science images as well as all of the dark frames.

With the array handling package numpy, we can define a 2D array slice to use for later.

cutter = np.s_[650:850, 580:780]

Now we can print out the sum of the pixels in each image cut out.

sci_list = ['jcmc11ctq_flt.fits',
            'jcmc11e6q_flt.fits']

print('Box Sums for Science Data:\n')

for file in sci_list:
    image = fits.getdata(file)
    img_slice = image[cutter]
    neatname = os.path.basename(file)
    print('{} --> {}'.format(neatname, np.sum(img_slice)))

print('\n----------------\n')
print('Box Sums for Dark Frames:\n')

for file in drk_list:
    image = fits.getdata(file)
    img_slice = image[cutter]
    neatname = os.path.basename(file)
    print('{} --> {}'.format(neatname, np.sum(img_slice)))
Box Sums for Science Data:

jcmc11ctq_flt.fits --> 428.6138000488281
jcmc11e6q_flt.fits --> 2379.35107421875

----------------

Box Sums for Dark Frames:

jcrx01itq_raw.fits --> 982
jcrx01ikq_raw.fits --> 302
jcrx01irq_raw.fits --> 644
jcrx01isq_raw.fits --> 777
jcrx01ioq_raw.fits --> 351
jcrx01ivq_raw.fits --> 1532
jcrx01j1q_raw.fits --> 3141
jcrx01imq_raw.fits --> 299
jcrx01j0q_raw.fits --> 2922
jcrx01iuq_raw.fits --> 1232
jcrx01j2q_raw.fits --> 3566
jcrx01iyq_raw.fits --> 2445
jcrx01ixq_raw.fits --> 2196
jcrx01ilq_raw.fits --> 348
jcrx01iwq_raw.fits --> 1904
jcrx01iqq_raw.fits --> 492
jcrx01ipq_raw.fits --> 347
jcrx01ijq_raw.fits --> 294
jcrx01iiq_raw.fits --> 300
jcrx01inq_raw.fits --> 324

It looks like the two dark frames that come closest to the science frames are jcrx01iqq and jcrx01iyq. We will use those to make a combined master dark frame. Note that for programs with more exposures, you will need to do this for each input image in the combined mosaic.

For better visualization, let’s take a look at one of our science images and matching dark frame. We will also highlight the dark rate extraction box in each plot.

plt_kwargs = {'norm': LogNorm(),
              'interpolation': 'nearest',
              'cmap': 'plasma',
              'origin': 'lower'}

fig, ax = plt.subplots(1, 2, figsize=(16, 12))

science = fits.getdata('jcmc11ctq_flt.fits')
dark = fits.getdata('jcrx01iqq_raw.fits')

ax[0].set_title('Science Data')
ax[1].set_title('Dark Frame')

ax[0].imshow(science, **plt_kwargs)
ax[1].imshow(dark, **plt_kwargs)

# Must define twice since artist objects can only be used once.
patch0 = Rectangle((680, 750), width=200, height=200, alpha=0.5)
patch1 = Rectangle((680, 750), width=200, height=200, alpha=0.5)

ax[0].add_patch(patch0)
ax[1].add_patch(patch1)
<matplotlib.patches.Rectangle at 0x7f829417ac90>
../../../_images/3d65de5ff796559473461463708bbf51c8a60e4b0d036b4550a3996299a52cca.png

To preserve important information in the header specific to the science image, such as the WCS solution, we will insert the data of the dark images into copies of the science images. We also must remember to adjust the exposure time of the copies of the science frames to that of the dark frames so that the drizzled products have the correct count rates.

flt_file = 'jcmc11ctq_flt.fits'
drk_file = 'jcrx01iiq_raw.fits'
new_file = 'dark1.fits'

shutil.copy(flt_file, new_file)
darkdat = fits.getdata(drk_file)
exptime = fits.getval(drk_file, 'exptime', ext=0)

with fits.open(new_file, mode='update') as hdu:
    
    hdu[1].data[:, :] = darkdat
    hdu[0].header['exptime'] = exptime
flt_file = 'jcmc11e6q_flt.fits'
drk_file = 'jcrx01iyq_raw.fits'
new_file = 'dark2.fits'

shutil.copy(flt_file, new_file)
darkdat = fits.getdata(drk_file)
exptime = fits.getval(drk_file, 'exptime', ext=0)

with fits.open(new_file, mode='update') as hdu:
    
    hdu[1].data[:, :] = darkdat
    hdu[0].header['exptime'] = exptime

We can now make the drizzled dark frame using the two individual dark frames we just made as inputs. The drizzle parameters are the same as the ones used to make the science drizzled products.

adriz_output = adriz(['dark1.fits', 'dark2.fits'], output='masterdark', **driz_kwargs)
No trailer file created...
AstroDrizzle log file: final.tra
AstroDrizzle Version 3.7.1 started at: 12:45:05.42 (18/09/2024)

==== Processing Step  Initialization  started at  12:45:05.427 (18/09/2024)
WARNING: No cte correction will be made for this SBC data.
WARNING: No cte correction will be made for this SBC data.
WCS Keywords

Number of WCS axes: 2
CTYPE : 'RA---TAN' 'DEC--TAN' 
CRVAL : 165.5587124859804 38.04320516616883 
CRPIX : 707.5000105276704 686.0000102072954 
CD1_1 CD1_2  : -5.080803516492815e-06 -4.733998760959866e-06 
CD2_1 CD2_2  : -4.733998760959866e-06 5.080803516492815e-06 
NAXIS : 1416  1373
********************************************************************************
*
*  Estimated memory usage:  up to 30 Mb.
*  Output image size:       1416 X 1373 pixels. 
*  Output image file:       ~ 22 Mb. 
*  Cores available:         1
*
********************************************************************************
==== Processing Step Initialization finished at 12:45:05.653 (18/09/2024)
==== Processing Step  Static Mask  started at  12:45:05.654 (18/09/2024)
==== Processing Step Static Mask finished at 12:45:05.654 (18/09/2024)
==== Processing Step  Subtract Sky  started at  12:45:05.655 (18/09/2024)
==== Processing Step Subtract Sky finished at 12:45:05.705 (18/09/2024)
==== Processing Step  Separate Drizzle  started at  12:45:05.70 (18/09/2024)
==== Processing Step Separate Drizzle finished at 12:45:05.706 (18/09/2024)
==== Processing Step  Create Median  started at  12:45:05.706 (18/09/2024)
==== Processing Step  Blot  started at  12:45:05.707 (18/09/2024)
==== Processing Step Blot finished at 12:45:05.707 (18/09/2024)
==== Processing Step  Driz_CR  started at  12:45:05.708 (18/09/2024)
==== Processing Step  Final Drizzle  started at  12:45:05.708 (18/09/2024)
WCS Keywords

Number of WCS axes: 2
CTYPE : 'RA---TAN' 'DEC--TAN' 
CRVAL : 165.5587124859804 38.04320516616883 
CRPIX : 707.5000105276704 686.0000102072954 
CD1_1 CD1_2  : -5.080803516492815e-06 -4.733998760959866e-06 
CD2_1 CD2_2  : -4.733998760959866e-06 5.080803516492815e-06 
NAXIS : 1416  1373
-Generating simple FITS output: masterdark_drz_sci.fits
Deleted all instances of WCS with key A in extensions [0]
Writing out image to disk: masterdark_drz_sci.fits
Writing out image to disk: masterdark_drz_wht.fits
==== Processing Step Final Drizzle finished at 12:45:06.728 (18/09/2024)
AstroDrizzle Version 3.7.1 is finished processing at 12:45:06.729 (18/09/2024).
   --------------------          --------------------
                   Step          Elapsed time
   --------------------          --------------------

         Initialization          0.2268 sec.
            Static Mask          0.0004 sec.
           Subtract Sky          0.0502 sec.
       Separate Drizzle          0.0004 sec.
          Create Median          0.0000 sec.
                   Blot          0.0003 sec.
                Driz_CR          0.0000 sec.
          Final Drizzle          1.0201 sec.
   ====================          ====================
                  Total          1.2982 sec.
No trailer file saved...

We will now display the images to confirm that they show similar elevated dark rates. You might want do display them in DS9 (or any other viewer) outside of this notebook so you can play with the stretch a bit and so you can see bigger versions of the images.

# Some plotting parameters
plt_kwargs = {'norm': LogNorm(vmin=1e-5, vmax=0.01),
              'interpolation': 'nearest',
              'cmap': 'plasma',
              'origin': 'lower'}

f165lp = fits.getdata('jcmc11010_drz_sci.fits')
masterdark = fits.getdata('masterdark_drz_sci.fits')

fig, ax = plt.subplots(1, 2, figsize=(16, 12))

ax[0].set_title('Drizzled Science Data')
ax[1].set_title('Drizzled Dark Frame')

ax[0].imshow(f165lp, **plt_kwargs)
ax[1].imshow(masterdark, **plt_kwargs)
<matplotlib.image.AxesImage at 0x7f8295b02d90>
../../../_images/942adae145c71cb9cf4c2d07c103caec914ba1cc45791d77fa5cbb8e375217e7.png

The images look comparable. We will now proceed to performing some photometric analysis to estimate the dark current in the source.

Photometry#


Now we will use the photutils package to set up the two apertures. We will use these apertures to measure the flux of different regions in the images.

The first aperture is centered on our target at (735, 710), and is shaped as an elliptical to encompass all of the flux from the source. The other aperture will be the same exact shape, but located near the edge of the detector at (200, 200).

aper = EllipticalAperture([(735, 710), (200, 200)], a=70, b=40, theta=0.5*np.pi)

Let’s overplot the two apertures in the images so you can see their locations.

fig, ax = plt.subplots(1, 2, figsize=(16, 12))

ax[0].set_title('Drizzled Science Data')
ax[1].set_title('Drizzled Dark Frames')

ax[0].imshow(f165lp, **plt_kwargs)
ax[1].imshow(masterdark, **plt_kwargs)

aper.plot(ax[0])
aper.plot(ax[1])
[<matplotlib.patches.Ellipse at 0x7f8293ff34d0>,
 <matplotlib.patches.Ellipse at 0x7f8293fc6190>]
../../../_images/d0a295320f50a9098d407c5035147081ba7f348a859797a3faa7ce200e4ccd09.png

Finally, we do the photometry using the two apertures on both images. We print out the tables to see the results.

f165lp_phot = aperture_photometry(f165lp, aper)
masterdark_phot = aperture_photometry(masterdark, aper)

sumdiff = f165lp_phot['aperture_sum'] - masterdark_phot['aperture_sum']

print('Science data photometry:\n')
print(f165lp_phot)
print('\n')

print('Dark frame photometry:\n')
print(masterdark_phot)
print('\n')

print('\nDifference of aperture sums (science - dark):\n')
print(sumdiff)
Science data photometry:

 id xcenter ycenter     aperture_sum   
      pix     pix                      
--- ------- ------- -------------------
  1   735.0   710.0   2.885625333666923
  2   200.0   200.0 0.09033901192133124


Dark frame photometry:

 id xcenter ycenter     aperture_sum   
      pix     pix                      
--- ------- ------- -------------------
  1   735.0   710.0  0.3220912920189489
  2   200.0   200.0 0.06844985468540221



Difference of aperture sums (science - dark):

    aperture_sum    
--------------------
   2.563534041647974
0.021889157235929033

The target aperture has 2.89 cts/sec, while the same aperture in the dark frame has 0.322 cts/sec. That means that ~11% of the flux in your source comes from dark current and should be subtracted out, leaving a flux for you source of 2.564 cts/sec.

Final Thoughts#


  1. The difference in flux in the second aperture (the one in the lower left portion of the image) shows that there is a small residual background of ~0.02 cts/sec in the science frame. This could be real background from the sky (and not dark current from the detector that you might want to account for properly in your flux and error budget.

  2. The dark frame we created does not have the exact same dark count rate as we measured in the science frame. You could try searching for other darks that more closely resemble your science frame.

  3. These problems can be avoided using a few strategies detailed in ISR ACS 2018-07 (Avila 2018).

For more help:#

More details may be found on the ACS website and in the ACS Instrument and Data Handbooks.

Please visit the HST Help Desk. Through the help desk portal, you can explore the HST Knowledge Base and request additional help from experts.


Top of Page Space Telescope Logo