Aligning HST images to an Absolute Reference Catalog#

This notebook requires creating and activating a virtual environment using the requirements file in this notebook's repository. Please also review the README file before using the notebook.

Table of Contents#

Introduction
Import Packages

1. Download the Observations with astroquery
    1.1 Check image header data
    1.2 Inspect the alignment

2. Query the Reference Catalogs
    2.1 Identify Coordinates
    2.2 SDSS Query
    2.3 Gaia Query

3. Align images with TweakReg
    3.1 SDSS Alignment
    3.2 Inspect the shift file and fit residuals
    3.3 Gaia Alignment
    3.4 Inspect the shift file and fit residuals
    3.5 Overplot Matched Sources on the image
    3.6 Update header with optimal parameters

4. Combine the images using AstroDrizzle

Conclusions
About this Notebook

Introduction#

The alignment of HST exposures is a critical step in image stacking or combination performed with software such as AstroDrizzle. Generally, a relative alignment is performed to align one or multiple images to another image that is designated as the reference image. The reference image is generally the deepest exposure and/or that covering the largest area of all the exposures. This process aligns the images to each other, but the pointing error of the observatory can still cause the images to have incorrect absolute astrometry. When absolute astrometry is desired, the images can be aligned to an external catalog with an absolute world coordinate system (WCS). In this example, we will provide a workflow to query catalogs such as SDSS and Gaia using the astroquery package, and then align the images to that catalog using TweakReg.

The workflow in this notebook for aligning images to Gaia is based on WFC3 ISR 2017-19: Aligning HST Images to Gaia: a Faster Mosaicking Workflow and contains a subset of the information and code found in this repository. For more information, see the notebook in that repository titled Gaia_alignment.ipynb.

For more information about TweakReg, see the other notebooks in this repository or the TweakReg Documentation.

For more information on Astroquery, see the other notebooks in this repository or the Astroquery Documentation.

Import Packages#

Table of Contents


The following Python packages are required to run the Jupyter Notebook:

  • os - change and make directories

  • glob - gather lists of filenames

  • shutil - remove directories and files

  • numpy - math and array functions

  • matplotlib - make figures and graphics

  • astropy - file handling, tables, units, WCS, statistics

  • astroquery - download data and query databases

  • drizzlepac - align and combine HST images

import os
import glob
import shutil
import numpy as np
import matplotlib.pyplot as plt
from collections import defaultdict
from IPython.display import Image, clear_output
import astropy.units as u
from astropy.io import ascii, fits
from astropy.table import Table
from astropy.units import Quantity
from astropy.coordinates import SkyCoord
from astropy.visualization import ZScaleInterval
from astroquery.mast import Observations
from astroquery.sdss import SDSS
from astroquery.gaia import Gaia
from drizzlepac import tweakreg, astrodrizzle
from drizzlepac.processInput import getMdriztabPars

%config InlineBackend.figure_format = 'retina' # Improves the resolution of figures rendered in notebooks.
Gaia.MAIN_GAIA_TABLE = 'gaiadr3.gaia_source' # Change this to the desired Gaia data release.
Gaia.ROW_LIMIT = 100000 # Show all of the matched sources in the printed tables.

# Set the locations of reference files and retrieve the MDRIZTAB recommended drizzle parameters.
os.environ['CRDS_SERVER_URL'] = 'https://hst-crds.stsci.edu'
os.environ['CRDS_SERVER'] = 'https://hst-crds.stsci.edu'
os.environ['CRDS_PATH'] = './crds_cache'
os.environ['iref'] = './crds_cache/references/hst/wfc3/'

The following task in the stsci.skypac package can be run with TEAL:
                                    skymatch                                    
The following tasks in the drizzlepac package can be run with TEAL:
    astrodrizzle       config_testbed      imagefindpars           mapreg       
       photeq            pixreplace           pixtopix            pixtosky      
  refimagefindpars       resetbits          runastrodriz          skytopix      
     tweakback            tweakreg           updatenpol

1. Download the Observations with astroquery#

Table of Contents

For this notebook, we will download WFC3/UVIS images of NGC 6791 from program 12692 in the F606W filter acquired in Visit 01. The three FLC images have been processed by the HST WFC3 pipeline (calwf3), which includes bias subtraction, dark current and CTE correction, cosmic-ray rejection, and flat-fielding:

        ibwb01xqq_flc.fits
        ibwb01xrq_flc.fits
        ibwb01xxq_flc.fits

MAST queries may be done using query_criteria, where we specify:

    \(\rightarrow\) obs_id, proposal_id, and filters

MAST data products may be downloaded by using download_products, where we specify:

    \(\rightarrow\) products = calibrated (FLT, FLC) or drizzled (DRZ, DRC) files

    \(\rightarrow\) type = standard products (CALxxx) or advanced products (HAP-SVM)

Depending on your connection speed this cell may take a few minutes to execute.
obs_ids = ['ibwb01*']
props = ['12692']
filts = ['F606W']

obsTable = Observations.query_criteria(obs_id=obs_ids, proposal_id=props, filters=filts)
products = Observations.get_product_list(obsTable)

data_prod = ['FLC']    # ['FLC','FLT','DRC','DRZ']                  
data_type = ['CALWF3'] # ['CALACS','CALWF3','CALWP2','HAP-SVM']    

Observations.download_products(products, productSubGroupDescription=data_prod, project=data_type, cache=True)
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/ibwb01xqq_flc.fits to ./mastDownload/HST/ibwb01xqq/ibwb01xqq_flc.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/ibwb01xrq_flc.fits to ./mastDownload/HST/ibwb01xrq/ibwb01xrq_flc.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/ibwb01xxq_flc.fits to ./mastDownload/HST/ibwb01xxq/ibwb01xxq_flc.fits ...
 [Done]
Table length=3
Local PathStatusMessageURL
str47str8objectobject
./mastDownload/HST/ibwb01xqq/ibwb01xqq_flc.fitsCOMPLETENoneNone
./mastDownload/HST/ibwb01xrq/ibwb01xrq_flc.fitsCOMPLETENoneNone
./mastDownload/HST/ibwb01xxq/ibwb01xxq_flc.fitsCOMPLETENoneNone

Move to the files to the local working directory:

for flc in glob.glob('./mastDownload/HST/*/*flc.fits'):
    flc_name = os.path.basename(flc)
    os.rename(flc, flc_name)
    
if os.path.exists('mastDownload'):
    shutil.rmtree('mastDownload')

1.1 Check image header data#

The cell below retrieves values for specific keywords from the first and second extensions of the image header. We see that the 1st exposure is 30 seconds and the 2nd and 3rd exposures are 360 seconds in duration. The 3rd exposure is dithered by approximately 82 arcseconds in the y-direction which is approximately the width of one WFC3 UVIS detector chip. A full list of header keywords is available in Section 2.4 of the WFC3 Data Handbook.

paths = sorted(glob.glob('*flc.fits'))
data = []
keywords_ext0 = ["ROOTNAME", "ASN_ID", "TARGNAME", "DETECTOR", "FILTER", "exptime", 
                 "ra_targ", "dec_targ", "postarg1", "postarg2", "DATE-OBS"]
keywords_ext1 = ["orientat"]

for path in paths:
    path_data = []
    for keyword in keywords_ext0:
        path_data.append(fits.getval(path, keyword, ext=0))
    for keyword in keywords_ext1:
        path_data.append(fits.getval(path, keyword, ext=1))
    data.append(path_data)
    
keywords = keywords_ext0 + keywords_ext1
table = Table(np.array(data), names=keywords, dtype=['str', 'str', 'str', 'str', 'str', 'f8', 'f8', 'f8', 'f8', 'f8', 'str', 'f8'])
table['exptime'].format = '7.1f'
table['ra_targ'].format = table['dec_targ'].format = '7.4f'
table['postarg1'].format = table['postarg2'].format = '7.3f'
table['orientat'].format = '7.2f'
table
Table length=3
ROOTNAMEASN_IDTARGNAMEDETECTORFILTERexptimera_targdec_targpostarg1postarg2DATE-OBSorientat
str32str32str32str32str32float64float64float64float64float64str32float64
ibwb01xqqNONENGC-6791UVISF606W30.0290.224837.80270.0000.0002011-10-20-56.27
ibwb01xrqNONENGC-6791UVISF606W360.0290.224837.80270.0000.0002011-10-20-56.27
ibwb01xxqNONENGC-6791UVISF606W360.0290.224837.80270.00081.6002011-10-20-56.25

1.2 Inspect the Alignment#

Check the active WCS solution in the image header. If the image is aligned to a catalog, the list the number of matches and the fit RMS in milli-arcseconds. Next, convert the fit RMS values to pixels for comparison with the alignment results performed later in this notebook using Tweakreg.

ext_0_keywords = ['DETECTOR', 'EXPTIME'] # extension 0 keywords.
ext_1_keywords = ['WCSNAME', 'NMATCHES', 'RMS_RA', 'RMS_DEC'] # extension 1 keywords.

# Define the detector plate scales in arcsec per pixel.
DETECTOR_SCALES = {
  'IR': 0.1283, 
  'UVIS': 0.0396, 
  'WFC': 0.05
}

formatted_data = {}
column_data = defaultdict(list)

for fits_file in sorted(glob.glob('*flc.fits')):
    column_data['filename'].append(fits_file)
    header0 = fits.getheader(fits_file, 0)
    header1 = fits.getheader(fits_file, 1)
    
    for keyword in ext_0_keywords:
        column_data[keyword].append(header0[keyword])
    for keyword in ext_1_keywords:
        if 'RMS' in keyword:
            value = np.around(header1[keyword], decimals=1)
        else:
            value = header1[keyword]
        column_data[keyword].append(value)
            
    for keyword in ['RMS_RA', 'RMS_DEC']:
        rms_value = header1[keyword] / 1000 / DETECTOR_SCALES[header0['DETECTOR']]
        column_data[f'{keyword}_pix'].append(np.round(rms_value, decimals=2))

wcstable = Table(column_data)
wcstable
Table length=3
filenameDETECTOREXPTIMEWCSNAMENMATCHESRMS_RARMS_DECRMS_RA_pixRMS_DEC_pix
str18str4float64str30int64float64float64float64float64
ibwb01xqq_flc.fitsUVIS30.0IDC_2731450pi-FIT_IMG_GAIAeDR34421.31.50.030.04
ibwb01xrq_flc.fitsUVIS360.0IDC_2731450pi-FIT_IMG_GAIAeDR34624.03.80.10.1
ibwb01xxq_flc.fitsUVIS360.0IDC_2731450pi-FIT_IMG_GAIAeDR34293.93.70.10.09

2. Query the Reference Catalogs#

Table of Contents

Now that we have the FITS images, we will download the reference catalogs from both SDSS and Gaia using astroquery.

2.1 Identify Coordinates#

We will first create a SkyCoord Object to point astroquery to where we are looking on the sky. Since our example uses data from NGC 6791, we will use the ra_targ and dec_targ keywords from the first image to get the coordinates of the object.

RA = table['ra_targ'][0]
Dec = table['dec_targ'][0]

2.2 SDSS Query#

We now give the RA and Dec values to an astropy SkyCoord object, which we will pass to the SDSS query. Additionally, we use an astropy Quantity object to create a radius for the SDSS query in physical units. We set the radius to 5 arcminutes to comfortably cover the area of our images. For reference, the UVIS detector field of view is approximately 2.7’\(\times\)2.7’ and a y-dither of 82 arcseconds covers a total area on the sky of approximately 2.7’\(\times\)4.1’.

coord = SkyCoord(ra=RA, dec=Dec, unit=(u.deg, u.deg))
radius = Quantity(5., u.arcmin)

Next, we perform the query via the SDSS.query_region method of astroquery.sdss. The spectro=False keyword argument means we want to exclude spectroscopic objects, as we are looking for objects to match with an image. In the fields parameter, we specify a list of fields we want returned by the query. In this case we only need the position, as well as a g-band magnitude if we want to remove very dim and/or bright objects from the catalog, as those are likely measured poorly. Details on selecting objects by magnitude may be found in the ‘Gaia_alignment’ notebook. Many other fields are available in the SDSS query and are documented on this webpage.

sdss_query = SDSS.query_region(coord, radius=radius, spectro=False, fields=['ra', 'dec', 'g'])
sdss_query
Table length=5426
radecg
float64float64float64
290.1589955239737.847385451942622.30409
290.20335374555537.80006907272319.35148
290.20746923100137.816532647322320.33007
290.22513834140537.737225118522619.70937
290.27028196766837.761850252355316.54826
290.27147537173537.730089579401615.66874
290.28591883059937.847893241655918.53543
290.2063829109137.857849826168219.08411
290.21766534848737.734467919973518.97118
290.22011946871337.874914639293517.08503
.........
290.23948446228137.879737966936924.90789
290.27765233083537.751792220829118.62893
290.28027468433537.766481199891518.14019
290.14381040005237.776039870594718.70891
290.19473628869737.777627512305317.86845
290.22029855626937.877823493936218.10617
290.25826212428137.810644938268917.33894
290.17787237233437.8422895598552-9999.0
290.19135670941337.829036784209718.48336
290.30457947059837.803447017201319.30075

We then need to save the catalog to use with TweakReg. As the returned value of the query is an astropy.Table, saving the file is straightforward:

sdss_query.write('sdss.cat', format='ascii.commented_header', overwrite=True)

2.3 Gaia Query#

Similarly to SDSS, we can query Gaia catalogs for our target via astroquery.gaia. This may be preferable in many cases because the spaced-based astrometry from Gaia is generally very accurate and precise. We can use the same coord and radius search parameters from our earlier SDSS query.

gaia_query = Gaia.query_object_async(coordinate=coord, radius=radius)
gaia_query
INFO: Query finished. [astroquery.utils.tap.core]
Table length=4865
solution_idDESIGNATIONSOURCE_IDrandom_indexref_epochrara_errordecdec_errorparallaxparallax_errorparallax_over_errorpmpmrapmra_errorpmdecpmdec_errorra_dec_corrra_parallax_corrra_pmra_corrra_pmdec_corrdec_parallax_corrdec_pmra_corrdec_pmdec_corrparallax_pmra_corrparallax_pmdec_corrpmra_pmdec_corrastrometric_n_obs_alastrometric_n_obs_acastrometric_n_good_obs_alastrometric_n_bad_obs_alastrometric_gof_alastrometric_chi2_alastrometric_excess_noiseastrometric_excess_noise_sigastrometric_params_solvedastrometric_primary_flagnu_eff_used_in_astrometrypseudocolourpseudocolour_errorra_pseudocolour_corrdec_pseudocolour_corrparallax_pseudocolour_corrpmra_pseudocolour_corrpmdec_pseudocolour_corrastrometric_matched_transitsvisibility_periods_usedastrometric_sigma5d_maxmatched_transitsnew_matched_transitsmatched_transits_removedipd_gof_harmonic_amplitudeipd_gof_harmonic_phaseipd_frac_multi_peakipd_frac_odd_winruwescan_direction_strength_k1scan_direction_strength_k2scan_direction_strength_k3scan_direction_strength_k4scan_direction_mean_k1scan_direction_mean_k2scan_direction_mean_k3scan_direction_mean_k4duplicated_sourcephot_g_n_obsphot_g_mean_fluxphot_g_mean_flux_errorphot_g_mean_flux_over_errorphot_g_mean_magphot_bp_n_obsphot_bp_mean_fluxphot_bp_mean_flux_errorphot_bp_mean_flux_over_errorphot_bp_mean_magphot_rp_n_obsphot_rp_mean_fluxphot_rp_mean_flux_errorphot_rp_mean_flux_over_errorphot_rp_mean_magphot_bp_rp_excess_factorphot_bp_n_contaminated_transitsphot_bp_n_blended_transitsphot_rp_n_contaminated_transitsphot_rp_n_blended_transitsphot_proc_modebp_rpbp_gg_rpradial_velocityradial_velocity_errorrv_method_usedrv_nb_transitsrv_nb_deblended_transitsrv_visibility_periods_usedrv_expected_sig_to_noiserv_renormalised_gofrv_chisq_pvaluerv_time_durationrv_amplitude_robustrv_template_teffrv_template_loggrv_template_fe_hrv_atm_param_originvbroadvbroad_errorvbroad_nb_transitsgrvs_maggrvs_mag_errorgrvs_mag_nb_transitsrvs_spec_sig_to_noisephot_variable_flaglbecl_lonecl_latin_qso_candidatesin_galaxy_candidatesnon_single_starhas_xp_continuoushas_xp_sampledhas_rvshas_epoch_photometryhas_epoch_rvhas_mcmc_gspphothas_mcmc_mscin_andromeda_surveyclassprob_dsc_combmod_quasarclassprob_dsc_combmod_galaxyclassprob_dsc_combmod_starteff_gspphotteff_gspphot_lowerteff_gspphot_upperlogg_gspphotlogg_gspphot_lowerlogg_gspphot_uppermh_gspphotmh_gspphot_lowermh_gspphot_upperdistance_gspphotdistance_gspphot_lowerdistance_gspphot_upperazero_gspphotazero_gspphot_lowerazero_gspphot_upperag_gspphotag_gspphot_lowerag_gspphot_upperebpminrp_gspphotebpminrp_gspphot_lowerebpminrp_gspphot_upperlibname_gspphotdist
yrdegmasdegmasmasmasmas / yrmas / yrmas / yrmas / yrmas / yrmas1 / um1 / um1 / ummasdegdegdegdegdegelectron / selectron / smagelectron / selectron / smagelectron / selectron / smagmagmagmagkm / skm / sdkm / sKlog(cm.s**-2)dexkm / skm / smagmagdegdegdegdegKKKlog(cm.s**-2)log(cm.s**-2)log(cm.s**-2)dexdexdexpcpcpcmagmagmagmagmagmagmagmagmag
int64objectint64int64float64float64float32float64float32float64float32float32float32float64float32float64float32float32float32float32float32float32float32float32float32float32float32int16int16int16int16float32float32float32float32int16boolfloat32float32float32float32float32float32float32float32int16int16float32int16int16int16float32float32int16int16float32float32float32float32float32float32float32float32float32boolint16float64float32float32float32int16float64float32float32float32int16float64float32float32float32float32int16int16int16int16int16float32float32float32float32float32int16int16int16int16float32float32float32float32float32float32float32float32int16float32float32int16float32float32int16float32objectfloat64float64float64float64boolboolint16boolboolboolboolboolboolboolboolfloat32float32float32float32float32float32float32float32float32float32float32float32float32float32float32float32float32float32float32float32float32float32float32float32objectfloat64
1636148068921376768Gaia DR3 2051293319932402944205129331993240294417104148122016.0290.22486113562830.1234355137.801929503613590.155340691.24091546330640860.159768727.76694876.0644784-0.62265434076184740.154743726.03242907928795450.18924683-0.0330014680.16417459-0.027278472-0.0201708860.07942047-0.046242356-0.19703287-0.076851850.00517138930.050814763394038950.407836413.483030.02.0511602e-1531False1.3016587--------------45260.26493552492200.0189501614.319122001.01388040.175694810.202403290.102880320.089386284-95.1318277.093792.6807811-30.180996False385492.167884547433860.9903239496.9766818.95708339106.264534883216615.385507619.73157320.27257239549.19942951062547.085240477.5131717.8985711.3317894020202.37400051.31548881.0585117--------------------------------------------NOT_AVAILABLE69.9876189105112910.914199739506339302.0419657477765459.01190915812673FalseFalse0FalseFalseFalseFalseFalseTrueFalseFalse1.4003378e-135.356044e-130.999967753653.5923591.57523744.39924.85524.49924.9965-0.4967-0.7199-0.1315653.0443491.94811283.51120.78120.64990.92020.53050.440.62660.31020.25810.3649MARCS0.0007449136479797354
1636148068921376768Gaia DR3 205129332421826380820512933242182638082149749782016.0290.223279727540841.049147237.802839447959031.4068799-0.60690774600642671.4172788-0.428220453.8347042-0.496949943918527371.433404-3.8023671866937022.183717-0.325595260.440830080.040013984-0.36232764-0.18673699-0.298557550.51452190.16053891-0.3253903-0.32209665133013212.3624072187.533173.49461981.735654495False--1.25838210.3532982-0.10796609-0.06725511-0.030691369-0.22952616-0.1275772916143.268870618610.06122743743.2264185171.14967740.235481130.32430920.357489230.4221966-95.1267274.99657-8.526287-16.614477False13288.73757662943781.199292273.9916220.8170990--------0------------------0--------------------------------------------------NOT_AVAILABLE69.987894390365210.915721621139165302.040099004613159.01311977354264FalseFalse0FalseFalseFalseFalseFalseFalseFalseFalse7.759076e-083.7895185e-090.9999943------------------------------------------0.0012108975043825237
1636148068921376768Gaia DR3 205129332422297228820512933242229722881644563802016.0290.222619948805970.03284164537.802707524747090.0394590.30530948963652580.0411276977.4234522.5961652-0.70126232753246160.041221105-2.49966104338574050.047550496-0.0475552980.165465030.021742897-0.0778674560.066915035-0.08765522-0.26260835-0.059347805-0.087691456-0.001331030738303812-2.1703348333.383270.00.031False1.4413656--------------44250.06704042482100.01503751812.04822000.92104670.138905180.189018130.109005130.0641752-88.3616572.3029565.923747-33.787945False3814179.2546590772062.25175741855.996816.63462431717.78303874708586.2028065276.9364317.251122393494.120407965024210.008291349.122615.8895511.247089302502301.36157040.616500850.7450695--------------------------------------------NOT_AVAILABLE69.9875410099388210.916132432650413302.039054790094259.01313187059839FalseFalse0TrueFalseFalseFalseFalseTrueTrueFalse1.0228448e-139.3310026e-110.99997414770.34474749.60454790.8184.17764.15964.19020.35460.32680.38052185.93162147.36252239.19260.35110.32590.37350.27010.25050.28750.14060.13030.1496PHOENIX0.001721026645501702
1636148068921376768Gaia DR3 2051293324222972416205129332422297241615999055882016.0290.22483097015550.05294129637.800586114058950.0648960550.218370438246445250.0670577663.25645262.3448853-0.58272005397828510.066188715-2.27132660016116360.0787358-0.0768301040.14136231-0.025055349-0.072975410.056257915-0.07904705-0.24559797-0.07446872-0.087429665-0.0335280339503923-0.037620842401.514340.00.031False1.4891585--------------45250.110738866492200.015343606515.8542000.997787240.152932750.214773120.129280220.08397196-94.64999470.080226.0968003-29.722376False3891751.03660763972771.49502251171.244317.57912846817.00652343411046.4622293126.4279718.057978441347.1552304503578.58573156.9063316.924351.235931803202901.13362690.47884940.6547775--------------------------------------------NOT_AVAILABLE69.9863784534824610.913632562752337302.041224263599659.01062093567249FalseFalse0TrueFalseFalseFalseFalseTrueTrueFalse6.110852e-115.145215e-130.99991324831.9014812.9594859.144.6614.64124.6768-0.4447-0.5218-0.37861595.51841545.62231641.48860.00870.00210.0210.00690.00170.01660.00360.00090.0088PHOENIX0.0020867793124382814
1636148068921376768Gaia DR3 205129331993479193620512933199347919361277407952016.0290.22256026388090.09856778437.803978470044570.122381570.186089237607969780.126199871.47455972.3556678-0.205917660343796220.12384657-2.3466506380483370.14836667-0.00343720640.17569284-0.028029243-0.0236887970.0831415-0.04713238-0.21456409-0.08000817-0.0190065170.07798844386038330.34466058403.298740.00.031False1.4745288--------------44260.20897423482100.0252598416.266506001.0116760.164512680.1805770.0794067760.08114059-89.37092680.655790.7836158-30.156015False377665.16671859144721.124127591.7184418.63004140296.016045706441965.845524350.6397819.16025441517.31486865230936.127615584.4235217.9635091.2227474000001.19674490.53021240.6665325--------------------------------------------NOT_AVAILABLE69.9886834952528110.916731652994313302.0396258357147359.014369244472704FalseFalse0FalseFalseFalseFalseFalseTrueFalseFalse1.0210282e-135.259736e-130.999994754590.27834575.7664603.02444.82554.79814.858-1.2317-1.4162-1.10091664.28761569.11961730.9150.01020.00260.02570.0080.0020.020.00430.00110.0107PHOENIX0.0021977590777958536
1636148068921376768Gaia DR3 205129332421654323220512933242165432324969746282016.0290.226819596871960.580043437.801127727705170.85181140.98661196885736370.84873931.1624444.5651155-0.442437662624050151.042491-4.5436251700190051.2715639-0.03034182-0.130354290.099056150.024711896-0.035745695-0.130549790.34666777-0.069062054-0.28526154-0.107861118901890-0.84486425177.74490.01.4856265e-1595False--1.55213480.21582161-0.00621337680.28042490.035700142-0.421011060.1781859222151.815063425800.02325232930.515776000.9543710.208770540.288419070.19270840.43569437-76.916684.362335-3.7503886-19.752798False189100.173459526538440.81711596122.5939320.6854861634.7777430671914948.3456144.167187721.4852891989.512291396926145.523087516.2069319.8681891.2407482000001.61709980.79980280.817297--------------------------------------------NOT_AVAILABLE69.9875754334518210.912457430119012302.044446154287159.01072318960171FalseFalse0FalseFalseFalseFalseFalseFalseFalseFalse1.5372552e-091.1275478e-090.99999505------------------------------------------0.002222460820739603
1636148068921376768Gaia DR3 2051293324222974464205129332422297446417984361052016.0290.22752493081420.2407992537.8034727751390350.30747050.445066524419969630.31342351.42001641.903844-0.77414992477356190.3071843-1.73934298252762920.374548080.0204875150.17073557-0.0363375430.0416509140.12431544-0.0061035734-0.18864702-0.065493080.0605290680.123350315382038020.5945536412.031980.90154610.957877495False--1.31476190.081987195-0.0167665020.1181909740.037860535-0.0898746-0.1086104344250.52942884482100.02776220820.245789041.020920.184008080.17896370.0736982750.10250232-90.67370685.22672-9.854181-30.444962False380207.208736176724160.7419642279.270519.896347991.399500159475617.501483412.1841920.4361829224.2019002106420411.06425320.26362818.8712981.5231086000001.56488420.5398351.0250492--------------------------------------------NOT_AVAILABLE69.9899708940742310.912983986333725302.0467059052208559.01283416072846FalseFalse0FalseFalseFalseFalseFalseFalseFalseFalse1.8206678e-092.9034798e-080.99998796------------------------------------------0.002298594469020544
1636148068921376768Gaia DR3 2051293324216539648205129332421653964816494576092016.0290.22423204098210.527127437.800395107394630.766553341.36327565842414340.67285462.02610732.2253373-0.97086458829061460.6578662-2.00238563025194070.98862535-0.334653530.08556003-0.27351620.26027030.1616140.21899539-0.265428070.0335000120.1458301-0.23663591243024301.2024069273.663820.00.095False--1.68184630.19114311-0.260752350.11348202-0.08333635-0.06321393-0.1521266128181.4304484321410.070790234.3759540151.05430190.245606850.44389820.229683370.165783-110.2577667.430465.994361-23.29821False244109.85521560273350.8030061136.8049620.5853160--------0------------------2--------------------------------------------------NOT_AVAILABLE69.9859924248062710.91397427439119302.040239437952359.01056325889003FalseFalse0FalseFalseFalseFalseFalseFalseFalseFalse------------------------------------------------0.0023210548670645637
1636148068921376768Gaia DR3 205129332422533606420512933242253360643980909602016.0290.223029396664630.4765001837.80034041154940.571635841.04655396175326130.605739831.72772840.34264810.0165256796874195530.6219764-0.342249354474437440.7205138-0.113586160.15291650.07183364-0.1044668260.054135017-0.07860708-0.17583270.10776901-0.16631071-0.164722253730365886.8070312487.4786.598426424.326495False--1.29765440.14454299-0.0576655570.0526744050.0332027340.010857275-0.0792269743251.0361372505000.1981419321.4078086305.61499450.16931350.215803350.101834080.09376588-92.0437770.390440.82470447-23.783632False298836.2066294848865.043918165.7851418.38158243492.858050365895965.083862396.94559518.60673741916.84299006550376.7629867135.5677617.3421571.6858286020201.26457980.225154881.0394249--------------------------------------------NOT_AVAILABLE69.9855183318727510.914804515213548302.038432481796159.01076433315906FalseFalse0FalseFalseFalseFalseFalseFalseFalseFalse9.514203e-111.056036e-050.99998164------------------------------------------0.0027187891427339723
...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................
1636148068921376768Gaia DR3 205129517964929459220512951796492945926421895312016.0290.299927459824860.05703522337.860930642772420.080387330.17438924499333810.07572372.30296782.3790622-0.67228334547202830.07357572-2.28209808869056370.099360675-0.0999062660.12198629-0.030872876-0.0162742180.002427859-0.013690395-0.4221675-0.0023941058-0.067306004-0.0941992836503650-1.037416346.2230.00.031False1.495752--------------42240.14341107452200.025017241177.41403200.96067420.243728770.232741280.065795440.091317765-73.8895378.48466-18.219536-40.47013False3641591.61507159842361.7503392909.318117.68277212981.254920764472445.3395221.64237617.859087111464.330951944686438.8011137.7394116.8337971.5365436005001.02528950.176315310.8489742--------------------------------------------NOT_AVAILABLE70.0680917156638810.886777120867162302.1836819237343559.052883463912785FalseFalse0FalseFalseFalseFalseFalseFalseTrueFalse2.2967477e-097.032779e-070.9999951------------------------------------------0.08315680702577756
1636148068921376768Gaia DR3 205129487041153561620512948704115356164309557012016.0290.17105427991030.08935162437.8742101388241540.108656990.36945862661639120.1138345153.24557669.114665-6.318711438864550.11210035-6.5689427509367870.13436429-0.0078412550.11561305-0.01169015-0.036353250.11398082-0.05428327-0.21885028-0.10007106-0.082207880.061571144399039270.13260055406.648620.229238990.4186275331False1.4996165--------------46270.18900031492200.0111670291.7622564001.00390740.267865330.107605250.104045730.08249484-87.4581875.753624-15.217908-38.780235False384779.80540032432371.0609461735.0094618.45740146362.82439226289376.34242157.2059818.939345563.9644400367635.256008107.29917.8697661.1884873000001.06953430.481899260.58763504--------------------------------------------NOT_AVAILABLE70.0348593367095410.984050062538527301.999871965901759.09291303817852FalseFalse0FalseFalseFalseFalseFalseTrueFalseFalse3.5958955e-115.3623333e-130.99999494889.59474878.37164901.45754.83924.82724.8515-1.6225-1.7671-1.52371828.68271778.36911860.66780.00450.00110.01080.00360.00090.00860.0020.00050.0047MARCS0.0831808340919906
1636148068921376768Gaia DR3 205128823468062873620512882346806287362753599892016.0290.12281252254180.1283277337.78191636316250.167706530.457119709271901150.164422722.780149515.87947-6.4918277134628470.16428103-14.4918510159381310.217646610.0027644680.12523003-0.100281990.0506576230.066664360.005579139-0.37272593-0.1188299060.00509953170.113982340904090-1.3663982383.077820.01.9997444e-1531False1.3034092--------------47270.3113583482300.0102933812.4593005000.951505840.170920090.13878890.058289340.13473459-116.17805583.694192.6145937-36.532135False407450.443527509372130.8725948516.211619.05326745103.687711523979545.13652420.18635720.29922545502.450091057051963.6524675137.5645617.9951631.3456466010002.3040621.24595831.0581036--------------------------------------------NOT_AVAILABLE69.933337119497310.977945740748442301.8806347601019459.014107067896354FalseFalse0FalseFalseFalseFalseFalseFalseFalseFalse3.8000676e-137.05338e-130.9998841------------------------------------------0.08322243598195858
1636148068921376768Gaia DR3 2051294870411542272205129487041154227216512471472016.0290.17501016654010.0669400837.876028424026390.0924856140.206037591795022120.085798652.40140822.2515461-0.39323372307182880.08297268-2.2169410942210960.118552975-0.062974490.14318395-0.0336323680.03577277-0.0341151470.00034698079-0.4515389-0.109378310.072311810.009728736377037430.06470177384.275020.00.095False--1.43710540.023171378-0.160518380.025070293-0.010794556-0.076078640.02811128343260.17073497482210.01102003161.055740151.00147950.29626180.182184160.144272090.11568721-84.17235675.77043-18.85028842.884228False3741241.3444017878721.2767242972.288617.9526370--------0------------------2--------------------------------------------------NOT_AVAILABLE70.037916500518210.982037682530848302.0066758819189759.093831343557795FalseFalse0FalseFalseFalseFalseFalseFalseFalseFalse------------------------------------------------0.08322879189943044
1636148068921376768Gaia DR3 2051295381516623744205129538151662374416678728482016.0290.28826290149020.1479390637.869130155367920.214395080.66087202543174460.19874353.32525098.816373-2.68849909261271060.19940175-8.396451784151660.26135257-0.121748830.108698270.062106630.00640506340.07006799-0.034789-0.362983440.027279017-0.00647773-0.083680145339033810.5362549369.12530.419998470.526066995False--1.33896930.05077461-0.0875690060.18131016-0.04555531-0.13339908-0.1101084639230.3739153422000.029168483178.31316001.0198720.21484670.206238150.126745220.11732818-75.8992779.33952-25.199-39.338097False346420.75381480136151.0356286406.2786619.12729639140.642327752087887.475215418.81448619.96825239478.46027347016049.10938352.52389518.048281.471412902202201.91997150.840955731.0790157--------------------------------------------NOT_AVAILABLE70.071483072491510.89864899377486302.170707879319159.063253212461184FalseFalse0FalseFalseFalseFalseFalseFalseFalseFalse8.597156e-115.9048273e-090.9999822------------------------------------------0.08324003047068324
1636148068921376768Gaia DR3 205128840647938969620512884064793896968567112482016.0290.11943540856380.168512537.803481150098460.219498960.240107422764838040.209612561.14548216.186268-2.25082905476555340.21421075-5.7622632191142280.27950856-0.041893690.055101015-0.160175030.1141437740.152777050.08570732-0.41045648-0.013016244-0.0259738710.04609114336903690-0.34031537371.165740.04.2135146e-1531False1.4944085--------------42250.40078756432100.00952052919.020708000.98650190.207523540.1126088050.0999876260.1787911-113.1324983.6748212.821432-33.98683False367332.18752621583050.86817735382.6263419.3839138136.828327815067187.068960719.35621619.99810240250.457301026567735.90949742.3821718.751061.1658645000001.24704170.614192960.63284874--------------------------------------------NOT_AVAILABLE69.9519040142854610.989775662227835301.8867764535164659.035605221497256FalseFalse0FalseFalseFalseFalseFalseFalseFalseFalse5.805757e-102.1388297e-100.9999928------------------------------------------0.0832529718271768
1636148068921376768Gaia DR3 20511071287985665282051107128798566528537854282016.0290.330072883004330.506318537.7990051965467660.608610032.595823033348330.727872253.566316815.07725-9.905819875496990.8522273-11.366537961753560.78330284-0.037682395-0.031865645-0.0061560404-0.09596210.24388646-0.024079807-0.0854026750.35197842-0.08016355-0.08673279257025702.9779897345.66052.60022932.656405295False--1.31030580.1573422-0.104157340.033458136-0.09948709-0.27206930.0419742731221.2133433331310.020845002138.17416001.13443110.183162150.099376230.235923510.23159805-121.59579582.81833-15.507539-25.80699False253128.700828741648650.86452293148.8691920.413414639.7179681588192548.3536894.754542421.34107410145.1834940460665815.550319.33637319.3431021.4366765000001.99797250.927661.0703125--------------------------------------------NOT_AVAILABLE70.0220719284697310.83820692421412302.1959655141427658.98684034920937FalseFalse0FalseFalseFalseFalseFalseFalseFalseFalse2.8843332e-121.4122469e-090.99999326------------------------------------------0.08326359931304438
1636148068921376768Gaia DR3 2051287964103728384205128796410372838416388941582016.0290.142786521877440.0349411637.750375252598580.042418790.183662435264695420.0434608464.22592832.248427-0.42816121111843630.04497176-2.20728378302159370.0509416830.03511510.055712797-0.14853671-0.0518249720.13105968-0.06969542-0.3188419-0.1026289-0.116389790.0937231142804262-0.6057206420.234160.00.031False1.5003259--------------49280.07342281512400.031589528.174233000.978412150.127623020.130485830.069852560.09125381-109.351384.420715-1.0416744-31.609655False4263374.64237302189351.917791759.651616.866798481618.8911817173066.9249954233.7750517.315498492463.80015246004726.911771356.464416.2688831.2098145020101.04661560.448699950.59791565--------------------------------------------NOT_AVAILABLE69.911482471427810.949949126990399301.893910006924458.97950356750648FalseFalse0TrueFalseFalseFalseFalseTrueTrueFalse1.02206966e-135.105279e-130.999981765064.9855040.86575095.1244.44244.42174.4671-0.2906-0.3312-0.25181867.08231788.14811945.35280.0180.00520.0370.01440.00420.02970.00760.00220.0157PHOENIX0.08328823877757062
1636148068921376768Gaia DR3 205128693331158003220512869333115800321354416172016.0290.171279999811650.0441494837.730910827203310.054037760.23575027131211040.0564599044.17553472.2787905-0.291095662000127650.055603746-2.2601215671869460.06337706-0.017252762-0.013258554-0.13981561-0.0514389950.18330099-0.08995912-0.30687386-0.12603593-0.084348590.07638833414041220.1777575428.26430.00.031False1.5031--------------47260.09099819502400.0322462736.7542543101.00541630.088989830.142247330.1057359050.10415334-109.95827577.314916.73328-30.988867False4072336.07857555510961.52014341536.748817.266148461127.33469313582178.7009325129.5648217.70841471705.03237555713997.5450583225.9800116.6685641.2124451020001.03984640.442262650.5975838--------------------------------------------NOT_AVAILABLE69.903703756235310.921169975727066301.926005109922858.95474875157075FalseFalse0TrueFalseFalseFalseFalseTrueTrueFalse1.0224553e-135.108376e-130.999977955002.62844981.45265036.26274.6024.55114.6331-0.8208-0.8784-0.75341583.6141512.62021733.18860.02430.00740.0520.01950.00590.04190.01040.00320.0225PHOENIX0.08330419708303734
1636148068921376768Gaia DR3 205128696766477542420512869676647754249543108992016.0290.19718073295320.2930343437.722252403860490.36113890.50806683107502220.371531931.36749172.9852633-0.48170314357243170.36552507-2.94614307473170720.442575660.142766830.072574675-0.1556142-0.054114140.09976298-0.10295689-0.3932338-0.21745701-0.036722640.23811138345034320.5666112368.388730.00.095False--1.43555590.09823673-0.0335834960.0606465040.054102458-0.074938364-0.1056569740230.66058314421900.017682025156.029001.02090810.261768220.102804990.153338370.21728502-115.630585-75.681816-4.198842-29.578873False341175.077760305204550.8039993217.758620.0792881881.092852127656889.7685028.30146220.56608621184.313332660139767.446383524.7520619.0840031.5159332000001.48208240.486797330.99528503--------------------------------------------NOT_AVAILABLE69.904914830410310.898961260981507301.9598224300295358.94095155723347FalseFalse0FalseFalseFalseFalseFalseFalseFalseFalse4.541147e-102.3747997e-080.9999982------------------------------------------0.08333124886811095

This query has returned very large number of columns, so we select the most useful columns to make the catalog easier to use with TweakReg.

reduced_query = gaia_query['ra', 'dec', 'phot_g_mean_mag']
reduced_query
Table length=4865
radecphot_g_mean_mag
degdegmag
float64float64float32
290.224861135628337.8019295036135918.957083
290.2232797275408437.8028394479590320.817099
290.2226199488059737.8027075247470916.63462
290.224830970155537.8005861140589517.579128
290.222560263880937.8039784700445718.630041
290.2268195968719637.8011277277051720.685486
290.227524930814237.80347277513903519.896347
290.224232040982137.8003951073946320.585316
290.2230293966646337.800340411549418.381582
.........
290.2999274598248637.8609306427724217.682772
290.171054279910337.87421013882415418.457401
290.122812522541837.781916363162519.053267
290.175010166540137.8760284240263917.952637
290.288262901490237.8691301553679219.127296
290.119435408563837.8034811500984619.38391
290.3300728830043337.79900519654676620.413414
290.1427865218774437.7503752525985816.866798
290.1712799998116537.7309108272033117.266148
290.197180732953237.7222524038604920.079288

Then we write this catalog to an ascii file for use with TweakReg.

reduced_query.write('gaia.cat', format='ascii.commented_header', overwrite=True)

3. Align with TweakReg#

Table of Contents

With the catalogs downloaded and the headers populated, we now need to run TweakReg with each catalog passed into the refcat parameter. The steps below compare the astrometric residuals obtained from aligning to each refcat. In each case, we set the updatehdr parameter to False to avoid altering the WCS of the images until we are satisfied with the alignment by inspecting both the TweakReg shift file and the astrometric residual plots.

3.1 SDSS Alignment#

refcat = 'sdss.cat'
wcsname = 'SDSS' # Specify the WCS name for this alignment.
cw = 3.5         # 2x the FWHM of the PSF = 3.5 pixels for ACS/WFC, WFC3/UVIS or 2.5 for WFC3/IR.

tweakreg.TweakReg('*flc.fits', # Pass the input images to tweakreg.
                  imagefindcfg={'threshold': 500., 'conv_width': cw},  # Detection parameters that vary for different datasets.
                  refcat=refcat,                # Use user supplied catalog (Gaia).
                  interactive=False,            # Don't show the interactive interface.
                  see2dplot=False,              # Suppress additional figures in this test.
                  shiftfile=True,               # Save output shift file to examine shifts later.
                  outshifts='SDSS_shifts.txt',  # Name of the shift file that will be saved.
                  wcsname=wcsname,              # Give our WCS a new name defined above.
                  reusename=True,               # Overwrite the WCS if it exists.
                  ylimit=1.5,
                  fitgeometry='rscale',         # Allow for translation, rotation, and plate scaling.
                  updatehdr=False)              # Don't update the header with new WCS until later.
clear_output()
../../../_images/aface2a8fc982fa17e555cdf83a07f0c55aac22eec644a84119cd10e1d4d0f1c.png ../../../_images/91a91c8da43b3796b69643baf06e2ca4555b416d08fb4a9339e5889584140e10.png
# If the alignment is unsuccessful then stop the notebook.

with open('SDSS_shifts.txt', 'r') as shift:
    for line_number, line in enumerate(shift, start=1):
        if "nan" in line:
            raise ValueError('nan found in line {} in shift file'.format(line_number))
        else:
            continue

3.2 Inspect the shift file and fit residuals#

We can look at the shift file to see how well the fit did (or we could open the output png images for more information). The columns are:

  • Filename, X Shift [pixels], Y Shift [pixels], Rotation [degrees], Scale, X RMS [pixels], Y RMS [pixels]

shift_table = Table.read('SDSS_shifts.txt',
                         format='ascii.no_header', 
                         names=['file', 'dx', 'dy', 'rot', 'scale', 'xrms', 'yrms'])

formats = ['.2f', '.2f', '.3f', '.5f', '.2f', '.2f']
for i, col in enumerate(shift_table.colnames[1:]):
    shift_table[col].format = formats[i]
shift_table
Table length=3
filedxdyrotscalexrmsyrms
str18float64float64float64float64float64float64
ibwb01xxq_flc.fits1.56-0.23359.9991.000080.380.52
ibwb01xrq_flc.fits1.480.550.0020.999950.370.36
ibwb01xqq_flc.fits1.450.710.0030.999930.370.33

Show the astrometric residual plots.

Image(filename='residuals_ibwb01xqq_flc.png', width=500, height=300)
../../../_images/c03b0619345e38072ae68ec79ce15e3430e92689e28efd7cc6bbe37fa5309203.png
Image(filename='residuals_ibwb01xrq_flc.png', width=500, height=300)
../../../_images/c23f8837ccb126a4decf0eb86207ec0fb840b55651effc4002302722b82eaba6.png
Image(filename='residuals_ibwb01xxq_flc.png', width=500, height=300)
../../../_images/e435e36c6cd41fef9fa43183ce57e84fe5aefd81be450274bd5a02259518c306.png

We can see in the plots above that the RMS is fairly large at about 0.5 pixels. This is generally considered a poor fit, with the desired RMS being < 0.1 pixels depending on the number of sources. This is likely because the SDSS astrometric precision is insufficient for a high-quality HST alignment. One approach would be to align the first image to SDSS and then align the remaining HST images to one another. This would improve both the absolute and relative alignment of the individual frames.

3.3 Gaia Alignment#

refcat = 'gaia.cat'
wcsname = 'Gaia' # Specify the WCS name for this alignment.
cw = 3.5         # 2x the FWHM of the PSF = 3.5 pixels for ACS/WFC, WFC3/UVIS or 2.5 for WFC3/IR.

tweakreg.TweakReg('*flc.fits', # Pass the input images to tweakreg.
                  imagefindcfg={'threshold': 500., 'conv_width': cw},  # Detection parameters that vary for different datasets.
                  refcat=refcat,                # Use user supplied catalog (Gaia).
                  interactive=False,            # Don't show the interactive interface.
                  see2dplot=False,              # Suppress additional figures in this test.
                  shiftfile=True,               # Save output shift file to examine shifts later.
                  outshifts='Gaia_shifts.txt',  # Name of the shift file that will be saved.
                  wcsname=wcsname,              # Give our WCS a new name defined above.
                  reusename=True,               # Overwrite the WCS if it exists.
                  ylimit=0.3,                   # The ylimit for the residuals plot.
                  fitgeometry='rscale',         # Allow for translation, rotation, and plate scaling.
                  searchrad=0.1,                # With many sources use a smaller search radius for convergence.
                  updatehdr=False)              # Don't update the header with new WCS until later.
clear_output()
../../../_images/7e397d3fb7470fa31aee1b52ab167d2afdb4aaa636dfabea30a56e982a7f8590.png ../../../_images/0e8f38ca4f1af66c3a00d42a533e6569b2b4992cfbf09ce072a1e08efeba6b8a.png
# If the alignment is unsuccessful then stop the notebook.

with open('Gaia_shifts.txt', 'r') as shift:
    for line_number, line in enumerate(shift, start=1):
        if "nan" in line:
            raise ValueError('nan found in line {} in shift file'.format(line_number))
        else:
            continue

3.4 Inspect the shift file and fit residuals#

We can similarly look at the shift file from alignment to the Gaia catalog. As expected, the Gaia catalog does quite a bit better, with rms residuals ~0.05 pixels.

The columns are: Filename, X Shift [pixels], Y Shift [pixels], Rotation [degrees], Scale, X RMS [pixels], Y RMS [pixels]

shift_table = Table.read('Gaia_shifts.txt',
                         format='ascii.no_header', 
                         names=['file', 'dx', 'dy', 'rot', 'scale', 'xrms', 'yrms'])

formats = ['.2f', '.2f', '.3f', '.5f', '.2f', '.2f']
for i, col in enumerate(shift_table.colnames[1:]):
    shift_table[col].format = formats[i]
shift_table
Table length=3
filedxdyrotscalexrmsyrms
str18float64float64float64float64float64float64
ibwb01xxq_flc.fits-0.210.090.0001.000000.030.05
ibwb01xrq_flc.fits-0.220.100.0001.000000.030.05
ibwb01xqq_flc.fits-0.220.10360.0001.000000.020.03

Print the Number of Gaia Matches per image from the TweakReg output.

rootname1 = 'ibwb01xqq'
rootname2 = 'ibwb01xrq'
rootname3 = 'ibwb01xxq'
match_tab1 = ascii.read(rootname1+'_flc_catalog_fit.match')  # load match file in astropy table
match_tab2 = ascii.read(rootname2+'_flc_catalog_fit.match')  # load match file in astropy table
match_tab3 = ascii.read(rootname3+'_flc_catalog_fit.match')  # load match file in astropy table
print(rootname1+': Number of Gaia Matches =', len(match_tab1))
print(rootname2+': Number of Gaia Matches =', len(match_tab2))
print(rootname3+': Number of Gaia Matches =', len(match_tab3))
ibwb01xqq: Number of Gaia Matches = 560
ibwb01xrq: Number of Gaia Matches = 565
ibwb01xxq: Number of Gaia Matches = 545

Compare the RMS and number of matches from TweakReg with the MAST alignment results.

wcstable
Table length=3
filenameDETECTOREXPTIMEWCSNAMENMATCHESRMS_RARMS_DECRMS_RA_pixRMS_DEC_pix
str18str4float64str30int64float64float64float64float64
ibwb01xqq_flc.fitsUVIS30.0IDC_2731450pi-FIT_IMG_GAIAeDR34421.31.50.030.04
ibwb01xrq_flc.fitsUVIS360.0IDC_2731450pi-FIT_IMG_GAIAeDR34624.03.80.10.1
ibwb01xxq_flc.fitsUVIS360.0IDC_2731450pi-FIT_IMG_GAIAeDR34293.93.70.10.09

The new Gaia solution looks better than the MAST solution, so we rerun TweakReg below and update the image headers with the new WCS solution.

For more details on absolute astrometry in HST images, see Section 4.5 in the DrizzlePac Handbook. Show the astrometric residual plots:

Image(filename='residuals_ibwb01xqq_flc.png', width=500, height=300)
../../../_images/9677e900a9c9a6e6cae0bd22e5605d08950d01edbe57517a7fa1ef3281957120.png
Image(filename='residuals_ibwb01xrq_flc.png', width=500, height=300)
../../../_images/f13237ff2bc7224f8b4c3c625fe9399a80b6d94ac22c8eb1af4d0bc690dfdc5a.png
Image(filename='residuals_ibwb01xxq_flc.png', width=500, height=300)
../../../_images/dd0f9dc38b10f1a235de7f0be631a20ec5a1275fd86f98ab43f4610e553e61fd.png

3.5 Overplot Matched Sources on the Image#

Now, let’s plot the sources that were matched between all images on the “bottom” UVIS chip. This is referred to as chip 2 or SCI, 1 or extension 1. The cell below reads in the *_fit.match file as an astropy table. Unfortunately, it doesn’t automatically name columns so we look at the header to grab the columns.

To verify that TweakReg obtained an acceptable fit between matched source catalogs, it is useful to inspect the results before updating the image header WCS. In the figure below, sources that are matched with Gaia are overplotted on one of the input FLC frames with the two chips merged together.

It can be useful to check that TweakReg locked onto stars and not hot pixels or other detector artifacts before proceeding to update the image header.

# Choose one of the files to view.
# rootname = 'ibwb01xqq'
rootname = 'ibwb01xrq'
# rootname = 'ibwb01xxq'

plt.figure(figsize=(7, 7), dpi=140)
chip1_data = fits.open(rootname+'_flc.fits')['SCI', 2].data
chip2_data = fits.open(rootname+'_flc.fits')['SCI', 1].data
fullsci = np.concatenate([chip2_data, chip1_data])
z1, z2 = ZScaleInterval().get_limits(fullsci)
plt.imshow(fullsci, cmap='Greys', origin='lower', vmin=z1, vmax=z2)

match_tab = ascii.read(rootname+'_flc_catalog_fit.match')  # Load the match file in astropy table.
match_tab_chip1 = match_tab[match_tab['col15'] == 2]  # Filter the table for sources on chip 1.
match_tab_chip2 = match_tab[match_tab['col15'] == 1]  # Filter the table for sources on chip 2.
x_cord1, y_cord1 = match_tab_chip1['col11'], match_tab_chip1['col12']
x_cord2, y_cord2 = match_tab_chip2['col11'], match_tab_chip2['col12']

plt.scatter(x_cord1, y_cord1+2051, s=50, edgecolor='r', facecolor='None', label='Matched Sources')
plt.scatter(x_cord2, y_cord2, s=50, edgecolor='r', facecolor='None')
plt.title(f'Matched sources UVIS to Gaia: N = {len(match_tab)}', fontsize=14)
plt.show()
../../../_images/16f967484634fb584fb33821eb1c044b98359ff4a0f1811f006c6453d3920b6d.png

3.6 Update header with optimal parameters#

Once you’ve decided on the optimal set of parameters for your alignment, it is safe to update the header of the input data, (assuming that the new solution is better than the MAST alignment.) To apply these transformations to the image, we simply need to run TweakReg the same as before, but set the parameter updatehdr equal to True:

refcat = 'gaia.cat'
wcsname = 'Gaia'                   # Specify the WCS name for this alignment
cw = 3.5                           # 2x the FWHM of the PSF = 3.5 for ACS/WFC, WFC3/UVIS or 2.5 for WFC3/IR 

tweakreg.TweakReg('*flc.fits', # Pass the input images to tweakreg.
                  imagefindcfg={'threshold': 500., 'conv_width': cw},  # Detection parameters that vary for different datasets.
                  refcat=refcat,                # Use user supplied catalog (Gaia).
                  interactive=False,            # Don't show the interactive interface.
                  see2dplot=False,              # Suppress additional figures in this test.
                  shiftfile=True,               # Save output shift file to examine shifts later.
                  outshifts='Gaia_shifts.txt',  # Name of the shift file that will be saved.
                  wcsname=wcsname,              # Give our WCS a new name defined above.
                  reusename=True,               # Overwrite the WCS if it exists.
                  ylimit=0.3,                   # The ylimit for the residuals plot.
                  fitgeometry='rscale',         # Allow for translation, rotation, and plate scaling.
                  searchrad=0.1,                # With many sources use a smaller search radius for convergence.
                  updatehdr=True)               # Update the header with new WCS solution.

clear_output()
../../../_images/7e397d3fb7470fa31aee1b52ab167d2afdb4aaa636dfabea30a56e982a7f8590.png ../../../_images/0e8f38ca4f1af66c3a00d42a533e6569b2b4992cfbf09ce072a1e08efeba6b8a.png

4. Combine the Images using AstroDrizzle#

Table of Contents

While the three sets of FLC files are now aligned, in this example we drizzle together only the two long exposures. When exposures are very different lengths, drizzling them together doesn’t work well when ‘EXP’ weighting is used. For objects that saturate in the long exposures, the problem occurs at the boundary where the signal transitions from only being present in short exposures near the core to being present at larger radii in the longer exposures. The result is a discontinuity in the PSF radial profile and a resulting flux that is too low in some regions. For photometry of saturated objects, the short exposures should be drizzled separately from the long exposures.

Next, we combine the images through drizzling, and retrieve some recommended values for this process from the MDRIZTAB reference file. The parameters in this file are different for each detector and are based on the number of input frames. These are a good starting point for drizzling and may be adjusted based on your science goals.

input_images = sorted(glob.glob('ibwb01x[rx]q_flc.fits'))

mdz = fits.getval(input_images[0], 'MDRIZTAB', ext=0).split('$')[1]

print('Searching for the MDRIZTAB file:', mdz)

get_mdriztab = os.system('crds sync --hst --files '+mdz+' --output-dir '+os.environ['iref'])
Searching for the MDRIZTAB file: ubi1853qi_mdz.fits
CRDS - INFO -  Symbolic context 'hst-operational' resolves to 'hst_1166.pmap'
CRDS - INFO -  Reorganizing 0 references from 'instrument' to 'flat'
CRDS - INFO -  Reorganizing from 'instrument' to 'flat' cache,  removing instrument directories.
CRDS - INFO -  Syncing explicitly listed files.
CRDS - INFO -  Fetching  ./crds_cache/references/hst/wfc3/ubi1853qi_mdz.fits       95.0 K bytes  (1 / 1 files) (0 / 95.0 K bytes)
CRDS - INFO -  0 errors
CRDS - INFO -  0 warnings
CRDS - INFO -  5 infos
def get_vals_from_mdriztab(input_images, 
                           kw_list=[
                               'driz_sep_bits', 
                               'combine_type', 
                               'driz_cr_snr', 
                               'driz_cr_scale', 
                               'final_bits']):
    
    'Get only selected parameters from the MDRIZTAB.'
    mdriz_dict = getMdriztabPars(input_images)
    
    requested_params = {}
    
    print('Outputting the following parameters:')
    for k in kw_list:
        requested_params[k] = mdriz_dict[k]
        print(k, mdriz_dict[k])
    
    return requested_params
selected_params = get_vals_from_mdriztab(input_images)
- MDRIZTAB: AstroDrizzle parameters read from row 2.
Outputting the following parameters:
driz_sep_bits 96
combine_type minmed
driz_cr_snr 3.5 3.0
driz_cr_scale 1.2 0.7
final_bits 96

Note that the parameter final_bits= ‘96’ is equivalent to final_bits= ‘64, 32’ because they are added in a bit-wise manner.

# To override any of the above values:
selected_params['driz_sep_bits'] = '256, 64, 32'
selected_params['final_bits'] = '256, 64, 32'
# selected_params['combine_type']  = 'median'
# selected_params['driz_cr_snr']   = '4.0 3.5'
# selected_params['driz_cr_scale'] = '1.5 1.2'

astrodrizzle.AstroDrizzle(input_images, 
                          output='f606w',
                          preserve=False,
                          clean=True, 
                          build=False,
                          context=False,
                          skymethod='match',
                          **selected_params)

clear_output()

Display the combined science and weight images. In this example there appear to be three detector chips due to the large dither that was used in the observations. We note that the exposures overlapped in the center region, as evidenced by the increased value of the weight map corresponding to a longer effective exposure time. In addition, the resulting mosaic is better cleaned of artifacts and cosmic rays in this region due to the multiple exposures, as seen by the uniform background in the science mosaic.

sci = fits.getdata('f606w_drc_sci.fits')
wht = fits.getdata('f606w_drc_wht.fits')

fig = plt.figure(figsize=(20, 20))
ax1 = fig.add_subplot(1, 2, 1)
ax2 = fig.add_subplot(1, 2, 2)

ax1.imshow(sci, vmin=-0.05, vmax=0.4, cmap='Greys_r', origin='lower')
ax2.imshow(wht, vmin=0.0, vmax=1000, cmap='Greys_r', origin='lower')

ax1.title.set_text('Science Image')
ax2.title.set_text('Weight Image')
ax1.set_xlabel('X (pixels)')
ax1.set_ylabel('Y (pixels)')
ax2.set_xlabel('X (pixels)')
ax2.set_ylabel('Y (pixels)')
Text(0, 0.5, 'Y (pixels)')
../../../_images/edf38a45927e6524cb5852e957baa37e73298f02c98ad360b38c0cdae858309b.png

For more details on inspecting the drizzled products after reprocessing, see Section 7.3 in the DrizzlePac Handbook.

Conclusions#

Table of Contents

Many other services have interfaces for querying catalogs that can also be used to align HST images. In general, Gaia works very well for HST due to its high precision, but can have a low number of sources in some regions, especially at high galactic latitudes. Aligning images to an absolute frame provides a way to make data comparable across many epochs/detectors/observatories, and in many cases, makes the alignment easier to complete.

About this Notebook#

Created: 14 Dec 2018;     V. Bajaj
Updated: 03 Jul 2024;     M. Revalski, V. Bajaj, & J. Mack

Source: GitHub spacetelescope/hst_notebooks

Additional Resources#

Below are some additional resources that may be helpful. Please send any questions through the HST Help Desk, selecting the DrizzlePac category.

Citations#

If you use Python packages such as astropy, astroquery, drizzlepac, matplotlib, or numpy for published research, please cite the authors.

Follow these links for more information about citing various packages:


Top of Page Space Telescope Logo