Aligning HST images to an Absolute Reference Catalog

This notebook requires creating and activating a virtual environment using the requirements file in this notebook's repository. Please also review the README file before using the notebook.

Table of Contents

Introduction
Import Packages

1. Download the Observations with astroquery
    1.1 Check image header data
    1.2 Inspect the alignment

2. Query the Reference Catalogs
    2.1 Identify Coordinates
    2.2 SDSS Query
    2.3 Gaia Query

3. Align images with TweakReg
    3.1 SDSS Alignment
    3.2 Inspect the shift file and fit residuals
    3.3 Gaia Alignment
    3.4 Inspect the shift file and fit residuals
    3.5 Overplot Matched Sources on the image
    3.6 Update header with optimal parameters

4. Combine the images using AstroDrizzle

Conclusions
About this Notebook

Introduction

The alignment of HST exposures is a critical step in image stacking or combination performed with software such as AstroDrizzle. Generally, a relative alignment is performed to align one or multiple images to another image that is designated as the reference image. The reference image is generally the deepest exposure and/or that covering the largest area of all the exposures. This process aligns the images to each other, but the pointing error of the observatory can still cause the images to have incorrect absolute astrometry. When absolute astrometry is desired, the images can be aligned to an external catalog with an absolute world coordinate system (WCS). In this example, we will provide a workflow to query catalogs such as SDSS and Gaia using the astroquery package, and then align the images to that catalog using TweakReg.

The workflow in this notebook for aligning images to Gaia is based on WFC3 ISR 2017-19: Aligning HST Images to Gaia: a Faster Mosaicking Workflow and contains a subset of the information and code found in this repository. For more information, see the notebook in that repository titled Gaia_alignment.ipynb.

For more information about TweakReg, see the other notebooks in this repository or the TweakReg Documentation.

For more information on Astroquery, see the other notebooks in this repository or the Astroquery Documentation.

Import Packages

Table of Contents

---

The following Python packages are required to run the Jupyter Notebook:

• os - change and make directories

• glob - gather lists of filenames

• shutil - remove directories and files

• numpy - math and array functions

• matplotlib - make figures and graphics

• astropy - file handling, tables, units, WCS, statistics

• astroquery - download data and query databases

• drizzlepac - align and combine HST images

import os
import glob
import shutil
import numpy as np
import matplotlib.pyplot as plt
from collections import defaultdict
from IPython.display import Image, clear_output
import astropy.units as u
from astropy.io import ascii, fits
from astropy.table import Table
from astropy.units import Quantity
from astropy.coordinates import SkyCoord
from astropy.visualization import ZScaleInterval
from astroquery.mast import Observations
from astroquery.sdss import SDSS
from astroquery.gaia import Gaia
from drizzlepac import tweakreg, astrodrizzle
from drizzlepac.processInput import getMdriztabPars

%config InlineBackend.figure_format = 'retina' # Improves the resolution of figures rendered in notebooks.
Gaia.MAIN_GAIA_TABLE = 'gaiadr3.gaia_source' # Change this to the desired Gaia data release.
Gaia.ROW_LIMIT = 100000 # Show all of the matched sources in the printed tables.

# Set the locations of reference files and retrieve the MDRIZTAB recommended drizzle parameters.
os.environ['CRDS_SERVER_URL'] = 'https://hst-crds.stsci.edu'
os.environ['CRDS_SERVER'] = 'https://hst-crds.stsci.edu'
os.environ['CRDS_PATH'] = './crds_cache'
os.environ['iref'] = './crds_cache/references/hst/wfc3/'

The following task in the stsci.skypac package can be run with TEAL:
                                    skymatch                                    

The following tasks in the drizzlepac package can be run with TEAL:
    astrodrizzle       config_testbed      imagefindpars           mapreg       
       photeq            pixreplace           pixtopix            pixtosky      
  refimagefindpars       resetbits          runastrodriz          skytopix      
     tweakback            tweakreg           updatenpol

1. Download the Observations with astroquery

Table of Contents

For this notebook, we will download WFC3/UVIS images of NGC 6791 from program 12692 in the F606W filter acquired in Visit 01. The three FLC images have been processed by the HST WFC3 pipeline (calwf3), which includes bias subtraction, dark current and CTE correction, cosmic-ray rejection, and flat-fielding:

        ibwb01xqq_flc.fits
        ibwb01xrq_flc.fits
        ibwb01xxq_flc.fits

---

MAST queries may be done using query_criteria, where we specify:

    \(\rightarrow\) obs_id, proposal_id, and filters

MAST data products may be downloaded by using download_products, where we specify:

    \(\rightarrow\) products = calibrated (FLT, FLC) or drizzled (DRZ, DRC) files

    \(\rightarrow\) type = standard products (CALxxx) or advanced products (HAP-SVM)

Depending on your connection speed this cell may take a few minutes to execute.

obs_ids = ['ibwb01*']
props = ['12692']
filts = ['F606W']

obsTable = Observations.query_criteria(obs_id=obs_ids, proposal_id=props, filters=filts)
products = Observations.get_product_list(obsTable)

data_prod = ['FLC']    # ['FLC','FLT','DRC','DRZ']                  
data_type = ['CALWF3'] # ['CALACS','CALWF3','CALWP2','HAP-SVM']    

Observations.download_products(products, productSubGroupDescription=data_prod, project=data_type, cache=True)

Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/ibwb01xqq_flc.fits to ./mastDownload/HST/ibwb01xqq/ibwb01xqq_flc.fits ...

 [Done]

Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/ibwb01xrq_flc.fits to ./mastDownload/HST/ibwb01xrq/ibwb01xrq_flc.fits ...

 [Done]

Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:HST/product/ibwb01xxq_flc.fits to ./mastDownload/HST/ibwb01xxq/ibwb01xxq_flc.fits ...

 [Done]

Table length=3

Local Path	Status	Message	URL
str47	str8	object	object
./mastDownload/HST/ibwb01xqq/ibwb01xqq_flc.fits	COMPLETE	None	None
./mastDownload/HST/ibwb01xrq/ibwb01xrq_flc.fits	COMPLETE	None	None
./mastDownload/HST/ibwb01xxq/ibwb01xxq_flc.fits	COMPLETE	None	None

Move to the files to the local working directory:

for flc in glob.glob('./mastDownload/HST/*/*flc.fits'):
    flc_name = os.path.basename(flc)
    os.rename(flc, flc_name)
    
if os.path.exists('mastDownload'):
    shutil.rmtree('mastDownload')

1.1 Check image header data

The cell below retrieves values for specific keywords from the first and second extensions of the image header. We see that the 1st exposure is 30 seconds and the 2nd and 3rd exposures are 360 seconds in duration. The 3rd exposure is dithered by approximately 82 arcseconds in the y-direction which is approximately the width of one WFC3 UVIS detector chip. A full list of header keywords is available in Section 2.4 of the WFC3 Data Handbook.

paths = sorted(glob.glob('*flc.fits'))
data = []
keywords_ext0 = ["ROOTNAME", "ASN_ID", "TARGNAME", "DETECTOR", "FILTER", "exptime", 
                 "ra_targ", "dec_targ", "postarg1", "postarg2", "DATE-OBS"]
keywords_ext1 = ["orientat"]

for path in paths:
    path_data = []
    for keyword in keywords_ext0:
        path_data.append(fits.getval(path, keyword, ext=0))
    for keyword in keywords_ext1:
        path_data.append(fits.getval(path, keyword, ext=1))
    data.append(path_data)
    
keywords = keywords_ext0 + keywords_ext1
table = Table(np.array(data), names=keywords, dtype=['str', 'str', 'str', 'str', 'str', 'f8', 'f8', 'f8', 'f8', 'f8', 'str', 'f8'])
table['exptime'].format = '7.1f'
table['ra_targ'].format = table['dec_targ'].format = '7.4f'
table['postarg1'].format = table['postarg2'].format = '7.3f'
table['orientat'].format = '7.2f'
table

Table length=3

ROOTNAME	ASN_ID	TARGNAME	DETECTOR	FILTER	exptime	ra_targ	dec_targ	postarg1	postarg2	DATE-OBS	orientat
str32	str32	str32	str32	str32	float64	float64	float64	float64	float64	str32	float64
ibwb01xqq	NONE	NGC-6791	UVIS	F606W	30.0	290.2248	37.8027	0.000	0.000	2011-10-20	-56.27
ibwb01xrq	NONE	NGC-6791	UVIS	F606W	360.0	290.2248	37.8027	0.000	0.000	2011-10-20	-56.27
ibwb01xxq	NONE	NGC-6791	UVIS	F606W	360.0	290.2248	37.8027	0.000	81.600	2011-10-20	-56.25

1.2 Inspect the Alignment

Check the active WCS solution in the image header. If the image is aligned to a catalog, the list the number of matches and the fit RMS in milli-arcseconds. Next, convert the fit RMS values to pixels for comparison with the alignment results performed later in this notebook using Tweakreg.

ext_0_keywords = ['DETECTOR', 'EXPTIME'] # extension 0 keywords.
ext_1_keywords = ['WCSNAME', 'NMATCHES', 'RMS_RA', 'RMS_DEC'] # extension 1 keywords.

# Define the detector plate scales in arcsec per pixel.
DETECTOR_SCALES = {
  'IR': 0.1283, 
  'UVIS': 0.0396, 
  'WFC': 0.05
}

formatted_data = {}
column_data = defaultdict(list)

for fits_file in sorted(glob.glob('*flc.fits')):
    column_data['filename'].append(fits_file)
    header0 = fits.getheader(fits_file, 0)
    header1 = fits.getheader(fits_file, 1)
    
    for keyword in ext_0_keywords:
        column_data[keyword].append(header0[keyword])
    for keyword in ext_1_keywords:
        if 'RMS' in keyword:
            value = np.around(header1[keyword], decimals=1)
        else:
            value = header1[keyword]
        column_data[keyword].append(value)
            
    for keyword in ['RMS_RA', 'RMS_DEC']:
        rms_value = header1[keyword] / 1000 / DETECTOR_SCALES[header0['DETECTOR']]
        column_data[f'{keyword}_pix'].append(np.round(rms_value, decimals=2))

wcstable = Table(column_data)
wcstable

Table length=3

filename	DETECTOR	EXPTIME	WCSNAME	NMATCHES	RMS_RA	RMS_DEC	RMS_RA_pix	RMS_DEC_pix
str18	str4	float64	str30	int64	float64	float64	float64	float64
ibwb01xqq_flc.fits	UVIS	30.0	IDC_2731450pi-FIT_IMG_GAIAeDR3	442	1.3	1.5	0.03	0.04
ibwb01xrq_flc.fits	UVIS	360.0	IDC_2731450pi-FIT_IMG_GAIAeDR3	462	4.0	3.8	0.1	0.1
ibwb01xxq_flc.fits	UVIS	360.0	IDC_2731450pi-FIT_IMG_GAIAeDR3	429	3.9	3.7	0.1	0.09

2. Query the Reference Catalogs

Table of Contents

Now that we have the FITS images, we will download the reference catalogs from both SDSS and Gaia using astroquery.

2.1 Identify Coordinates

We will first create a SkyCoord Object to point astroquery to where we are looking on the sky. Since our example uses data from NGC 6791, we will use the ra_targ and dec_targ keywords from the first image to get the coordinates of the object.

RA = table['ra_targ'][0]
Dec = table['dec_targ'][0]

2.2 SDSS Query

We now give the RA and Dec values to an astropy SkyCoord object, which we will pass to the SDSS query. Additionally, we use an astropy Quantity object to create a radius for the SDSS query in physical units. We set the radius to 5 arcminutes to comfortably cover the area of our images. For reference, the UVIS detector field of view is approximately 2.7’\(\times\)2.7’ and a y-dither of 82 arcseconds covers a total area on the sky of approximately 2.7’\(\times\)4.1’.

coord = SkyCoord(ra=RA, dec=Dec, unit=(u.deg, u.deg))
radius = Quantity(3., u.arcmin)

Next, we perform the query via the SDSS.query_region method of astroquery.sdss. The spectro=False keyword argument means we want to exclude spectroscopic objects, as we are looking for objects to match with an image. In the fields parameter, we specify a list of fields we want returned by the query. In this case we only need the position, as well as a g-band magnitude if we want to remove very dim and/or bright objects from the catalog, as those are likely measured poorly. Details on selecting objects by magnitude may be found in the ‘Gaia_alignment’ notebook. Many other fields are available in the SDSS query and are documented on this webpage.

sdss_query = SDSS.query_region(coord, radius=radius, spectro=False, fields=['ra', 'dec', 'g'])
sdss_query

Table length=2140

ra	dec	g
float64	float64	float64
290.175257483306	37.7757023742629	19.10387
290.175260270816	37.8293581828758	-9999.0
290.175268441296	37.7757076932531	18.98614
290.175307121632	37.7548760313867	19.49425
290.175317556537	37.8293308028686	18.26527
290.175318784872	37.8293349727083	-9999.0
290.175320686698	37.7548746619173	19.7736
290.175328125755	37.7625936217729	19.44313
290.175333152015	37.8293270812477	18.38934
...	...	...
290.274703632708	37.7877877835155	18.8502
290.274704603188	37.7891141064558	18.63407
290.274721361389	37.7877737666928	19.01597
290.27472930088	37.7890657093451	18.74797
290.274794783948	37.7993797493131	16.67577
290.274800891914	37.7993938284574	16.11324
290.274807588715	37.799392577291	16.08992
290.274815152976	37.7993736308255	16.13595
290.27482391543	37.7993757145458	16.16263
290.274839705192	37.8423495653678	20.26493

We then need to save the catalog to use with TweakReg. As the returned value of the query is an astropy.Table, saving the file is straightforward:

sdss_query.write('sdss.cat', format='ascii.commented_header', overwrite=True)

2.3 Gaia Query

Similarly to SDSS, we can query Gaia catalogs for our target via astroquery.gaia. This may be preferable in many cases because the spaced-based astrometry from Gaia is generally very accurate and precise. We can use the same coord and radius search parameters from our earlier SDSS query.

gaia_query = Gaia.query_object_async(coordinate=coord, radius=radius)
gaia_query

INFO: Query finished. [astroquery.utils.tap.core]

Table length=2331

solution_id	DESIGNATION	SOURCE_ID	random_index	ref_epoch	ra	ra_error	dec	dec_error	parallax	parallax_error	parallax_over_error	pm	pmra	pmra_error	pmdec	pmdec_error	ra_dec_corr	ra_parallax_corr	ra_pmra_corr	ra_pmdec_corr	dec_parallax_corr	dec_pmra_corr	dec_pmdec_corr	parallax_pmra_corr	parallax_pmdec_corr	pmra_pmdec_corr	astrometric_n_obs_al	astrometric_n_obs_ac	astrometric_n_good_obs_al	astrometric_n_bad_obs_al	astrometric_gof_al	astrometric_chi2_al	astrometric_excess_noise	astrometric_excess_noise_sig	astrometric_params_solved	astrometric_primary_flag	nu_eff_used_in_astrometry	pseudocolour	pseudocolour_error	ra_pseudocolour_corr	dec_pseudocolour_corr	parallax_pseudocolour_corr	pmra_pseudocolour_corr	pmdec_pseudocolour_corr	astrometric_matched_transits	visibility_periods_used	astrometric_sigma5d_max	matched_transits	new_matched_transits	matched_transits_removed	ipd_gof_harmonic_amplitude	ipd_gof_harmonic_phase	ipd_frac_multi_peak	ipd_frac_odd_win	ruwe	scan_direction_strength_k1	scan_direction_strength_k2	scan_direction_strength_k3	scan_direction_strength_k4	scan_direction_mean_k1	scan_direction_mean_k2	scan_direction_mean_k3	scan_direction_mean_k4	duplicated_source	phot_g_n_obs	phot_g_mean_flux	phot_g_mean_flux_error	phot_g_mean_flux_over_error	phot_g_mean_mag	phot_bp_n_obs	phot_bp_mean_flux	phot_bp_mean_flux_error	phot_bp_mean_flux_over_error	phot_bp_mean_mag	phot_rp_n_obs	phot_rp_mean_flux	phot_rp_mean_flux_error	phot_rp_mean_flux_over_error	phot_rp_mean_mag	phot_bp_rp_excess_factor	phot_bp_n_contaminated_transits	phot_bp_n_blended_transits	phot_rp_n_contaminated_transits	phot_rp_n_blended_transits	phot_proc_mode	bp_rp	bp_g	g_rp	radial_velocity	radial_velocity_error	rv_method_used	rv_nb_transits	rv_nb_deblended_transits	rv_visibility_periods_used	rv_expected_sig_to_noise	rv_renormalised_gof	rv_chisq_pvalue	rv_time_duration	rv_amplitude_robust	rv_template_teff	rv_template_logg	rv_template_fe_h	rv_atm_param_origin	vbroad	vbroad_error	vbroad_nb_transits	grvs_mag	grvs_mag_error	grvs_mag_nb_transits	rvs_spec_sig_to_noise	phot_variable_flag	l	b	ecl_lon	ecl_lat	in_qso_candidates	in_galaxy_candidates	non_single_star	has_xp_continuous	has_xp_sampled	has_rvs	has_epoch_photometry	has_epoch_rv	has_mcmc_gspphot	has_mcmc_msc	in_andromeda_survey	classprob_dsc_combmod_quasar	classprob_dsc_combmod_galaxy	classprob_dsc_combmod_star	teff_gspphot	teff_gspphot_lower	teff_gspphot_upper	logg_gspphot	logg_gspphot_lower	logg_gspphot_upper	mh_gspphot	mh_gspphot_lower	mh_gspphot_upper	distance_gspphot	distance_gspphot_lower	distance_gspphot_upper	azero_gspphot	azero_gspphot_lower	azero_gspphot_upper	ag_gspphot	ag_gspphot_lower	ag_gspphot_upper	ebpminrp_gspphot	ebpminrp_gspphot_lower	ebpminrp_gspphot_upper	libname_gspphot	dist
				yr	deg	mas	deg	mas	mas	mas		mas / yr	mas / yr	mas / yr	mas / yr	mas / yr																	mas				1 / um	1 / um	1 / um								mas					deg								deg	deg	deg	deg			electron / s	electron / s		mag		electron / s	electron / s		mag		electron / s	electron / s		mag							mag	mag	mag	km / s	km / s								d	km / s	K	log(cm.s**-2)	dex		km / s	km / s		mag	mag				deg	deg	deg	deg															K	K	K	log(cm.s**-2)	log(cm.s**-2)	log(cm.s**-2)	dex	dex	dex	pc	pc	pc	mag	mag	mag	mag	mag	mag	mag	mag	mag
int64	object	int64	int64	float64	float64	float32	float64	float32	float64	float32	float32	float32	float64	float32	float64	float32	float32	float32	float32	float32	float32	float32	float32	float32	float32	float32	int16	int16	int16	int16	float32	float32	float32	float32	int16	bool	float32	float32	float32	float32	float32	float32	float32	float32	int16	int16	float32	int16	int16	int16	float32	float32	int16	int16	float32	float32	float32	float32	float32	float32	float32	float32	float32	bool	int16	float64	float32	float32	float32	int16	float64	float32	float32	float32	int16	float64	float32	float32	float32	float32	int16	int16	int16	int16	int16	float32	float32	float32	float32	float32	int16	int16	int16	int16	float32	float32	float32	float32	float32	float32	float32	float32	int16	float32	float32	int16	float32	float32	int16	float32	object	float64	float64	float64	float64	bool	bool	int16	bool	bool	bool	bool	bool	bool	bool	bool	float32	float32	float32	float32	float32	float32	float32	float32	float32	float32	float32	float32	float32	float32	float32	float32	float32	float32	float32	float32	float32	float32	float32	float32	object	float64
1636148068921376768	Gaia DR3 2051293319932402944	2051293319932402944	1710414812	2016.0	290.2248611356283	0.12343551	37.80192950361359	0.15534069	1.2409154633064086	0.15976872	7.7669487	6.0644784	-0.6226543407618474	0.15474372	6.0324290792879545	0.18924683	-0.033001468	0.16417459	-0.027278472	-0.020170886	0.07942047	-0.046242356	-0.19703287	-0.07685185	0.0051713893	0.050814763	394	0	389	5	0.407836	413.48303	0.0	2.0511602e-15	31	False	1.3016587	--	--	--	--	--	--	--	45	26	0.26493552	49	22	0	0.01895016	14.319122	0	0	1.0138804	0.17569481	0.20240329	0.10288032	0.089386284	-95.13182	77.09379	2.6807811	-30.180996	False	385	492.16788454743386	0.9903239	496.97668	18.957083	39	106.26453488321661	5.3855076	19.731573	20.272572	39	549.1994295106254	7.0852404	77.51317	17.898571	1.3317894	0	2	0	2	0	2.3740005	1.3154888	1.0585117	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	NOT_AVAILABLE	69.98761891051129	10.914199739506339	302.04196574777654	59.01190915812673	False	False	0	False	False	False	False	False	True	False	False	1.4003378e-13	5.356044e-13	0.99996775	3653.592	3591.5752	3744.3992	4.8552	4.4992	4.9965	-0.4967	-0.7199	-0.1315	653.0443	491.9481	1283.5112	0.7812	0.6499	0.9202	0.5305	0.44	0.6266	0.3102	0.2581	0.3649	MARCS	0.0007449136479797354
1636148068921376768	Gaia DR3 2051293324218263808	2051293324218263808	214974978	2016.0	290.22327972754084	1.0491472	37.80283944795903	1.4068799	-0.6069077460064267	1.4172788	-0.42822045	3.8347042	-0.49694994391852737	1.433404	-3.802367186693702	2.183717	-0.32559526	0.44083008	0.040013984	-0.36232764	-0.18673699	-0.29855755	0.5145219	0.16053891	-0.3253903	-0.32209665	133	0	132	1	2.3624072	187.53317	3.4946198	1.7356544	95	False	--	1.2583821	0.3532982	-0.10796609	-0.06725511	-0.030691369	-0.22952616	-0.12757729	16	14	3.2688706	18	6	1	0.061227437	43.226418	5	17	1.1496774	0.23548113	0.3243092	0.35748923	0.4221966	-95.12672	74.99657	-8.526287	-16.614477	False	132	88.7375766294378	1.1992922	73.99162	20.817099	0	--	--	--	--	0	--	--	--	--	--	--	--	--	--	0	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	NOT_AVAILABLE	69.9878943903652	10.915721621139165	302.0400990046131	59.01311977354264	False	False	0	False	False	False	False	False	False	False	False	7.759076e-08	3.7895185e-09	0.9999943	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--		0.0012108975043825237
1636148068921376768	Gaia DR3 2051293324222972288	2051293324222972288	164456380	2016.0	290.22261994880597	0.032841645	37.80270752474709	0.039459	0.3053094896365258	0.041127697	7.423452	2.5961652	-0.7012623275324616	0.041221105	-2.4996610433857405	0.047550496	-0.047555298	0.16546503	0.021742897	-0.077867456	0.066915035	-0.08765522	-0.26260835	-0.059347805	-0.087691456	-0.0013310307	383	0	381	2	-2.1703348	333.38327	0.0	0.0	31	False	1.4413656	--	--	--	--	--	--	--	44	25	0.06704042	48	21	0	0.015037518	12.04822	0	0	0.9210467	0.13890518	0.18901813	0.10900513	0.0641752	-88.36165	72.302956	5.923747	-33.787945	False	381	4179.254659077206	2.2517574	1855.9968	16.63462	43	1717.7830387470858	6.2028065	276.93643	17.251122	39	3494.1204079650242	10.008291	349.1226	15.889551	1.2470893	0	25	0	23	0	1.3615704	0.61650085	0.7450695	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	NOT_AVAILABLE	69.98754100993882	10.916132432650413	302.0390547900942	59.01313187059839	False	False	0	True	False	False	False	False	True	True	False	1.0228448e-13	9.3310026e-11	0.9999741	4770.3447	4749.6045	4790.818	4.1776	4.1596	4.1902	0.3546	0.3268	0.3805	2185.9316	2147.3625	2239.1926	0.3511	0.3259	0.3735	0.2701	0.2505	0.2875	0.1406	0.1303	0.1496	PHOENIX	0.001721026645501702
1636148068921376768	Gaia DR3 2051293324222972416	2051293324222972416	1599905588	2016.0	290.2248309701555	0.052941296	37.80058611405895	0.064896055	0.21837043824644525	0.067057766	3.2564526	2.3448853	-0.5827200539782851	0.066188715	-2.2713266001611636	0.0787358	-0.076830104	0.14136231	-0.025055349	-0.07297541	0.056257915	-0.07904705	-0.24559797	-0.07446872	-0.087429665	-0.03352803	395	0	392	3	-0.037620842	401.51434	0.0	0.0	31	False	1.4891585	--	--	--	--	--	--	--	45	25	0.110738866	49	22	0	0.0153436065	15.8542	0	0	0.99778724	0.15293275	0.21477312	0.12928022	0.08397196	-94.649994	70.08022	6.0968003	-29.722376	False	389	1751.0366076397277	1.4950225	1171.2443	17.579128	46	817.0065234341104	6.4622293	126.42797	18.057978	44	1347.155230450357	8.58573	156.90633	16.92435	1.2359318	0	32	0	29	0	1.1336269	0.4788494	0.6547775	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	NOT_AVAILABLE	69.98637845348246	10.913632562752337	302.0412242635996	59.01062093567249	False	False	0	True	False	False	False	False	True	True	False	6.110852e-11	5.145215e-13	0.9999132	4831.901	4812.959	4859.14	4.661	4.6412	4.6768	-0.4447	-0.5218	-0.3786	1595.5184	1545.6223	1641.4886	0.0087	0.0021	0.021	0.0069	0.0017	0.0166	0.0036	0.0009	0.0088	PHOENIX	0.0020867793124382814
1636148068921376768	Gaia DR3 2051293319934791936	2051293319934791936	127740795	2016.0	290.2225602638809	0.098567784	37.80397847004457	0.12238157	0.18608923760796978	0.12619987	1.4745597	2.3556678	-0.20591766034379622	0.12384657	-2.346650638048337	0.14836667	-0.0034372064	0.17569284	-0.028029243	-0.023688797	0.0831415	-0.04713238	-0.21456409	-0.08000817	-0.019006517	0.07798844	386	0	383	3	0.34466058	403.29874	0.0	0.0	31	False	1.4745288	--	--	--	--	--	--	--	44	26	0.20897423	48	21	0	0.02525984	16.266506	0	0	1.011676	0.16451268	0.180577	0.079406776	0.08114059	-89.370926	80.65579	0.7836158	-30.156015	False	377	665.1667185914472	1.124127	591.71844	18.630041	40	296.01604570644196	5.8455243	50.63978	19.160254	41	517.3148686523093	6.1276155	84.42352	17.963509	1.2227474	0	0	0	0	0	1.1967449	0.5302124	0.6665325	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	NOT_AVAILABLE	69.98868349525281	10.916731652994313	302.03962583571473	59.014369244472704	False	False	0	False	False	False	False	False	True	False	False	1.0210282e-13	5.259736e-13	0.99999475	4590.2783	4575.766	4603.0244	4.8255	4.7981	4.858	-1.2317	-1.4162	-1.1009	1664.2876	1569.1196	1730.915	0.0102	0.0026	0.0257	0.008	0.002	0.02	0.0043	0.0011	0.0107	PHOENIX	0.0021977590777958536
1636148068921376768	Gaia DR3 2051293324216543232	2051293324216543232	496974628	2016.0	290.22681959687196	0.5800434	37.80112772770517	0.8518114	0.9866119688573637	0.8487393	1.162444	4.5651155	-0.44243766262405015	1.042491	-4.543625170019005	1.2715639	-0.03034182	-0.13035429	0.09905615	0.024711896	-0.035745695	-0.13054979	0.34666777	-0.069062054	-0.28526154	-0.1078611	189	0	189	0	-0.84486425	177.7449	0.0	1.4856265e-15	95	False	--	1.5521348	0.21582161	-0.0062133768	0.2804249	0.035700142	-0.42101106	0.17818592	22	15	1.8150634	25	8	0	0.023252329	30.515776	0	0	0.954371	0.20877054	0.28841907	0.1927084	0.43569437	-76.9166	84.362335	-3.7503886	-19.752798	False	189	100.17345952653844	0.81711596	122.59393	20.685486	16	34.777743067191494	8.345614	4.1671877	21.485289	19	89.51229139692614	5.5230875	16.20693	19.868189	1.2407482	0	0	0	0	0	1.6170998	0.7998028	0.817297	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	NOT_AVAILABLE	69.98757543345182	10.912457430119012	302.0444461542871	59.01072318960171	False	False	0	False	False	False	False	False	False	False	False	1.5372552e-09	1.1275478e-09	0.99999505	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--		0.002222460820739603
1636148068921376768	Gaia DR3 2051293324222974464	2051293324222974464	1798436105	2016.0	290.2275249308142	0.24079925	37.803472775139035	0.3074705	0.44506652441996963	0.3134235	1.4200164	1.903844	-0.7741499247735619	0.3071843	-1.7393429825276292	0.37454808	0.020487515	0.17073557	-0.036337543	0.041650914	0.12431544	-0.0061035734	-0.18864702	-0.06549308	0.060529068	0.123350315	382	0	380	2	0.5945536	412.03198	0.9015461	0.9578774	95	False	--	1.3147619	0.081987195	-0.016766502	0.118190974	0.037860535	-0.0898746	-0.10861043	44	25	0.52942884	48	21	0	0.027762208	20.245789	0	4	1.02092	0.18400808	0.1789637	0.073698275	0.10250232	-90.673706	85.22672	-9.854181	-30.444962	False	380	207.20873617672416	0.7419642	279.2705	19.896347	9	91.39950015947561	7.5014834	12.18419	20.436182	9	224.20190021064204	11.064253	20.263628	18.871298	1.5231086	0	0	0	0	0	1.5648842	0.539835	1.0250492	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	NOT_AVAILABLE	69.98997089407423	10.912983986333725	302.04670590522085	59.01283416072846	False	False	0	False	False	False	False	False	False	False	False	1.8206678e-09	2.9034798e-08	0.99998796	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--		0.002298594469020544
1636148068921376768	Gaia DR3 2051293324216539648	2051293324216539648	1649457609	2016.0	290.2242320409821	0.5271274	37.80039510739463	0.76655334	1.3632756584241434	0.6728546	2.0261073	2.2253373	-0.9708645882906146	0.6578662	-2.0023856302519407	0.98862535	-0.33465353	0.08556003	-0.2735162	0.2602703	0.161614	0.21899539	-0.26542807	0.033500012	0.1458301	-0.23663591	243	0	243	0	1.2024069	273.66382	0.0	0.0	95	False	--	1.6818463	0.19114311	-0.26075235	0.11348202	-0.08333635	-0.06321393	-0.15212661	28	18	1.4304484	32	14	1	0.0707902	34.375954	0	15	1.0543019	0.24560685	0.4438982	0.22968337	0.165783	-110.25776	67.43046	5.994361	-23.29821	False	244	109.8552156027335	0.8030061	136.80496	20.585316	0	--	--	--	--	0	--	--	--	--	--	--	--	--	--	2	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	NOT_AVAILABLE	69.98599242480627	10.91397427439119	302.0402394379523	59.01056325889003	False	False	0	False	False	False	False	False	False	False	False	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--		0.0023210548670645637
1636148068921376768	Gaia DR3 2051293324225336064	2051293324225336064	398090960	2016.0	290.22302939666463	0.47650018	37.8003404115494	0.57163584	1.0465539617532613	0.60573983	1.7277284	0.3426481	0.016525679687419553	0.6219764	-0.34224935447443744	0.7205138	-0.11358616	0.1529165	0.07183364	-0.104466826	0.054135017	-0.07860708	-0.1758327	0.10776901	-0.16631071	-0.16472225	373	0	365	8	86.80703	12487.478	6.598426	424.3264	95	False	--	1.2976544	0.14454299	-0.057665557	0.052674405	0.033202734	0.010857275	-0.07922697	43	25	1.0361372	50	50	0	0.19814193	21.407808	63	0	5.6149945	0.1693135	0.21580335	0.10183408	0.09376588	-92.04377	70.39044	0.82470447	-23.783632	False	298	836.206629484886	5.043918	165.78514	18.381582	43	492.85805036589596	5.0838623	96.945595	18.606737	41	916.8429900655037	6.7629867	135.56776	17.342157	1.6858286	0	2	0	2	0	1.2645798	0.22515488	1.0394249	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	NOT_AVAILABLE	69.98551833187275	10.914804515213548	302.0384324817961	59.01076433315906	False	False	0	False	False	False	False	False	False	False	False	9.514203e-11	1.056036e-05	0.99998164	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--		0.0027187891427339723
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
1636148068921376768	Gaia DR3 2051288101537870720	2051288101537870720	1476096901	2016.0	290.1654951838555	0.04821452	37.78544763786987	0.057633974	0.22503724093996813	0.060696572	3.7075775	2.275607	-0.40649463749074166	0.060702868	-2.2390065331160773	0.06900224	0.012334322	0.13113154	-0.11899681	-0.019961989	0.14481837	-0.038124166	-0.29374352	-0.14935088	-0.11253025	0.07077164	419	0	419	0	-0.67899024	410.35037	0.0	0.0	31	False	1.504531	--	--	--	--	--	--	--	48	27	0.09853348	49	23	0	0.019995905	15.846279	0	0	0.9756978	0.15598494	0.12121681	0.09078951	0.13566518	-107.63154	79.53522	2.3636649	-34.418354	False	417	1993.7156577629107	1.4394238	1385.079	17.43821	40	950.0458805863858	6.864714	138.39555	17.89418	39	1436.0270711844762	5.677532	252.93156	16.854988	1.196797	0	0	0	1	0	1.0391922	0.45597076	0.58322144	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	NOT_AVAILABLE	69.95160438021598	10.949157081403966	301.94561639491445	59.0085358362026	False	False	0	True	False	False	False	False	True	True	False	1.0226445e-13	5.119075e-13	0.9999763	5141.5874	5123.6333	5158.2056	4.6326	4.6194	4.6433	-0.1554	-0.2038	-0.1121	1939.21	1896.9001	1979.4012	0.0037	0.0009	0.0107	0.003	0.0007	0.0086	0.0016	0.0003	0.0045	PHOENIX	0.04992738302391301
1636148068921376768	Gaia DR3 2051105376451505792	2051105376451505792	883322466	2016.0	290.25702221163755	0.18905094	37.75972204677077	0.2666551	-0.039959702467232905	0.25163692	-0.15879904	2.4071147	-0.49144532612992103	0.24496886	-2.3564132165553557	0.34456575	0.12697652	0.14626107	-0.19946665	-0.0483627	0.07724031	-0.09694064	-0.30264643	-0.12386921	0.12871955	0.22539671	336	0	334	2	0.6659974	366.43585	0.53870255	0.5023208	95	False	--	1.3167961	0.06423852	-0.042650696	0.016197763	0.06316402	-0.07971905	-0.026673468	40	24	0.49691302	44	21	0	0.04652881	11.265161	0	6	1.0250903	0.21310079	0.28677547	0.15462908	0.17873366	-106.79698	-85.40637	-17.572538	-26.361528	False	332	290.82182538107634	0.9811591	296.4064	19.5283	30	191.56420702782418	7.9658885	24.048065	19.632757	30	383.7427884655358	10.2831545	37.317616	18.287794	1.9782112	0	30	0	30	0	1.3449631	0.104457855	1.2405052	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	NOT_AVAILABLE	69.96033287236655	10.87285183333383	302.06762743476895	58.96444565627617	False	False	0	False	False	False	False	False	False	False	False	1.600451e-09	6.6841253e-06	0.9999911	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--		0.04993419538093678
1636148068921376768	Gaia DR3 2051105715754593536	2051105715754593536	367649089	2016.0	290.280804620236	0.14492457	37.779546916641095	0.17963423	0.21468976829977565	0.18964018	1.1320901	7.283765	-1.5459199132772414	0.18082064	-7.117820261813646	0.226899	0.0821018	0.19335385	-0.19755834	-0.0073253466	0.17681892	-0.059489112	-0.13821003	-0.10181402	0.08707409	0.12708417	339	0	338	1	0.13918142	351.19504	0.0	0.0	95	False	--	1.3965503	0.052453425	-0.21597628	0.046664923	-0.14694694	0.0073276316	-0.10843161	40	25	0.3193467	44	18	0	0.025737958	3.3489754	0	0	1.0044005	0.28954354	0.21789046	0.10168418	0.21477832	-101.08861	-86.16012	-9.762876	-25.57112	False	339	447.7702296111591	0.9872021	453.57504	19.059729	40	189.71597841117108	4.778628	39.70093	19.643282	40	375.7994248842881	5.9547777	63.10889	18.310505	1.2629589	0	2	0	1	0	1.332777	0.5835533	0.7492237	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	NOT_AVAILABLE	69.98687073758735	10.864652528833176	302.1130527278516	58.97852325474902	False	False	0	False	False	False	False	False	False	False	False	9.687291e-11	2.1935703e-10	0.9999868	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--		0.049937021935667826
1636148068921376768	Gaia DR3 2051105342091729152	2051105342091729152	987208462	2016.0	290.2498342848316	0.18311875	37.7568154080175	0.23919971	0.33264640961489067	0.24093883	1.380626	6.841415	-3.039507836669035	0.23630942	-6.1291394683796065	0.29052094	0.06648603	0.18299167	-0.13769636	-0.05914548	0.17252389	-0.109444715	-0.24172258	-0.12743072	0.02975334	0.1879744	378	0	377	1	0.6412554	409.2934	0.0	0.0	31	False	1.5193098	--	--	--	--	--	--	--	44	26	0.4198297	47	22	0	0.036963172	7.785712	0	0	1.0226985	0.20777619	0.22200799	0.12704432	0.11301937	-103.070724	-89.776115	-15.491943	-22.853712	False	364	274.33505719729413	0.83240443	329.56943	19.591663	20	108.42793269198275	8.1400175	13.320356	20.250689	22	229.26346946948766	7.0309343	32.607822	18.84706	1.2309451	0	0	0	0	0	1.4036293	0.6590252	0.7446041	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	NOT_AVAILABLE	69.95513531037714	10.876684519101982	302.055500002796	58.96316234659864	False	False	0	False	False	False	False	False	False	False	False	7.055214e-10	2.4849628e-10	0.9999974	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--		0.049944646819559776
1636148068921376768	Gaia DR3 2051294595528631808	2051294595528631808	132742956	2016.0	290.20159605323926	0.31878126	37.849139504585956	0.4108245	-0.12391565776920735	0.4221808	-0.29351324	3.416373	-1.2441356333735205	0.42176288	-3.1817810079389437	0.5004318	-0.050365023	-0.0008751949	-0.17113145	0.052433796	0.13154244	0.061325245	-0.26776782	0.0703754	-0.04012821	-0.025681598	349	0	347	2	2.601649	440.67697	1.641707	2.0908484	31	False	1.4571096	--	--	--	--	--	--	--	41	24	0.7057606	43	21	0	0.018866189	31.54997	0	0	1.1001017	0.2407212	0.19015062	0.13248968	0.1401775	-98.95326	79.32095	-10.734245	-22.811646	False	346	162.80630029410034	0.72370195	224.9632	20.15819	40	88.09942109357075	4.8307185	18.237333	20.47611	39	109.63341109058099	5.6637025	19.357199	19.648039	1.2145281	0	0	0	0	0	0.8280716	0.31792068	0.5101509	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	NOT_AVAILABLE	70.02264706458212	10.951397166233019	302.0320582619724	59.06231257269266	False	False	0	False	False	False	False	False	False	False	False	1.8466157e-08	7.57946e-10	0.99999225	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--		0.04995017327286985
1636148068921376768	Gaia DR3 2051293564734766208	2051293564734766208	1140836395	2016.0	290.284611215241	0.19846809	37.818928955959755	0.25827777	0.42437517750308906	0.26023453	1.6307412	2.672299	-0.36837091118114074	0.25271687	-2.646787449087477	0.31571978	0.022631811	0.07644129	-0.0386861	-0.027465278	0.08704854	-0.076738395	-0.2122061	-0.011657735	0.07329119	0.13596117	373	0	371	2	0.69725716	399.0092	0.50564337	0.4083656	95	False	--	1.4507254	0.06923322	-0.094798625	0.1284554	0.025951082	-0.15272623	-0.07501197	45	26	0.448276	49	21	0	0.0034285483	126.74221	0	0	1.0249964	0.2242597	0.1924402	0.0709199	0.12980221	-98.00924	89.90393	-16.552565	-28.187826	False	375	265.7468947091863	0.9801038	271.14157	19.626196	19	186.72408355470947	17.84828	10.461741	19.660542	17	322.6965427610224	18.00704	17.920576	18.47591	1.9169391	0	2	0	0	0	1.1846313	0.034345627	1.1502857	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	NOT_AVAILABLE	70.0242520847949	10.879224379025027	302.13914736517705	59.01566265745925	False	False	0	False	False	False	False	False	False	False	False	2.237545e-09	1.1975736e-05	0.999988	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--		0.04997304885611112
1636148068921376768	Gaia DR3 2051293560440219776	2051293560440219776	468406816	2016.0	290.2824764003652	0.0670952	37.82319670158857	0.08790515	0.19609972636397568	0.08809279	2.2260587	2.2592366	-0.3504044686282836	0.08420435	-2.2318976044885916	0.10411392	-0.0277822	0.11649344	-0.0013309642	-0.0026663947	0.041511085	-0.01673569	-0.2758432	-0.04604975	-0.034664087	0.025199417	401	0	400	1	-0.95538807	383.92657	0.0	0.0	31	False	1.495298	--	--	--	--	--	--	--	46	27	0.1470113	51	22	0	0.01771481	8.017093	0	0	0.965368	0.23983715	0.21379092	0.057209257	0.11334882	-95.99321	84.87185	-0.7996727	-28.648691	False	406	1125.789695176469	1.2112837	929.4187	18.058723	45	517.7398808963094	5.841485	88.63155	18.553263	44	810.5151900044447	7.3433576	110.373924	17.475992	1.179843	0	1	0	0	0	1.0772705	0.49453926	0.58273125	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	NOT_AVAILABLE	70.0274044867681	10.882611695924751	302.1382120297565	59.02022616662701	False	False	0	False	False	False	False	False	True	True	False	1.0224728e-13	5.244856e-13	0.999979	5021.658	4976.7427	5065.875	4.6656	4.6308	4.6852	-0.2925	-0.4153	-0.1754	2265.6306	2154.7927	2422.6208	0.0069	0.0017	0.0176	0.0055	0.0013	0.0141	0.0029	0.0007	0.0075	PHOENIX	0.04997598495418403
1636148068921376768	Gaia DR3 2051105887561931520	2051105887561931520	489297215	2016.0	290.28771982216256	0.17982018	37.79753057383014	0.22851922	-0.27637032867327577	0.23563609	-1.1728692	2.7092662	-0.8106479890183471	0.23371251	-2.5851446903556123	0.29344675	0.1637247	0.2333921	-0.052740883	-0.0014309319	0.17818415	-0.04178299	-0.1668674	0.030660324	0.1277136	0.24382767	360	0	359	1	0.45686695	382.51477	0.0	0.0	31	False	1.4419978	--	--	--	--	--	--	--	42	26	0.42486578	46	19	0	0.024028761	19.193428	0	0	1.0162723	0.23008826	0.213876	0.09316541	0.14170524	-89.87326	-80.582855	-20.517448	-27.920107	False	364	312.1578148653012	0.8630008	361.71207	19.451431	43	106.71067735738295	5.8664865	18.189878	20.268023	42	250.00603032889998	8.702527	28.727982	18.75302	1.1427448	0	1	0	1	0	1.5150032	0.81659126	0.69841194	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	NOT_AVAILABLE	70.00576776990025	10.867630795568862	302.1326139962143	58.99438818311647	False	False	0	False	False	False	False	False	False	False	False	3.923956e-10	6.2316663e-10	0.9999929	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--		0.04998339321858548
1636148068921376768	Gaia DR3 2051105376451534080	2051105376451534080	349005785	2016.0	290.2620306861585	0.21345817	37.76225637755539	0.28413698	0.304847155398599	0.27989152	1.0891618	2.7098396	-0.14449493714268458	0.27666575	-2.7059844059821785	0.3406538	0.111298405	0.07204159	-0.12538862	-0.055207342	0.13312708	-0.1043947	-0.21191615	-0.16984196	0.055460937	0.2138439	348	0	344	4	-0.37459254	341.67722	0.0	0.0	95	False	--	1.4468051	0.07294632	-0.062292464	0.06712199	0.040067807	-0.10036108	-0.08958374	41	25	0.49400654	44	19	0	0.024567783	3.541193	1	0	0.98464596	0.2542177	0.2283579	0.12611242	0.1156371	-111.30088	-87.94184	-12.496002	-21.521162	False	334	235.45663259991187	0.7688483	306.2459	19.75759	36	85.92625106757575	6.0261345	14.258933	20.503227	35	175.13321253443684	5.9210563	29.578035	19.139475	1.1087369	0	1	0	0	0	1.3637524	0.7456379	0.6181145	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	NOT_AVAILABLE	69.9644203275566	10.870404794991602	302.0763416918218	58.96583000941361	False	False	0	False	False	False	False	False	False	False	False	3.0029546e-10	7.6057516e-10	0.9999992	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--	--		0.04999419931408359

This query has returned very large number of columns, so we select the most useful columns to make the catalog easier to use with TweakReg.

reduced_query = gaia_query['ra', 'dec', 'phot_g_mean_mag']
reduced_query

Table length=2331

ra	dec	phot_g_mean_mag
deg	deg	mag
float64	float64	float32
290.2248611356283	37.80192950361359	18.957083
290.22327972754084	37.80283944795903	20.817099
290.22261994880597	37.80270752474709	16.63462
290.2248309701555	37.80058611405895	17.579128
290.2225602638809	37.80397847004457	18.630041
290.22681959687196	37.80112772770517	20.685486
290.2275249308142	37.803472775139035	19.896347
290.2242320409821	37.80039510739463	20.585316
290.22302939666463	37.8003404115494	18.381582
...	...	...
290.1654951838555	37.78544763786987	17.43821
290.25702221163755	37.75972204677077	19.5283
290.280804620236	37.779546916641095	19.059729
290.2498342848316	37.7568154080175	19.591663
290.20159605323926	37.849139504585956	20.15819
290.284611215241	37.818928955959755	19.626196
290.2824764003652	37.82319670158857	18.058723
290.28771982216256	37.79753057383014	19.451431
290.2620306861585	37.76225637755539	19.75759

Then we write this catalog to an ascii file for use with TweakReg.

reduced_query.write('gaia.cat', format='ascii.commented_header', overwrite=True)

3. Align with TweakReg

Table of Contents

With the catalogs downloaded and the headers populated, we now need to run TweakReg with each catalog passed into the refcat parameter. The steps below compare the astrometric residuals obtained from aligning to each refcat. In each case, we set the updatehdr parameter to False to avoid altering the WCS of the images until we are satisfied with the alignment by inspecting both the TweakReg shift file and the astrometric residual plots.

3.1 SDSS Alignment

refcat = 'sdss.cat'
wcsname = 'SDSS' # Specify the WCS name for this alignment.
cw = 3.5         # 2x the FWHM of the PSF = 3.5 pixels for ACS/WFC, WFC3/UVIS or 2.5 for WFC3/IR.

tweakreg.TweakReg('*flc.fits', # Pass the input images to tweakreg.
                  imagefindcfg={'threshold': 500., 'conv_width': cw},  # Detection parameters that vary for different datasets.
                  refcat=refcat,                # Use user supplied catalog (Gaia).
                  interactive=False,            # Don't show the interactive interface.
                  see2dplot=False,              # Suppress additional figures in this test.
                  shiftfile=True,               # Save output shift file to examine shifts later.
                  outshifts='SDSS_shifts.txt',  # Name of the shift file that will be saved.
                  wcsname=wcsname,              # Give our WCS a new name defined above.
                  reusename=True,               # Overwrite the WCS if it exists.
                  ylimit=1.5,
                  fitgeometry='rscale',         # Allow for translation, rotation, and plate scaling.
                  updatehdr=False)              # Don't update the header with new WCS until later.
clear_output()

# If the alignment is unsuccessful then stop the notebook.

with open('SDSS_shifts.txt', 'r') as shift:
    for line_number, line in enumerate(shift, start=1):
        if "nan" in line:
            raise ValueError('nan found in line {} in shift file'.format(line_number))
        else:
            continue

3.2 Inspect the shift file and fit residuals

We can look at the shift file to see how well the fit did (or we could open the output png images for more information). The columns are:

• Filename, X Shift [pixels], Y Shift [pixels], Rotation [degrees], Scale, X RMS [pixels], Y RMS [pixels]

shift_table = Table.read('SDSS_shifts.txt',
                         format='ascii.no_header', 
                         names=['file', 'dx', 'dy', 'rot', 'scale', 'xrms', 'yrms'])

formats = ['.2f', '.2f', '.3f', '.5f', '.2f', '.2f']
for i, col in enumerate(shift_table.colnames[1:]):
    shift_table[col].format = formats[i]
shift_table

Table length=3

file	dx	dy	rot	scale	xrms	yrms
str18	float64	float64	float64	float64	float64	float64
ibwb01xqq_flc.fits	1.47	0.69	0.003	0.99993	0.37	0.34
ibwb01xxq_flc.fits	1.57	-0.14	360.000	1.00010	0.38	0.54
ibwb01xrq_flc.fits	1.48	0.56	0.002	0.99995	0.37	0.36

Show the astrometric residual plots.

Image(filename='residuals_ibwb01xqq_flc.png', width=500, height=300)

Image(filename='residuals_ibwb01xrq_flc.png', width=500, height=300)

Image(filename='residuals_ibwb01xxq_flc.png', width=500, height=300)

We can see in the plots above that the RMS is fairly large at about 0.5 pixels. This is generally considered a poor fit, with the desired RMS being < 0.1 pixels depending on the number of sources. This is likely because the SDSS astrometric precision is insufficient for a high-quality HST alignment. One approach would be to align the first image to SDSS and then align the remaining HST images to one another. This would improve both the absolute and relative alignment of the individual frames.

3.3 Gaia Alignment

refcat = 'gaia.cat'
wcsname = 'Gaia' # Specify the WCS name for this alignment.
cw = 3.5         # 2x the FWHM of the PSF = 3.5 pixels for ACS/WFC, WFC3/UVIS or 2.5 for WFC3/IR.

tweakreg.TweakReg('*flc.fits', # Pass the input images to tweakreg.
                  imagefindcfg={'threshold': 500., 'conv_width': cw},  # Detection parameters that vary for different datasets.
                  refcat=refcat,                # Use user supplied catalog (Gaia).
                  interactive=False,            # Don't show the interactive interface.
                  see2dplot=False,              # Suppress additional figures in this test.
                  shiftfile=True,               # Save output shift file to examine shifts later.
                  outshifts='Gaia_shifts.txt',  # Name of the shift file that will be saved.
                  wcsname=wcsname,              # Give our WCS a new name defined above.
                  reusename=True,               # Overwrite the WCS if it exists.
                  ylimit=0.3,                   # The ylimit for the residuals plot.
                  fitgeometry='rscale',         # Allow for translation, rotation, and plate scaling.
                  searchrad=0.1,                # With many sources use a smaller search radius for convergence.
                  updatehdr=False)              # Don't update the header with new WCS until later.
clear_output()

# If the alignment is unsuccessful then stop the notebook.

with open('Gaia_shifts.txt', 'r') as shift:
    for line_number, line in enumerate(shift, start=1):
        if "nan" in line:
            raise ValueError('nan found in line {} in shift file'.format(line_number))
        else:
            continue

3.4 Inspect the shift file and fit residuals

We can similarly look at the shift file from alignment to the Gaia catalog. As expected, the Gaia catalog does quite a bit better, with rms residuals ~0.05 pixels.

The columns are: Filename, X Shift [pixels], Y Shift [pixels], Rotation [degrees], Scale, X RMS [pixels], Y RMS [pixels]

shift_table = Table.read('Gaia_shifts.txt',
                         format='ascii.no_header', 
                         names=['file', 'dx', 'dy', 'rot', 'scale', 'xrms', 'yrms'])

formats = ['.2f', '.2f', '.3f', '.5f', '.2f', '.2f']
for i, col in enumerate(shift_table.colnames[1:]):
    shift_table[col].format = formats[i]
shift_table

Table length=3

file	dx	dy	rot	scale	xrms	yrms
str18	float64	float64	float64	float64	float64	float64
ibwb01xqq_flc.fits	-0.22	0.10	360.000	1.00000	0.03	0.03
ibwb01xxq_flc.fits	-0.21	0.10	0.000	1.00000	0.03	0.05
ibwb01xrq_flc.fits	-0.22	0.10	0.000	1.00000	0.03	0.05

Print the Number of Gaia Matches per image from the TweakReg output.

rootname1 = 'ibwb01xqq'
rootname2 = 'ibwb01xrq'
rootname3 = 'ibwb01xxq'
match_tab1 = ascii.read(rootname1+'_flc_catalog_fit.match')  # load match file in astropy table
match_tab2 = ascii.read(rootname2+'_flc_catalog_fit.match')  # load match file in astropy table
match_tab3 = ascii.read(rootname3+'_flc_catalog_fit.match')  # load match file in astropy table
print(rootname1+': Number of Gaia Matches =', len(match_tab1))
print(rootname2+': Number of Gaia Matches =', len(match_tab2))
print(rootname3+': Number of Gaia Matches =', len(match_tab3))

ibwb01xqq: Number of Gaia Matches = 565
ibwb01xrq: Number of Gaia Matches = 567
ibwb01xxq: Number of Gaia Matches = 538

Compare the RMS and number of matches from TweakReg with the MAST alignment results.

wcstable

Table length=3

filename	DETECTOR	EXPTIME	WCSNAME	NMATCHES	RMS_RA	RMS_DEC	RMS_RA_pix	RMS_DEC_pix
str18	str4	float64	str30	int64	float64	float64	float64	float64
ibwb01xqq_flc.fits	UVIS	30.0	IDC_2731450pi-FIT_IMG_GAIAeDR3	442	1.3	1.5	0.03	0.04
ibwb01xrq_flc.fits	UVIS	360.0	IDC_2731450pi-FIT_IMG_GAIAeDR3	462	4.0	3.8	0.1	0.1
ibwb01xxq_flc.fits	UVIS	360.0	IDC_2731450pi-FIT_IMG_GAIAeDR3	429	3.9	3.7	0.1	0.09

The new Gaia solution looks better than the MAST solution, so we rerun TweakReg below and update the image headers with the new WCS solution.

For more details on absolute astrometry in HST images, see Section 4.5 in the DrizzlePac Handbook. Show the astrometric residual plots:

Image(filename='residuals_ibwb01xqq_flc.png', width=500, height=300)

Image(filename='residuals_ibwb01xrq_flc.png', width=500, height=300)

Image(filename='residuals_ibwb01xxq_flc.png', width=500, height=300)

3.5 Overplot Matched Sources on the Image

Now, let’s plot the sources that were matched between all images on the “bottom” UVIS chip. This is referred to as chip 2 or SCI, 1 or extension 1. The cell below reads in the *_fit.match file as an astropy table. Unfortunately, it doesn’t automatically name columns so we look at the header to grab the columns.

To verify that TweakReg obtained an acceptable fit between matched source catalogs, it is useful to inspect the results before updating the image header WCS. In the figure below, sources that are matched with Gaia are overplotted on one of the input FLC frames with the two chips merged together.

It can be useful to check that TweakReg locked onto stars and not hot pixels or other detector artifacts before proceeding to update the image header.

# Choose one of the files to view.
# rootname = 'ibwb01xqq'
rootname = 'ibwb01xrq'
# rootname = 'ibwb01xxq'

plt.figure(figsize=(7, 7), dpi=140)
chip1_data = fits.open(rootname+'_flc.fits')['SCI', 2].data
chip2_data = fits.open(rootname+'_flc.fits')['SCI', 1].data
fullsci = np.concatenate([chip2_data, chip1_data])
z1, z2 = ZScaleInterval().get_limits(fullsci)
plt.imshow(fullsci, cmap='Greys', origin='lower', vmin=z1, vmax=z2)

match_tab = ascii.read(rootname+'_flc_catalog_fit.match')  # Load the match file in astropy table.
match_tab_chip1 = match_tab[match_tab['col15'] == 2]  # Filter the table for sources on chip 1.
match_tab_chip2 = match_tab[match_tab['col15'] == 1]  # Filter the table for sources on chip 2.
x_cord1, y_cord1 = match_tab_chip1['col11'], match_tab_chip1['col12']
x_cord2, y_cord2 = match_tab_chip2['col11'], match_tab_chip2['col12']

plt.scatter(x_cord1, y_cord1+2051, s=50, edgecolor='r', facecolor='None', label='Matched Sources')
plt.scatter(x_cord2, y_cord2, s=50, edgecolor='r', facecolor='None')
plt.title(f'Matched sources UVIS to Gaia: N = {len(match_tab)}', fontsize=14)
plt.show()

3.6 Update header with optimal parameters

Once you’ve decided on the optimal set of parameters for your alignment, it is safe to update the header of the input data, (assuming that the new solution is better than the MAST alignment.) To apply these transformations to the image, we simply need to run TweakReg the same as before, but set the parameter updatehdr equal to True:

refcat = 'gaia.cat'
wcsname = 'Gaia'                   # Specify the WCS name for this alignment
cw = 3.5                           # 2x the FWHM of the PSF = 3.5 for ACS/WFC, WFC3/UVIS or 2.5 for WFC3/IR 

tweakreg.TweakReg('*flc.fits', # Pass the input images to tweakreg.
                  imagefindcfg={'threshold': 500., 'conv_width': cw},  # Detection parameters that vary for different datasets.
                  refcat=refcat,                # Use user supplied catalog (Gaia).
                  interactive=False,            # Don't show the interactive interface.
                  see2dplot=False,              # Suppress additional figures in this test.
                  shiftfile=True,               # Save output shift file to examine shifts later.
                  outshifts='Gaia_shifts.txt',  # Name of the shift file that will be saved.
                  wcsname=wcsname,              # Give our WCS a new name defined above.
                  reusename=True,               # Overwrite the WCS if it exists.
                  ylimit=0.3,                   # The ylimit for the residuals plot.
                  fitgeometry='rscale',         # Allow for translation, rotation, and plate scaling.
                  searchrad=0.1,                # With many sources use a smaller search radius for convergence.
                  updatehdr=True)               # Update the header with new WCS solution.

clear_output()

4. Combine the Images using AstroDrizzle

Table of Contents

While the three sets of FLC files are now aligned, in this example we drizzle together only the two long exposures. When exposures are very different lengths, drizzling them together doesn’t work well when ‘EXP’ weighting is used. For objects that saturate in the long exposures, the problem occurs at the boundary where the signal transitions from only being present in short exposures near the core to being present at larger radii in the longer exposures. The result is a discontinuity in the PSF radial profile and a resulting flux that is too low in some regions. For photometry of saturated objects, the short exposures should be drizzled separately from the long exposures.

Next, we combine the images through drizzling, and retrieve some recommended values for this process from the MDRIZTAB reference file. The parameters in this file are different for each detector and are based on the number of input frames. These are a good starting point for drizzling and may be adjusted based on your science goals.

input_images = sorted(glob.glob('ibwb01x[rx]q_flc.fits'))

mdz = fits.getval(input_images[0], 'MDRIZTAB', ext=0).split('$')[1]

print('Searching for the MDRIZTAB file:', mdz)

!crds sync --hst --files {mdz} --output-dir {os.environ['iref']}

Searching for the MDRIZTAB file: ubi1853qi_mdz.fits

CRDS - INFO -  Symbolic context 'hst-latest' resolves to 'hst_1192.pmap'
CRDS - INFO -  Reorganizing 0 references from 'instrument' to 'flat'
CRDS - INFO -  Reorganizing from 'instrument' to 'flat' cache,  removing instrument directories.
CRDS - INFO -  Syncing explicitly listed files.

CRDS - INFO -  Fetching  ./crds_cache/references/hst/wfc3/ubi1853qi_mdz.fits       95.0 K bytes  (1 / 1 files) (0 / 95.0 K bytes)

CRDS - INFO -  0 errors
CRDS - INFO -  0 warnings
CRDS - INFO -  5 infos

def get_vals_from_mdriztab(input_images, 
                           kw_list=[
                               'driz_sep_bits', 
                               'combine_type', 
                               'driz_cr_snr', 
                               'driz_cr_scale', 
                               'final_bits']):
    
    'Get only selected parameters from the MDRIZTAB.'
    mdriz_dict = getMdriztabPars(input_images)
    
    requested_params = {}
    
    print('Outputting the following parameters:')
    for k in kw_list:
        requested_params[k] = mdriz_dict[k]
        print(k, mdriz_dict[k])
    
    return requested_params

selected_params = get_vals_from_mdriztab(input_images)

- MDRIZTAB: AstroDrizzle parameters read from row 2.
Outputting the following parameters:
driz_sep_bits 96
combine_type minmed
driz_cr_snr 3.5 3.0
driz_cr_scale 1.2 0.7
final_bits 96

Note that the parameter final_bits= ‘96’ is equivalent to final_bits= ‘64, 32’ because they are added in a bit-wise manner.

# To override any of the above values:
selected_params['driz_sep_bits'] = '256, 64, 32'
selected_params['final_bits'] = '256, 64, 32'
# selected_params['combine_type']  = 'median'
# selected_params['driz_cr_snr']   = '4.0 3.5'
# selected_params['driz_cr_scale'] = '1.5 1.2'

astrodrizzle.AstroDrizzle(input_images, 
                          output='f606w',
                          preserve=False,
                          clean=True, 
                          build=False,
                          context=False,
                          skymethod='match',
                          **selected_params)

clear_output()

Display the combined science and weight images. In this example there appear to be three detector chips due to the large dither that was used in the observations. We note that the exposures overlapped in the center region, as evidenced by the increased value of the weight map corresponding to a longer effective exposure time. In addition, the resulting mosaic is better cleaned of artifacts and cosmic rays in this region due to the multiple exposures, as seen by the uniform background in the science mosaic.

sci = fits.getdata('f606w_drc_sci.fits')
wht = fits.getdata('f606w_drc_wht.fits')

fig = plt.figure(figsize=(20, 20))
ax1 = fig.add_subplot(1, 2, 1)
ax2 = fig.add_subplot(1, 2, 2)

ax1.imshow(sci, vmin=-0.05, vmax=0.4, cmap='Greys_r', origin='lower')
ax2.imshow(wht, vmin=0.0, vmax=1000, cmap='Greys_r', origin='lower')

ax1.title.set_text('Science Image')
ax2.title.set_text('Weight Image')
ax1.set_xlabel('X (pixels)')
ax1.set_ylabel('Y (pixels)')
ax2.set_xlabel('X (pixels)')
ax2.set_ylabel('Y (pixels)')

Text(0, 0.5, 'Y (pixels)')

For more details on inspecting the drizzled products after reprocessing, see Section 7.3 in the DrizzlePac Handbook.

Conclusions

Table of Contents

Many other services have interfaces for querying catalogs that can also be used to align HST images. In general, Gaia works very well for HST due to its high precision, but can have a low number of sources in some regions, especially at high galactic latitudes. Aligning images to an absolute frame provides a way to make data comparable across many epochs/detectors/observatories, and in many cases, makes the alignment easier to complete.

About this Notebook

Created: 14 Dec 2018;     V. Bajaj
Updated: 03 Jul 2024;     M. Revalski, V. Bajaj, & J. Mack

Source: GitHub spacetelescope/hst_notebooks

Additional Resources

Below are some additional resources that may be helpful. Please send any questions through the HST Help Desk, selecting the DrizzlePac category.

• WFC3 Website

• WFC3 Data Handbook

• WFC3 Instrument Handbook

Citations

If you use Python packages such as astropy, astroquery, drizzlepac, matplotlib, or numpy for published research, please cite the authors.

Follow these links for more information about citing various packages:

• Citing astropy

• Citing astroquery

• Citing drizzlepac

• Citing matplotlib

• Citing numpy

---

Top of Page