WFSS Spectra Part 0: Optimal Extraction#
Use case: optimal extraction of grism spectra; redshift measurement; emission-line maps. Simplified version of JDox Science Use Case # 33.
Data: JWST simulated NIRISS images from MIRAGE, run through the JWST calibration pipeline; galaxy cluster.
Tools: specutils, astropy, pandas, emcee, lmfit, corner, h5py.
Cross-intrument: NIRSpec
Documentation: This notebook is part of a STScI’s larger post-pipeline Data Analysis Tools Ecosystem.
Introduction#
This notebook is 1 of 4 in a set focusing on NIRISS WFSS data: 1. 1D optimal extraction since the JWST pipeline only provides a box extraction. Optimal extraction improves S/N of spectra for faint sources. 2. Combine and normalize 1D spectra. 3. Cross correlate galaxy with template to get redshift. 4. Spatially resolved emission line map.
This notebook will start with post-pipeline products of NIRISS WFSS, 2D rectified spectra, from spec level3.
Optimal extraction requires source morphology along the cross-dispersion direction, which will be retrieved from direct images taken along with WFSS observations. Morphology along dispersion direction is also essential to infer the spectral resolution, which will be obtained using template fitting to get redshift and stellar population in notebook #3 of this set.
Note: We here assume reduction of the 2D rectified spectrum has been performed at a decent level, i.e. there is no contaminating flux from other sources on the target 2D spectrum, and that background is already subtracted.
%matplotlib inline
import os
import numpy as np
from scipy.ndimage import rotate
from scipy.optimize import curve_fit
from astropy.io import fits
import astropy.units as u
import astropy.wcs as wcs
from astropy.io import ascii
from specutils import Spectrum1D
from astropy.nddata import StdDevUncertainty
import specutils
print('specutils', specutils.__version__)
import astropy
print('astropy', astropy.__version__)
specutils 1.15.0
astropy 6.1.1
The version should be#
specutils 1.11.0
astropy 5.3.4
import matplotlib.pyplot as plt
import matplotlib as mpl
mpl.rcParams['savefig.dpi'] = 80
mpl.rcParams['figure.dpi'] = 80
0.Download and load data:#
These include pipeline processed data for NIRISS, as well as photometric catalog from image step3.
if not os.path.exists('./pipeline_products'):
import zipfile
import urllib.request
boxlink = 'https://data.science.stsci.edu/redirect/JWST/jwst-data_analysis_tools/NIRISS_lensing_cluster/pipeline_products.zip'
boxfile = './pipeline_products.zip'
urllib.request.urlretrieve(boxlink, boxfile)
zf = zipfile.ZipFile(boxfile, 'r')
zf.extractall()
DIR_DATA = './pipeline_products/'
# Output directory;
DIR_OUT = './output/'
if not os.path.exists(DIR_OUT):
os.mkdir(DIR_OUT)
# Filter for detection and science image;
filt_det = 'f200w'
# Image array from direct image. This is for optimal extraction and masking.
# This image should already be sky-subtracted; otherwise, you will encounter a wrong result with optimal extraction.
infile = f'{DIR_DATA}l3_nis_{filt_det}_i2d_skysub.fits'
hdu = fits.open(infile)
# This is just for error array;
infile = f'{DIR_DATA}l3_nis_{filt_det}_i2d.fits'
hdu_err = fits.open(infile)
data = hdu[0].data
imwcs = wcs.WCS(hdu[0].header, hdu)
err = hdu_err[2].data
weight = 1/np.square(err)
# Segmentation map;
# This can be prepared by running Photutils, if the pipeline does not generate one.
segfile = f'{DIR_DATA}l3_nis_{filt_det}_i2d_seg.fits'
seghdu = fits.open(segfile)
segdata = seghdu[0].data
/tmp/ipykernel_1875/874489203.py:25: RuntimeWarning: divide by zero encountered in divide
weight = 1/np.square(err)
# Load catalog from image level3;
# to obtain source position in pixel coordinate.
catfile = f'{DIR_DATA}l3_nis_{filt_det}_cat.ecsv'
fd = ascii.read(catfile)
fd
id | xcentroid | ycentroid | sky_centroid | aper_bkg_flux | aper_bkg_flux_err | aper30_flux | aper30_flux_err | aper50_flux | aper50_flux_err | aper70_flux | aper70_flux_err | aper_total_flux | aper_total_flux_err | aper30_abmag | aper30_abmag_err | aper50_abmag | aper50_abmag_err | aper70_abmag | aper70_abmag_err | aper_total_abmag | aper_total_abmag_err | aper30_vegamag | aper30_vegamag_err | aper50_vegamag | aper50_vegamag_err | aper70_vegamag | aper70_vegamag_err | aper_total_vegamag | aper_total_vegamag_err | CI_30_50 | CI_50_70 | CI_30_70 | is_star | sharpness | roundness | nn_dist | nn_abmag | isophotal_flux | isophotal_flux_err | isophotal_abmag | isophotal_abmag_err | isophotal_vegamag | isophotal_vegamag_err | isophotal_area | semimajor_sigma | semiminor_sigma | ellipticity | orientation | sky_orientation | sky_bbox_ll | sky_bbox_ul | sky_bbox_lr | sky_bbox_ur |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
pix | pix | deg,deg | Jy | Jy | Jy | Jy | Jy | Jy | Jy | Jy | Jy | Jy | pix | Jy | Jy | pix2 | pix | pix | deg | deg | deg,deg | deg,deg | deg,deg | deg,deg | |||||||||||||||||||||||||||||
int64 | float64 | float64 | SkyCoord | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float32 | float64 | float64 | float64 | float32 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | float64 | SkyCoord | SkyCoord | SkyCoord | SkyCoord |
1 | 1901.4791 | 616.6617 | 3.569653944921018,-30.405070186723776 | 3.483045e-08 | nan | 8.130363e-08 | nan | 1.921404e-07 | nan | 3.765590e-07 | nan | nan | nan | 26.624725 | nan | 25.690953 | nan | 24.960418 | nan | nan | nan | 24.915445 | nan | 23.981673 | nan | 23.251138 | nan | nan | nan | 0.9338 | 0.7305 | 1.6643 | nan | 0.643053 | 0.050833 | 687.142856 | 22.245099 | 3.217875e-06 | nan | 22.631077 | nan | 20.921797 | nan | 83.0 | 2.300009 | 2.249340 | 0.022030 | -68.272405 | 22.344833 | 3.569758007135362,-30.405165289935546 | 3.5697605503538763,-30.404964734861768 | 3.56952547091111,-30.405163096272904 | 3.5695280146070885,-30.404962541203844 |
2 | 1215.5473 | 657.4411 | 3.5841636319830568,-30.40446267443482 | 4.591611e-08 | nan | 1.550019e-07 | nan | 3.638838e-07 | nan | 6.725310e-07 | nan | nan | nan | 25.924158 | nan | 24.997593 | nan | 24.330719 | nan | nan | nan | 24.214878 | nan | 23.288313 | nan | 22.621439 | nan | nan | nan | 0.9266 | 0.6669 | 1.5934 | nan | 0.666481 | 0.076069 | 687.142856 | 22.631077 | 4.591563e-06 | nan | 22.245099 | nan | 20.535819 | nan | 95.0 | 2.968964 | 1.778262 | 0.401050 | 89.747234 | 180.364471 | 3.5842476045730627,-30.404590023798825 | 3.5842510319996697,-30.404316539215852 | 3.5840573485032308,-30.404588250016765 | 3.58406077646256,-30.404314765438812 |
3 | 381.9250 | 984.5165 | 3.601859711428753,-30.398662437780448 | 4.268341e-08 | nan | 1.404209e-07 | nan | 3.172360e-07 | nan | 6.277053e-07 | nan | nan | nan | 26.031420 | nan | 25.146544 | nan | 24.405610 | nan | nan | nan | 24.322140 | nan | 23.437264 | nan | 22.696330 | nan | nan | nan | 0.8849 | 0.7409 | 1.6258 | nan | 0.672185 | 0.331872 | 673.406986 | 16.320658 | 4.647145e-06 | nan | 22.232034 | nan | 20.522754 | nan | 101.0 | 2.880019 | 1.989674 | 0.309146 | -89.237463 | 1.379775 | 3.6019516675347565,-30.39879122365138 | 3.601955045183127,-30.39851773861101 | 3.6017402841066257,-30.398789281273874 | 3.60174366234683,-30.398515796238758 |
4 | 1100.5677 | 1408.4408 | 3.5867654957418873,-30.390792851961756 | 4.182255e-08 | nan | 1.977441e-07 | nan | 4.634087e-07 | nan | 8.557904e-07 | nan | nan | nan | 25.659741 | nan | 24.735090 | nan | 24.069081 | nan | nan | nan | 23.950461 | nan | 23.025810 | nan | 22.359801 | nan | nan | nan | 0.9247 | 0.6660 | 1.5907 | nan | 0.675891 | 0.239616 | 759.750606 | 22.245099 | 4.479109e-06 | nan | 22.272021 | nan | 20.562741 | nan | 78.0 | 2.864331 | 1.524197 | 0.467870 | 89.968483 | 180.585720 | 3.586849890460872,-30.390920199626773 | 3.586853310121526,-30.390646714975425 | 3.586659660970119,-30.390918429617493 | 3.586663081163341,-30.39064494497111 |
5 | 662.4288 | 1685.1702 | 3.596088874688556,-30.385833272645225 | 3.108490e-08 | nan | 6.850841e-07 | nan | 1.445057e-06 | nan | 2.486505e-06 | nan | nan | nan | 24.310640 | nan | 23.500287 | nan | 22.911026 | nan | nan | nan | 22.601360 | nan | 21.791007 | nan | 21.201746 | nan | nan | nan | 0.8104 | 0.5893 | 1.3996 | nan | 0.584634 | -0.067064 | 334.076772 | 20.727265 | 5.441945e-06 | nan | 22.060615 | nan | 20.351335 | nan | 71.0 | 1.874113 | 1.820229 | 0.028752 | 10.431093 | 101.048330 | 3.5961919899951726,-30.38591938323127 | 3.5961942522168715,-30.385737059975185 | 3.5959594989853887,-30.38591723639035 | 3.5959617616410102,-30.38573491313818 |
6 | 319.4990 | 1885.3035 | 3.60338188710553,-30.382251101963146 | 3.326749e-08 | nan | 6.893335e-07 | nan | 1.514847e-06 | nan | 2.523116e-06 | nan | nan | nan | 24.303927 | nan | 23.449078 | nan | 22.895157 | nan | nan | nan | 22.594647 | nan | 21.739798 | nan | 21.185877 | nan | nan | nan | 0.8548 | 0.5539 | 1.4088 | nan | 0.581781 | 0.158721 | 111.504109 | 22.028328 | 5.666923e-06 | nan | 22.016632 | nan | 20.307352 | nan | 75.0 | 1.894424 | 1.866322 | 0.014834 | 14.267744 | 104.884981 | 3.603486459620283,-30.38233965159385 | 3.603488932995949,-30.382139095884135 | 3.6032539769794103,-30.38233751766547 | 3.6032564508323546,-30.382136961959965 |
7 | 266.9182 | 1983.6317 | 3.604515260247416,-30.380468545121307 | 3.310798e-08 | nan | 7.155692e-07 | nan | 1.476385e-06 | nan | 2.520043e-06 | nan | nan | nan | 24.263371 | nan | 23.477001 | nan | 22.896480 | nan | nan | nan | 22.554091 | nan | 21.767721 | nan | 21.187200 | nan | nan | nan | 0.7864 | 0.5805 | 1.3669 | nan | 0.593763 | 0.061278 | 111.504109 | 22.016632 | 5.606203e-06 | nan | 22.028328 | nan | 20.319048 | nan | 74.0 | 1.893807 | 1.847921 | 0.024230 | 89.641888 | 180.259126 | 3.6046074836711655,-30.380562964468123 | 3.6046099547002646,-30.380362408741654 | 3.6043961396388418,-30.38056102635311 | 3.6043986111017956,-30.38036047063047 |
Make a broadband flux catalog.#
For a convenient reason, here we compile catalogs into a flux catalog, which will be used in the following notebook (01b).
To run this cell, you will need a photometric catalog of sources, that list sources position and flux for each filter. For now, I use this catalog prepared in another notebook. (“sources_extend_01.cat”)
This catalog can also be used for generic phot-z/SED fitting softwares, like EAZY and gsf (see notebook No.04).
For now, we use an input catalog.#
filts = ['f115w', 'f150w', 'f200w', 'f090w', 'f435w', 'f606w', 'f814w', 'f105w', 'f125w', 'f140w', 'f160w']
eazy_filts = [309, 310, 311, 308, 1, 4, 6, 202, 203, 204, 205]
magzp = 25.0 # magnitude zeropoint in the catalog.
# Read catalog;
fd_input = ascii.read(f'{DIR_DATA}sources_extend_01.cat')
ra_input = fd_input['x_or_RA']
dec_input = fd_input['y_or_Dec']
# Header;
fw = open(f'{DIR_OUT}l3_nis_flux.cat', 'w')
fw.write('# id')
for ff in range(len(filts)):
fw.write(f' F{eazy_filts[ff]} E{eazy_filts[ff]}')
fw.write('\n')
# Contents;
for ii in range(len(fd['id'])):
rtmp = np.sqrt((fd['sky_centroid'].ra.value[ii] - ra_input[:])**2 + (fd['sky_centroid'].dec.value[ii] - dec_input[:])**2)
iix = np.argmin(rtmp)
for ff in range(len(filts)):
if ff == 0:
fw.write(f"{fd['id'][ii]}")
mag = fd_input[f'niriss_{filts[ff]}_magnitude'][iix]
flux_nu = 10**((mag-magzp)/(-2.5))
# Currently, the catalog does not provide proper error;
# Assuming a 5% error for flux.
flux_err_nu = flux_nu * 0.05
fw.write(f' {flux_nu:.5e} {flux_err_nu:.5e}')
fw.write('\n')
fw.close()
1.Load 2D spectrum;#
# Which filter, grating, and object?
filt = 'f200w'
# grism = 'G150R'
grism = 'G150C'
id = '00004'
# Zero-indexed number for dither --- the test data here has two dither positions, so 0 or 1.
ndither = 0
file_2d = f'{DIR_DATA}l3_nis_{filt}_{grism}_s{id}_cal.fits'
hdu_2d = fits.open(file_2d)
# Align grism direction
# - x-direction = Dispersion (wavelength) direction.
# - y-direction = Cross-dispersion.
# in this notebook.
if grism == 'G150C':
# If spectrum is horizontal;
data_2d = hdu_2d[ndither*7+1].data
dq_2d = hdu_2d[ndither*7+2].data
err_2d = hdu_2d[ndither*7+3].data
wave_2d = hdu_2d[ndither*7+4].data
else:
data_2d = rotate(hdu_2d[ndither*7+1].data, 90)
dq_2d = rotate(hdu_2d[ndither*7+2].data, 90)
err_2d = rotate(hdu_2d[ndither*7+3].data, 90)
wave_2d = rotate(hdu_2d[ndither*7+4].data, 90)
# Get position angle of observation;
hd_2d = hdu_2d[1].header
PA_V3 = hd_2d['PA_V3']
PA_V3
0.0
plt.imshow(data_2d, vmin=0, vmax=1000)
<matplotlib.image.AxesImage at 0x7fdeaad47f50>
![../../../_images/d2c8bca1a1d0b51ac343bcfb811b9f5cf03c59286dc1a5c5d8d5d32b2f4af642.png](../../../_images/d2c8bca1a1d0b51ac343bcfb811b9f5cf03c59286dc1a5c5d8d5d32b2f4af642.png)
# Get light profile of the source;
# Again, y is for cross-dispersion, and x is for dispersion directions.
y2d, x2d = data_2d.shape[:]
# Cut out segmentation map;
iix = np.where(fd['id'] == int(id))[0][0]
# Target position from image 3 catalog;
ycen = fd['ycentroid'][iix].value
xcen = fd['xcentroid'][iix].value
# Cutout size = y direction of 2D spectrum;
rsq = y2d
sci_cut = data[int(ycen-rsq/2.+0.5):int(ycen+rsq/2.+0.5), int(xcen-rsq/2.+0.5):int(xcen+rsq/2.+0.5)]
seg_cut = segdata[int(ycen-rsq/2.+0.5):int(ycen+rsq/2.+0.5), int(xcen-rsq/2.+0.5):int(xcen+rsq/2.+0.5)]
# Rotate image for PA of Grism observation;
if grism == 'G150C':
sci_rot = rotate(sci_cut, PA_V3)
else:
sci_rot = rotate(sci_cut, PA_V3+90)
WFSS grism is dispersed in a direction of x-axis in the plot below.#
plt.imshow(sci_rot, vmin=0, vmax=1.0)
plt.title('Direct image')
plt.xlabel('Wavelength direction >>>', color='r', fontsize=18)
plt.ylabel('Cross-dispersion direction >>>', color='r', fontsize=18)
Text(0, 0.5, 'Cross-dispersion direction >>>')
![../../../_images/8a9b86a5e1b32e6c8673a78e4ca63365e6ead07d7295fae7ddc29c628ed1a5cb.png](../../../_images/8a9b86a5e1b32e6c8673a78e4ca63365e6ead07d7295fae7ddc29c628ed1a5cb.png)
2.Get light profile at different x position — This will be used for optimal extraction.#
for ii in range(sci_rot.shape[1]):
flux_tmp = sci_rot[:, ii]
xx_tmp = np.arange(0, len(sci_rot[:, ii]), 1)
plt.plot(xx_tmp, flux_tmp, label='x={}'.format(ii))
plt.legend(loc=0, fontsize=8)
<matplotlib.legend.Legend at 0x7fdea7b3aed0>
![../../../_images/b192d182f5d64f090b3cb44776297c07c6da4a3085e290247c0f77b711fbcd8e.png](../../../_images/b192d182f5d64f090b3cb44776297c07c6da4a3085e290247c0f77b711fbcd8e.png)
# Sum along x (disperse) direction
flux_y = np.zeros(len(sci_rot[:, 0]), 'float')
for ii in range(sci_rot.shape[0]):
flux_y[ii] = np.sum(sci_rot[ii, :])
# Sky subtraction, if needed.
# sky = np.mean([flux_y[0], flux_y[-1]])
# Normalize;
flux_y[:] /= flux_y.sum()
plt.plot(xx_tmp, flux_y)
plt.xlabel('y-position')
plt.ylabel('Source Flux')
Text(0, 0.5, 'Source Flux')
![../../../_images/8a11dd90dcbb20bfc7a605845476b5b816034db0d8cd35b48d52ccd0bc1a337c.png](../../../_images/8a11dd90dcbb20bfc7a605845476b5b816034db0d8cd35b48d52ccd0bc1a337c.png)
3.One-dimensional extraction;#
Show pipeline 1D extraction as an example;#
# Normal extraction;
flux_disp1 = np.zeros(x2d, 'float')
err_disp1 = np.zeros(x2d, 'float')
wave_disp1 = np.zeros(x2d, 'float')
for ii in range(x2d): # Wavelength direction.
mask_tmp = (dq_2d[:, ii] == 0) & (err_2d[:, ii] > 0)
# Sum within a box;
flux_disp1[ii] = np.sum(data_2d[:, ii][mask_tmp])
err_disp1[ii] = np.sqrt(np.sum(err_2d[:, ii][mask_tmp]**2))
wave_disp1[ii] = wave_2d[0, ii]
plt.errorbar(wave_disp1, flux_disp1, yerr=err_disp1)
plt.xlim(1.7, 2.3)
(1.7, 2.3)
![../../../_images/438626eaa7587289aa362ab7a9c81224b8232adf415c5798a33f98cba052f648.png](../../../_images/438626eaa7587289aa362ab7a9c81224b8232adf415c5798a33f98cba052f648.png)
Optimal extraction;#
# Following Horne(1986, PASP, 98, 609);
flux_disp = np.zeros(x2d, 'float')
err_disp = np.zeros(x2d, 'float')
wave_disp = np.zeros(x2d, 'float')
# Sigma clipping.
sig = 5.0
for ii in range(x2d): # ii : wavelength element.
# Mask;
# 1. DQ array
# 2. error value
# 3. CR detection
mask_tmp = (dq_2d[:, ii] == 0) & (err_2d[:, ii] > 0) & ((data_2d[:, ii] - flux_y[:] * flux_disp1[ii])**2 < sig**2 * err_2d[:, ii]**2)
ivar = 1. / err_2d[:, ii]**2
num = flux_y[:] * data_2d[:, ii] * ivar
den = flux_y[:]**2 * ivar
flux_disp[ii] = num[mask_tmp].sum(axis=0) / den[mask_tmp].sum(axis=0)
err_disp[ii] = np.sqrt(1./den[mask_tmp].sum(axis=0))
wave_disp[ii] = wave_2d[0, ii]
plt.errorbar(wave_disp, flux_disp, yerr=err_disp)
plt.xlim(1.7, 2.3)
(1.7, 2.3)
![../../../_images/832de4293c06f0fe336d60fff685f0e47990d3190f98b0e750a3a019ec1abe6c.png](../../../_images/832de4293c06f0fe336d60fff685f0e47990d3190f98b0e750a3a019ec1abe6c.png)
# Compare;
plt.errorbar(wave_disp, flux_disp, yerr=err_disp, color='r', label='Optimal')
plt.errorbar(wave_disp1, flux_disp1, yerr=err_disp1, color='b', alpha=0.5, label='Box')
plt.ylim(-10, 20000)
plt.legend(loc=0)
plt.xlabel('Wavelength')
Text(0.5, 0, 'Wavelength')
![../../../_images/da5c88f03a1a07b964c7f7ba5b686c7ab2c7dfa88870ba137a2e67d7bd8ae18e.png](../../../_images/da5c88f03a1a07b964c7f7ba5b686c7ab2c7dfa88870ba137a2e67d7bd8ae18e.png)
4.Write 1d spectrum out to a file;#
file_1d = f'{DIR_OUT}l3_nis_{filt}_{grism}_s{id}_1d_opt.fits'
# Now make it into a Spectrum1D instance.
obs = Spectrum1D(spectral_axis=wave_disp*u.um,
flux=flux_disp*u.MJy,
uncertainty=StdDevUncertainty(err_disp), unit='MJy')
obs.write(file_1d, format='tabular-fits', overwrite=True)
5.Light profile along x-axis = Resolution of dispersed spectrum;#
As WFSS does not have a slit, any dispersed spectrum is affected by source morphology. The estimate on the effective spectral resolution will be needed in the following notebook. And we here try to estimate it beforehand;
for ii in range(sci_rot.shape[0]):
flux_tmp = sci_rot[ii, :]
xx_tmp = np.arange(0, len(sci_rot[ii, :]), 1)
plt.plot(xx_tmp, flux_tmp, label=f'y={ii}')
plt.legend(loc=1, fontsize=8)
plt.xlabel('Wavelength direction')
plt.title('Source light profile along dispersed direction', fontsize=14)
Text(0.5, 1.0, 'Source light profile along dispersed direction')
![../../../_images/9bd3d16c04721638f4a0960d445059f1a8282db7d66f37aec32b256c718a3c59.png](../../../_images/9bd3d16c04721638f4a0960d445059f1a8282db7d66f37aec32b256c718a3c59.png)
*Unless you are interested in spatially resolved spectra, you can stack and get light profile as a good approximation.#
# Sum along cross-disperse direction
flux_x = np.zeros(len(sci_rot[0, :]), 'float')
for ii in range(sci_rot.shape[0]):
flux_x[ii] = np.sum(sci_rot[:, ii])
# Normalize;
flux_x[:] /= flux_x.sum()
plt.plot(xx_tmp, flux_x, label='Convolution kernel')
plt.legend(loc=2)
<matplotlib.legend.Legend at 0x7fdea7b42ed0>
![../../../_images/3bf96a4adc856625502a335355927b3067e7326b80cf9a10762dcaad63535726.png](../../../_images/3bf96a4adc856625502a335355927b3067e7326b80cf9a10762dcaad63535726.png)
Fit with a moffat function;#
# Fitting function with Moffat
# Moffat fnc.
def moffat(xx, A, x0, gamma, alp):
yy = A * (1. + (xx-x0)**2/gamma**2)**(-alp)
return yy
def fit_mof(xx, lsf):
# xx = lsf * 0
# for ii in range(len(lsf)):
# xx[ii] = ii - len(lsf)/2.
popt, pcov = curve_fit(moffat, xx, lsf)
return popt
def LSF_mof(xsf, lsf, f_plot=True):
'''
Input:
=======
xsf : x axis for the profile.
lsf : light profile.
'''
# for ii in range(len(sci[0, :])):
# lsf[ii] = np.mean(sci_rot[int(height/2.-5):int(height/2.+5), ii])
# xsf[ii] = ii - len(lsf)/2.
try:
A, xm, gamma, alpha = fit_mof(xsf, lsf)
except RuntimeError:
print('Fitting failed.')
A, xm, gamma, alpha = -1, -1, -1, -1
pass
if A > 0:
lsf_mod = moffat(xsf, A, 0, gamma, alpha)
if f_plot:
yy = moffat(xsf, A, xm, gamma, alpha)
plt.plot(xsf, yy, 'r.', ls='-', label='Data')
plt.plot(xsf, lsf_mod, 'b+', ls='-', label=f'Model: gamma={gamma:2f}\nalpha={alpha:2f}')
plt.legend()
plt.show()
return A, xm, gamma, alpha
# LSF, line spread function
iix_peak = np.argmax(flux_x)
xx_tmp_shift = xx_tmp - xx_tmp[iix_peak]
A, xm, gamma, alpha = LSF_mof(xx_tmp_shift, flux_x)
![../../../_images/93843c2ea67d91b3e9720ff8f8f2bed9ac261c801c3f26b73230c4936e93b8a8.png](../../../_images/93843c2ea67d91b3e9720ff8f8f2bed9ac261c801c3f26b73230c4936e93b8a8.png)
# Write it down;
# Tha parameters are in unit of pixel.
fm = open(f'{DIR_OUT}l3_nis_{filt}_{grism}_s{id}_moffat.txt', 'w')
fm.write('# A x0 gamma alp\n')
fm.write('# Moffat function\n')
fm.write(f'{A:.3f} {xm:.3f} {gamma:.3f} {alpha:.3f}\n')
fm.close()
Repeat for other filters, other objects.#
The following big colum executes the same processes above for other filters and dither position.#
grism = 'G150C'
id = '00004'
DIR_OUT = './output/'
if not os.path.exists(DIR_OUT):
os.mkdir(DIR_OUT)
filts = ['f115w', 'f150w', 'f200w']
ndithers = np.arange(0, 2, 1)
sig = 5.0
for filt in filts:
print(filt)
# 2d spectrum;
file_2d = f'{DIR_DATA}l3_nis_{filt}_{grism}_s{id}_cal.fits'
hdu_2d = fits.open(file_2d)
for ndither in ndithers:
print(ndither)
if grism == 'G150C':
# If spectrum is horizontal;
data_2d = hdu_2d[ndither*7+1].data
dq_2d = hdu_2d[ndither*7+2].data
err_2d = hdu_2d[ndither*7+3].data
wave_2d = hdu_2d[ndither*7+4].data
else:
data_2d = rotate(hdu_2d[ndither*7+1].data, 90)
dq_2d = rotate(hdu_2d[ndither*7+2].data, 90)
err_2d = rotate(hdu_2d[ndither*7+3].data, 90)
wave_2d = rotate(hdu_2d[ndither*7+4].data, 90)
y2d, x2d = data_2d.shape[:]
plt.close()
plt.imshow(data_2d, vmin=0, vmax=300)
plt.show()
# Re-extract 2d image;
# if ndither == 0:
rsq = y2d
sci_cut = data[int(ycen-rsq/2.+0.5):int(ycen+rsq/2.+0.5), int(xcen-rsq/2.+0.5):int(xcen+rsq/2.+0.5)]
seg_cut = segdata[int(ycen-rsq/2.+0.5):int(ycen+rsq/2.+0.5), int(xcen-rsq/2.+0.5):int(xcen+rsq/2.+0.5)]
# Not sure if the offset in extractioin box is bug ;
if grism == 'G150C':
sci_rot = rotate(sci_cut, PA_V3+0)
else:
sci_rot = rotate(sci_cut, PA_V3+0+90)
# This is for spectral resolution;
# Get light profile along the x-axis
for ii in range(sci_rot.shape[0]):
flux_tmp = sci_rot[ii, :]
xx_tmp = np.arange(0, len(sci_rot[ii, :]), 1)
# Sum along cross-disperse direction
flux_x = np.zeros(len(sci_rot[0, :]), 'float')
for ii in range(sci_rot.shape[0]):
flux_x[ii] = np.sum(sci_rot[ii, :])
# Normalize;
flux_x[:] /= flux_x.sum()
# LSF
iix_peak = np.argmax(flux_x)
xx_tmp_shift = xx_tmp - xx_tmp[iix_peak]
A, xm, gamma, alpha = LSF_mof(xx_tmp_shift, flux_x)
if ndither == 0:
# Write it down;
fm = open(f'{DIR_OUT}l3_nis_{filt}_{grism}_s{id}_moffat.txt', 'w')
fm.write('# A x0 gamma alp\n')
fm.write('# Moffat function\n')
fm.write(f'{A:.3f} {xm:.3f} {gamma:.3f} {alpha:.3f}\n')
fm.close()
# This is for Optimal extraction;
# Sum along x (disperse) direction
flux_y = np.zeros(len(sci_rot[:, 0]), 'float')
for ii in range(sci_rot.shape[0]):
flux_y[ii] = np.sum(sci_rot[ii, :])
# Normalize;
flux_y[:] /= flux_y.sum()
# Following Horne;
flux_disp = np.zeros(x2d, 'float')
err_disp = np.zeros(x2d, 'float')
wave_disp = np.zeros(x2d, 'float')
for ii in range(x2d):
# Mask;
# 1. DQ array
# 2. error value
# 3. CR detection
mask_tmp = (dq_2d[:, ii] == 0) & (err_2d[:, ii] > 0)
ivar = 1. / err_2d[:, ii]**2
num = flux_y[:] * data_2d[:, ii] * ivar
den = flux_y[:]**2 * ivar
flux_disp[ii] = num[mask_tmp].sum(axis=0)/den[mask_tmp].sum(axis=0)
err_disp[ii] = np.sqrt(1./den[mask_tmp].sum(axis=0))
wave_disp[ii] = wave_2d[0, ii]
plt.close()
con_plot = (wave_disp > 0)
plt.errorbar(wave_disp[con_plot], flux_disp[con_plot], yerr=err_disp[con_plot])
plt.ylim(-0, 3000)
plt.show()
# Wirte:
# Now make it into a Spectrum1D instance.
file_1d = f'{DIR_OUT}l3_nis_{filt}_{grism}_s{id}_ndither{ndither}_1d_opt.fits'
if wave_disp[1] - wave_disp[0] < 0:
obs = Spectrum1D(spectral_axis=wave_disp[::-1]*u.um,
flux=flux_disp[::-1]*u.MJy,
uncertainty=StdDevUncertainty(err_disp[::-1]), unit='MJy')
else:
obs = Spectrum1D(spectral_axis=wave_disp*u.um,
flux=flux_disp*u.MJy,
uncertainty=StdDevUncertainty(err_disp), unit='MJy')
obs.write(file_1d, format='tabular-fits', overwrite=True)
f115w
0
![../../../_images/de6647b4dded3f29f76943be1a0600587558f017a3a82414e411bbebab08d050.png](../../../_images/de6647b4dded3f29f76943be1a0600587558f017a3a82414e411bbebab08d050.png)
![../../../_images/120ee15a12fa6ee216a135858fb8d8bea3a683391b4f0091976c60f2e1baa7fa.png](../../../_images/120ee15a12fa6ee216a135858fb8d8bea3a683391b4f0091976c60f2e1baa7fa.png)
![../../../_images/e7e2f5e6c5076b138152e4204680f9f74626acbd8b9b49c639d4a251a1b83800.png](../../../_images/e7e2f5e6c5076b138152e4204680f9f74626acbd8b9b49c639d4a251a1b83800.png)
1
![../../../_images/c873330b746d7a4ed06b04f7317b6cb62cd5214ae13a2782e536b7a52b187e74.png](../../../_images/c873330b746d7a4ed06b04f7317b6cb62cd5214ae13a2782e536b7a52b187e74.png)
![../../../_images/120ee15a12fa6ee216a135858fb8d8bea3a683391b4f0091976c60f2e1baa7fa.png](../../../_images/120ee15a12fa6ee216a135858fb8d8bea3a683391b4f0091976c60f2e1baa7fa.png)
![../../../_images/86021fb2c1d42fbf05aa9139c2aadcbdee8feba202c2bf449cfd8a22d25e5cbd.png](../../../_images/86021fb2c1d42fbf05aa9139c2aadcbdee8feba202c2bf449cfd8a22d25e5cbd.png)
f150w
0
![../../../_images/7a0c2af540d65ed76d7f0b4a3bec51402bdd44b49f347c54e8f0fc40d804fd08.png](../../../_images/7a0c2af540d65ed76d7f0b4a3bec51402bdd44b49f347c54e8f0fc40d804fd08.png)
![../../../_images/120ee15a12fa6ee216a135858fb8d8bea3a683391b4f0091976c60f2e1baa7fa.png](../../../_images/120ee15a12fa6ee216a135858fb8d8bea3a683391b4f0091976c60f2e1baa7fa.png)
![../../../_images/bc397b5e28817ee93721f46507cf9517d8866c49646fe391f2e8402eb23f1258.png](../../../_images/bc397b5e28817ee93721f46507cf9517d8866c49646fe391f2e8402eb23f1258.png)
1
![../../../_images/171e4972d8fe3674826d23a932a1ecd2e43a174b2f757d5f00c85a907689d1a1.png](../../../_images/171e4972d8fe3674826d23a932a1ecd2e43a174b2f757d5f00c85a907689d1a1.png)
![../../../_images/120ee15a12fa6ee216a135858fb8d8bea3a683391b4f0091976c60f2e1baa7fa.png](../../../_images/120ee15a12fa6ee216a135858fb8d8bea3a683391b4f0091976c60f2e1baa7fa.png)
![../../../_images/76561c94b33f4d23d4139908684321e932136f12b2d320e5f8dd7de6a060505b.png](../../../_images/76561c94b33f4d23d4139908684321e932136f12b2d320e5f8dd7de6a060505b.png)
f200w
0
![../../../_images/b5d94fb2a936eaed9f35298dde88b548daea28dab5c31f482a0e28603b6d7079.png](../../../_images/b5d94fb2a936eaed9f35298dde88b548daea28dab5c31f482a0e28603b6d7079.png)
![../../../_images/98b942ba54fa5a6250c88928ac666631f818062ae7bacd96eb40571a23d10258.png](../../../_images/98b942ba54fa5a6250c88928ac666631f818062ae7bacd96eb40571a23d10258.png)
![../../../_images/b906e485eff14cd5e969b6d3ab0301dcb41e5395d979f3210efa1a6c93683051.png](../../../_images/b906e485eff14cd5e969b6d3ab0301dcb41e5395d979f3210efa1a6c93683051.png)
1
![../../../_images/9afdc9b1ed02663e0448b54a8221a38ef7df9b1b4fcc25bdc145e5325e95a9f9.png](../../../_images/9afdc9b1ed02663e0448b54a8221a38ef7df9b1b4fcc25bdc145e5325e95a9f9.png)
![../../../_images/120ee15a12fa6ee216a135858fb8d8bea3a683391b4f0091976c60f2e1baa7fa.png](../../../_images/120ee15a12fa6ee216a135858fb8d8bea3a683391b4f0091976c60f2e1baa7fa.png)
![../../../_images/397dae30cf5075164b7620d8d6e99b76fe12ca515d6aad0fe825bf6f4f81d751.png](../../../_images/397dae30cf5075164b7620d8d6e99b76fe12ca515d6aad0fe825bf6f4f81d751.png)
Another object;#
Absorption line galaxy
grism = 'G150C'
id = '00003'
DIR_OUT = './output/'
filts = ['f115w', 'f150w', 'f200w']
ndithers = np.arange(0, 2, 1) # There are four dithers in the data set;
sig = 5.0
for filt in filts:
print(filt)
# 2d spectrum;
file_2d = f'{DIR_DATA}l3_nis_{filt}_{grism}_s{id}_cal.fits'
hdu_2d = fits.open(file_2d)
for ndither in ndithers:
print(ndither)
if grism == 'G150C':
# If spectrum is horizontal;
data_2d = hdu_2d[ndither*7+1].data
dq_2d = hdu_2d[ndither*7+2].data
err_2d = hdu_2d[ndither*7+3].data
wave_2d = hdu_2d[ndither*7+4].data
else:
data_2d = rotate(hdu_2d[ndither*7+1].data, 90)
dq_2d = rotate(hdu_2d[ndither*7+2].data, 90)
err_2d = rotate(hdu_2d[ndither*7+3].data, 90)
wave_2d = rotate(hdu_2d[ndither*7+4].data, 90)
y2d, x2d = data_2d.shape[:]
plt.close()
plt.imshow(data_2d, vmin=0, vmax=20)
plt.show()
# Re-extract 2d image;
# if ndither == 0:
rsq = y2d
sci_cut = data[int(ycen-rsq/2.+0.5):int(ycen+rsq/2.+0.5), int(xcen-rsq/2.+0.5):int(xcen+rsq/2.+0.5)]
seg_cut = segdata[int(ycen-rsq/2.+0.5):int(ycen+rsq/2.+0.5), int(xcen-rsq/2.+0.5):int(xcen+rsq/2.+0.5)]
# Not sure if the offset in extraction box is bug ;
if grism == 'G150C':
sci_rot = rotate(sci_cut, PA_V3+0)
else:
sci_rot = rotate(sci_cut, PA_V3+0+90)
# This is for spectral resolution;
# Get light profile along the x-axis
for ii in range(sci_rot.shape[0]):
flux_tmp = sci_rot[ii, :]
xx_tmp = np.arange(0, len(sci_rot[ii, :]), 1)
# Sum along cross-disperse direction
flux_x = np.zeros(len(sci_rot[0, :]), 'float')
for ii in range(sci_rot.shape[0]):
flux_x[ii] = np.sum(sci_rot[ii, :])
# Normalize;
flux_x[:] /= flux_x.sum()
# LSF
iix_peak = np.argmax(flux_x)
xx_tmp_shift = xx_tmp - xx_tmp[iix_peak]
A, xm, gamma, alpha = LSF_mof(xx_tmp_shift, flux_x)
if ndither == 0:
# Write it down;
fm = open(f'{DIR_OUT}l3_nis_{filt}_{grism}_s{id}_moffat.txt', 'w')
fm.write('# A x0 gamma alp\n')
fm.write('# Moffat function\n')
fm.write(f'{A:.3f} {xm:.3f} {gamma:.3f} {alpha:.3f}\n')
fm.close()
# This is for Optimal extraction;
# Sum along x (disperse) direction
flux_y = np.zeros(len(sci_rot[:, 0]), 'float')
for ii in range(sci_rot.shape[0]):
flux_y[ii] = np.sum(sci_rot[ii, :])
# Normalize;
flux_y[:] /= flux_y.sum()
# Following Horne;
flux_disp = np.zeros(x2d, 'float')
err_disp = np.zeros(x2d, 'float')
wave_disp = np.zeros(x2d, 'float')
for ii in range(x2d):
# Mask;
# 1. DQ array
# 2. error value
# 3. CR detection
mask_tmp = (dq_2d[:, ii] == 0) & (err_2d[:, ii] > 0)
ivar = 1. / err_2d[:, ii]**2
num = flux_y[:] * data_2d[:, ii] * ivar
den = flux_y[:]**2 * ivar
flux_disp[ii] = num[mask_tmp].sum(axis=0)/den[mask_tmp].sum(axis=0)
err_disp[ii] = np.sqrt(1./den[mask_tmp].sum(axis=0))
wave_disp[ii] = wave_2d[0, ii]
plt.close()
con_plot = (wave_disp > 0)
plt.errorbar(wave_disp[con_plot], flux_disp[con_plot], yerr=err_disp[con_plot])
plt.ylim(-20, 100)
plt.show()
# Write:
# Now, make it into a Spectrum1D instance.
file_1d = f'{DIR_OUT}l3_nis_{filt}_{grism}_s{id}_ndither{ndither}_1d_opt.fits'
if wave_disp[1] - wave_disp[0] < 0:
obs = Spectrum1D(spectral_axis=wave_disp[::-1]*u.um,
flux=flux_disp[::-1]*u.MJy,
uncertainty=StdDevUncertainty(err_disp[::-1]), unit='MJy')
else:
obs = Spectrum1D(spectral_axis=wave_disp*u.um,
flux=flux_disp*u.MJy,
uncertainty=StdDevUncertainty(err_disp), unit='MJy')
obs.write(file_1d, format='tabular-fits', overwrite=True)
f115w
0
![../../../_images/a8223b96ffa1cadb4f89955ce1711519d5d683a3a40197e7fa2a407bbeda2d7e.png](../../../_images/a8223b96ffa1cadb4f89955ce1711519d5d683a3a40197e7fa2a407bbeda2d7e.png)
![../../../_images/fae01c79054c4b75ba4d2cf60e442c9b0eb00f455b1bbe8dc134f18a71df38d8.png](../../../_images/fae01c79054c4b75ba4d2cf60e442c9b0eb00f455b1bbe8dc134f18a71df38d8.png)
![../../../_images/74b2a12f81d7ed2e167cf72a31d74a3f10adb65d2c6ad6f649c357d8f8e0c5fa.png](../../../_images/74b2a12f81d7ed2e167cf72a31d74a3f10adb65d2c6ad6f649c357d8f8e0c5fa.png)
1
![../../../_images/b71836c78057c0a08e25967e34d6a03ebe3e8c572df3add33a2c662ac36b9440.png](../../../_images/b71836c78057c0a08e25967e34d6a03ebe3e8c572df3add33a2c662ac36b9440.png)
![../../../_images/fae01c79054c4b75ba4d2cf60e442c9b0eb00f455b1bbe8dc134f18a71df38d8.png](../../../_images/fae01c79054c4b75ba4d2cf60e442c9b0eb00f455b1bbe8dc134f18a71df38d8.png)
![../../../_images/763e6a067b4c2961d807714917ee3a350cb7243db5ae368b7b97d3caeed577d3.png](../../../_images/763e6a067b4c2961d807714917ee3a350cb7243db5ae368b7b97d3caeed577d3.png)
f150w
0
![../../../_images/2fee79a7e32046699ba5dee7e657597c235d55d16dc09e8650cca7119050e6c9.png](../../../_images/2fee79a7e32046699ba5dee7e657597c235d55d16dc09e8650cca7119050e6c9.png)
![../../../_images/fae01c79054c4b75ba4d2cf60e442c9b0eb00f455b1bbe8dc134f18a71df38d8.png](../../../_images/fae01c79054c4b75ba4d2cf60e442c9b0eb00f455b1bbe8dc134f18a71df38d8.png)
![../../../_images/2b26650527f511a950c849cb8909e369538c8a263cde295d83c75d7383842af2.png](../../../_images/2b26650527f511a950c849cb8909e369538c8a263cde295d83c75d7383842af2.png)
1
![../../../_images/a5d9a85f62e626f51447149a9ab83bfea462a6d96e96ce9ecfccdc9640911c26.png](../../../_images/a5d9a85f62e626f51447149a9ab83bfea462a6d96e96ce9ecfccdc9640911c26.png)
![../../../_images/49ba48ed87633669a53962d0c29c8ccc903dae504bde627b6286ac6b1f757c60.png](../../../_images/49ba48ed87633669a53962d0c29c8ccc903dae504bde627b6286ac6b1f757c60.png)
![../../../_images/bf9e7da28705a872d49be041ced75d793e9448fc2a17e12801f6e8fdeaa3e957.png](../../../_images/bf9e7da28705a872d49be041ced75d793e9448fc2a17e12801f6e8fdeaa3e957.png)
f200w
0
![../../../_images/c54bc7796f0bc4aab87b0adf920b5b400f0f2f17f5b879f0433b6cb9723ab23d.png](../../../_images/c54bc7796f0bc4aab87b0adf920b5b400f0f2f17f5b879f0433b6cb9723ab23d.png)
![../../../_images/fae01c79054c4b75ba4d2cf60e442c9b0eb00f455b1bbe8dc134f18a71df38d8.png](../../../_images/fae01c79054c4b75ba4d2cf60e442c9b0eb00f455b1bbe8dc134f18a71df38d8.png)
![../../../_images/00eab39b5da4541496e5ba33c576866960d99417ff4222dc5202752d873d679a.png](../../../_images/00eab39b5da4541496e5ba33c576866960d99417ff4222dc5202752d873d679a.png)
1
![../../../_images/1c0c28fee5f44ce577a0c83e478b458df5a81d41f15ff6c224569ee35c3f72fd.png](../../../_images/1c0c28fee5f44ce577a0c83e478b458df5a81d41f15ff6c224569ee35c3f72fd.png)
![../../../_images/fae01c79054c4b75ba4d2cf60e442c9b0eb00f455b1bbe8dc134f18a71df38d8.png](../../../_images/fae01c79054c4b75ba4d2cf60e442c9b0eb00f455b1bbe8dc134f18a71df38d8.png)
![../../../_images/ddb5240f8b5d8101953eb937084f5a6e665afe9711a25211d9c55fbe080d5b28.png](../../../_images/ddb5240f8b5d8101953eb937084f5a6e665afe9711a25211d9c55fbe080d5b28.png)