Installing the Hubble Advanced Spectral Products Script

Installing the Hubble Advanced Spectral Products Script#

This Notebook is designed to walk you through downloading, installing, and using the Hubble Advanced Spectral Products (HASP) co-add script.#

Learning Goals:#

By the end of this tutorial, you will:

  • Download conda and create a conda environment

  • Download and install the co-add script

  • Learn how to run the script

  • Understand the naming conventions of the co-added output files

Table of Contents#

0. Introduction

1. Downloading and Installing conda

- 1.1 Installing conda

- 1.2 Creating a conda Environment

2. Downloading and Installing the HASP Script

3. Running the Co-add Script

- 3.1 Running the Script

- 3.2 Understanding the Output Files

0. Introduction#

The Hubble Advanced Spectral Products (HASP) code is a script that co-adds spectra of the same target within programs. This software is able to co-add data taken with the spectrographs onboard the Hubble Space Telescope (HST); the Space Telescope Imaging Spectrograph (STIS) and the Cosmic Origins Spectrograph (COS). The Hubble Spectroscopic Legacy Archive (HSLA) uses this script to co-add these instruments’ data from the MAST archive to create high-quality spectra with a broad wavelength coverate (whenever possible from the ultraviolet to the near-infrared) that is publicly available for the scientific community. These custom co-addition notebooks will instruct users on how to produce their own co-adds in cases where the MAST archive data needs special processing or is rejected by the default filters used in the co-add script.

The script first co-adds the observations for each grating for a given program, then it combines all gratings for the observation set. Finally, it co-adds the spectra of each observation set in the program to produce a fully co-added spectra for each target in a program.

This notebook focuses primarily on the installation of the co-add code, and provides a quick overview on its usage. To see an example of downloading COS and STIS datasets, running the script, and analyzing the output, please check out our notebook CoaddTutorial.ipynb.

1. Downloading and Installing conda#

1.1 Installing conda#

Conda is a package, dependency, and environment manager that runs on Windows, Mac, and Linux. Conda allows us to easily install Python packages and create isolated conda environments, each with their own packages and dependencies. By switching between conda environments, we avoid conflicts between different versions of packages.

We will create a conda environment to install the packages needed for the HASP script. We can first check if you have conda already installed by running the command in the terminal:

!conda --version

Running the command should print out the version of conda that you have installed (e.g. conda 23.7.2). If the command returns a statement saying that the package is unknown, then you do not have conda installed on your machine. You will need to download one of the conda distributions.

There are a few different distributions of conda that you can install, depending on your preferences. Anaconda is one distribution that carries a lot of pre-installed packages, some of which you won’t use. Miniconda is another distribution of conda that contains only the minimum packages. Finally, the Mamba disctribution of conda is similar to Miniconda, but uses different parallelization and cache algorithms to increase speed and optimize memory.

Once you have installed one of these clients, try running the above cell again to confirm that conda is installed.

1.2 Creating a Conda Environment#

Once you’ve installed conda, we can create a conda environment. We will download all of the packages needed to run the HASP script in a new environment that we will create, called hasp-env. We will use this environment for all of the tutorial notebooks.

The first step is to add the conda-forge channel to the list of avaible conda channels. Channels are where conda packages are stored and downloaded from, and conda-forge allows us to download additional packages for the code that the default conda channel may not have available. We can add this channel by running the following command in the terminal:

!conda config --add channels conda-forge

We can now create the conda environment, hasp-env. Note that this can take several minutes to run:

!yes | conda create --name hasp-env python=3.11 notebook jupyterlab numpy astropy astroquery matplotlib

We also downloaded some additional packages that we will need outside of the HASP script to analyze the data. Once we activate the conda environment, as seen below, we can download the HASP script and run it. Note that you should run this in a terminal rather than in the cell below. Depending on your shell and operating system settings, you may need to restart your Jupyter notebook application in order for your environment to be fully activated in your notebook.

!conda activate hasp-env

Now that we created and activated a conda environment, we can now begin to download the HASP code.

2. Downloading and Installing the HASP Script#

We will download the HASP wrapper script from the HASP Github repository. Downloading this script will create a hasp directory, which will contain all of the required code.

You will only have to run one of the two options below, depending on your computer and Git settings.

Option A, using pip:#

pip install clones the reposoitory and install the packages according to the setup configuration of the repository. You may need to create a Personal Access Token (PAT) to use instead of your Github account’s password, since support for password authentication was removed in 2021. You can learn how to create a PAT here.

pip install git+https://github.com/spacetelescope/hasp.git

Option B, using git clone:#

While git clone still downloads the repository, it differs from pip in that it also downloads all of the history and files of the repository. If you have a password-protected SSH key, rather than a PAT, you can clone the repo by running the following code in your terminal. If you want to setup a SSH key, you can learn how to do so here.

git clone git@github.com:spacetelescope/hasp.git

After you’ve downloaded the code using Options A or B, run the command below to download the script’s dependencies:#

!pip install ./hasp/.

This will install additional dependencies using the pyproject.toml file.

To run the wrapper, We only need to download the following COS and/or STIS files:

  • X1D - the one-dimensional extracted product spectra.

  • SX1 - the one-dimensional extracted spectra from combined or cosmic-ray rejected images. This file is only produced with STIS data.

Make sure that all of these files, for every spectra you wish to abut, are in the same directory. The script will only co-add the files within this directory.

3. Running the Co-add Script#

3.1 Running the Script#

Now that we have the wrapper installed, we can now run the co-add script.

Command-line Option

Value

--input_directory (-i)

The path to the directory that contains the data to be co-added (required)

--output_directory (-o)

The path to the directory that will contain the newly co-added products** (required)

--threshold (-t)

The threshold for flux-based filtering (optional)

--snrmax (-s)

The maximum SNR for flux-based filtering (optional)

--no_keyword_filtering (-k)

Disable keyword based filtering (except for STIS PRISM data, which is always filtered) (optional)

**Note: If you wish to exclude certain data files from the co-add, you can just remove them from your input directory before you run the script.

To finally run the script, open your terminal, activate your conda environment, and run:

swrapper -i <YOUR_DATA_DIR> -o <YOUR_PRODUCT_DIR>

You should now have created the co-added spectra for your desired program.

3.2 Understanding the Output Files#

The script produces multiple different files with abutted spectra. Currently, the script outputs abutted products for a single program. It first creates co-added spectra for each grating of a single observation set:

hst_programID_instrument_targetname_grating_obset_cspec.fits

It then co-adds the spectra of all gratings for a single observation set:

hst_programID_instrument_targetname_allGratings_obset_cspec.fits

Finally, it co-adds all abutted observation sets’ spectra to create a final co-added product for a single target:

hst_programID_instrument_targetname_allGratings_cspec.fits

An example of this will be below. These filenames are the output files for a STIS GD71 dataset that is co-added in the CoaddTutorial.ipynb notebook example. Here, the programID is 7656, the instrument is STIS, and the targetname is gd71.

Step

Filename

Description

1

hst_7656_stis_gd71_g140l_o4a520_cspec.fits

Co-adding all G140L observations for the observation set, O4A520.

2

hst_7656_stis_gd71_g140l-g230l-g430l-g750l_o4a520_cspec.fits

Co-adding all observations taken at every grating for the observation set, O4A520.

3

hst_7656_stis_gd71_g140l-g230l-g430l-g750l_o4a5_cspec.fits

Co-adding all GD71 observations at each grating for this program, O4A5.

Note: HST file naming conventions use a combination of three letters and/or numbers to have a unique association between a PI’s proposal ID and program ID, meaning that o4a5 at the end of hst_7656_stis_gd71_g140l-g230l-g430l-g750l_o4a5_cspec.fits is essentially the program ID for our example. Check out more information on the MAST HST file naming convention page

About this Notebook#

Author: Sierra Gomez (sigomez@stsci.edu)

Updated on: 12/04/2023

This tutorial was generated to be in compliance with the STScI style guides and would like to cite the Jupyter guide in particular.

Space Telescope Logo
Contents