Space Telescope Science Institute
Day: Sep 24-25, 2015
Time: 9:00 am - 5:00 pm
Room: Boardroom
Instructors: Erik Bray, Justin Ely
Helpers: Joseph Long
General Information
Software Carpentry's mission is to help scientists and engineers get more research done in less time and with less pain by teaching them basic lab skills for scientific computing. This hands-on workshop will cover basic concepts and tools, including program design, version control, data management, and task automation. Participants will be encouraged to help one another and to apply what they have learned to their own research problems.
For more information on what we teach and why, please see our paper "Best Practices for Scientific Computing".
Who: Faculty, staff, and post-docs in the INS division. You don't need to have any previous knowledge of the tools that will be presented at the workshop, though you will be expected to have some experience working with the command line shell.
Where: 3700 San Martin Drive, Baltimore, Maryland 21218. Get directions with OpenStreetMap or Google Maps.
Requirements: Participants must bring a laptop with a few specific software packages installed (listed below). They are also required to abide by Software Carpentry's Code of Conduct.
Contact: Please mail embray@stsci.edu for more information.
Schedule
A loose schedule is outlined below, but we're really here to teach the areas +of most interest to the students. As such, the topics for each section may +change based on student feedback. Coffee breaks will be interpersed throughout. Lunch will be at noon.
Day 1
| Setup and introductions |
| Discussion of Unix shell with intermediate tips |
| Introduction to scientific programming and data analysis with Python |
| Split: Python for beginners |
| Split: Building programs with Python |
| Testing with Python |
Day 2
| Version control with Git |
| Managing data with SQL |
| Programming for databases in Python |
| Wrap-up |
Etherpad: https://swcarpentry.etherpad.mozilla.org/2015-09-24-stsci.
We will use this Etherpad for chatting, taking notes, and sharing URLs and bits of code.
Syllabus
The Unix Shell
- Files and directories
- History and tab completion
- Pipes and redirection
- Looping over files
- Creating and running shell scripts
- Finding things
- Reference...
Programming in Python
- Using libraries
- Working with arrays
- Reading and plotting data
- Creating and using functions
- Loops and conditionals
- Defensive programming
- Using Python from the command line
- Reference...
Version Control with Git
- Creating a repository
- Recording changes to files:
add,commit, ... - Viewing changes:
status,diff, ... - Ignoring files
- Working on the web:
clone,pull,push, ... - Resolving conflicts
- Open licenses
- Where to host work, and why
- Reference...
Managing Data with SQL
- Reading and sorting data
- Filtering with
where - Calculating new values on the fly
- Handling missing values
- Combining values using aggregation
- Combining information from multiple tables using
join - Creating, modifying, and deleting data
- Programming with databases
- Reference...
Setup
To participate in this Software Carpentry workshop, you will need access to the software described below.
Most of the software outlined below should be pre-installed on your system (particularly if you're using an STScI-provided laptop). However, you should go over each section and veryify that you do indeed have it installed.
The Bash Shell
Bash is a commonly-used shell that gives you the power to do simple tasks more quickly.
Windows
Download the Git for Windows installer. Run the installer. Important: on the 6th page of the installation wizard (the page titled `Configuring the terminal emulator...`) select `Use Windows' default console window`. If you forgot to do this programs that you need for the workshop will not work properly. If this happens rerun the installer and select the appropriate option. This will provide you with both Git and Bash in the Git Bash program.
Mac OS X
The default shell in all versions of Mac OS X is bash, so no
need to install anything. You access bash from the Terminal
(found in
/Applications/Utilities). You may want to keep
Terminal in your dock for this workshop.
Linux
The default shell is usually Bash, but if your
machine is set up differently you can run it by opening a
terminal and typing bash. There is no need to
install anything.
Git
Git is a version control system that lets you track who made changes to what when and has options for easily updating a shared or public version of your code on github.com. You will need a supported web browser (current versions of Chrome, Firefox or Safari, or Internet Explorer version 9 or above).
Windows
Git should be installed on your computer as part of your Bash install (described above).
Mac OS X
For OS X 10.9 and higher, install Git for Mac
by downloading and running the most recent "mavericks" installer from
this list.
After installing Git, there will not be anything in your /Applications folder,
as Git is a command line program.
For older versions of OS X (10.5-10.8) use the
most recent available installer labelled "snow-leopard"
available here.
Linux
If Git is not already available on your machine you can try to
install it via your distro's package manager. For Debian/Ubuntu run
sudo apt-get install git and for Fedora run
sudo yum install git.
Python
Python is a popular language for scientific computing, and great for general-purpose programming as well. For this course we'll be using the standard STScI science package Ureka. If you're using an STScI laptop, this should already be installed.
We will teach Python using the IPython notebook, a programming environment that runs in a web browser. For this to work you will need a reasonably up-to-date browser. The current versions of the Chrome, Safari and Firefox browsers are all supported (some older browsers, including Internet Explorer version 9 and below, are not).
SQLite
SQL is a specialized programming language used with databases. We use a simple database manager called SQLite in our lessons.
Windows
The Software Carpentry Windows Installer installs SQLite for Windows. If you used the installer to configure nano, you don't need to run it again.
Mac OS X
SQLite comes pre-installed on Mac OS X.
Linux
SQLite comes pre-installed on Linux.