Omar Laurino - Sherpa, Python, Astronomy coevolution - ADASS XXVI Trieste

Sherpa, Python, and Astronomy
A successful co-evolution

Omar Laurino

and the Sherpa team

October 19th, 2016

Sherpa

Chandra X-Ray Observatory, July 1999
Chandra X-Ray Observatory, July 1999

Sherpa Features

Fitting a Spectrum
Fitting a Spectrum

Fit N-D data sets in the Poisson/Gaussian regime, on Linux and OS X

Sherpa Features

A Python Function User Model
A Python Function User Model

Build complex model expressions, including instrument response and your own Python functions

Sherpa Features

2D Projection of the Parameter Space
2D Projection of the Parameter Space

Calculate confidence levels and visualize parameter space

Sherpa Features

Comparison of Optimization Methods
Comparison of Optimization Methods

Choose a robust optimization method for the fit:

Levenberg-Marquardt
Nelder-Mead Simplex
Differential Evolution

Sherpa Features

MCMC vs 2D Projection
MCMC vs 2D Projection

Perform Bayesian analysis using the Markov Chain Monte Carlo

Sherpa Features

Awesome Custom Analysis
Awesome Custom Analysis

Use Python to create complex analysis tasks and extend the built-in functionality.

Sherpa: from CIAO to Standalone

Chandra Interactive Analysis of Observations
Chandra Interactive Analysis of Observations

How the Landscape has changed

Software Development

Software Development

New Tools

Distributed Version Control git, GitHub, GitLab, Mercurial, BitBucket
Distributed Version Control
git, GitHub, GitLab,
Mercurial, BitBucket

Automated Quality Assurance

Continuous Integration
Travis CI & Co.

Coverage Analysis
Coveralls & Co.

Static Code Analysis
Code Climate & Co.

New Tools

Tool-Assisted Lightweight Code Reviews
Tool-Assisted Lightweight Code Reviews

New Tools

Packaging and Portability
Packaging and Portability
Automation and Reproducibility
Automation and Reproducibility

Astronomical Software Development

Modernizing Sherpa

Challenges

UI vs API design and documentation

Some users prefer:

load_data '3c273.pi'
fit
IOError: [Errno 2] No such file or directory: '3c273.pi'

Challenges

Other Users (and all Developers) prefer:

from sherpa.astro.ui.utils import Session
s = Session()
s.load_data(input)
s.fit()
...
/soft/ciao-4.8/lib/python2.7/site-packages/sherpa/utils/__init__.pyc in is_binary_file(filename)
   1132     1024 bytes of the file.
   1133     """
-> 1134     fd = open(filename, 'r')
   1135     lines = fd.readlines(1024)
   1136     fd.close()

IOError: [Errno 2] No such file or directory: '3c273.pi'

Challenges

Portability
Performance
Coding Style (PEP8)
Open-ended Dependencies
Python 2 and 3
CIAO vs Standalone

Challenges

Infrastructure
vs
Science

A New Ecosystem

Projects using Sherpa

Multiwavelength Analysis with Sherpa

The Future

⇒ Your code and science here ⇐
  • Migrate documentation to sphinx
  • Integrated docker pipeline
  • On demand Sherpa cloud notebooks: move Sherpa to the Data!
  • Sherpa as a pathfinder for Chandra software projects

Try it yourself

Sherpa Workshop at Grapevine AAS

QR Code

http://cxc.cfa.harvard.edu/contrib/sherpa/

https://github.com/sherpa/sherpa

Binder