Reading and Writing binary data

astropy.io.fits

astropy.io.fits is a Python module developed at STScI to read and write all types of FITS files.

External resource!

The astropy.io.fits tutorial itself is good for our purpose and since this is the internet I will not reinvent the wheel. Read the manual then come back to this page for an exercise.

Exercise

Use the following code to download a FITS file for this exercise:

from astropy.extern.six.moves.urllib import request
import tarfile
url = 'http://python4astronomers.github.com/core/core_examples.tar'
tarfile.open(fileobj=request.urlopen(url), mode='r|').extractall()
cd py4ast/core
ls

Read in the FITS file. Find the time and date of the observation. Then use plt.imshow() to display the intensity array using some sensible minimum and maximum value so that the spectrum is visible.

Click to Show/Hide Solution

Here is a possible solution:

from astropy.io import fits
hdus = fits.open('3c120_stis.fits.gz')
hdus.info()
head = hdus[0].header
head.keys()             # lists all keywords in a dictionary
head['TDATEOBS']
head['TTIMEOBS']
img = hdus[1].data      # Intensity data

plt.clf()
plt.imshow(img, origin = 'lower', vmin = -10, vmax = 65)
plt.colorbar()

You might recognize this piece of code. It was used before in the A Crash Course in Scientific Python: 2D STIS Reduction part of the tutorial, but now you should understand the astropy.io.fits commands in more detail.

Reading IDL .sav files

IDL is still a very common tool in astronomy. While IDL packages exist to read and write data in simple (ASCII) or standardized file formats (FITS), that users of all platforms can use, IDL also offers a binary file format with an undocumented, proprietary structure. However, acess to this file format (usually called .sav) is very simple and convenient in IDL. Therefore, many IDL users dump their data in this way and you might be forced to read .sav files a collegue has sent you.

Here is an examplary .sav file. If you have trouble downloading the file, then use IPython:

from astropy.extern.six.moves.urllib import request
url = 'http://python4astronomers.github.com/_downloads/myidlfile.sav'
open('myidlfile.sav', 'wb').write(request.urlopen(url).read())
ls

What can you do?

1. Convert your collegue to use a different file format.
2. Read that file in python.

With any relatively recent version scipy (at least 0.9) then this is a matter of two lines:

from scipy.io.idl import readsav
data = readsav('myidlfile.sav')

Exercise: Where is your data?

idlsave already prints some information on the screen while reading the file. Inspect the object data, find out how you use it and plot the x and y data in it.

Click to Show/Hide Solution

data is a dictionary and all the variables in the .sav file are fields in this dictionary. You get a list with data.keys(). Then, this is easy:

plt.plot(data['x'], data['y'])

Note that idlsave cannot write files and that it will fail to read if the .sav file contains special structures like system variables or compiled IDL code.