OBSOLETE
This page is now obsolete. It has been replaced by this page: https://www.gemini.edu/observing/phase-iii/reducing-data .
Please update your bookmark.
Gemini Data Reduction Software
Gemini offers data reduction software for its facility instruments. The Gemini IRAF package and the DRAGONS platform are the official data reduction software supported by Gemini.
For the next several years we will be transitioning from the IRAF platform to our new Python-based DRAGONS platform. As time goes by, more and more instruments and modes will be supported by DRAGONS. During this transition, it is possible that users will require both platforms depending on the data they have obtained.
- Latest version of Gemini IRAF: v1.15 (June 2022)
- Latest version of DRAGONS: v3.0.4 (November 2022)
Which platform should I use?
Gemini IRAF currently still support all the instruments and modes. However for any imaging data from current instruments, we recommend using DRAGONS. DRAGONS currently support imaging data reduction, but does not support any spectroscopy, for now.
| GMOS Imaging | DRAGONS |
| NIRI Imaging | DRAGONS |
| GNIRS Keyhole Imaging | DRAGONS |
| Flamingos-2 Imaging | DRAGONS |
| GSAOI Imaging | DRAGONS, plus Disco-Stu |
| GMOS longslit spectroscopy | Gemini IRAF for science reduction, DRAGONS for quicklook |
| Any other spectroscopy | Gemini IRAF |
| Decommisioned Instruments | Gemini IRAF |
Download and Install
DRAGONS and Gemini IRAF, support software and dependencies, are all installed the same way using the Anaconda distribution system with conda-forge.
Requirements
- Gemini IRAF
-
- Python 3.7
- 32-bit compatibility libraries
- Linux, equilvalent to CentOS 7 and above
- Or Mac OS X 10.13 to 10.14.0
To run Gemini IRAF on Mac OS X 10.14.1 and above, you will need to use a virtual machine. See https://gemini-iraf-vm-tutorial.readthedocs.io for instructions.
- DRAGONS
-
- Python 3.7
- Linux, equivalent to CentOS 7 and above
- Or Mac OS X 10.13 and above
- Intel Anaconda
Please note starting DRAGONS v3.0, Python 2.7 is no longer supported. Python 3 is now required.
See the installation instructions here:
Announcements
Switch to conda-forge for installing data reduction software
January 12, 2023
STScI has recently announced that it will end support for the main Astroconda channel, used for installing Gemini data reduction packages, on 1 February 2023.
We were already planning to base our upcoming DRAGONS 3.1 release on the more up-to-date package set at conda-forge and have been working for the past few weeks to ensure that dependencies of Gemini IRAF and DRAGONS 3.0 can also be installed using conda-forge, in place of STScI's Astroconda channel and Anaconda's "defaults". As of writing, these instructions also apply to the latest (July 2022) version of the Gemini IRAF VM.
All users are advised to remove STScI's channel from your Anaconda configuration as follows:
conda config --remove channels http://ssb.stsci.edu/astroconda
Until this is done, you may find it impossible to install or update any conda packages at all after 1 February.
You should also add conda-forge and Gemini's public channel as follows -- even if you already have them defined (to make sure they appear in the correct order of precedence):
conda config --add channels conda-forge conda config --add channels http://astroconda.gemini.edu/public
When you next need to update packages, instead of doing so within your existing ("geminiconda" or similar) environment, you should create a new environment, to get a consistent set of packages from the new channels; Gemini will not be able to troubleshoot problems due to installation from an unsupported mixture of channels.
An environment with DRAGONS and Gemini IRAF can be created in much the same way as previously, omitting any "--no-channel-priority" flag. You will no longer be able to install the "stsci" meta-package alongside our software, however, since STScI is dropping support for it. If you need both Gemini and STScI data reduction packages, you should create separate conda environments for each one and follow STScI's instructions at the above link, to set up "stenv". Since DS9 was previously part of "stsci", you should now include that package separately. You should also specify "dragons" explicitly, to help ensure that the latest version gets picked up.
In summary, you can type the following on Linux:
conda create -n geminiconda3 python=3.7 iraf-all pyraf-all ds9 dragons
(where "geminiconda3" is whatever you decide to call the new environment, to distinguish it from the old one).
On MacOS 10.15+, you should omit the IRAF-related packages, since they can no longer run there natively, without the IRAF (Linux) VM:
conda create -n dragons python=3.7 ds9 dragons
Anyone still wanting to run PyRAF natively on MacOS 10.13-10.14 should specify "python=3.7.9" to get a suitable combination of packages on those old OSs, as a workaround for several conda-forge and conda limitations.
Because the older Python 2 stack is heavily dependent on Astroconda, it may no longer be possible to maintain Python 2 environments after January, but you may continue using any that you already have, until something needs updating; all new environments should use Python 3 (see our announcement from 23 August 2022).
We appreciate your patience while any remaining wrinkles are ironed out; any further updates will be announced here.
DRAGONS v3.0.4 Patch Release Available
November 15, 2022
This patch release includes several small fixes and improvements, many related to the Quality Assessment Pipeline run internally at Gemini. Provenance for flux calibration is now included. The patch is recommended to all but not critical for most.
For a complete list of changes since v3.0.3 see:
https://dragons.readthedocs.io/en/v3.0.4/changes.html
PyRAF on Python 3
August 23, 2022
Following some collaboration with the "IRAF Community" project earlier this year, we are pleased to make available PyRAF 2.2.1, the first version to pass all our testing on Python 3. This runs roughly as fast as the Python 2 version, thanks to improvements by Ole Streicher.
This version is already included on the new IRAF VM for Apple machines (see yesterday's announcement). On Linux, it can be installed by specifying "python=3.7" instead of "python=2.7" in the "conda create" command -- see:
Linux builds are also available for Python 3.8 & 3.9, but using 3.7 allows you to install it alongside the current public DRAGONS version. PyRAF 2.2.1 is NOT compatible with Python 2, where the latest version will remain at 2.1.15.
New IRAF VM for MacOS
August 22, 2022
A new virtual machine implementation for running IRAF on MacOS 10.14+ is now available. You can find updated instructions at:
https://gemini-iraf-vm-tutorial.readthedocs.io
This should provide a reliable and relatively convenient way of running Gemini IRAF on recent Apple computers with an M1/M2 processor (though M2 has not been tested yet). However, this comes with an order-of-magnitude speed penalty, because of the need to emulate an Intel CPU on ARM64, which is unavoidable without having 64-bit ports of Gemini IRAF and its dependencies (such as STSDAS). On Apple machines with an Intel CPU, the virtualization overhead is much smaller.
DRAGONS v3.0.3 Patch Release Available
July 8, 2022
A patch release of DRAGONS is available. This patch adds better handling of the GMOS-S data following the January failure of amplifier #5. For that data, the whole amplifier is automatically masked as bad pixels ensuring that those pixels are not used in calculations. The patch also includes bug fixes, small improvements, and fixes to the documentation.
For a complete list of changes since v3.0.1 see:
https://dragons.readthedocs.io/en/v3.0.3/changes.html
Gemini IRAF v1.15 with new support for Flamingos 2 MOS
June 8, 2022
A new version of Gemini IRAF v1.15 with new support for Flamingos 2 MOS (Multi-Object Spectroscopy) data is available.
Along with Flamingos 2 MOS support, the new release includes the changes added in the v1.14 patch released two months ago, as well as several other bug fixes.
The package is distributed via conda. See the installation instructions page for details.
Revisions relative to v1.14 are summarized here: gemini_v115_rev.txt
Gemini IRAF Patch release for v1.14 to support new Flamingos 2 fitlers
April 19, 2022
A patch release for Gemini IRAF v1.14 is available.
The replacement of two filters and the reshuffling of another in Flamingos 2 necessitated additions and modifications to lookup tables and modifications to the code. This patch release contains only the affected files. The patch is to be applied on-top of an already installed Gemini IRAF v1.14 package.
Installation instructions and content of the patch are available in gemini_v114_patch1.txt.
Download: gemini_v114_patch1.tar.gz
DRAGONS v3.0.1 Patch Release Available
December 6, 2021
A patch release of DRAGONS is available. It includes bug fixes, small improvements, and fixes to the documentation.
For a complete list of changes since v3.0.0 see:
https://dragons.readthedocs.io/en/v3.0.1/changes.html
DRAGONS v3.0 Released
October 12, 2021
A new version of DRAGONS has been released. This version offers improvements and fixes to the imaging support. New is this version is support for GMOS longslit spectroscopy for quicklook purposes. This new spectroscopy support is not approved for science quality reduction, but can be use for "quicklook" inspection of new data. A quicklook reduction is intended to check the overall quality of the data, such as confirming the presence of the target(s) and whether the expected S/N has been achieved. It is particularly useful for assessing ToO observations quickly. We are continuing to develop DRAGONS and the next major release will support science-quality reduction of GMOS longslit data, with optional interactive tools for key reduction steps as well as additional routines such as flagging cosmic rays and performing the slit function correction.
For a complete list of changes since v2.1 see:
https://dragons.readthedocs.io/en/v3.0.0/changes.html
Virtual machine image for running IRAF under recent MacOS releases
May 19, 2020
A CentOS 7 virtual machine image (OVA file) is now available to facilitate running Astroconda IRAF under MacOS 10.15+, which no longer supports running the necessary 32-bit binaries natively. This comes with Anaconda 2019.10, Gemini IRAF 1.14, DRAGONS 2.1.0 and other packages from Astroconda pre-installed. Users of MacOS 10.14 affected by the Tk bug that causes a desktop session logout when displaying graphics may also want to install this guest distribution as a workaround.
Please see instructions at https://gemini-iraf-vm-tutorial.readthedocs.io.
New DRAGONS Patch Release
April 20, 2020
A bug fix release of DRAGONS is now available. In release 2.1.1, we have fixed bugs and typos found by the users and ourselves since the initial release. We have also added compatibility with astropy v4. If you already have DRAGONS installed, you can update by doing conda install dragons=2.1.1. If you need to install DRAGONS for the first time, please see the Download and Installation Instructions.
This update also contains an update of disco_stu.
For information and tutorials on DRAGONS, see the "DRAGONS Information" section.
DRAGONS First Public Release!
October 31, 2019
It is with great delight that we are announcing the first public release of Gemini's new Python-base data reduction platform, DRAGONS, Data Reduction for Astronomy from Gemini Observatory North and South. This project has been many years in the making. DRAGONS offers a more streamlined approached to the data reduction of Gemini data, compared to the Gemini IRAF package.
This release, version 2.1.0, supports imaging reduction only, for the current facility instruments. For spectroscopy data, please continue to use Gemini IRAF for the time being. Work is on-going regarding spectroscopy-support in DRAGONS but it will be a while before it is publicly available for science-quality reduction.
To download DRAGONS: Download and Installation Instructions
For information and tutorials on DRAGONS, see the "DRAGONS Information" section.
IMPORTANT: MacOS 10.14.6 and 10.15 incompatibilities with data reduction software
October 11, 2019
As of this week's v10.15 release, MacOS is no longer capable of running the 32-bit Astroconda IRAF distribution needed by Gemini IRAF. For the time being, Gemini IRAF users on Apple machines are advised to continue using MacOS 10.14 or earlier, or to install Astroconda in a virtual machine with a compatible OS. Gemini will look into providing a ready-made VM image to help with this while we are migrating our data reduction tools to Python. Furthermore, MacOS 10.14.6 suffers from a bug that can cause a desktop session logout when attempting to display plots or images with PyRAF, DS9, Matplotlib or other software that uses Tk. We suggest that PyRAF users on 10.14 avoid updating their OS until such time as this problem is resolved by Apple and/or we can determine a reliable workaround (check here for further announcements). IRAF CL is unaffected, but note that we are no longer testing it routinely and are aware of occasional failures with Gemini IRAF.
Gemini IRAF Information
Use Gemini IRAF to reduce Gemini facility instrument data, including decommissioned instruments. Get to the Gemini IRAF page for a Description of the package, details of its content, and list of releases and revision history.
However, for imaging data from currently active instruments, we recommend the use of DRAGONS (below) instead.
Use DRAGONS to reduce imaging data from the currently active Gemini facility instruments: GMOS, NIRI, Flamingos-2, GSAOI, GNIRS (keyhole). For spectroscopy data, you must use Gemini IRAF. However, now you can obtain a quicklook reduction of GMOS longslit spectroscopy data. A quicklook reduction is intended to check the overall quality of the data, not to be used as a science quality reduction.
The DRAGONS documentation is hosted on readthedocs.org.
- Tutorials
- User Manuals
- Programmer Manuals
Science quality verification for DRAGONS imaging modes is discussed in this report.
The primary reference to be cited by users of DRAGONS is:
View the recording of the demo of DRAGONS 3.1 for the science quality reduction of GMOS longslit data to see what's coming next...
Other Gemini Data Reduction Software
- Disco-Stu - Distortion Correction and Stacking Utility
Disco-Stu is a software package for GSAOI images. This standalone package, written in python, will align and stack images that have already been processed by DRAGONS or the Gemini IRAF gareduce task. The current release is v1.3.7.- To install:
conda install disco_stu
Note that disco_stu is installed by default when doing a full install of Gemini IRAF and DRAGONS with conda.- Documentation
- Disco-Stu Manual
- gemini_calmgr - Gemini Local Calibration Manager
The Gemini Local Calibration Manager,
gemini_calmgr, is a python package that DRAGONS uses to associate data to reduce with the best processed calibration available. The Local Calibration Manager handles a lightweight database where the users can upload information about the processed calibration they have produced. DRAGONS can then use the calibration association rules to identify and retrieve automatically the master bias, the master flat, etc. that the data requires. The calibration association rules are the same as those used in the Gemini Observatory Archive.The package
gemini_calmgris installed automatically as part of a DRAGONS installation.
Need help?
To get help with Gemini IRAF or DRAGONS, please use the Gemini Helpdesk system. Use the Gemini IRAF category even for DRAGONS questions.
If you find a bug with DRAGONS, please consider reporting on the DRAGONS Github issues portal. You will need a Github account.
For comment, suggestions, and general feedback on DRAGONS, please add a comment to this decidicated post on the Gemini Data Reduction User Forum.