Data Reduction

Content owned by kchiboucas

Announcements

Announcements prior to 2020
View All GRACES News

These pages present the most up-to-date information about the data format and the data reduction.

Data format

This page describes:

  1. How to get the data from the Gemini Observatory Archive (GOA)
  2. How to get the associated calibrations from GOA
  3. What information can be found in the header
  4. What the data look like
  5. Format of processed data files from Opera (also available from GOA)

Data reduction

This page describes different options to get extracted spectra. 

 Data Format

How to get the data from Gemini Observatory Archive (GOA)

  1. Access the GOA search form and either Login to GOA or Request an Account
  2. Search for your program ID
  3. Follow the instructions to download your data

How to get associated calibrations

Biases/arcs/flats:

The GRACES calibration data are not supported with the GOA for the function "Load Associated Calibrations". They need to be selected manually. There are two good ways to do that:

Entering the calibration progID
  1. Select Instrument = GRACES
  2. Select Program ID = GN-CALYYYYYMMDD, where YYYYMMDD is the UT date that corresponds to when the science frames were ontained.
Picking the calibrations one by one
  1. Select Instrument = GRACES
  2. Enter the UTC Date that corresponds to the date when the science frames were obtained.
  3. Select the Obs. Type to be Bias, Flat or Arc, which are the three baseline calibration provided.
If your data were taken on different nights, it is required to use the calibrations for all the corresponding nights, as calibrations cannot be used for science frames observed more than 1 day before or after.

Finding flux standards (and moon calibrator - object 301):

For data taken in 17B or earlier:
  1. Select Instrument = GRACES
  2. Select Program ID = GN-YYYYS-SV-171, where YYYY is the year and S is the A or B semester.
  3. Under Advanced options, include Science Verification data
For data taken in 18A or later:
  1. Select Instrument = GRACES
  2. Select Program ID = GN-YYYYS-CAL-171, where YYYY is the year and S is the A or B semester.
  3. Under Advanced options, include Engineering data

The headers

Changes have been made to the GRACES headers due to the changes from 1 Amp to 2 Amp. Data taken before June 2017 were taken in 1 Amp mode. From June 2017, data are taken in 2 Amp mode. Note these are nominal values provided by CFHT.:

AMPLIST = 'a ' / List of amplifiers for this image
AMPLIST = 'a,b ' / List of amplifiers for this image

CCDSEC = '[1:2048,1:4608]' / Read out area of the detector (unbinned)
CCDSEC = '[21:2068,1:4608]' / Read out area of the detector (unbinned)

DATASEC = '[1:2048,1:4608]' / Imaging area of the detector
DATASEC = '[21:2068,1:4608]' / Imaging area of the detector

BIASSEC = '[2049:2080,1:4608]' / Overscan (bias) area of the detector
is replaced by whole new keywords:
TRIMSEC = '[21:2068,4:4605]' / Useful imaging area of the detector
BSECA = '[1:20,1:4608]' / Overscan (bias) area from Amp A
BSECB = '[2069:2088,1:4608]' / Overscan (bias) area from Amp B
CSECA = '[21:1044,1:4608]' / Section in full CCD for DSECA
CSECB = '[1045:2068,1:4608]' / Section in full CCD for DSECB
DSECA = '[21:1044,1:4608]' / Imaging area from Amp A
DSECB = '[1045:2068,1:4608]' / Imaging area from Amp B
TSECA = '[21:1044,4:4605]' / Trim section for Amp A
TSECB = '[1045:2068,4:4605]' / Trim section for Amp B

GAINA = 1.30 / Amp A gain (electrons/ADU)
RDNOISEA= 4.20 / Amp A read noise (electrons)
GAINA = 1.30 / Amp A gain (electrons/ADU)
GAINB = 1.30 / Amp B gain (electrons/ADU)
RDNOISEA= 4.20 / Amp A read noise (electrons)
RDNOISEB= 4.10 / Amp B read noise (electrons)

(Note, these are nominal values provided by CFHT. Our measured values are found here.)

In Slow readout mode:
RDTIME = 60.00 / Read out time (sec)
EREADSPD= 'Slow: 2.90e noise, 1.20e/ADU, 60s' / ESPaDOnS det read out
RDTIME = 30.00 / Read out time (sec)
EREADSPD= 'Slow: 2.90e noise, 1.15e/ADU, 30s' / ESPaDOnS det read out
In Normal readout mode:
RDTIME = 38.00 / Read out time (sec)
EREADSPD= 'Normal: 4.20e noise, 1.30e/ADU, 38s' / ESPaDOnS det read out
RDTIME = 19.00 / Read out time (sec)
EREADSPD= 'Normal: 4.15e noise, 1.30e/ADU, 19s' / ESPaDOnS det read

How the data look like

The full GRACES spectral is 400-1,050 nm. The useful range is 420-1,010nm.

Raw 2D spectrum of star HIP 57258 (GRACES phase I first light).

This figure shows the raw 2D spectrum of the A3 star HIP 57258. It also is the GRACES phase I first light! 

For data processed with Opera:

About the i.fits and m.fits files

The fits files can be read using the usual softwares, like python or IDL. They contain multiple columns, and the number of columns depends on the spectroscopic mode. The format follows the standards established by the pipeline Libre-ESpRIT used at CFHT.

FITS keywords identify the type of reduction and content of the file:


REDUCTIO= 'Intensity' / Type of reduction
FILENAME " Base filename at reduction"
DATE " UTC Date of reduction"
UTIME " UTC time of reduction"
HSTTIME " Local time in Hawaii of reduction"
FILENAMO " Base filename at acquisition"
DATE " UTC Date of reduction"
UTIME " UTC time of reduction"
HSTTIME " Local time in Hawaii of reduction"

For the 1-fiber spectroscopic mode, the i.fits files have:

Normalized spectra

COL1 = 'Wavelength' / Normalized
COL2 = 'Intensity' / Normalized
COL3 = 'ErrorBar' / Normalized

Unormalized spectra (counts are in electrons)

COL4 = 'Wavelength' / UnNormalized
COL5 = 'Intensity' / UnNormalized
COL6 = 'ErrorBar' / UnNormalized

Normalized and unormalized spectra, but without autoccorection of the wavelength calibration using the telluric lines

COL7 = 'Wavelength' / Normalized, no autowave correction
COL8 = 'Intensity' / Normalized, no autowave correction
COL9 = 'ErrorBar' / Normalized, no autowave correction
COL10 = 'Wavelength' / UnNormalized, no autowave correction
COL11 = 'Intensity' / UnNormalized, no autowave correction
COL12 = 'ErrorBar' / UnNormalized, no autowave correction

For the 2-fiber spectroscopic mode, the i.fits files have:

Normalized spectra

COL1 = 'Wavelength' / Normalized
COL2 = 'Target ' / Normalized - Target (science fiber) corrected by the Sky (sky fiber)
COL3 = 'Sky' / Normalized - Sky as observed in the science fiber (sky fiber)
COL4 = 'Target+Sky ' / Normalized - Target+Sky spectrum (science fiber)
COL5 = 'ErrorBar1' / Normalized
COL6 = 'ErrorBar2' / Normalized
COL7 = 'ErrorBar3' / Normalized

Unormalized spectra (counts are in electrons)

COL8 = 'Wavelength' / UnNormalized
COL9 = 'Target ' / UnNormalized - Target (science fiber) corrected by the Sky (sky fiber)
COL10 = 'Sky' / UnNormalized - Sky as observed in the science fiber (sky fiber)
COL11 = 'Target+Sky ' / UnNormalized - Target+Sky spectrum (target fiber)
COL12 = 'ErrorBar1' / UnNormalized
COL13 = 'ErrorBar2' / UnNormalized
COL14 = 'ErrorBar3' / UnNormalized

Normalized and unormalized spectra, but without autoccorection of the wavelength calibration using the telluric lines

COL15 = 'Wavelength' / Normalized, no autowave correction
COL16 = 'Target ' / Normalized, no autowave correction
COL17 = 'Target+sky' / Normalized, no autowave correction
COL18 = 'Sky ' / Normalized, no autowave correction
COL19 = 'ErrorBar1' / Normalized, no autowave correction
COL20 = 'ErrorBar2' / Normalized, no autowave correction
COL21 = 'ErrorBar3' / Normalized, no autowave correction
COL22 = 'Wavelength' / UnNormalized, no autowave correction
COL23 = 'Target ' / UnNormalized, no autowave correction
COL24 = 'Target+sky' / UnNormalized, no autowave correction
COL25 = 'Sky ' / UnNormalized, no autowave correction
COL26 = 'ErrorBar1' / UnNormalized, no autowave correction
COL27 = 'ErrorBar2' / UnNormalized, no autowave correction
COL28 = 'ErrorBar3' / UnNormalized, no autowave correction

For the 2-fiber spectroscopic mode, the m.fits files have:

COL1 = 'Order ' / Order number
COL2 = 'NElements' / Number of elements in order
COL3 = 'NBeams ' / Number of beams (2 for 2-f)
COL4 = 'ElementIndex' / Index of this element
COL5 = 'Wavelength' / Uncorrected wavelength (nm)
COL6 = 'Tell ' / Wavelength with telluric correction applied (nm)
COL7 = 'RVel ' / Heliocentric wavelength correction (nm)
COL8 = 'XCorr ' / Cross-correlation
COL9 = 'RawFlux ' / Raw flux
COL10 = 'RawFluxVar' / Raw flux variance
COL11 = 'NormalizedFlux' / Normalized flux
COL12 = 'NormalizedFluxVar' / Normalized flux variance
COL13 = 'FcalFlux' / Calibrated flux
COL14 = 'FcalFluxVar' / Calibrated flux variance
COL15 = 'Beam ' / Beam number (starting from 0)  0 = sky fiber, for col 16 - 21
COL16 = 'BeamRawFlux' / Faw beam flux for the beam (electron)
COL17 = 'BeamRawFluxVar' / Raw beam flux variance (electron)
COL18 = 'BeamNormalizedFlux' / Normalized beam flux
COL19 = 'BeamNormalizedFluxVar' / Normalized beam flux variance
COL20 = 'BeamFcalFlux' / UnNormalized beam flux (electron)
COL21 = 'BeamFcalFluxVar' / UnNormalized beam flux variance (electron)
COL22 = 'Beam ' / Beam number (starting from 0) 1 = target fiber, no sky subtraction, for col 23 - 28
COL23 = 'BeamRawFlux' / Faw beam flux for the beam (electron)
COL24 = 'BeamRawFluxVar' / Raw beam flux variance (electron)
COL25 = 'BeamNormalizedFlux' / Normalized beam flux
COL26 = 'BeamNormalizedFluxVar' / Normalized beam flux variance
COL27 = 'BeamFcalFlux' / UnNormalized beam flux (electron)
COL28 = 'BeamFcalFluxVar' / UnNormalized beam flux variance (electron)

For the 1-fiber spectroscopic mode, the m.fits files have:

COL1 = 'Order ' / Order number
COL2 = 'NElements' / Number of elements in order
COL3 = 'NBeams ' / Number of beams  (1 for 1-f)
COL4 = 'ElementIndex' / Index of this element
COL5 = 'Wavelength' / Uncorrected wavelength (nm)
COL6 = 'Tell ' / Wavelength with telluric correction applied (nm)
COL7 = 'RVel ' / Heliocentric wavelength correction (nm)
COL8 = 'XCorr ' / Cross-correlation
COL9 = 'RawFlux ' / Raw flux (electron)
COL10 = 'RawFluxVar' / Raw flux variance (electron)
COL11 = 'NormalizedFlux' / Normalized flux
COL12 = 'NormalizedFluxVar' / Normalized flux variance
COL13 = 'FcalFlux' / UnNormalized flux (electron)
COL14 = 'FcalFluxVar' / UnNormalized flux variance (electron)

Data Reduction

 Two pipelines were developed completely independently for GRACES data. One is called OPERA, the other DRAGraces. Both are open source, and are described further down on this page.

The recommended approach to get reliable results using GRACES is to compare extracted spectra using both OPERA and DRAGraces. OPERA is not supported by the Gemini observatory, but an internal version is run at the end of each GRACES run, and the extracted spectra are available through the Gemini Observatory Archive (GOA). See the Data Format page for information about these processed files. For any question regarding spectra reduced with OPERA, please email graces_science@gemini.edu. DRAGraces is a simple IDL script specialized for GRACES spectra. For any question or request about GRACES, please email achene@gemini.edu.
 

The OPERA pipeline


OPERA - Open source Pipeline for ESPaDOnS Reduction and Analysis. 
The Canada-France-Hawaii Telescope (CFHT) developed A software reduction, name OPERA, that is specialized  for ESPaDOnS spectrograph extraction. That same pipeline can be adapted to also reduce GRACES data. The latest version can be downloaded on SourceForge (https://sourceforge.net/projects/opera-pipeline/).  Reduced data from Opera is available on the GOA (more information on the Opera output can be found on the GRACES data format page).

This document from CFHT, OPERAperformance.pdf, describes the performance of Opera in comparison with Libre-Esprit on Espadons data, which is also relevant for GRACES data reduction.

The DRAGRACES pipeline


Written in IDL, the pipeline is detailed in the AJ publication DRAGraces: A pipeline for the GRACES high-resolution spectrograph at Gemini (please refer to this paper when the pipeline was used for your reduction). It is also briefly discussed on the DR forum. It is barely more than a thousand lines long, and everyone is invited to download, use and modify it as needed. It is requested that no helpdesk ticket is sent regarding DRAGraces, as questions and requests will be better addressed sent directly to its author, André-Nicolas Chené. 

Newest version available: v1.4 available on github. This version 1) has an improved, more robust trace finding, 2) corrects for cosmic rays the Bias, Flat and Science frames, and 3) produces better header in the reduced spectra.

Older version available: v1.3.1 available on github. This version is modified to deal with the shift in the spectra on the detector post repair in late 2018. It can be used for spectra both pre- and post-repair and includes some other minor bug fixes. It also has an improved way to handle the path to the data given to the option "dir", avoiding any confusions caused by how the path is written.

 DRAGRACES - Data Reduction and Analysis for GRACES. 

Comparing both pipelines

  OPERA
 DRAGraces
 Open source
 YES
 YES
 Orders extracted
  22 to 57 
  22 to 56
 Reduction
  bias, flat
  overscan, bias, flat
 Slit tilt correction
  YES
  YES
 Wavelength calibration
  YES
  YES
Wavelength correct
using telluric lines
  YES
  NO
 Output file
fits spectrum
(one value per column)
 fits spectrum
(one order per extension)
 Language
  C (parameters tunned in a Makefile) 
  IDL (parameters given as values in the command line)


NOTES:
  1. The values given in the table are current, and will be updated as the pipelines are developed. Developement is expected by Gemini users who download the script, improve it, and share their work.
  2. We have found that the throughput for the target and sky fibers is different.  Neither Opera nor DRAGRACES currently takes this into account.  For data reduced with Opera, we found that Target+sky fiber / Sky fiber = 1.27 (median value), while for DRAGRACES the median value was 1.24.  This factor should be used to scale the Sky spectra to match that of the sky in the target fiber.

Do it yourself

The high-resolution spectrograph GRACES produces data that are comparable to any echelle spectrograph. It can be reduced reasonably well using IRAF.  However, for optimal resolution power (IRAF cannot deal with ESPaDOnS tilted slice image) and signal-to-noise, it is better to use a dedicated software. 
For any question about the raw data format, please contact graces_science@gemini.edu.