1.1 New data notification Emails
We send automatic emails in the morning following observing to notify PIs and/or Co-Is that new data were taken for their program. This facility is provided by our internal data handling systems. Email addresses to send notifications to are updated from the Observing Database daily. If you would like to add email addresses to receive the notifications, please simply add them to the "PI / PC Email" field in the top level of your Gemini Science Program in the OT and store your program to the database.
Email notifications for data taken during the night go out at 8:00am in the morning following the observing night. The email contains a summary table of all the relevant data files.
These Emails are triggered purely on the presence of science data files from night time observing. These data will not generally have been Quality Assessed, or their metadata validated, at the time the Email notifications go out. Data Quality and FITS header information in these files will be updated as QA proceeds. Data should be retrieved from the Gemini Observatory Archive in the normal manner.
1.2 Gemini Observatory Archive
The Gemini Observatory Archive (GOA) located at https://archive.gemini.edu is now the primary conduit to obtain data from Gemini Observatory. The Help and About pages also linked at the top of the archive page give more information.
The GOA provides a simple yet powerful interface to search for and download science data, calibrations and observation logs both for Gemini program PIs and Co-Is, and also for people searching for Gemini data by observation details such as instrument configuration, sky coordinates, observing date etc. Paul Hirst is the GOA project lead.
Data Flow and Distribution
All Gemini data distribution to users is via the archive web interface. Data are transferred to the archive automatically during observing, and each file is typically available for download from the archive within a minute or two of the observation completing. Data Quality Assessment proceeds over the following days and the archive is updated if QA states or other metadata are updated.
The archive will automatically find all the calibration data associated with search results, and presents this in the associated calibrations tab in the search form. Additionally, it's possible to search for calibration data directly in the search form simply by setting the search fields appropriately.
The archive contains mostly raw data. Some data is processed by the observatory and this processed data is uploaded to the archive. Notably, this includes GMOS bias frames and imaging twilight flat fields, along with MOS preimaging data. Other processed data is produced and archived on a somewhat ad-hoc basis.
Gemini metadata, including sky coordinates, target name and instrument configuration are in general immediately world public, and the full FITS headers of proprietary data files can be viewed by any archive user. This is a deliberate policy and allows people to avoid writing proposals that would duplicate observations whose data are still proprietary. The actual pixel values are proprietary and only made available to the project they were observed for, typically for 12 months (18 prior to semester 2016A). A few projects (for example some of the exoplanet campaign projects with NICI and GPI) have been granted a proprietary metadata status where certain items of metadata (sky coordinates and target name), and the full FITS header text, assume the same proprietary privacy restrictions as the pixel values.
Downloading proprietary data
The archive is open to the public to search for data, and download any data which is no longer proprietary. If you are a gemini PI or Co-I and would like to access proprietary data for your program you will need to complete the following steps:
- Create an account on the archive
- Register your Gemini program with your archive account. To do this you need the program ID and the ODB key you were given in your time award notification email.
More details on how do to this are provided in the User Accounts section of the archive help page
Getting data automatically (e.g., as part of reduction pipeline)
Please, read point 11 from the GOA help. You can also find an example of a script from the AAT Observational Techniques Workshop 2016.
Transition from the Gemini Science Archive
Gemini science data were initially stored in the Gemini Science Archive (GSA), operated by the Canadian Astronomy Data Center (CADC). From November 2015, all data are instead stored in, and accessed via, the Gemini Observatory Archive, which is operated by the Observatory.
All Gemini data has been transferred into the new archive. However, user accounts and program registrations are not transferable - you will need to create an account on the new archive and register any programs from which you would like to download proprietary data with that new account.
Unlike the GSA, the GOA does not have the concept of "PI packages". PIs simply search for and download their data (and optionally the associated calibrations) by project ID using the main search form.