Quality Control

Quality control of Gemini data is achieved by performing data quality assessment (QA). The QA processes are described below:

• The queue, observations and raw data
• Real-time QA
• Off-line data processing
• Off-line QA
• Final approval, repeating observations, re-scheduling and data distribution

Quality Assessment, in this form at least, is applicable only to queue and QuickStart service mode data.

The queue, observations and raw data

Observations awaiting execution are stored in the Observing Database. The database contains ingested Phase I proposals and Phase II details defined by the PI. The database also contains the status of an observation, including its location within the QA process.

Execution is carried out at the observation level, not the science program level. That is, flexibility to respond to changing conditions, for example, is provided by switching between programs. Thus an observation is the minimum schedulable entity. Each observation may generate many datasets (when executed by the OCS, one OBSERVE command generates one dataset). Each dataset is a single FITS file. For example, multiple repetitions of a mosaic or offsets between object and sky can form one observation.

Real-time Data QA

Real-time quality assessment is performed by the observer taking the data (usually at the summit). This is intended to be a very simple level of assessment and is applied to all datasets. There are two parts: (a) identify whether or not the dataset meets the observing condition criteria defined by the PI and (b) test against simple Gemini criteria to determine whether the data is usable for doing science.

For all queue observations, the PI will have specified four observing conditions: image quality, cloud cover, water vapor content and sky background. The conditions are defined to be within one of several percentile bins. By comparing the requested and current conditions the QA identifies whether the PI criteria are met. Therefore the process adds four header keywords to each dataset (RAW(IQ|CC|WV|BG) with the values being the current observing condition bin (e.g. 20-percentile). Another header keyword identifies whether the overall PI requirements are met (RAWPIREQ, with values of yes|no|unknown). The "unknown" value is required because it may not be possible to determine e.g. the image quality without significant data processing.

The criterion of scientific utility is whether or not the dataset contains useful scientific information as determined by simple visual inspection. To pass this test it is not necessary that the dataset be usable in the program for which it was taken (which is why it is denoted as a Gemini, not PI, criterion). The intent is to ensure that all usable data enters the science archive. For example, data taken in which the shutter failed to open or the dome caused severe vignetting would be classified as "bad". An exposure in which only three of the four detector quadrants returned useable data or in which the target was saturated might be serviceable for another program. This process adds one header item to the dataset: RAWGEMQA with a value of (bad|usable|unknown). If the dataset is bad then the observation is re-scheduled in the queue. (NB: it need not be re-observed immediately depending on the reason for bad data).

The observation status in the database is updated after QA.

Off-line Data Processing

Off-line data processing is eventually to be performed in an integrated data processing pipeline. Until that is available, off-line data processing is performed semi-manually at sea level by the Gemini data analysts using the same data processing software scripts that form the elemental components of the pipeline. The details of the processing recipe depend on the instrument configuration, telescope and instrument sequencing. The observation status in the database is updated after processing.

Off-line Data QA

The final image (or spectrum) derived from off-line data processing is subjected to further QA. For the immediate future, the only PI criteria for overall success is total integration time; longer term the intent is to adopt measurements of precision e.g. flux limit. The image quality of the final stacked data may differ from that requested by the PI, even if the individual datasets are in accordance, and is verified. If the criteria are not met the observation may be re-queued (see the next section).

The Gemini criterion is that the observation was executed as defined by the PI i.e. the correct instrument configuration (e.g. filter) and target (co-ordinates) was used. These attributes are assessed by comparing the data headers with the proposal. If the criteria are not met the observation may be re-queued (see the next section).

The process adds further headers to the final QA data product: FINALINT, FINALIQ, PICONFIG, PITARGET all with values of (pass|fail).

The observation status in the database is updated after QA.

Final approval, repeating observations, re-scheduling and distribution

The Gemin Contact Scientist (CS) is responsible for approving the distribution and packaging of the data.

If the off-line data QA is passed (i.e. all header values are "pass"), the real-time data QA is passed (i.e. header values are "OK" and "usable") and there are no significant issues with data processing then the data is packaged and distributed to the PI and to the science archive with the normal proprietary period.

(a) Criteria for repeating observations

Observations will be scheduled to be repeated if there was significant error by the Gemini Staff, technical failure or a substantial change in the observing conditions during execution. Examples of "significant error" are incorrect instrument configuration (wrong filter, grating, slit etc.) or the target is not in the field of view. A "substantial change" in the observing conditions means that the conditions degraded by at least 50% from those requested during the majority of the observation. An example would be a delivered image quality of 0.45 arcsec when 0.3 arcsec was requested.

The value of 50% is set to provide some margin during early ("shared-risks") science operations. The intent is to reduce this value to zero as experience with the telescope, instrument and environmental conditions are obtained.

The decision on whether an observation passes this criteria is made by the Gemini CS for that program, in consultation with the Instrument Specialist and Observer if necessary, but does not involve the PI.

An observation that was defined incorrectly by the PI will not be repeated.

(b) Re-scheduled observations

As is the case for any queue-mode programs, there is no guarantee that a re-scheduled observation will be repeated. The re-scheduled observation will have the same weighting in the queue as the original observation.

Feedback is added into the description of the science program so that the problem can be avoided when it is re-observed.

(c) Data access and time accounting

When an observation is scheduled to be repeated, the data from the original observation will be distributed to the PI and to the science archive with the normal proprietary period. The original observation does not count towards the PI's time allocation.

In all cases the status of the observation in the database is updated.