III. DATA QUALITY ASSURANCE AND PROCESSING 3.1 Introduction Data quality assurance and processing techniques are discussed in this chapter. Where possible, detailed calibration and/or processing methods are described. Applicable methods for ancillary data are presented from the point in processing at which the data were obtained. 3.2 CTD Systems 3.2.1 CTD Sensor Calibration Sea-Bird Electronics, Inc. CTD systems are designed in such a way as to permit easy processing and reprocessing of the raw data with new calibration coefficients. Subsequently, following a cruise, if the sensor calibration is in question, or the calibration is rather old, the raw data can be easily reprocessed with new calibration coefficients. During this study, the Sea-Bird 911 Plus CTD sensors were calibrated at the manufacturer's facilities on numerous occasions. The primary and/or secondary temperature and conductivity sensors were calibrated in December 1996, March 1997, February and May 1998, and again in February/March 1999. In all cases, the conductivity and temperature sensors were found to be within the manufacturer's specifications. Pressure sensor calibrations were done in August 1996 and again in January 1999. Table 3.2-1 summarizes the manufacturer's specifications. Table 3.2-1. Manufacturer’s specifications for SeaBird Electronics,Inc., 911 Plus CTD. Sensor Initial Accuracy Stability Resolution (per month) Conductivity 0.003 mmho/cm 0.002 mmho/cm 0.0004 mmho/cm Temperature 0.002ºC 0.0003ºC 0.0002ºC Pressure 0.015% FS 0.0015% FS 0.001% FS 10,000 psia (0.102M) (1.02M) As a further CTD calibration check, water samples were drawn from mixed layers (where possible) for each cast for each cruise to determine a separate calculated CTD salinity minus bottle salinity intercomparison. These results (by cruise) for the 911 Plus CTD are presented in Table 3.2-2. The bottle salinities were run at Texas A&M University. The resulting data reveal that the calculated mean salinity differences for these mixed-layer comparisons for each cruise (including the pump plumbing plagued cruise of December 1998) ranged between +0.009 and - 0.014 psu. Subsequently, no additional processing or reprocessing of the 911 Plus CTD data was required. Unfortunately, efforts to overcome the December 1998 pump plumbing problem by various means of lagging and offsetting the data were unsuccessful in eliminating the noisy salinity signal. Subsequently, these salinity data were discarded. A Sea-Bird SBE 19 SeaCat CTD was used, but only for a small number of casts (Stations 12-15 and Station 41) during the last cruise (March-April 1999) when the 911 Plus CTD was unavailable due to wire termination problems. The SeaCat's specifications are presented in Table 3.2-3. This instrument had previously been calibrated in January 1998, but was post calibrated in May 1999. Prior to recalibration, a TS plot revealed a significant offset in the data. Following processing and recalibration, the corrected data plot compared favorably with the TS plot for the 911 Plus. This comparison is shown in Figure 3.2-1. Table 3.2-2. Time (in months) of most recent calibration of 911 Plus CTD sensors before and after each cruise during the DeSoto Canyon Eddy Intrusion Study. Cruise 10,000 psia Primary Primary Secondary Secondary Dates Pressure Temp. Conduct. Temp, Conduct. 03/18/97 - 03/28/97 7/22 3/11 3/11 0/11 0/11 07/08/97 - 07/19/97 11/18 7/7 7/7 4/7 4/7 11/11/97 - 11/22/97 15/14 8/NA 8/NA 8/3 8/3 03/31/98 - 04/11/98 20/9 1/NA 1/3 1/3 1/NA 08/04/98 - 08/15/98 24/5 3/6 3/7 1/6 1/7 12/01/98 - 12/13/98 28/1 7/2 7/3 5/2 5/3 03/30/99 - 04/06/99 2/NA 1/NA 1/NA 1/NA 1/NA NA = NotAvailable Table 3.2-3. Mixed layer 911 Plus CTD salinity minus Bottle salinity comparisons by cruise during the DeSoto Canyon Eddy Intrusion Study. Cruise ID Dates CTD - Bottle Std. Dev. (psu) (psu) PE 9722 03/18/97 – 03/28/97 -0.014 +/-0.007 PE 9803 07/08/97 – 07/19/97 -0.009 +/-0.008 PE 9820 11/10/97 – 11/22/97 -0.007 +/-0.008 PE 9830 03/31/98 – 04/10/98 -0.002 +/-0.007 PE 9908 08/03/98 – 08/14/98 0.001 +/-0.010 PE 9923 12/01/98 – 12/13/98 0.009 +/-0.013 PE 9932 03/29/99 – 04/06/99 -0.002 +/-0.006 3.2.2 CTD Data Processing Raw CTD data files were ingested into SAIC’s Physical Oceanographic Data Base Management (PODBM) system. In this database the type of station (CTD or XBT) was identified as well as the latitude, longitude, depth of cast and water depth for each cast. These data were then averaged to one-meter levels and sigma-t, temperature and salinity data were plotted versus depth for each cast to serve as an initial quality check. Finally, a composite Temperature-Salinity (T-S) plot was produced for each cruise. 3.3 XBT System 3.3.1 XBT Probe Calibration The T-7 XBT probe is rated by the manufacturer as having a temperature accuracy of ? 0.2 ?C over the range -2 to +35 ?C. No effort was made to provide an independent calibration check of this accuracy, but CTD data collected at the same location as XBT data (but not at the same time) during the second Feature Survey in December 1998 ranged from 0.1?C to 0.7?C lower in temperature than the corresponding XBT data. These results were derived from a comparison of temperatures at 25 and 40m depth in a 50 meter thick mixed layer along a repeated section that was sampled twice over twelve hours. Separately, it has been well documented (Hanawa et al. 1995) that the manufacturer's original drop-rate equation (z = 6.472 t - 0.00216 t2) under-estimates the actual drop rate of the T-7 XBT probe, tending to artificially lift the isotherms approximately 3.4 % from their actual depths. The data obtained in this program were collected using this same historical standard equation: Subsequently, in processing, a linear correction (Z1 = 1.0336 z (from Hanawa et al. 1995)) was applied to more accurately present the depth/temperature pairs. 3.3.2 XBT Data Processing SAIC uses a systematic procedure for the processing of CTD and XBT profiles. The following list summarizes the processing steps for CTD/XBT profile data respectively. Data are logged into the SAIC Physical Oceanographic Data Base and the cruise is assigned a unique cruise ID. Next, the data are read into binary disk files on the local server. These files are created by storing the data scans as ASCII characters in a sequential file using the system file utility program. During sequential file creation, checks are made on the input parameters (temperature, conductivity and depth) for large spikes, data gaps and number of data scans for each individual cast. Vertical profiles of temperature and conductivity are plotted and checked for spikes or obviously questionable data. The bad data scans are removed or the entire cast is discarded depending on the number of bad data points. T/S diagrams are created for each cast to assure consistency with known characteristics of water masses in the Gulf. From the corrected data set, both vertical and horizontal contour maps are produced for individual sections and the entire cruise, respectively. These maps are checked visually against the profiles as a quality control measure. Other data products (geostrophic velocity, integrated transports, dynamic heights) are produced as required for further analysis. 3.4 Altimetry Sea surface height (SSH) analysis maps were produced from archival altimetric measurements based on the latest versions of the TOPEX and ERS geophysical data records (GDRs). TOPEX data were obtained from the Physical Oceanography Distributed Active Archive Center (PO DAAC) at the Jet Propulsion Laboratory, and ERS data from the "Centre ERS d'Archivage et de Traitement"(CERSAT), the French Processing and Archiving Facility for ERS-1 and ERS-2. Both data sets are processed in as consistent a fashion as possible to produce accurate analysis maps based on the blended tandem altimetric observations. 3.4.1 Data Processing The TOPEX data were corrected using standard corrections supplied on the JPL/PO.DAAC TOPEX GDRs, including inverted barometer, electromagnetic bias, ionosphere and wet/dry tropospheric corrections, as recommended in the GDR handbook (Callahan, 1993). The mean sea surface included on the GDRs (Basic and Rapp, 1992) was subtracted from each sub-satellite data point to apply an implicit cross-track geoid gradient correction. Ocean tides were removed using the tidal solution derived from the Colorado Center for Astrodynamics Research (CCAR) barotropic tide model (version 1.0) assimilating TOPEX data (Tierney et al., 1998). The ERS-1 and ERS-2 Altimeter Ocean Products (ALTOPR) CD-ROMs were obtained from CERSAT. The ERS altimeter data were corrected using standard corrections supplied on the ALTOPR GDRs, including inverted barometer, electromagnetic bias, ionosphere and wet/dry tropospheric corrections. The data were also corrected using the CCAR Version 1.0 tide model to be consistent with the TOPEX processing. Each cycle of corrected 10-day repeat TOPEX and 35-day repeat ERS data was linearly interpolated to reference ground tracks based on computed orbits for the satellites. The TOPEX reference track was based on a ground track computed for cycle 18, with a fixed spacing of the sub- satellite reference points at once per second along-track which is approximately a 5.77 km spacing over the Gulf of Mexico. The ERS 35-day reference ground track is based on 1/second along-track points computed for cycle 6 of the ERS-1 Multidisciplinary 1 mission phase. No gridding of the non-repeat ERS-1 data was performed. An empirical orbit error correction was applied to the ERS along-track data, after gridding when appropriate, to remove residual ERS orbit error. An empirical correction of the TOPEX data was not needed; however, to consistently "filter" both data sets the empirical correction was also applied to the TOPEX data. This correction was based on an along-track "loess" filter, which removed a running least squares fit of a tilt plus bias within a sliding window from the along-track data. The filter window is approximately 15 degrees of latitude (200 second along-track), passing the short wavelength mesoscale signals while filtering the longer wavelength orbit and environmental correction errors. This processing procedure references the along-track data to an accurate high resolution mean sea surface based on altimeter data collected from the TOPEX/POSEIDON, ERS-1 and GEOSAT Exact Repeat missions (Yi, 1995). By referencing the data to an independent mean sea surface, a climatology based on multiple altimeter mission data sets can be developed to include past, present and future altimeter data referenced in the same manner. This includes the historical GEOSAT data (Berger et al., 1996) and the TOPEX/POSEIDON and ERS-2 data available during the DeSoto Canyon field program. 3.4.2 Blended T/P and ERS 1&2 Climatology An archive of the Gulf of Mexico SSH maps from January 1, 1993 to April 30, 1999 has been produced from the corrected sea surface height anomalies from TOPEX/POSEIDON and ERS-1&-2. Daily analysis maps of height anomaly relative to the mean sea surface have been created using an obective analysis procedure (Cressman, 1959) to interpolate the along-track data to a 1/4 degree grid over the Gulf. The method used an iterative difference-correction scheme to update an initial guess field and converge to a final gridded map. A multigrid procedure was used to provide the initial guess. The complete multigrid procedure is described in an appendix to Hendricks et al. (1996). Five Cressman iterations were used with radii of influences of 200, 175, 150, 125 and 100km, while employing a 100 km spatial decorrelation length scale in the isotropic Cressman weighting function. Data were also weighted in time using a 12- day decorrelation time scale relative to the analysis date. A model mean is added the sea surface height anomaly maps to estimate the total dynamic topography. This mean was computed for the time period 1993 through 1998 from a data assimilation hindcast of the general circulation in the Gulf. The 1993 through 1998 hindcast was made for the MMS Deepwater Reanalysis and Synthesis Project by Lakshmi Kantha and Jei Choi using the University of Colorado Princeton Ocean Model (CUPOM). The residual mean in the sea surface height anomaly maps over the time period 1993-1998 is removed before adding the model mean to produce the synthetic height estimate. A sample map is shown in Figure 3.4-1. 3.5 Moored Instrumentation 3.5.1 Moored Instrumentation Calibration Each of the moored instruments deployed in this program, with the exception of the Aanderaa RCM current meters, was calibrated and serviced by its manufacturer prior to initial deployment. The Aanderaa instruments were serviced and recalibrated by Environmental Sensors, Inc., (ESI) formerly the Canadian distributor and service provider for Aanderaa instruments in North America. In the cases of the Hugr?n Seamon Mini Temperature Recorders, the RDI 300 KHz WorkHorse ADCPs and the Sea-Bird MicroCats, these instruments were all newly purchased for the program. Periodically, over the course of the program, a number of instruments were returned to the various manufacturers or ESI for servicing and/or re-calibration. At the conclusion of the field effort, most instruments were returned to the same entities for a post-program servicing and recalibration. Table 3.5-1 summarizes the manufacturer's specifications and the following sections discuss specific calibration issues for each type instrument. In addition to the above, the conductivity data from the Aanderaa, General Oceanics and InterOcean current meters, when deployed in the upper 200m of the water column, tended to show some data quality degradation over the four-month deployment period due to fouling. Subsequently, the calculated salinity data obtained from the temperature and conductivity data collected by these current meters were adjusted following comparisons with nearby CTD casts taken at the beginning and end of each mooring deployment. 3.5.1.1 Aanderaa Current Meters All of the Aanderaa current meters were calibrated at ESI for wide range temperature, wide range conductivity and direction as well as an operational test of speed in February or July 1997 and again at the conclusion of the field program in June or July 1999. Temperature calibration stability over the study period was excellent for all but two of the instruments. These two fell just outside the rated accuracy (?0.05?C) with differences on the order of ?0.07?C when compared at the high end (near 28?C) of the calibration scale. Conductivity calibration stability was significantly worse, as only three instruments were within the rated accuracy (?0.074 mmho/cm) at the conclusion of the program. Seven sensors had drifted producing conductivity differences ranging from ?0.078 to ?0.230 mmho/cm and one sensor had drifted by as much as 1.241 mmho/cm. Two conductivity sensors had failed during the program and were replaced with newly calibrated sensors. A compass calibration check revealed that all of the instruments were within the stated calibration accuracy (within ?5?) over the course of the program, and an operational test of speed (counting rotor turns) revealed that all of the instruments were functioning properly prior to their initial deployment and following final recovery. 3.5.1.2 General Oceanics Current Meters The Niskin winged current meters were calibrated by the manufacturer in January or February 1997 and again in August 1999. Three units were removed from service due to malfunctions or a leak prior to the final calibration check and two units still require a final servicing. To date, all of the tested instruments were within their respective sensor specifications at the conclusion of the field program. Table 3.5-1. Manufacturer’s specifications for moored instruments used during DeSoto Canyon Eddy Intrusion Study. Temp. Temp. Conduct. Conduct. Speed Speed Direction Pressure Pressure Range (?C) Accuracy/ Range Accuracy/ Range Accuracy/ Accuracy/ Range Accuracy/ Resolution (mmho/cm) Resolution (cm/s) Resolution (Degrees) (psia) Resolution (?C) (mmho/cm) (cm/s) (psia) Aanderaa -0.34 to +/-0.05/0.1% 0-74 +/-0.074/0.074 2-295 +/- 1 cm/s or +/-5 for 5-100 cm/s 0-3000 +/- 1% Range/ RCM 7/8 32.17 Range +/- 2% of speed/ +/17.5 for 2.5 to 5 0-5000 0.1% Range Current Meter starting speed and 100 is 2 cm/s to 200 cm/s/0.035 General -5 to 45 +/-0.25/0.016 0-75 +/-2.5/0.1 0-300 +/-1/1 +/-2/1 0-2000 +/-0.5% Range/ Oceanics 0.1% Range MK2 Current Meter Hugrun -2 to 40 +/-0.1/0.025 Temperature Recorder InterOcean -5 to 45 Thermistor 5-65 +/-0.2/0.01 0-350 +/-2% of reading +/-2/0.5 S4 Current +/-0.05/0.003 +/-1cm/s/0.2 Meter Standard cm/s +/-0.2/0.05 RDI Narrowband -5 to 45 +/-0.2/0.012 0-1000 +/-0.2% of measured +/-5/0.005 ADCP velocity +/-0.5cm/s RDI Workhorse -5 to 45 +/-0.4/0.01 0-1000 +/-0.5% of measured +/-5/0.01 ADCP velocity +/-0.5cm/s SeaBird -5 to 40 +/-0.002/0.0001 0-100 +/-0.003/month 0-300 +/-0.15%Range/ MicroCat Res 0.0001 0.002%Range CT Recorder SeaBird -5 to 35 +/-0.01/0.001 0-70 +/-0.01/0.001 0-300 +/-0.15%Range/ SeaCat 0.002%Range CT Recorder 3.5.1.3 Hugrun Temperature Recorders These instruments were calibrated by the manufacturer in January 1997 and again in July 1999. A comparison of reprocessed data from the last deployment with both the new and old calibration coefficients revealed that there had been no significant change/drift in calibration over the study period. All comparisons were found to be within the instrument's rated accuracy, generally varying by no more than ? 0.025?C. Subsequently, no adjustments were made in these data following instrument recalibration. 3.5.1.4 InterOcean Current Meters The S4 current meters were initially calibrated by the manufacturer between March 1997 and March 1998. Two were recalibrated in July and November 1998 and one of these same units (that had been recovered by a shrimping vessel) was recalibrated again in June 1999 following final recovery. Two other units were returned to MMS after only one deployment. All of the instruments were equipped with conductivity sensors. 3.5.1.5 RDI ADCPs All of the instruments were calibrated/serviced by the manufacturer in January or February 1997, and all but one of the recovered units were serviced again in June or November 1999. One instrument is yet to be serviced. The remaining eleven instruments were found to be within specification, except that four of the WorkHorse units (197, 200, 209 and 211) were found to exceed the manufacturer suggested accuracy of less than ?5 degrees for compass error when tested in a downward- looking mode. However, these same compasses were within specifications when tested in an upward-looking mode, which is how each was used in this study. Subsequently, all of the ADCP data are considered to be within instrument specifications, and there is no indication of any problem with data from the unit which is still awaiting servicing at RDI. 3.5.1.6 SeaBird CT Recorders The SeaCat and MicroCat instruments were calibrated between December 1996 and June 1997, prior to initial deployment, and again in May or June 1999 following final recovery. Three of the SeaCat units were borrowed from the manufacturer and only deployed once during the initial deployment period, and a fourth borrowed unit was only deployed once during the fifth deployment period. The conductivity cells on all of these instruments were protected by antifouling cylinders which were replaced after a maximum of eight months use (two deployments). These cylinders appear to have worked well as there was no buildup of biological material inside the conductivity cells. In addition, the post-program calibrations reveal that the total conductivity sensor drift over the two-year field effort was less than 0.045 mmho/cm at 60 mmho/cm and less than 0.020 mmho/cm at 30 mmho/cm. Temperature drift on these same units was less than 0.003?C. Subsequently, the salinity drift is on the order of 0.020 to 0.050 psu. 3.5.2 Moored Instrumentation Data Processing The following instruments were used on the moorings: InterOcean’s S-4, Hugrun’s Seamon-mini, Sea-Bird’s SeaCat and MicroCat, RDI’s ADCP, Aanderaa’s RCM-7&8 and General Oceanics’ MK2. Each required a somewhat different processing scheme to bring the data into SAIC’s database. Some instruments, such as the S4, have their calibrations internally recorded. Others, like the Aanderaas, have the calibrations applied during data processing. The SAIC database utilizes a consistent format whereby each filename has two data fields representing either vector or scalar data. For a vector, the u-v components are presented. Scalar data may be either a single parameter in each of the two fields, such as temperature, or two different parameters, such as temperature and salinity. Times for all data are in GMT. Once in SAIC’s database, each file for each deployment is analyzed for proper processing, missing or erroneous data, calibration errors or trends in the data. Missing data may be patched using similar data. Erroneous data may be bridged by interpolation. Data, such as salinity, may require detrending due to drift of the sensor. This is accomplished by applying an offset to the time series by comparison to adjacent CTD records, as required. Once the raw time series files have been cleaned up, they are filtered using 3-HLP and 40-HLP filters. The vector data are rotated to align the data with the local bathymetry. All time series data for all deployments are concatenated into a single record covering the entire two-year study period. Where data are missing, the existing files are concatenated into as long a record as possible. Some single deployment files remain due to data gaps. 3.6 Drifting-Buoy Data Drifter data from MMS included all transmissions received from each drifter and each position of the drifter as determined by Service Argos. Interactive procedures were then used to remove all duplicate positions, verify the validity of each position fix, sort the data into a time- ordered sequence and archive the data into the PODBM system. Once loaded onto the system, a final visual check was made of the data by plotting each individual buoy trajectory on a high-resolution map of the study area containing detailed bathymetry and coastline. This was used to identify spurious changes in the buoy's movement which could then be removed.