Introduction
The V3 Working Group (V3WG) of the WOCE Data Products Committee (DPC) met in early November 2001 to discuss data and inventory reporting conventions the WOCE data system should adopt to insure consistency across all DAC’s; and consequently, enhance the integration of the data sets by those wishing to combine several data sets.
The V3WG recommends all WOCE netCDF data files for the V3 product comply with the conventions as outlined below. These conventions are a superset of the Cooperative Ocean/Atmosphere Research Data Service (COARDS) netCDF standard. http://ferret.wrc.noaa.gov/noaa_coop/coop_cdf_profile.html
The COARDS standard has received significant support from many research and institutional centers. It is being adopted for a variety of data sets, including climate and forecasting products. Additionally, it is widely recognized by numerous software plotting and analysis packages to provide more automatic processing, plotting, and analysis capabilities.
Additionally, the V3WG has now specified the composition of the WOCE V3 inventory files. Each DAC is REQUIRED to provide one entry in the table for each netCDF file they produce for the V3 dataset. These inventory files are the cornerstones for the off-line and on-line search/integration tools that are envisioned to be released in concert with the V3 datasets.
This document presents the following specifications/conventions:
Any exceptions to the REQUIRED specifications should be brought to the immediate attention of the V3WG.
Listed here are important requirements for COARDS-compliant netCDF files that the DAC’s must address. DAC’s should consult the COARDS standards (see URL above) and/or the V3WG for additional details and questions.
These six variables are REQUIRED in every netCDF file (.nc file) for every DAC (see Table 1). They are essential to for a successful integrated search.
woce_date
woce_time
time
latitude
longitude
depth or pressure (if depth = 0 you must set it as depth=0)NOTE: In the case where there is a single latitude, longitude, time, and/or depth value per file, then a corresponding single-element array should be utilized as indicated below. This information should NOT be included as an attribute. See 1c for example.
NOTE: variable time must be of the form “minutes since 1900-1-1 0:0:0”. Other time increments may be used, e.g. “days since 1900-1-1”, provided it unambiguously identifies the observation time. Other reference start dates are acceptable, but we encourage the use of “1900-1-1 0:0:0”.
Each of the six REQUIRED variables listed above MUST have these 4 attributes and must follow this EXACT convention (including case sensitivity)
long_name=
units=
data_min=
data_max=NOTE: data_min/max attributes refer to the min/max of the variable data in this file only.
Where the long name and unit is given in TABLE 1 below – they must be followed exactly. For example the long name for woce_date MUST BE WOCE date and the units MUST BE yyyymmdd UTC.
If any or all of the dimensions of a data variable have the interpretations of time, “height or depth", "latitude", or "longitude" then those dimensions should appear in the relative order time, then depth, then latitude, then longitude in the Common Data Format Language (CDL) definition corresponding to the file, e.g.,
Example (attributes etc excluded for brevity):
dimensions:
time = 1 ;
depth = 1004 ;
latitude = 1 ;
longitude = 1 ;
variables: (data type is example)
long time(time) ;
float depth(depth) ;
float latitude(latitude) ;
float longitude(longitude) ;
float T_20(time, depth, latitude, longitude) ;
The following 6 attributes are not REQUIRED, but IF they are in the netCDF files they MUST follow this EXACT convention (including case sensitivity).
valid_range OR valid_min and valid_max
C_format
FORTRAN_format
_FillValue
missing_value
| VARIABLE | ATTRIBUTE | ||
| time | long_name = time | REQUIRED | |
| units = days since 1900-1-1 0:0:0 | REQUIRED | ||
| data_min = | REQUIRED | ||
| data_max = | REQUIRED | ||
| valid_min = | (or valid_range) | Not REQUIRED. If you have it, it must call it this. | |
| valid_max = | Not REQUIRED. If you have it, it must call it this. | ||
| C_format = | Not REQUIRED. If you have it, it must call it this. | ||
| FORTRAN_format = | Not REQUIRED. If you have it, it must call it this. | ||
| _FillValue = | Not REQUIRED. If you have it, it must call it this. | ||
| missing_value | REQUIRED | ||
| depth or pressure | long_name = depth or pressure | REQUIRED | |
| units = meters | REQUIRED | ||
| positive= down or up | REQUIRED | ||
| data_min = | REQUIRED | ||
| data_max = | REQUIRED | ||
| valid_min = | (or valid_range) | Not REQUIRED. If you have it, it must call it this. | |
| valid_max = | Not REQUIRED. If you have it, it must call it this. | ||
| C_format = | Not REQUIRED. If you have it, it must call it this. | ||
| FORTRAN_format = | Not REQUIRED. If you have it, it must call it this. | ||
| _FillValue = | Not REQUIRED. If you have it, it must call it this. | ||
| missing_value | Not REQUIRED. If you have it, it must call it this. | ||
| latitude | long_name = latitude | REQUIRED | |
| units = degrees_N | REQUIRED | ||
| data_min = | REQUIRED | ||
| data_max = | REQUIRED | ||
| valid_min = | (or valid_range) | Not REQUIRED. If you have it, it must call it this. | |
| valid_max = | Not REQUIRED. If you have it, it must call it this. | ||
| C_format = | Not REQUIRED. If you have it, it must call it this. | ||
| FORTRAN_format = | Not REQUIRED. If you have it, it must call it this. | ||
| _FillValue = | Not REQUIRED. If you have it, it must call it this. | ||
| missing_value | Not REQUIRED. If you have it, it must call it this. | ||
| longitude | long_name = longitude | REQUIRED | |
| units = degrees_E | REQUIRED | ||
| data_min = | REQUIRED | ||
| data_max = | REQUIRED | ||
| valid_min = | (or valid_range) | Not REQUIRED. If you have it, it must call it this. | |
| valid_max = | Not REQUIRED. If you have it, it must call it this. | ||
| C_format = | Not REQUIRED. If you have it, it must call it this. | ||
| FORTRAN_format = | Not REQUIRED. If you have it, it must call it this. | ||
| _FillValue = | Not REQUIRED. If you have it, it must call it this. | ||
| missing_value | Not REQUIRED. If you have it, it must call it this. | ||
| woce_date | long_name = WOCE date | REQUIRED | |
| units = yyyymmdd UTC | REQUIRED | ||
| data_min = | REQUIRED | ||
| data_max = | REQUIRED | ||
| valid_min = | (or valid_range) | Not REQUIRED. If you have it, it must call it this. | |
| valid_max = | Not REQUIRED. If you have it, it must call it this. | ||
| C_format = | Not REQUIRED. If you have it, it must call it this. | ||
| FORTRAN_format = | Not REQUIRED. If you have it, it must call it this. | ||
| _FillValue = | Not REQUIRED. If you have it, it must call it this. | ||
| missing_value | Not REQUIRED. If you have it, it must call it this. | ||
| woce_time | long_name = WOCE time of day | REQUIRED | |
| units = hhmmss.dd UTC | REQUIRED | ||
| data_min = | REQUIRED | ||
| data_max = | REQUIRED | ||
| valid_min = | (or valid_range) | Not REQUIRED. If you have it, it must call it this. | |
| valid_max = | Not REQUIRED. If you have it, it must call it this. | ||
| C_format = | Not REQUIRED. If you have it, it must call it this. | ||
| FORTRAN_format = | Not REQUIRED. If you have it, it must call it this. | ||
| _FillValue = | Not REQUIRED. If you have it, it must call it this. | ||
| missing_value | Not REQUIRED. If you have it, it must call it this. |
A study of all DACS indicated that there are 6 variables most common to more than one DAC. As such they should be consistently named and given consistent attributes (see Table 2).
temperature
sst
salinity
u (current, not winds)
v (current, not winds)
sea_levelFor those DACS with these variables the variables MUST follow the conventions in Table 2 below. Where the long name and unit is given in the tables below – they must be followed exactly. For example the long name for sst MUST BE temperature and the units MUST BE degree C.
All data variables other than these 6 common data variables must follow the conventions in Table 3.
| VARIABLE | ATTRIBUTE | ||
| temperature | long_name = temperature | REQUIRED | |
| units = degree C | REQUIRED | ||
| data_min = | REQUIRED | ||
| data_max = | REQUIRED | ||
| valid_min = | (or valid_range) | Not REQUIRED. If you have it, it must call it this. | |
| valid_max = | Not REQUIRED. If you have it, it must call it this. | ||
| C_format = | Not REQUIRED. If you have it, it must call it this. | ||
| FORTRAN_format = | Not REQUIRED. If you have it, it must call it this. | ||
| _FillValue = | Not REQUIRED. If you have it, it must call it this. | ||
| missing_value | Not REQUIRED. If you have it, it must call it this. | ||
| sst | long_name = sea surface temperature | REQUIRED | |
| units = degree C | REQUIRED | ||
| data_min = | REQUIRED | ||
| data_max = | REQUIRED | ||
| valid_min = | (or valid_range) | Not REQUIRED. If you have it, it must call it this. | |
| valid_max = | Not REQUIRED. If you have it, it must call it this. | ||
| C_format = | Not REQUIRED. If you have it, it must call it this. | ||
| FORTRAN_format = | Not REQUIRED. If you have it, it must call it this. | ||
| _FillValue = | Not REQUIRED. If you have it, it must call it this. | ||
| missing_value | Not REQUIRED. If you have it, it must call it this. | ||
| salinity | long_name = | REQUIRED but long name up to DAC | |
| units = | REQUIRED | ||
| data_min = | REQUIRED | ||
| data_max = | REQUIRED | ||
| valid_min = | (or valid_range) | Not REQUIRED. If you have it, it must call it this. | |
| valid_max = | Not REQUIRED. If you have it, it must call it this. | ||
| C_format = | Not REQUIRED. If you have it, it must call it this. | ||
| FORTRAN_format = | Not REQUIRED. If you have it, it must call it this. | ||
| _FillValue = | Not REQUIRED. If you have it, it must call it this. | ||
| missing_value | Not REQUIRED. If you have it, it must call it this. | ||
| u | long_name = | REQUIRED but long name up to DAC (e.g. eastward velocity component) | |
| units = meters per second | REQUIRED | ||
| data_min = | REQUIRED | ||
| data_max = | REQUIRED | ||
| valid_min = | (or valid_range) | Not REQUIRED. If you have it, it must call it this. | |
| valid_max = | Not REQUIRED. If you have it, it must call it this. | ||
| C_format = | Not REQUIRED. If you have it, it must call it this. | ||
| FORTRAN_format = | Not REQUIRED. If you have it, it must call it this. | ||
| _FillValue = | Not REQUIRED. If you have it, it must call it this. | ||
| missing_value | Not REQUIRED. If you have it, it must call it this. | ||
| v | long_name = | REQUIRED but long name up to DAC (e.g. northward velocity component) | |
| units = meters per second | REQUIRED | ||
| data_min = | REQUIRED | ||
| data_max = | REQUIRED | ||
| valid_min = | (or valid_range) | Not REQUIRED. If you have it, it must call it this. | |
| valid_max = | Not REQUIRED. If you have it, it must call it this. | ||
| C_format = | Not REQUIRED. If you have it, it must call it this. | ||
| FORTRAN_format = | Not REQUIRED. If you have it, it must call it this. | ||
| _FillValue = | Not REQUIRED. If you have it, it must call it this. | ||
| missing_value | Not REQUIRED. If you have it, it must call it this. | ||
| sea_level | long_name = | REQUIRED but long name up to DAC | |
| units = meters | REQUIRED | ||
| data_min = | REQUIRED | ||
| data_max = | REQUIRED | ||
| valid_min = | (or valid_range) | Not REQUIRED. If you have it, it must call it this. | |
| valid_max = | Not REQUIRED. If you have it, it must call it this. | ||
| C_format = | Not REQUIRED. If you have it, it must call it this. | ||
| FORTRAN_format = | Not REQUIRED. If you have it, it must call it this. | ||
| _FillValue = | Not REQUIRED. If you have it, it must call it this. | ||
| missing_value | Not REQUIRED. If you have it, it must call it this. | ||
| wind_speed | long_name = | REQUIRED but long name up to DAC | |
| units = meters per second | REQUIRED | ||
| data_min = | REQUIRED | ||
| data_max = | REQUIRED | ||
| valid_min = | (or valid_range) | Not REQUIRED. If you have it, it must call it this. | |
| valid_max = | Not REQUIRED. If you have it, it must call it this. | ||
| C_format = | Not REQUIRED. If you have it, it must call it this. | ||
| FORTRAN_format = | Not REQUIRED. If you have it, it must call it this. | ||
| _FillValue = | Not REQUIRED. If you have it, it must call it this. | ||
| missing_value | Not REQUIRED. If you have it, it must call it this. |
| VARIABLE | ATTRIBUTE | ||
| variable_name | long_name = | REQUIRED but long name up to DAC | |
| units = | REQUIRED | ||
| data_min = | REQUIRED | ||
| data_max = | REQUIRED | ||
| valid_min = | (or valid_range) | Not REQUIRED. If you have it, it must call it this. | |
| valid_max = | Not REQUIRED. If you have it, it must call it this. | ||
| C_format = | Not REQUIRED. If you have it, it must call it this. | ||
| FORTRAN_format = | Not REQUIRED. If you have it, it must call it this. | ||
| _FillValue = | Not REQUIRED. If you have it, it must call it this. | ||
| missing_value | Not REQUIRED. If you have it, it must call it this. |
ALL DACS MUST include this global attribute in each .nc netCDF file
•WOCE_Version = “3.0”;
•Conventions = “COARDS/WOCE”; //
Cooperative Ocean/Atmosphere Research Data Service
In addition, if you have these in please include them as distinct global attributes using these conventions:
•EXPOCODE
=
THE CDV3 will be able
to be searched by EXPOCODE to recover all data from that cruise. IF you have
it, please include it as a separate global attribute
•DAC_ID
=
Most DACS identify
the datasets with an inhouse number. e.g. the ADCP has a SADCP_ID field which
uniquely identifies it to the DAC. Same for floats, sea_level... It is that
field that would allow a user to go from the CD, ask the DAC for a chunk of
data and get exactly what you are looking for. Not REQUIRED - but if the DAC
has it it makes getting what you are looking for from the DAC that much easier.
•file_source =
The name of the file
from which the .nc file was created.
•file_date =
The date of the file
from which .nc file was created
•instrument =
The instrument type.
This is up to each DAC
Inventories are the tab delimited ASCII text files created by each DAC and supplied to the WDIU for use in the CDV3 search tool. Every netCDF file in the WOCE V3 data set will have a single entry (i.e. line) in the inventory files. These inventory tables will facilitate searching the WOCE data base according to user parameters. The tables will also serve as the foundation for forming DODS URL’s. The first line of every inventory must be a column header line, indicating what is in each column. Every variable is separated by a TAB and each line is ended with a new line (NL).
The inventories for the UOT low density program are a bit different from the rest of the DACS because of the size of the dataset - to try to help in the integrated search. See 4c for details. UOT High Density should be treated as the other DAC datasets in 4a and 4b.
Thirteen fields are REQUIRED in every inventory file for every DAC other than the UOT Low Density dataset (see 4c). These REQUIRED fields are the bare minimum fields essential to create a successful search across data holdings from all DAC’s. The information contained within the inventory must be only for valid data.
| cd_name | The name of the DAC example whp or float so the search tool can say – go to the float directory/CD for this data |
| file_path | from the root … example= ./data/atlantic/ note the direction of the slashes=/ and that it begins and ends with a “./” |
| file_compressed_name | compressed file name … example file.zip \ |
| file_name | .nc filename example 10689.nc |
| woce_date_min | using woce_date variable (yyyymmdd) |
| woce_date_max | using woce_date variable (yyyymmdd) |
| latitude_min | |
| latitude_max | |
| longitude_westmost | |
| longitude_eastmost | |
| depth_min or pressure_min | |
| depth_max or pressure_max | |
| EXPOCODE |
In addition the "GOOD" min and max of the six variables found to be common among the DACS should be listed in the inventories of the DACS that have them:
temperature_min
temperature_max
sst_min
sst_max
salinity_min
salinity_max
u_min
u_max
v_min
v_max
sea_level_min
sea_level_max
wind_speed_min
wind_speed_max
Because of the size of the UOT Low Density dataset, it was felt that the best way to be able to deal with this for an integrated search was a Top table inventory and a related Sub table inventory. The Top table inventory is an inventory of the ZIP files. The Sub table inventory is of the individual netCDF files contained within those ZIP files, related by file_compressed_name. The Top table will thus have the min and max of the entire ZIP file, while the sub-table will have the min and max of every individual file within that ZIP file. The Sub table will not even get searched unless the Top table indicates that it falls within the search parameters.
UOT High Density should follow the conventions listed for the DACS above in 4a and 4b.
| cd_name | The name of the DAC example uot so the search tool can say – go to the uot directory for this data |
| file_path | from the root … example= ./data/atlantic/1993 note the direction of the slashes=/ and that it begins and ends with a “./” |
| file_compressed_name | compressed file name … example q1_tar.gz |
| woce_date_min | of the entire compressed file using woce_date variable (yyyymmdd) |
| woce_date_max | of the entire compressed file using woce_date variable (yyyymmdd) |
| latitude_min | of the entire compressed file |
| latitude_max | of the entire compressed file |
| longitude_westmost | of the entire compressed file |
| longitude_eastmost | of the entire compressed file |
| depth_min | of the entire compressed file |
| depth_max | of the entire compressed file |
| temperature_min | of the entire compressed file |
| temperature_max | of the entire compressed file |
| salinity_min | of the entire compressed file |
| salinity_max | of the entire compressed file |
The SUB table of the UOT inventory should be of every individual netCDF file in the ZIP files. It should follow this convention.
| cd_name | The name of the DAC example uot so the search tool can say – go to the uot directory for this data |
| file_path | from the root … example= ./data/atlantic/1993 note the direction of the slashes=/ and that it begins and ends with a “./” |
| file_compressed_name | compressed file name … example q1_tar.gz (relates to the top table) |
| file_name | file name of individual data.nc file within zip file |
| date | of the .nc file using woce_date variable (yyyymmdd) |
| latitude | of the .nc file |
| longitude | of the .nc file |
| depth_min | of the .nc file |
| depth_max | of the .nc file |
| temperature_min | of the .nc file |
| temperature_max | of the .nc file |
| salinity_min | of the .nc file |
| salinity_max | of the .nc file |