WOCE CD Version 3.0 data and inventory conventions
WOCE Data Products Committee V3 Working Group
March 19, 2002

WOCE CD Version 3.0 data and inventory conventions 1
1. netCDF Conventions for REQUIRED variables in WOCE V3 netCDF files 2
  1.a. REQUIRED Variables 2
  1b. REQUIRED Attributes 2
  1.c Order of dimensions: 3
  1.d. Other Attribute conventions 3
TABLE 1: The 6 REQUIRED variables, REQUIRED attributes and other attributes. 3
2. netCDF conventions for the data variables common to multiple DAC’s 5
  2.a Common data variables 5
  2.b other data variables 5
TABLE 2: The common data variables and their REQUIRED attributes. 5
TABLE 3: Other data variables and their REQUIRED attributes. 7
3. GLOBAL ATTRIBUTES 7
4. Requirements for the WOCE CD Version 3.0 inventory files 8
  4.a REQUIRED inventory fields (other than UOT LD  – see 4c) 8
  4.b DAC-dependant REQUIRED inventory fields 9
  4.c UOT REQUIRED inventory fields 9
  4.c.1 UOT TOP TABLE REQUIRED inventory fields 9
  4.c.2 UOT SUB TABLE REQUIRED inventory fields 10

Introduction

The V3 Working Group (V3WG) of the WOCE Data Products Committee (DPC) met in early November 2001 to discuss data and inventory reporting conventions the WOCE data system should adopt to insure consistency across all DAC’s; and consequently, enhance the integration of the data sets by those wishing to combine several data sets.

The V3WG recommends all WOCE netCDF data files for the V3 product comply with the conventions as outlined below. These conventions are a superset of the Cooperative Ocean/Atmosphere Research Data Service (COARDS) netCDF standard.   http://ferret.wrc.noaa.gov/noaa_coop/coop_cdf_profile.html

The COARDS standard has received significant support from many research and institutional centers. It is being adopted for a variety of data sets, including climate and forecasting products. Additionally, it is widely recognized by numerous software plotting and analysis packages to provide more automatic processing, plotting, and analysis capabilities.

Additionally, the V3WG has now specified the composition of the WOCE V3 inventory files. Each DAC is REQUIRED to provide one entry in the table for each netCDF file they produce for the V3 dataset. These inventory files are the cornerstones for the off-line and on-line search/integration tools that are envisioned to be released in concert with the V3 datasets.

This document presents the following specifications/conventions:

  1. netCDF conventions for the REQUIRED variables in data (netCDF) files (see Table 1)
  2. netCDF conventions for the common data variables found in data (netCDF) files (see Table 2)
  3. REQUIRED global attributes for netCDF files
  4. requirements for the WOCE CD Version 3.0 inventory files

Any exceptions to the REQUIRED specifications should be brought to the immediate attention of the V3WG.

1. netCDF Conventions for REQUIRED variables in WOCE V3 netCDF files

Listed here are important requirements for COARDS-compliant netCDF files that the DAC’s must address. DAC’s should consult the COARDS standards (see URL above) and/or the V3WG for additional details and questions.

1.a. REQUIRED Variables

These six variables are REQUIRED in every netCDF file (.nc file) for every DAC (see Table 1).  They are essential to for a successful integrated search.

woce_date
woce_time
time
latitude
longitude
depth or pressure (if depth = 0 you must set it as depth=0)

NOTE: In the case where there is a single latitude, longitude, time, and/or depth value per file, then a corresponding single-element array should be utilized as indicated below. This information should NOT be included as an attribute. See 1c for example.

NOTE: variable time must be of the form “minutes since 1900-1-1 0:0:0”. Other time increments may be used, e.g. “days since 1900-1-1”, provided it unambiguously identifies the observation time. Other reference start dates are acceptable, but we encourage the use of “1900-1-1 0:0:0”.

1b. REQUIRED Attributes

Each of the six REQUIRED variables listed above MUST have these 4 attributes and must follow this EXACT convention (including case sensitivity)

long_name=
units=
data_min=
data_max=

NOTE: data_min/max attributes refer to the min/max of the variable data in this file only.

Where the long name and unit is given in TABLE 1 below – they must be followed exactly. For example the long name for woce_date MUST BE WOCE date and the units MUST BE yyyymmdd UTC.

1.c Order of dimensions:

If any or all of the dimensions of a data variable have the interpretations of time, “height or depth", "latitude", or "longitude" then those dimensions should appear in the relative order time, then depth, then latitude, then longitude in the Common Data Format Language (CDL) definition corresponding to the file, e.g.,

Example (attributes etc excluded for brevity):

dimensions:
time = 1 ;
depth = 1004 ;
latitude = 1 ;
longitude = 1 ;

variables: (data type is example)
long time(time) ;
float depth(depth) ;
float latitude(latitude) ;
float longitude(longitude) ;
float T_20(time, depth, latitude, longitude) ;

1.d. Other Attribute conventions

The following 6 attributes are not REQUIRED, but IF they are in the netCDF files they MUST follow this EXACT convention (including case sensitivity).

valid_range OR valid_min and valid_max
C_format
FORTRAN_format
_FillValue
missing_value

TABLE 1: The 6 REQUIRED variables, REQUIRED attributes and other attributes. (red attributes are REQUIRED in the EXACT format shown)

VARIABLE ATTRIBUTE    
time long_name = time   REQUIRED
  units = days since 1900-1-1 0:0:0   REQUIRED
  data_min =   REQUIRED
  data_max =   REQUIRED
  valid_min = (or valid_range) Not REQUIRED. If you have it, it must call it this.
  valid_max =   Not REQUIRED. If you have it, it must call it this.
  C_format =   Not REQUIRED. If you have it, it must call it this.
  FORTRAN_format =   Not REQUIRED. If you have it, it must call it this.
  _FillValue =   Not REQUIRED. If you have it, it must call it this.
  missing_value   REQUIRED
       
depth or pressure long_name = depth or pressure   REQUIRED
  units = meters   REQUIRED
  positive= down or up   REQUIRED
  data_min =   REQUIRED
  data_max =   REQUIRED
  valid_min = (or valid_range) Not REQUIRED. If you have it, it must call it this. 
  valid_max =   Not REQUIRED. If you have it, it must call it this.
  C_format =   Not REQUIRED. If you have it, it must call it this.
  FORTRAN_format =   Not REQUIRED. If you have it, it must call it this.
  _FillValue =   Not REQUIRED. If you have it, it must call it this.
  missing_value   Not REQUIRED. If you have it, it must call it this.
       
latitude long_name = latitude   REQUIRED
  units = degrees_N   REQUIRED
  data_min =   REQUIRED
  data_max =   REQUIRED
  valid_min = (or valid_range) Not REQUIRED. If you have it, it must call it this.
  valid_max =   Not REQUIRED. If you have it, it must call it this.
  C_format =   Not REQUIRED. If you have it, it must call it this.
  FORTRAN_format =   Not REQUIRED. If you have it, it must call it this.
  _FillValue =   Not REQUIRED. If you have it, it must call it this.
  missing_value   Not REQUIRED. If you have it, it must call it this.
       
longitude long_name = longitude   REQUIRED
  units = degrees_E   REQUIRED
  data_min =   REQUIRED
  data_max =   REQUIRED
  valid_min = (or valid_range) Not REQUIRED. If you have it, it must call it this.
  valid_max =   Not REQUIRED. If you have it, it must call it this.
  C_format =   Not REQUIRED. If you have it, it must call it this.
  FORTRAN_format =   Not REQUIRED. If you have it, it must call it this.
  _FillValue =   Not REQUIRED. If you have it, it must call it this.
  missing_value   Not REQUIRED. If you have it, it must call it this.
       
woce_date long_name = WOCE date   REQUIRED
  units = yyyymmdd UTC   REQUIRED
  data_min =   REQUIRED
  data_max =   REQUIRED
  valid_min = (or valid_range) Not REQUIRED. If you have it, it must call it this.
  valid_max =   Not REQUIRED. If you have it, it must call it this.
  C_format =   Not REQUIRED. If you have it, it must call it this.
  FORTRAN_format =   Not REQUIRED. If you have it, it must call it this.
  _FillValue =   Not REQUIRED. If you have it, it must call it this.
  missing_value   Not REQUIRED. If you have it, it must call it this.
       
woce_time long_name = WOCE time of day   REQUIRED
  units = hhmmss.dd UTC   REQUIRED
  data_min =   REQUIRED
  data_max =   REQUIRED
  valid_min = (or valid_range) Not REQUIRED. If you have it, it must call it this.
  valid_max =   Not REQUIRED. If you have it, it must call it this.
  C_format =   Not REQUIRED. If you have it, it must call it this.
  FORTRAN_format =   Not REQUIRED. If you have it, it must call it this.
  _FillValue =   Not REQUIRED. If you have it, it must call it this.
  missing_value   Not REQUIRED. If you have it, it must call it this.

2. netCDF conventions for the data variables common to multiple DAC’s

2.a Common data variables

A study of all DACS indicated that there are 6 variables most common to more than one DAC. As such they should be consistently named and given consistent attributes (see Table 2).

temperature
sst
salinity
u (current, not winds)
v (current, not winds)
sea_level

For those DACS with these variables the variables MUST follow the conventions in Table 2 below.  Where the long name and unit is given in the tables below – they must be followed exactly. For example the long name for sst MUST BE temperature and the units MUST BE degree C.

2.b other data variables

All data variables other than these 6 common data variables must follow the conventions in Table 3.

TABLE 2: The common data variables and their REQUIRED attributes. (red attributes are REQUIRED in the EXACT format shown)

VARIABLE ATTRIBUTE    
temperature long_name = temperature   REQUIRED
  units = degree C   REQUIRED
  data_min =   REQUIRED
  data_max =   REQUIRED
  valid_min = (or valid_range) Not REQUIRED. If you have it, it must call it this.
  valid_max =   Not REQUIRED. If you have it, it must call it this.
  C_format =   Not REQUIRED. If you have it, it must call it this.
  FORTRAN_format =   Not REQUIRED. If you have it, it must call it this.
  _FillValue =   Not REQUIRED. If you have it, it must call it this.
  missing_value   Not REQUIRED. If you have it, it must call it this.
       
sst long_name = sea surface temperature   REQUIRED
  units = degree C   REQUIRED
  data_min =   REQUIRED
  data_max =   REQUIRED
  valid_min = (or valid_range) Not REQUIRED. If you have it, it must call it this.
  valid_max =   Not REQUIRED. If you have it, it must call it this.
  C_format =   Not REQUIRED. If you have it, it must call it this.
  FORTRAN_format =   Not REQUIRED. If you have it, it must call it this.
  _FillValue =   Not REQUIRED. If you have it, it must call it this.
  missing_value   Not REQUIRED. If you have it, it must call it this.
       
salinity long_name =   REQUIRED but long name up to DAC
  units =   REQUIRED
  data_min =   REQUIRED
  data_max =   REQUIRED
  valid_min = (or valid_range) Not REQUIRED. If you have it, it must call it this.
  valid_max =   Not REQUIRED. If you have it, it must call it this.
  C_format =   Not REQUIRED. If you have it, it must call it this.
  FORTRAN_format =   Not REQUIRED. If you have it, it must call it this.
  _FillValue =   Not REQUIRED. If you have it, it must call it this.
  missing_value   Not REQUIRED. If you have it, it must call it this.
       
u long_name =   REQUIRED but long name up to DAC (e.g. eastward velocity component)
  units = meters per second   REQUIRED
  data_min =   REQUIRED
  data_max =   REQUIRED
  valid_min = (or valid_range) Not REQUIRED. If you have it, it must call it this.
  valid_max =   Not REQUIRED. If you have it, it must call it this.
  C_format =   Not REQUIRED. If you have it, it must call it this.
  FORTRAN_format =   Not REQUIRED. If you have it, it must call it this.
  _FillValue =   Not REQUIRED. If you have it, it must call it this.
  missing_value   Not REQUIRED. If you have it, it must call it this.
       
v long_name =   REQUIRED but long name up to DAC (e.g. northward velocity component)
  units = meters per second   REQUIRED
  data_min =   REQUIRED
  data_max =   REQUIRED
  valid_min = (or valid_range) Not REQUIRED. If you have it, it must call it this.
  valid_max =   Not REQUIRED. If you have it, it must call it this.
  C_format =   Not REQUIRED. If you have it, it must call it this.
  FORTRAN_format =   Not REQUIRED. If you have it, it must call it this.
  _FillValue =   Not REQUIRED. If you have it, it must call it this.
  missing_value   Not REQUIRED. If you have it, it must call it this.
       
sea_level long_name =   REQUIRED but long name up to DAC
  units = meters   REQUIRED
  data_min =   REQUIRED
  data_max =   REQUIRED
  valid_min = (or valid_range) Not REQUIRED. If you have it, it must call it this.
  valid_max =   Not REQUIRED. If you have it, it must call it this.
  C_format =   Not REQUIRED. If you have it, it must call it this.
  FORTRAN_format =   Not REQUIRED. If you have it, it must call it this.
  _FillValue =   Not REQUIRED. If you have it, it must call it this.
  missing_value   Not REQUIRED. If you have it, it must call it this.
       
wind_speed long_name =   REQUIRED but long name up to DAC
  units = meters per second   REQUIRED
  data_min =   REQUIRED
  data_max =   REQUIRED
  valid_min = (or valid_range) Not REQUIRED. If you have it, it must call it this.
  valid_max =   Not REQUIRED. If you have it, it must call it this.
  C_format =   Not REQUIRED. If you have it, it must call it this.
  FORTRAN_format =   Not REQUIRED. If you have it, it must call it this.
  _FillValue =   Not REQUIRED. If you have it, it must call it this.
  missing_value   Not REQUIRED. If you have it, it must call it this.

TABLE 3: Other data variables and their REQUIRED attributes. (red attributes are REQUIRED in the EXACT format shown)

VARIABLE ATTRIBUTE    
variable_name long_name =   REQUIRED but long name up to DAC
  units =   REQUIRED
  data_min =   REQUIRED
  data_max =   REQUIRED
  valid_min = (or valid_range) Not REQUIRED. If you have it, it must call it this.
  valid_max =   Not REQUIRED. If you have it, it must call it this.
  C_format =   Not REQUIRED. If you have it, it must call it this.
  FORTRAN_format =   Not REQUIRED. If you have it, it must call it this.
  _FillValue =   Not REQUIRED. If you have it, it must call it this.
  missing_value   Not REQUIRED. If you have it, it must call it this.

3. GLOBAL ATTRIBUTES

ALL DACS MUST include this global attribute in each .nc netCDF file

•WOCE_Version = “3.0”;
•Conventions = “COARDS/WOCE”; // Cooperative Ocean/Atmosphere Research Data Service

In addition, if you have these in please include them as distinct global attributes using these conventions:

•EXPOCODE =
THE CDV3 will be able to be searched by EXPOCODE to recover all data from that cruise.  IF you have it, please include it as a separate global attribute

•DAC_ID = 
Most DACS identify the datasets with an inhouse number. e.g. the ADCP has a SADCP_ID field which uniquely identifies it to the DAC.  Same for floats, sea_level...  It is that field that would allow a user to go from the CD, ask the DAC for a chunk of data and get exactly what you are looking for. Not REQUIRED - but if the DAC has it it makes getting what you are looking for from the DAC that much easier.

•file_source =
The name of the file from which the .nc file was created.

•file_date =
The date of the file from which .nc file was created

•instrument =
The instrument type. This is up to each DAC

4. Requirements for the WOCE CD Version 3.0 inventory files

Inventories are the tab delimited ASCII text files created by each DAC and supplied to the WDIU for use in the CDV3 search tool. Every netCDF file in the WOCE V3 data set will have a single entry (i.e. line) in the inventory files. These inventory tables will facilitate searching the WOCE data base according to user parameters. The tables will also serve as the foundation for forming DODS URL’s. The first line of every inventory must be a column header line, indicating what is in each column. Every variable is separated by a TAB and each line is ended with a new line (NL).

The inventories for the UOT low density program are a bit different from the rest of the DACS because of the size of the dataset - to try to help in the integrated search. See 4c for details.  UOT High Density should be treated as the other DAC datasets in 4a and 4b.

4.a REQUIRED inventory fields (other than UOT LD  – see 4c)

Thirteen fields are REQUIRED in every inventory file for every DAC other than the UOT Low Density dataset (see 4c). These REQUIRED fields are the bare minimum fields essential to create a successful search across data holdings from all DAC’s. The information contained within the inventory must be only for valid data.

cd_name The name of the DAC example whp or float so the search tool can say – go to the float directory/CD for this data
file_path from the root … example= ./data/atlantic/  note the direction of the slashes=/ and that it begins and ends with a “./”
file_compressed_name  compressed file name … example file.zip \
file_name .nc filename   example 10689.nc
woce_date_min using woce_date variable (yyyymmdd)
woce_date_max using woce_date variable (yyyymmdd)
latitude_min  
latitude_max  
longitude_westmost  
longitude_eastmost  
depth_min or pressure_min  
depth_max or pressure_max  
EXPOCODE  

4.b DAC-dependant REQUIRED inventory fields

In addition the "GOOD" min and max of the six variables found to be common among the DACS should be listed in the inventories of the DACS that have them:

temperature_min
temperature_max
sst_min
sst_max
salinity_min
salinity_max
u_min
u_max
v_min
v_max
sea_level_min
sea_level_max
wind_speed_min
wind_speed_max

4.c UOT REQUIRED inventory fields

Because of the size of the UOT Low Density dataset, it was felt that the best way to be able to deal with this for an integrated search was a Top table inventory and a related Sub table inventory.  The Top table inventory is an inventory of the ZIP files.  The Sub table inventory is of the individual netCDF files contained within those ZIP files, related by file_compressed_name.  The Top table will thus have the min and max of the entire ZIP file, while the sub-table will have the min and max of every individual file within that ZIP file. The Sub table will not even get searched unless the Top table indicates that it falls within the search parameters. 

UOT High Density should follow the conventions listed for the DACS above in 4a and 4b.

4.c.1 UOT TOP TABLE REQUIRED inventory fields

The top table of the UOT inventory should have one line for every ZIP file – that is for every basin per quarter.  It should follow this convention.

 

cd_name The name of the DAC example uot so the search tool can say – go to the uot directory for this data
file_path from the root … example= ./data/atlantic/1993  note the direction of the slashes=/ and that it begins and ends with a “./”
file_compressed_name  compressed file name … example q1_tar.gz
woce_date_min of the entire compressed file using woce_date variable (yyyymmdd)
woce_date_max of the entire compressed file using woce_date variable (yyyymmdd)
latitude_min of the entire compressed file
latitude_max of the entire compressed file
longitude_westmost of the entire compressed file
longitude_eastmost of the entire compressed file
depth_min of the entire compressed file
depth_max of the entire compressed file
temperature_min of the entire compressed file
temperature_max of the entire compressed file
salinity_min of the entire compressed file
salinity_max of the entire compressed file

 

4.c.2 UOT SUB TABLE REQUIRED inventory fields

The SUB table of the UOT inventory should be of every individual netCDF file in the ZIP files.  It should follow this convention.

cd_name The name of the DAC example uot so the search tool can say – go to the uot directory for this data
file_path from the root … example= ./data/atlantic/1993  note the direction of the slashes=/ and that it begins and ends with a “./”
file_compressed_name  compressed file name … example q1_tar.gz (relates to the top table)
file_name file name of individual data.nc file within zip file
date of the .nc file using woce_date variable (yyyymmdd)
latitude of the .nc file
longitude of the .nc file
depth_min of the .nc file
depth_max of the .nc file
temperature_min of the .nc file
temperature_max of the .nc file
salinity_min of the .nc file
salinity_max of the .nc file