The WOCE Inventory and Inventory Search Tool

The inventory is at the heart of the initial attempt to integrate the data sets assembled during WOCE. Its implementation depended crucially on the decision to ensure that all data were available in NetCDF format, though some are additionally available in ASCII.

There is a row in the inventory for every NetCDF file in the WOCE data set. Every row includes the full path name of the file, the earliest and latest dates, the westernmost and easternmost longitudes, and the northernmost and southernmost latitudes of the data in the file. Additionally, the range of values of the primary parameters are listed. For example, the inventory entry of a current meter record lists limiting values of u, v and T, the latter being commonly measured on a mooring. Consult the table for a full description of the parameter fields for different measurement types.

The inventory is included on these disks both as an Microsoft Access database and in a tab-delimited form arranged in separate tables corresponding closely to the datasets from the various Data Assembly Centres. We assume that individual groups might wish to download individual tables and generate their own searches. The search tool discussed below will, we hope, be the first of a series of tools which colleagues will develop, adapt and make available as they work further with the datasets and inventories.

WOCE Integrated Search Tool (WIST)

<install WIST>

Search Tool

The objective of the search tool is to return the full path names of NetCDF files satisfying the search input criteria. The intention is that analysis programs such as Matlab can use the returned path names to extract NetCDF files from the DVDs. Hyperlinks in the output provide access to DODS servers holding online copies of the datasets or to local copies of the DVDs.

The tool emphasizes the search for specific oceanographic parameter(s) although sub-selection allows for inclusion, or not, of specific measurement types. For example if hydrographic quality temperature and salinity are required then only the hydrographic box under each of those parameters on the left-hand side need be ticked. Note that sea level range is not usually useful as the measurement datum is local to the instrument. The limits of a search area are defined by a box on a Mercator chart. The bounding box can be defined either:

a)   by entering the coordinates in degrees of the bounding westernmost and easternmost longitudes and pressing the enter key. For example, 160W and –160.0 are alternate forms as is 200.0 although the range +/-180.0 is preferred. Likewise the formats 60.0N or +60.0 are acceptable for latitude with numerical values in range +/-90 the range, or

b)   by clicking and dragging to create an area on the map. The sides and corners of the search box can be rescaled or the whole box moved by dragging its centre icon. The map scrolls left or right when a search boundary reaches the edge of the map. The default centre of 160W should minimize the need to scroll.

High Density WOCE Upper Ocean Thermal (XBT) lines are included in the "UOT all" database so only one of these options is selectable at a time.

All dates, depths, and expocodes are searched if the corresponding boxes are left empty but parameter boxes must be explicitly ticked to be included in the search. (Empty parameter range boxes allow a search over the full range of the ticked observations).

Expocodes up to 12 characters in length identify specific WOCE cruises.
Characters 1-2 identify the country in alphabetical and code order
Characters 1-4 identify the ship in alphabetical and code order.
The remaining characters identify the cruise by number or name and, the leg by a number following an underscore. For example, for code 06MT30_1, the "06" means Germany, "MT" identifies the ship the Meteor, "30" indicates cruise number 30, and "_1" means Leg 1.

A partial expocode can be entered starting at its first character. Thus, by using the expocode box alone, an entry of 06 will pick out all datasets associated with German cruises, while entering 06MT will return all Meteor cruises. Note that some mooring and float datasets have been associated with hydrographic cruises in their vicinity. The % symbol can, if required, be used at the start as a wild card thus allowing a search for characters in the middle of the expocode.

If there is an expocode entry then only those datasets in which they may be present will be searched. (Hydrography, ADCP, Surface Meteorology, Current Meter, Sea Surface Salinity, and Floats)

Dates. The default minimum value is 1950 01 01 and the maximum is 2002 12 31. The text values "year", "month", or "day" always take their default values. So a minimum "1990 month day" and maximum "1990 month day" will search all 1990.

To start the search click on the Search Button at the bottom of the GUI. It should turn dark gray and remain so until the search is complete. If it turns light gray without indicating a finish then the program has crashed and should be restarted.

On completion of the search a pop-up box displays the number of successful "hits" on NetCDF files meeting the search criteria with the option to cancel and vary the settings before resuming the search, or accept (OK) and display a list of the hits.

The File/Save search results button offers three save formats and a summary of the input parameters of the search with xxx as a chosen file name.

xxx.txt gives a tab delimited record of the full path of the NetCDF file.

xxx.htm provides a link to the selected DODS server or local file(see below)

xxx.lis formats the links as list of DODs compatible URL text strings that may be useful in batch processing of files in an application.

xxx_input.txt provides a summary of the input settings of the search.

The structure of each record is

  DAC Name
  Full path name
  File name
  Zip file name (present if individual files have been compressed on the DVD)

The Reset button returns the interface to its initial state.

Access to files on DODS Servers or on a local hard drive

The DODS servers make the WOCE data resource available independently of the DVDs and additionally will include updates and amendments to the published DVD data sets.

Servers handling the WOCE datasets are selectable from the Options button. You may add new addresses to the dods.cfg file in the package. The default address is the first.

A File/Save Results in html format will provide an active link to the DODS server. The link assumes that files that are compressed on the DVDs have been uncompressed on the server.

It is possible, instead of making a DODS server link, to add a local address such as c:\ (in Windows) in the dods.cfg file. This will then place a link to a local copy of the dataset in the saved output .htm file. The user should set up whatever application (ncdump for example) is needed to handle the .nc file extension.

Messages

You must select at least one DAC to retrieve Results.

Occurs when one clicks the search button without having any parameters selected.

An error has occurred that requires the program to be restarted.
Click okay to end this session.

Occurs when the connection to the database (JDataStore) fails for any reason.

The ResultSet that you have requested is too Large.
Please further refine your search.

Occurs when the number of records that are being returned is too large to fit into the memory that the program has available. May require a reload of the program.

The ResultSet you have requested is too large to be displayed.
You may still use the Save feature to save results.

Occurs when the number of records can be held inside the program, but displaying the information is too much of a load on the program. However, the records can still be saved out to a file for viewing.

Searching all of UOT may take a while, see help file for additional information.

Occurs when a user searches on either of the UOT(All), temperature or salinity. This simply serves as a reminder to the user that due to the large size of the UOT dataset, the time till results return can be much longer than for the the other DACs (also see Notes).

Start Date must be earlier than End Date

Obvious!

Notes

The requirement to operate entirely within RAM puts a limit on the number of "hits" than can be handled. This only significantly affects the search over all one million XBTs (UOT all). All the other datasets ( ~135,000) are handled speedily.

The UOT records are organized by ocean, by quarter, and by whether or not they have received full quality control. A very large percentage of the records arise from the Pacific TAO array for which the DAC has included the daily observations. A count of the remaining records from the global WOCE trans-ocean lines amounts to 235,000 records (25%).

There are approximately 240 UOT zip files. We have found that version 2 of the program can generally handle with about 20 UOT zip files without crashing although this is a function of memory size. An alternate figure is that it seems practical to recover ~140,00 records at a time.

After a crash the program may need to be ended from the Command Shell (Ctrl C in WIndows)