4 km AVHRR Pathfinder Project

AVHRR Pathfinder Version 5.0 cloud-screened files: HDF to GIS-ready format

The following documentation describes the steps used within ArcGIS v10 with the Spatial Analyst tool to produce yearly averages from monthly day/night Pathfinder Version 5.0 cloud-screened HDF4 files. While these cloud-screened files are derived from official Pathfinder Version 5.0 data, they are NOT part of the official Version 5.0 collection. To access this collection use the Version5.0/ directory. The utilization of the Pathfinder Version 5.0 cloud-screened files saves the user from having to a) mask out cloud-contaminated pixels and b) account for "day" and "night" data files when averaging the data. For more information on their provenance, contents, or relationship to official Pathfinder Version 5.0 data, please contact Ken Casey at Kenneth.Casey@noaa.gov.

If you use Version 5 Pathfinder data, please cite the following publication: Casey, K.S., T.B. Brandon, P. Cornillon, and R. Evans (2010). "The Past, Present and Future of the AVHRR Pathfinder SST Program", in Oceanography from Space: Revisited, eds. V. Barale, J.F.R. Gower, and L. Alberotanza, Springer. DOI: 10.1007/978-90-481-8681-5_16. (Click here for a PDF copy)

NOAA makes no warranty regarding these data, expressed or implied, nor does the fact of distribution constitute such a warranty. NOAA and NODC cannot assume liability for any damages caused by any errors or omissions in these data, nor as a result of the failure of these data to function on a particular system. Documentation is provided as a courtesy and does not constitute endorsement of a particular corporation, software, or product.

Cloud-screened Pathfinder Version 5.0 annual averages methodology:

• Developed from monthly day/night Pathfinder Version 5.0 cloud-screened HDF4 files (http://data.nodc.noaa.gov/pathfinder/Version5.0_CloudScreened
• Produced by directly importing files into ArcGIS 10 in combination with ArcGIS Spatial Analyst Tool(s)
• Original data of quality flags 4-7 suitable for general use studies and depicting trends
• Final products will include monthly files plus annual mean SST as GIS files; units converted to degrees C.
• The basic procedure is to convert pixel values for EACH monthly layer to degrees Celsius, then average the twelve months together for each year.
• Final products are UNPROJECTED files. In order to assign a projection, users may use the Warp tool within ArcGIS 10, use a script, or the Marine Geospatial Ecology Tools (http://code.env.duke.edu/projects/mget*).

 

STEP 1 - Twelve months (one year) of Pathfinder Version 5 cloud-screened data (http://www.nodc.noaa.gov/SatelliteData/pathfinder4km/available.html) are added as layers within ArcGIS 10. Use data from the /FullRes subdirectory. Data are HDF4 files.






 

STEP 2 - As ArcGIS 10 imports each monthly file, a message will appear asking the user to select a data set. Select SST4_daynitavg (subdataset 6) for EACH time the message appears, for a total of twelve times. (This step selects day/night data of quality 4-7).



 

STEP 3 - On the initial import, ArcGIS v10 attempts to colorize the entire range of HDF4 data type; as a result the data will appear as a black box.



By having ArcGIS 10 calculate the histogram for EACH monthly data layer, a realistic set of pixel values will display (0 - approx. 500). The first step in working with the twelve HDF layers is to have ArcGIS 10 calculate the histogram for EACH monthly layer (this may be slow).

 

In ArcGIS 10: In the layer Table of Contents screen, right-click on a monthly layer name > Left-click Properties > Left-click Symbology > Under the Stretch drop-down menu, Left-click Histogram Equalize (click YES when a message appears stating that the histogram does not exist, do you want to calculate the histogram?).



 

After calculating the histogram for each monthly layer, a global view with pixel values should be visible for each month.


 

STEP 4 - After calculating the histogram for each month, ArcGIS should display a pixel value range for each layer. Data of pixel values less than 16 are unrealistic, so ArcGIS 10 Spatial Analyst Tool Suite is utilized to set all pixels 15 and less to NULL (NoData) for EACH monthly layer. Removing these artifact pixels may be accomplished by various methods in the Spatial Analyst tool suite, including the SetNull, Con, and Reclassification tools. Here, the Reclassification tool is employed to set unrealistic pixel values to NoData.

In ArcGIS 10: Open Arc Toolbox > Open Spatial Analyst Tools > Select Reclass > Left Double-click Reclassify. In the Reclassify window:

1. Input raster = Monthly_filename.hdf:6
2. Reclass field = Value
3. Reclassification = Set Old values as 0 - 15, New values as NoData. Leave NoData as NoData. DELETE all other rows in the Reclassification (click on small left box in row to highlight, and delete. Multiple rows can be selected with the SHIFT or CONTROL keys).
4. Use the Save button to save the reclassification so you can reuse it (choose any file name)
5. Output raster = File name of your choice
6. Leave the Change missing values to NoData (optional) UNCHECKED
7. Set NoData values for each monthly data layer (for a total of twelve monthly files with pixels marked NoData).



 

STEP 5 - After setting unrealistic pixels to NoData, a data month should display a pixel value range of 17 to app. 500.



These values will be converted to degrees Celsius. (The HDF files are 16-bit files, and pixel values can range from 0 to 65535 (2 to the 16th power). However, realistic pixel values for SST will always be less than 600 or so. SST in degrees C = (0.075 * pixel value) - 3.0). EACH monthly file now marked with NoData values is converted to degrees Celsius using ArcGIS' Spatial Analyst Raster Calculator.

In ArcGIS: Open ArcToolbox > Open Spatial Analyst Tools > Select Map Algebra > Left Double-click Raster Calculator. In the Raster Calculator window:

1. Map Algebra Expression = Double click a Monthly filename (e.g. select a file from the files created in the previous step marked with NoData values) * .075 - 3
2. Output raster = Filename of your choice




After running the raster calculator, a layer should appear with a temperature range of -1.725 to approx. 35 (these are pixel values converted to degrees Celsius). The conversion should be run on each monthly layer marked with NoData values from the previous step, for a total of twelve months in degrees Celsius. These layers will be used to calculate a final yearly SST average.



 

STEP 6 - After converting EACH monthly file to degrees Celsius, the twelve monthly files are averaged together using the ArcGIS Spatial Analyst Local Cell Statistics Tool.

In ArcGIS: Open ArcToolbox > Open Spatial Analyst Tools > Select Local > Left Double-click Cell Statistics.

In the Cell Statistics Window:

1. Input rasters or constant values = use the dropdown list or Browse button on the left to add ALL TWELVE MONTHS of data layers previously converted to degrees Celsius.
2. Output raster = Filename of your choice
3. Overlay statistic (optional) = MEAN
4. CHECK the Ignore NoData in calculations box (this box MUST be checked ON)



 

The final averaged layer, and the monthly layers, may be colorized via ArcGIS' Properties dialogue box as desired. Note that that these are unprojected layers, a projection must be assigned by the user separately (the Pathfinder Version 5 HDF files do carry projection information, however ArcGIS 10 cannot read this information at this time).



 

---

To receive notification of its release and to keep abreast of all Pathfinder news, subscribe to the Pathfinder RSS feed.

***

The 4 km Pathfinder effort at NODC and the University of Miami's Rosenstiel School of Marine and Atmospheric Science* (RSMAS) is an extension of and improvement on the sea surface temperature (SST) fields from the older NOAA/NASA AVHRR Oceans Pathfinder project. The Version 5.0, 5.1, and new Version 5.2 data are developed at RSMAS and NODC. In the 4 km Pathfinder project, some important shortcomings in the old 9 km data have been corrected, and the entire 1981-2009 time series reprocessed at the 4 km Global Area Coverage (GAC) level, the highest resolution possible globally. All of these data are available through NODC's ftp, http, OPeNDAP, and THREDDS access systems. Follow the links below to learn more about this effort.

 

Access the Pathfinder RSS Feed What is RSS?

4 km Home	Version 5 User Guide	Example Code and HDF Tools
Current Status	Available Data	Quality Assurance