NOAA Logo National Centers for Environmental Information

formerly the National Oceanographic Data Center (NODC)...  more on NCEI

NOAA Satellite and Information Service

You are here: HomeSubmit › Data Collecting Policy

NCEI Data Collecting Policy

March 1, 2018

Purpose

The policy defines the types of data that will be acquired for long-term preservation, and states limitations that may affect acceptance of environmental data at NOAA's National Centers for Environmental Information (NCEI).

General Policy and Guidance

  1. NCEI archives data in accordance with the NOAA Procedure for Scientific Records Appraisal and Archive Approval and the scope of the NCEI archives as documented in this policy. Priority is given to original environmental data over any products derived from such data. NCEI considers the following characteristics in its archive decisions:
    • NOAA or NESDIS mandate
    • Relevance and criticality to NOAA or NESDIS mission
    • Archive-ready data (well structured and adheres to established NCEI standards)
    • Broad user base
    • Relevance to NCEI development of use-inspired products
    • Multiple users (i.e., research, monitoring, relevant to a variety of sectors/users)
    • Uniqueness
    • Completeness
    • Intrinsic value
  2. NCEI constitutes the official NOAA archive. NCEI maintains, processes, distributes and provides long-term stewardship for most of NOAA's environmental and geospatial data, and provides a broad range of user services. NCEI is operated by NESDIS but performs data preservation and stewardship on behalf of the entire agency. NCEI supports U.S. environmental, societal, and economic well-being through stewardship of the Nation's weather, climate, coastal, oceanic, and geophysical information.
  3. NCEI seeks to ensure that environmental data collected by NOAA observing systems at a significant cost to the American public are preserved for future use. In addition, NCEI acquires data that directly complements these data, including data, metadata, and products (and their inputs) produced by NCEI. To this end, NCEI enters into agreements to archive and steward data to meet its mission and to preserve the global environmental data record. Taken together, the data collected at NCEI informs user communities and the general public, benefiting an array of scientific disciplines and sectors of the U.S. economy.
  4. NCEI recognizes that other Federal agencies are responsible for data collection and research in various environmental disciplines. Seeking to complement the archives and missions of other agencies rather than provide redundant archival and information services to the public, NCEI acquires observations, reanalysis/forecast model output, information products and correlative data as appropriate, pertaining to the following general categories1:
    • Weather/climate
    • Oceanographic
    • Marine geophysics
    • Atmospheric/ocean chemistry
    • Atmospheric/ocean indicators
    • Bathymetry
    • Cryospheric
    • Geomagnetic
    • Marine ecosystems
    • Marine geology
    • Paleoclimatology
    • Soil moisture/temperature
    • Space weather
    • Sun-earth interactions
    • Surface characterization
    • Surface water
    • Tropical cyclones
    • Tsunami events
    • Upper air
    • Vegetation indices
    • Volcanic events
    • Weather events
    • Related socio-economic data
  5. Given the abundance of environmental data and information collected and produced, NCEI must carefully consider each acquisition. This is especially true given current automated data collection processes, the sheer volume of modeled datasets, and high-resolution remotely-sensed data. NCEI will, where appropriate and feasible, attempt to assist NOAA and partner scientists in meeting federal open data initiatives. Archival decisions are made after careful consideration of the factors in this policy, as well as a complete consideration of the human, technical, and financial inputs required and the availability and prioritization of limited resources.

FAQ

  1. Can I archive non-NOAA data?
    NCEI accepts most environmental data regardless of its funding or affiliation. Our science stewards can walk you through our non-NOAA data submission criteria.
  2. What if my data don’t fit NCEI’s categories?
    Please contact us if you have environmental or climatological data that doesn’t cleanly fit an NCEI category.
  3. What can I do if my data get rejected?
    Some data are rejected because of simple errors, and can be resubmitted after the requested changes are made. If your data are rejected because it doesn't fit NCEI's archive, consider submitting to another data repository.
  4. What data formats does NCEI accept?
    The following data types and formats are recommended for preserving your data. This list isn’t comprehensive, and we can sometimes accept other file types. Please contact our science stewards if your submission format isn’t listed.

    Archive Recommended Formats

    Data Type(s) Accepted format(s) Comment(s)
    profiles
    (t, s, d, chl, etc.)
    ASCII
    .csv
    netCDF with CF conventions
    For tabular data (e.g., .csv), header information should clearly indicate the content of each column and be accompanied by a data dictionary to facilitate verifying data validity.
    time series ASCII
    .csv
    netCDF with CF conventions
    satellite processed data
    and reanalysis products
    netCDF4 (classic) with CF conventions
    netCDF3 with CF conventions
    HDF4
    HDF5
    See NCEI NetCDF Templates v2.0 documentation
    Geographic Information Systems Geo TIFF
    OpenGIS GML
    ESRI shapefiles (.shp, .dbf, .shx)
    OGC KML
    Other formats are acceptable, refer to guidance from NARA.
    digital video
    motion pictures
    MPEG-2000
    MPEG-2
    MPEG-4_AVC (H.264)
    MPEG-4_V (H.264)
    MPEG-1
    .dv
    Other formats are acceptable, refer to guidance from NARA.
    digital still images TIFF (Version 4, 5, or 6) Uncompressed Bitmap
    TIFF Uncompressed File with Exif metadata
    Portable Network Graphics (PNG)
    Graphics Interchange Format version 89a (GIF)
    Other formats are acceptable, refer to guidance from NARA.
    documentation (reports, summaries) PDF/A-1b
    ODP Version 1.0
    ASCII text
    Other formats are acceptable, refer to guidance from NARA.
  5. Is there a cost to archive data?
    You may need funding for a submission depending on it’s size, complexity, and affiliation. Our science stewards can help determine if there is a cost to archive your data.

Additional questions? Contact our science stewards before submitting your data.

Authority

  1. NOAA/NESDIS Environmental Data Management Planning Policy.
    https://www.nesdis.noaa.gov/sites/default/files/asset/document/npd_6010_01a.pdf
  2. NOAA Procedure for Scientific Records Appraisal and Archive Approval, 2008.
    https://nosc.noaa.gov/EDMC/documents/NOAA_Procedure_document_final_12-16-1.pdf
  3. NOAA Administrative Order (NAO) 212-15, Management of Environmental Data and Information, 2010.
    https://nosc.noaa.gov/EDMC/nao_212-15.php
  4. NESDIS Environmental Data Management Planning Procedural Directive, Version 2.0.1. 2015.
    https://nosc.noaa.gov/EDMC/PD.DMP.php

1 Selected and adapted from science keywords in the NASA Global Change Master Directory. See https://earthdata.nasa.gov/about/gcmd/global-change-master-directory-gcmd-keywords