===== ONE BAD ZIP FILE - REPLACE WITH FILE FROM 20130904: Email report: /nodc/users/nhall/CCHDO/20131119/data/1-data/atl/a16s/ct1/a16s_00243_00003_ct1.csv bad CRC 00bfbc9c (should be c493a12a) FROM UNZIP LOG: UNZIP: /nodc/users/nhall/CCHDO/20131119/data/0-data/atl/a16s/a16s_ct1.zip TO: /nodc/users/nhall/CCHDO/20131119/data/1-data/atl/a16s ABA$ unzip -t /nodc/users/nhall/CCHDO/20131119/data/0-data/atl/a16s/a16s_ct1.zip Archive: /nodc/users/nhall/CCHDO/20131119/data/0-data/atl/a16s/a16s_ct1.zip testing: a16s_00243_00003_ct1.csv bad CRC 00bfbc9c (should be c493a12a) At least one error was detected in /nodc/users/nhall/CCHDO/20131119/data/0-data/atl/a16s/a16s_ct1.zip. No errors detected in compressed data of /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/atl/a16s/a16s_ct1.zip. cp /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/atl/a16s/a16s_ct1.zip /nodc/users/nhall/CCHDO/20131119/data/0-data/atl/a16s/a16s_ct1.zip ===== ZERO LENGTH (16): /data/repeat/pacific/pr20/pr20_b/pr20_b_nc_ctd.zip 2002-05-10 FAILED:Zero Length /data/repeat/pacific/pr20/pr20_b/pr20_b_nc_hyd.zip 2002-06-05 FAILED:Zero Length /data/repeat/pacific/pr21/pr21_a/pr21_asu.txt 2001-12-03 FAILED:Zero Length /data/repeat/pacific/pr21/pr21_a/pr21_a_ct1.zip 2001-12-03 FAILED:Zero Length /data/co2clivar/atlantic/ar10/ar10_18HU19930405/18HU19930405_hy1.csv 2010-08-27 FAILED:Zero Length /data/repeat/atlantic/ar08/ar08_e/ar08_e_hy1.csv 2002-02-01 FAILED:Zero Length /data/repeat/atlantic/ar08/ar08_e/ar08_e_nc_ctd.zip 2002-10-11 FAILED:Zero Length /data/repeat/atlantic/ar08/ar08_e/ar08_e_nc_hyd.zip 2002-06-05 FAILED:Zero Length /data/repeat/atlantic/ar08/ar08_f/ar08_fsu.txt 2001-12-12 FAILED:Zero Length /data/repeat/atlantic/ar08/ar08_f/ar08_f_ct1.zip 2003-02-18 FAILED:Zero Length /data/repeat/atlantic/ar08/ar08_f/ar08_f_hy1.csv 2009-08-12 FAILED:Zero Length /data/repeat/atlantic/ar08/ar08_f/ar08_f_nc_ctd.zip 2003-02-18 FAILED:Zero Length /data/repeat/atlantic/ar08/ar08_g/ar08_gsu.txt 2001-12-12 FAILED:Zero Length /data/co2clivar/atlantic/bats/ars01/33H420100202/33H420100202_ars20_ct1.zip 2010-03-17 FAILED:Zero Length /data/co2clivar/atlantic/bats/ars01/33H420100202/33H420100202_ars20_nc_ctd.zip 2010-03-17 FAILED:Zero Length /data/co2clivar/southern/sr03/sr03_09AR20110104/09AR20110104_nc_hyd.zip 2013-10-16 FAILED:Zero Length LOOK FOR REPLACEMENT FILES IN CRAWL FROM 20130904 /data/repeat/pacific/pr20/pr20_b/pr20_b_nc_ctd.zip 2002-05-10 FAILED:Zero Length cp /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/pac/pr20_b/pr20_b_nc_ctd.zip /data/repeat/pacific/pr20/pr20_b/pr20_b_nc_hyd.zip 2002-06-05 FAILED:Zero Length cp /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/pac/pr20_b/pr20_b_nc_hyd.zip /data/repeat/pacific/pr21/pr21_a/pr21_asu.txt 2001-12-03 FAILED:Zero Length cp /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/pac/pr21_a/pr21_asu.txt /data/repeat/pacific/pr21/pr21_a/pr21_a_ct1.zip 2001-12-03 FAILED:Zero Length cp /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/pac/pr21_a/pr21_a_ct1.zip /data/co2clivar/atlantic/ar10/ar10_18HU19930405/18HU19930405_hy1.csv 2010-08-27 FAILED:Zero Length ?? /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/atl/ .... (DOES NOT EXIST IN EARLIER CRAWL) /data/repeat/atlantic/ar08/ar08_e/ar08_e_hy1.csv 2002-02-01 FAILED:Zero Length cp /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/atl/ar08_e/ar08_e_hy1.csv /data/repeat/atlantic/ar08/ar08_e/ar08_e_nc_ctd.zip 2002-10-11 FAILED:Zero Length cp /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/atl/ar08_e/ar08_e_nc_ctd.zip /data/repeat/atlantic/ar08/ar08_e/ar08_e_nc_hyd.zip 2002-06-05 FAILED:Zero Length cp /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/atl/ar08_e/ar08_e_nc_hyd.zip /data/repeat/atlantic/ar08/ar08_f/ar08_fsu.txt 2001-12-12 FAILED:Zero Length cp /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/atl/ar08_f/ar08_fsu.txt /data/repeat/atlantic/ar08/ar08_f/ar08_f_ct1.zip 2003-02-18 FAILED:Zero Length cp /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/atl/ar08_f/ar08_f_ct1.zip /data/repeat/atlantic/ar08/ar08_f/ar08_f_hy1.csv 2009-08-12 FAILED:Zero Length cp /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/atl/ar08_f/ar08_f_hy1.csv /data/repeat/atlantic/ar08/ar08_f/ar08_f_nc_ctd.zip 2003-02-18 FAILED:Zero Length cp /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/atl/ar08_f/ar08_f_nc_ctd.zip /data/repeat/atlantic/ar08/ar08_g/ar08_gsu.txt 2001-12-12 FAILED:Zero Length cp /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/atl/ar08_g/ar08_gsu.txt /data/co2clivar/atlantic/bats/ars01/33H420100202/33H420100202_ars20_ct1.zip 2010-03-17 FAILED:Zero Length ?? /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/atl/ars01/ /data/co2clivar/atlantic/bats/ars01/33H420100202/33H420100202_ars20_nc_ctd.zip 2010-03-17 FAILED:Zero Length ?? /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/atl/ars01/ /data/co2clivar/southern/sr03/sr03_09AR20110104/09AR20110104_nc_hyd.zip 2013-10-16 FAILED:Zero Length ?? /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/soc/sr03_09AR20110104/ FIX: 12/16 can be replaced with files from 20130904 4/16 cannot be found in 20130904 files (marked ?? above) TRY BY HAND ON CDIP WEB SITE: /data/co2clivar/atlantic/ar10/ar10_18HU19930405/18HU19930405_hy1.csv Appears to be same cruise as ar10_d -- leave it out. /data/co2clivar/atlantic/bats/ars01/33H420100202/33H420100202_ars20_ct1.zip Expocode 32KZBATS_33 -- no data, no further info on web site -- leave it out /data/co2clivar/atlantic/bats/ars01/33H420100202/33H420100202_ars20_nc_ctd.zip Expocode 32KZBATS_33 -- no data, no further info on web site -- leave it out /data/co2clivar/southern/sr03/sr03_09AR20110104/09AR20110104_nc_hyd.zip Expocode 09AR20110104 -- no NetCDF Hydrographic data on web site -- leave it out COPY TO PROJECTS DIRECTORY: mkdir /nodc/projects/gtspp/data/Process/CCHDO/data_131119 cd /nodc/projects/gtspp/data/Process/CCHDO/data_131119 cp -r -p /nodc/users/nhall/CCHDO/20131119/data/about . cp -r -p /nodc/users/nhall/CCHDO/20131119/data/0-data . mkdir 0-data/atl/ar08_f COPY REPLACEMENT FILES -- 1 FAULTY ZIP, 12 ZERO-LENGTH: cp -p /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/atl/a16s/a16s_ct1.zip 0-data/atl/a16s/a16s_ct1.zip cp -p /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/pac/pr20_b/pr20_b_nc_ctd.zip 0-data/pac/pr20_b/pr20_b_nc_ctd.zip cp -p /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/pac/pr20_b/pr20_b_nc_hyd.zip 0-data/pac/pr20_b/pr20_b_nc_hyd.zip cp -p /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/pac/pr21_a/pr21_asu.txt 0-data/pac/pr21_a/pr21_asu.txt cp -p /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/pac/pr21_a/pr21_a_ct1.zip 0-data/pac/pr21_a/pr21_a_ct1.zip cp -p /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/atl/ar08_e/ar08_e_hy1.csv 0-data/atl/ar08_e/ar08_e_hy1.csv cp -p /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/atl/ar08_e/ar08_e_nc_ctd.zip 0-data/atl/ar08_e/ar08_e_nc_ctd.zip cp -p /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/atl/ar08_e/ar08_e_nc_hyd.zip 0-data/atl/ar08_e/ar08_e_nc_hyd.zip cp -p /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/atl/ar08_f/ar08_fsu.txt 0-data/atl/ar08_f/ar08_fsu.txt cp -p /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/atl/ar08_f/ar08_f_ct1.zip 0-data/atl/ar08_f/ar08_f_ct1.zip cp -p /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/atl/ar08_f/ar08_f_hy1.csv 0-data/atl/ar08_f/ar08_f_hy1.csv cp -p /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/atl/ar08_f/ar08_f_nc_ctd.zip 0-data/atl/ar08_f/ar08_f_nc_ctd.zip cp -p /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/0-data/atl/ar08_g/ar08_gsu.txt 0-data/atl/ar08_g/ar08_gsu.txt CORRECT THIS MD5 IN md5_digest.log FOR BAD ZIP FILE (this is the good summary string): ABA$ pwd /nodc/users/nhall/CCHDO/ziperr_201304/data_20130904/about ABA$ grep "a16s_ct1.zip" md5_digest.log 318MSAVE5 atl/a16s/a16s_ct1.zip 1 8d80368916355f695b96273520de0c05 1316149 ABA$ CORRECT 12 LOG ENTRIES AND MD5 ENTRIES FOR ZERO-LENGTH FILES: grep "pr20_b_nc_ctd.zip" *.log download.log:/data/repeat/pacific/pr20/pr20_b/pr20_b_nc_ctd.zip 2002-05-10 OK 21OR287_1 pac/pr20_b/pr20_b_hy1.csv 1 aeed5d079b917466332786e3dab0cb4b 52745 md5_digest.log:21OR287_1 pac/pr20_b/pr20_b_nc_ctd.zip 1 c29f832dcf39cb001926346be5038388 328378 grep "pr20_b_nc_hyd.zip" *.log download.log:/data/repeat/pacific/pr20/pr20_b/pr20_b_nc_hyd.zip 2002-06-05 OK 21OR287_1 pac/pr20_b/pr20_b_nc_ctd.zip 1 c29f832dcf39cb001926346be5038388 328378 md5_digest.log:21OR287_1 pac/pr20_b/pr20_b_nc_hyd.zip 1 344deacbc6e369d4559ba948f68dc385 53998 grep "pr21_asu.txt" *.log download.log:/data/repeat/pacific/pr21/pr21_a/pr21_asu.txt 2001-12-03 OK 21OR257_1 pac/pr20_a/pr20_a_nc_hyd.zip 1 86dd4ab41ed49d5b541fd94134558b23 87170 md5_digest.log:21OR266_2 pac/pr21_a/pr21_asu.txt 1 b364fac7225a63ca81ee23851c50ebb4 2809 grep "pr21_a_ct1.zip" *.log download.log:/data/repeat/pacific/pr21/pr21_a/pr21_a_ct1.zip 2001-12-03 OK 21OR266_2 pac/pr21_a/pr21_asu.txt 1 b364fac7225a63ca81ee23851c50ebb4 2809 md5_digest.log:21OR266_2 pac/pr21_a/pr21_a_ct1.zip 1 9b05fecc7604597604f35d7748df6b1b 56762 grep "ar08_e_hy1.csv" *.log download.log:/data/repeat/atlantic/ar08/ar08_e/ar08_e_hy1.csv 2002-02-01 OK 08BD0993_1 atl/ar08_e/ar08_e_ct1.zip 1 2c8652ea6d348628685342ee9086c16d 98159 md5_digest.log:08BD0993_1 atl/ar08_e/ar08_e_hy1.csv 1 1c676bbeaba303a172acfc7f01549cc9 38139 grep "ar08_e_nc_ctd.zip" *.log download.log:/data/repeat/atlantic/ar08/ar08_e/ar08_e_nc_ctd.zip 2002-10-11 OK 08BD0993_1 atl/ar08_e/ar08_e_hy1.csv 1 1c676bbeaba303a172acfc7f01549cc9 38139 md5_digest.log:08BD0993_1 atl/ar08_e/ar08_e_nc_ctd.zip 1 79802dce6f2e793c8afdd7e207705da7 149787 grep "ar08_e_nc_hyd.zip" *.log download.log:/data/repeat/atlantic/ar08/ar08_e/ar08_e_nc_hyd.zip 2002-06-05 OK 08BD0993_1 atl/ar08_e/ar08_e_nc_ctd.zip 1 79802dce6f2e793c8afdd7e207705da7 149787 md5_digest.log:08BD0993_1 atl/ar08_e/ar08_e_nc_hyd.zip 1 4e308cbc1a04881c3b7bb647f043220f 37488 grep "ar08_fsu.txt" *.log download.log:/data/repeat/atlantic/ar08/ar08_f/ar08_fsu.txt 2001-12-12 OK 08BD0993_1 atl/ar08_e/ar08_e_nc_hyd.zip 1 4e308cbc1a04881c3b7bb647f043220f 37488 md5_digest.log:08BD0394_1 atl/ar08_f/ar08_fsu.txt 1 2942bb0f5eec57a201843ceaea0598d3 11515 grep "ar08_f_ct1.zip" *.log download.log:/data/repeat/atlantic/ar08/ar08_f/ar08_f_ct1.zip 2003-02-18 OK 08BD0394_1 atl/ar08_f/ar08_fsu.txt 1 2942bb0f5eec57a201843ceaea0598d3 11515 md5_digest.log:08BD0394_1 atl/ar08_f/ar08_f_ct1.zip 1 11c49140117a0a716f5acd9edd1e3c09 75792 grep "ar08_f_hy1.csv" *.log download.log:/data/repeat/atlantic/ar08/ar08_f/ar08_f_hy1.csv 2009-08-12 OK 08BD0394_1 atl/ar08_f/ar08_f_ct1.zip 1 11c49140117a0a716f5acd9edd1e3c09 75792 md5_digest.log:08BD0394_1 atl/ar08_f/ar08_f_hy1.csv 1 a04b821b54451d829ddb6957ad7703e2 135513 grep "ar08_f_nc_ctd.zip" *.log download.log:/data/repeat/atlantic/ar08/ar08_f/ar08_f_nc_ctd.zip 2003-02-18 OK 08BD0394_1 atl/ar08_f/ar08_f_hy1.csv 1 a04b821b54451d829ddb6957ad7703e2 135513 md5_digest.log:08BD0394_1 atl/ar08_f/ar08_f_nc_ctd.zip 1 78c81a9f19aeeb0442b8ebb958bcd38a 141571 grep "ar08_gsu.txt" *.log download.log:/data/repeat/atlantic/ar08/ar08_g/ar08_gsu.txt 2001-12-12 OK 08BD0394_1 atl/ar08_f/ar08_f_nc_ctd.zip 1 78c81a9f19aeeb0442b8ebb958bcd38a 141571 md5_digest.log:08EH1094_1 atl/ar08_g/ar08_gsu.txt 1 621691d1281676ab2cad5360d5371b00 3647 HERE ARE THE THREE GROUPS OF LINES TO REPLACE IN MD5 SUMMARY EACH GROUP WITH PRECEDING "GOOD" MD5 SUMMARY LINE: 21OR287_1 pac/pr20_b/pr20_b_hy1.csv 1 aeed5d079b917466332786e3dab0cb4b 52745 21OR287_1 pac/pr20_b/pr20_b_nc_ctd.zip 1 c29f832dcf39cb001926346be5038388 328378 21OR287_1 pac/pr20_b/pr20_b_nc_hyd.zip 1 344deacbc6e369d4559ba948f68dc385 53998 21OR257_1 pac/pr20_a/pr20_a_nc_hyd.zip 1 86dd4ab41ed49d5b541fd94134558b23 87170 21OR266_2 pac/pr21_a/pr21_asu.txt 1 b364fac7225a63ca81ee23851c50ebb4 2809 21OR266_2 pac/pr21_a/pr21_a_ct1.zip 1 9b05fecc7604597604f35d7748df6b1b 56762 08BD0993_1 atl/ar08_e/ar08_e_ct1.zip 1 2c8652ea6d348628685342ee9086c16d 98159 08BD0993_1 atl/ar08_e/ar08_e_hy1.csv 1 1c676bbeaba303a172acfc7f01549cc9 38139 08BD0993_1 atl/ar08_e/ar08_e_nc_ctd.zip 1 79802dce6f2e793c8afdd7e207705da7 149787 08BD0993_1 atl/ar08_e/ar08_e_nc_hyd.zip 1 4e308cbc1a04881c3b7bb647f043220f 37488 08BD0394_1 atl/ar08_f/ar08_fsu.txt 1 2942bb0f5eec57a201843ceaea0598d3 11515 08BD0394_1 atl/ar08_f/ar08_f_ct1.zip 1 11c49140117a0a716f5acd9edd1e3c09 75792 08BD0394_1 atl/ar08_f/ar08_f_hy1.csv 1 a04b821b54451d829ddb6957ad7703e2 135513 08BD0394_1 atl/ar08_f/ar08_f_nc_ctd.zip 1 78c81a9f19aeeb0442b8ebb958bcd38a 141571 08EH1094_1 atl/ar08_g/ar08_gsu.txt 1 621691d1281676ab2cad5360d5371b00 3647 The log and the md5 are corrected in /nodc/projects/gtspp/data/Process/CCHDO/data_131119/about Now need to run unzip on it... Copy it back to /nodc/users/nhall/CCHDO/20131119 first.. cd /nodc/projects/gtspp/data/Process/CCHDO/data_131119 cp -r -p /nodc/users/nhall/CCHDO/20131119/data/about . cp -r -p /nodc/users/nhall/CCHDO/20131119/data/0-data . ABA$ pwd /nodc/users/nhall/CCHDO/20131119 ABA$ ls atstart.txt cchdo_20131119.log crawlcch_20120425.pl data ABA$ mv data data_a ABA$ mkdir data_b ABA$ ls atstart.txt cchdo_20131119.log crawlcch_20120425.pl data data_a ABA$ cd data ABA$ cp -r -p /nodc/projects/gtspp/data/Process/CCHDO/data_131119/about . ABA$ cp -r -p /nodc/projects/gtspp/data/Process/CCHDO/data_131119/0-data . ABA$ crawlcch_20120425.pl -nocrawl > ccunz_20131119.log ABA$ COPY THE RESULTS TO PROJECTS DIRECTORY (for further testing): cd /nodc/projects/gtspp/data/Process/CCHDO/data_131119 cp -r -p /nodc/users/nhall/CCHDO/20131119/data/about . cp -r -p /nodc/users/nhall/CCHDO/20131119/data/1-data . LOOKS TO BE A SUCCESS -- NO ZIP ERRORS DETECTED, UNZIP PROBLEMS LIMITED TO THE "SKIPPED" FILES: CRAWL QUALITY REPORT: DOWNLOAD FAILURES (3): SKIPPED (3): /data/co2clivar/atlantic/ar04/holder/ar04_06MT200206su.txt 2009-06-15 FAILED:SKIPPED (holder) /data/co2clivar/atlantic/ar04/holder/ar04_06MT200206_ct1.zip 2009-06-15 FAILED:SKIPPED (holder) /data/co2clivar/atlantic/ar04/holder/ar04_06MT200206_hy1.csv 2009-06-15 FAILED:SKIPPED (holder) UNZIP/COPY FAILURES (3, 3 due to download failure): NOTE: UNZIP/COPY ERRORS = DOWNLOAD FAILURES (3) No unexpected additional unzip/copy errors MD5 SUMMARY MATCH FAILURES (111): EXTRA TRIES TO MATCH MD5 REQUIRED FOR 111 FILES: 1 TRIES: 3558 2 TRIES: 108 3 TRIES: 3 DONE 10:08 AM 12/11/2013 Due to some bad formatting in the CTD files, inventory script was modified.. working copy is now inv04a.pl Do the inventories in the Process directory: cd /nodc/projects/gtspp/data/Process/CCHDO/data_131119 cp /nodc/users/nhall/CCHDO/20131119/data/inv04a.pl . cp /nodc/users/nhall/CCHDO/20131119/data/boundaries.pl . inv04a.pl -ctd inv04a.pl -bot boundaries.pl -limits -i lldt_bot.txt > limits_bot.txt boundaries.pl -lldis -i lldt_bot.txt > lldis_bot.txt boundaries.pl -llall -i lldt_bot.txt > llall_bot.txt boundaries.pl -limits -i lldt_ctd.txt > limits_ctd.txt boundaries.pl -lldis -i lldt_ctd.txt > lldis_ctd.txt boundaries.pl -llall -i lldt_ctd.txt > llall_ctd.txt Created by inv04a.pl: inv_ctd.txt Inventory of CTD data lldt_ctd.txt lon, lat, date, time for CTD data inv_bot.txt Inventory of bottle data lldt_bot.txt lon, lat, date, time for bottle data Created by boundaries.pl: limits_ctd.txt lon-lat and date-time limits for CTD data llall_ctd.txt All lon-lat pairs for CTD data lldis_ctd.txt Distinct (sorted) lon-lat pairs for CTD data limits_bot.txt lon-lat and date-time limits for bottle data llall_bot.txt All lon-lat pairs for bottle data lldis_bot.txt Distinct (sorted) lon-lat pairs for bottle data lldt_bot_err.txt Reports of impossible position or date -- 91 instances found in bottle data (not included in limits or lon-lat pairs files) ============================================== Post notice to help desk ticket: https://intra.nodc.noaa.gov/cgi-bin/helpdesk/ismd?function=view&ticket=17841 ============================================== 1:10 PM 12/11/2013 The CCHDO collection at /nodc/projects/gtspp/data/Process/CCHDO/data_131119 is now complete (12 files were picked up from a crawl done 20130904, and the log and MD5 values were edited, so a MD5 check should work). The inventory script and boundaries script have also been run on it. The following inventory files are available in data_131119/: Created by inv04a.pl: inv_ctd.txt Inventory of CTD data lldt_ctd.txt lon, lat, date, time for CTD data inv_bot.txt Inventory of bottle data lldt_bot.txt lon, lat, date, time for bottle data Created by boundaries.pl: limits_ctd.txt lon-lat and date-time limits for CTD data llall_ctd.txt All lon-lat pairs for CTD data lldis_ctd.txt Distinct (sorted) lon-lat pairs for CTD data limits_bot.txt lon-lat and date-time limits for bottle data llall_bot.txt All lon-lat pairs for bottle data lldis_bot.txt Distinct (sorted) lon-lat pairs for bottle data lldt_bot_err.txt Reports of impossible position or date -- 91 instances found in bottle data (not included in limits or lon-lat pairs files) These files should allow the archive process to continue, starting at step 7 (rename or copy the appropriate files in the list above) and 8 (open the two limits files in the above list, and copy contents as needed); or just start at the SOP step 7, using the inv04a.pl and boundaries.pl scripts in the data_131119/ directory. If anyone finds any more errors or anomalies that I might be able to do something about, please let me know. Last week, on Dec. 6, at the end of the ODIP-2 workshop, I spoke to Steve Diggs about this, telling him that, while retrieval appears to be better than 99% reliable, the number of remaining errors in the downloaded files (blank files, zip file errors) make automation of the process impossible. He says he is now consulting with the programmer who built the CCHDO file access Application programming interface, and should be able to give me a simplified method of access that may work better.