From: Israel Muñiz Date: Wed, Oct 16, 2019 at 1:04 PM Subject: Re: NCEI Data Submission To: Zachary Mason - NOAA Affiliate Hi, Zachary, I was reviewing the file and the error in the fill values, but apparently I think it may be something due to the programs used to open the files, since the associated value I put when creating the files in the statistical program R is the same as I get when using other programs like QGIS (I attach a screenshot). So sorry for the delay I was trying to correct it but apparently it is not easy and probably will continue to occur if different programs are used to open the files, I think that if it is necessary to include a legend in the metadata that says that the fill values can be interpreted as underscores or NA in some programs. The other aspect linked to the name of the files has already been modified, these are found as attachments. Thank you very much for your support and again sorry for the delay. Greetings Israel On Wed, Oct 9, 2019 at 12:42 PM Israel Muñiz wrote: Hi Zachary, thank you so much for your response and review. The aspect related to the filling values in NaN zones was an error, I can modify the NetCDF metadata files to specify that the NaN is = underscores. Thanks for the observation. The other aspects are relatively simple so I hope to be able to send you the files before Friday of this week. Greetings and thanks for the support in the revision, hopefully soon these data will be available. Israel On Fri, Oct 4, 2019 at 8:33 AM Zachary Mason - NOAA Affiliate wrote: Hi Israel, I've completed the evaluation of all of your data files. I think that everything is just about ready to go. I have a couple of final questions but nothing so major as to derail the archival process. It looks like you have specified a fill value for use in your files but it looks like you are only using underscores and not this fill value. Is there a reason for this? While this is not a serious usability problem it would be better to be consistent. If editing these files is too much of an issue at this point we can just put a note somewhere in the metadata so that the user knows to be aware of this. Also, would it be possible to edit the variable names in the DHW frequency files to not include the "=" character? Maybe the word "equals" could be a good substitute? Thanks again for doing all of this follow-up work. And I apologize for the late replies. I spent about half of September out on work-related travel and I'm just now catching up on everything.  Best, On Wed, Sep 11, 2019 at 10:11 AM Zachary Mason - NOAA Affiliate wrote: Israel, Thanks for taking a second look at this. I will review these files and reach out if I have any further questions. Sorry for the delay, I've been out of the office on travel. Best, On Fri, Sep 6, 2019 at 1:37 PM Israel Muñiz wrote: Hello Zachary, I hope everything goes great, first of all, an apology for the delay in sending the data. Then I send you the final files, these I have to send them by google drive as they are too heavy. I also send you the answer to the point-by-point. Greetings and thank you very much for all the support Israel Point-by-point response to comments   I have been assigned to your recent data submission to the archive at NCEI. I've looked through the files that you sent over and I have a few questions/comments that need to be resolved before we can begin archiving the data.   We thank the constructive criticism and special attention during the review of the data files.   Although I'm having trouble accessing the actual error reports right now due to some technical issues, here are a few of my notes that should get you started on adjusting these files to comply with NCEI guidelines.   - All spaces in filenames should be replaced with underscores   This is ready.   - If applicable, all variables should have a standard_name attribute   This is ready.   - Variables should not have spaces or slashes in their names   This is ready. - Geospatial bounding box attributes should be added (longitude/latitude min and max)   Thanks for the observation, these characteristics already came in the description of the longitude and latitude variables, but we have added this information also in the global attributes.   - All reference documents should be listed under the references section and not the summary or variable description section   Thank you for the observation, these modifications I cannot make since I do not have access to the edition of the information uploaded on the platform, I ask you please that you made this change or allow me to edit this information, in order not to submit another package of data.   - File ID, date created, date modified are all attributes that should be added. This is ready, we have added this information in the global attributes.   - Time coverage attribute should be added This is ready, we have added this information in the time and global attributes.   - Time units should conform to UDUNITS standards   This is ready, we have added this information in the global attributes.   - Variable descriptions should be added to each variable. I see most (if not all) descriptions are found in a separate Word document, but it would be great if they could be added to the NetCDF files as well.   This is ready, we have added this information in the global attributes.   I'm not an expert on this type of data so this might be a dumb question but how can each value listed be matched with the pixel that it is supposed to represent? Should there be information on this somewhere? This question applies to any variable with a "per pixel" measurement.   I honestly don’t understand your question regarding the listed values and the pixels they represent. If you are referring to the geographic coordinates of each pixel and its corresponding value of the different variables listed, this is correctly linked in the NetCDF file, so opening the data as if they were a raster is possible to visualize and perform a spatial analysis of the values in each of the corresponding pixels. If you refer to the theoretical frame behind each of the variables, the calculation of each of the values is based on different bibliographic references cited in the package submitted in the NCEI, but this bibliography was not included in the NetCDF file.    Here is the link of the published manuscript corresponding to the generation and analysis of these data; I hope it can be of help. https://www.nature.com/articles/s41598-019-47307-0      Finally, the Daily ecoregional statistics of DHW.csv file could use a data dictionary. I'm not entirely sure that I understand what is being described in this file. The data dictionary can be a spreadsheet or text file that lists and describes each column header. Additionally, any codes or abbreviations should be defined in this document.   This is ready, we have added a data dictionary file that corresponds to the csv files.  Annual_maximum_Degree_Heating_Weeks_value_from_...Error! Filename not specified.  Daily_ecoregional_statistics_of_DHW.csvError! Filename not specified.  Data dictionary.docxError! Filename not specified.  Frequency_of_annual_maximum_Degree_Heating_Week...Error! Filename not specified.  Frequency_of_annual_maximum_Degree_Heating_Week...Error! Filename not specified.  Heat-stress_Regions_(HSR).ncError! Filename not specified.  Maximum_Degree_Heating_Weeks_value_per_pixel_fr...Error! Filename not specified.  Maximum_DHW_value_per_pixel_from_1987_to_2017.ncError! Filename not specified.  Methods summary for Three decades of heat stres...Error! Filename not specified.  Reef_locations_mask.ncError! Filename not specified.  Trend_of_annual_maximum_Degree_Heating_Weeks.ncError! Filename not specified.  Year_in_which_the_maximum_Degree_Heating_Weeks_...Error! Filename not specified. On Wed, Aug 14, 2019 at 7:50 PM Israel Muñiz wrote: Thank you so much, Zachary.  I'll send you the files ASAP. Cheers and thanks Israel On Tue, Aug 13, 2019 at 7:23 AM Zachary Mason - NOAA Affiliate wrote: Israel, Sorry for the delay in getting back to you and thanks for the information. When you make edits to the file you will have to send me the new version via email. I can switch the files on my end so that you don't have to create a new submission and a different package. Let me know if you have any questions about any of the changes that should be made. Thanks again, On Tue, Aug 6, 2019 at 11:52 AM Israel Muñiz wrote: Hey, Zachary, I hope you’re having a good day. First, I thank you for the excellent review of the information submitted, this will definitely help users to understand it much better. I apologize since I could not check the mail because I was sampling in the field and did not have much access to the Internet. I am working on the observations and suggestions concerning the metadata of the NetCDF files, of which I have no doubts. However, in the necessary's case edits on the NCEI page, I can’t make them since apparently the submitted data packages can’t be edited, so I imagine I have to copy the data and submit a different package, right? I honestly don’t understand your question regarding the listed values and the pixels they represent. If you are referring to the geographic coordinates of each pixel and its corresponding value of the different variables listed, this is correctly linked in the NetCDF file, so opening the data as if they were a raster is possible to visualize and perform a spatial analysis of the values in each of the corresponding pixels. If you refer to the theoretical frame behind each of the variables, the calculation of each of the values is based on different bibliographic references cited in the package submitted in the NCEI, but this bibliography was not included in the NetCDF file.  Here is the link of the published manuscript corresponding to the generation and analysis of these data; I hope it can be of help. https://www.nature.com/articles/s41598-019-47307-0   Again, thank you very much for the observations. Cheers,  Israel ----------------------------------------------- Aarón Israel Muñiz-Castillo Doctoral Student  Laboratorio de Ecología de Ecosistemas Arrecifales Coralinos CINVESTAV Unidad Mérida On Wed, Jul 31, 2019 at 1:05 PM Zachary Mason - NOAA Affiliate wrote: Hello Aaron and Jesus, I have been assigned to your recent data submission to the archive at NCEI. I've looked through the files that you sent over and I have a few questions/comments that need to be resolved before we can begin archiving the data. Although I'm having trouble accessing the actual error reports right now due to some technical issues, here are a few of my notes that should get you started on adjusting these files to comply with NCEI guidelines. - All spaces in filenames should be replaced with underscores - If applicable, all variables should have a standard_name attribute - Variables should not have spaces or slashes in their names - Geospatial bounding box attributes should be added (longitude/latitude min and max) - Time coverage attribute should be added - Time units should conform to UDUNITS standards - All reference documents should be listed under the references section and not the summary or variable description section - File ID, date created, date modified are all attributes that should be added. - Variable descriptions should be added to each variable. I see most (if not all) descriptions are found in a separate Word document, but it would be great if they could be added to the NetCDF files as well. I'm not an expert on this type of data so this might be a dumb question but how can each value listed be matched with the pixel that it is supposed to represent? Should there be information on this somewhere? This question applies to any variable with a "per pixel" measurement. Finally, the Daily ecoregional statistics of DHW.csv file could use a data dictionary. I'm not entirely sure that I understand what is being described in this file. The data dictionary can be a spreadsheet or text file that lists and describes each column header. Additionally, any codes or abbreviations should be defined in this document. Please let me know if you have any questions about any of this. Best, --  Zachary Mason 301-713-4862 CoRIS Data and Information Management Specialist