LAST CHANGED : 2023/07/12 00:35:56 CRUISE NAME(S) : SH1800 CRUISE DATES : 2018/02/01 17:42:32 to 2018/02/02 19:09:36 SHIP NAME : unspecified PORTS : unspecified CHIEF SCIENTIST : unspecified DATABASE NAME : aship DATA FILES : sh2018_031_63451.raw to sh2018_032_64800.raw STATUS : to do done ------ ----------- averaged [ ] loaded [ ] NOTE: heading correction instrument exists NOTE: time-dependent heading corrections applied IN the ensembles (see cal/rotate/ens_hcorr.ang) check heading correction [ ] calibration [ ] edited [ ] re-check heading correction [ ] check editing [ ] figures [ ] INSTRUMENT : os75 ACQUISITION : PROGRAM : uhdas PROCESSING: : python LOGGING : PARAMETERS : BT : bottom track mode (on or off) SI : sampling interval or averaging period for ensemble (sec) NB : number of bins BL : bin length (m) TD : transducer depth (m) BK : blanking length (m) HO : heading offset applied by DAS (deg) HB : heading bias (deg) CRPH : compensation for roll-pitch-heading, 1:on, 0:off) yy/mm/dd hh:mm:ss BT SI NB BL TD BK HO HB CRPH 2018/02/01 17:42:32 off 300 90 8 5 8 42.50 0.00 0001 HEADING : PRIMARY : heading from gyro CORRECTION : heading correction from posmv NOTE: time-dependent heading corrections applied IN the ensembles (see cal/rotate/ens_hcorr.ang) POSITIONS : gps positions from posmv CALIBRATION : (check original processing parameters) (1) transducer alignment original transducer alignment: additional rotation 0 final transducer angle is: (original transducer angle) - (rotate_angle) (2) scale factor original scale factor 1 additional scale factor (none) (3) ADCP (dx=starboard, dy=fwd) meters from GPS original: xducer_dx xducer_dy correction 0 0 final offset ? ? final = original + corrections COMMENTS : gaps in heading correction? scattering layers? bubbles? underway bias? PROCESSOR : I.M. Persistent --- original processing parameters from dbinfo.txt ---------- <...> NOTE: Replace the text below with the final dbinfo.txt content <...> after you are done with all postprocessing. Change the <...> comment to say "final processing parameters" ## (determined from "sonar"): model = os ## (determined from "sonar"): frequency = 75 ## (determined from "sonar"): instname = os75 ## (determined from "sonar"): pingtype = nb badbeam None beam_order None beamangle 30 configtype python cruisename SH1800 datatype uhdas dbname aship ens_len 300 fixfile aship.gps hcorr_inst posmv pgmin 50 pingpref None proc_engine python ref_method refsm refuv_smoothwin 3 refuv_source nav sonar os75nb txy_file aship.gps xducer_dx 0 xducer_dy 0 yearbase 2018 <...> Please consider submitting your final processed data to JASADCP <...> (Joint Carchive for SHipblard ADCP) data so other people can benefit <...> from all the work that went into this. GoogleSearch for <...> "JASADCP shipboard ADCP" in your browser. <...> <...> NOTE: Markup description: <...> - Narration is preceeded by these characters: <...> <...> - Anything with those characters at the beginning of a line will be <...> stripped out by JASADCP before archiving. <...> ================================================== POST PROCESSING STEPS: ================================================== <...> Post processing commands start in the os75nb directory. <...> If the cruise dates are wrong in the header, adjust manually. <...> The time range of processed data can be found in os75nb/adcpdb/aship.tr <...> This template covers most common situations when processing UHDAS data in CODAS. <...> For more detail, the latest version of the CODAS manual can be found at: <...> https://currents.soest.hawaii.edu/docs/adcp_doc/codas_doc/index.html -------------------------- ### 1. Check visual oddities -------------------------- <...> Perform a visual first pass of the data looking for the following problems: <...> (a) good heading correction for all of the ADCP data (any gaps?) <...> (b) sharp transitions in velocities <...> (i) on station vs underway (transducer angle or scale factor) <...> (ii) vertical stripes when transitioning between underway and <...> stopped (an offset between ADCP and GPS is not accounted for) <...> (iii) biases when underway with poor PG (will need to apply uvship) <...> (c) gaps due to "free inertial" flag in Seapath (heading exists, <...> but positions are missing, so no ship speed) <...> (d) false bottoms near steep topography: symptom strong scattering <...> layer is mis-identified as the bottom, resulting in shorter <...> profiles that stop at the scattering layer, but data on both <...> sides is much deeper <...> <...> When to post-process vs reprocess (eg. with adcp_database_maker.py) <...> You should reprocess if <...> (a) heading correction: if there are gaps and there is another accurate <...> heading device, you could reprocess using that other device <...> (b) sharp transitions in velocities <...> (ii) vertical stripes when transitioning between underway and stopped <...> (likely an offset between ADCP and GPS is not accounted for) <...> (iii) biases when underway with poor PG (will need to apply uvship) <...> If the offset is greater than 2m in x or y AND if there are periods <...> when the ship is underway in heavy weather, then you should reprocess <...> ensuring that you use the final xducer_dx and xducer_dy. Then the <...> 'uvship' algorithm can be attempted. Otherwise do not use it <...> (c) there are gaps due to "free inertial" flag in Seapath. In this case, <...> pick another position device, and make sure you get its xducer_dx and <...> xducer_dy correct. You might have to process twice, once to estimate <...> the values, and a second time to use them. <...> (d) false bottoms near steep topography: If the Percent Good is BLUE below <...> the scattering layers, you can only get those parts back by reprocessing <...> and NEVER looking for the bottom. Then you have to edit out the bottom <...> using the threshold tool for Bottom Selector. If the PG is red or orange, <...> you can resurrect those profiles by using "reset editing". <...> <...> Otherwise you may postprocess the cruise. <...> <...> Heading correction: <...> Check that we have a good heading correction for all of the ADCP data. <...> <...> There should be no holes in the heading correction graphs <...> (cal/rotate/ens_hcorr*.png), where "valid corrections" are green <...> circles and "bad corrections" are red crosses. Gaps in the plots where <...> there are no symbols are okay, no data was collected there. <...> <...> If there are red crosses on the heading correction graphs <...> (cal/rotate/ens_hcorr_*.png) then they need to be patched in the cal/rotate dir <...> using patch_hcorr.py. <...> -------------------------- ### Run this: (to look for gaps in the cruise track) plot_nav.py nav/a*.gps ### Run this: (to look for gaps in the heading correction, examine the character of <...> the watertrack and bottom track calibration, etc figview.py ### Run this: (looking for missing heading correction values or missing positions) dataviewer.py <...> NOTE: IF there are holes in the heading correction plots, you need to <...> run "patch_hcorr.py" to interpolate across the gaps. Do this BEFOE doing any <...> further post-processing ### Run this: (to interpolate for missing heading correction fixes) cd cal/rotate patch_hcorr.py cd ../.. ### Run this: (to check watertrack calibration after patching hcorr) catwt.py ### Run this: (to check watertrack calibration after patching hcorr) catbt.py ### Run this: (to check dxdy statistics calibration after patching hcorr) catxy.py --------------------- ### 2. ADCP calibration --------------------- <...> Calibrate the dataset in sum by checking the calibration values <...> and applying a fix to the necessary categories if necessary. <...> <...> When looking at a water track or bottom track calibration, we want to see <...> statistics inside the following values: <...> <...> median_tolerance <...> <...> - amplitude range .997 to 1.003 # 0.3% <...> - phase/angle range -0.05 to 0.05 # 1/2 degree <...> - offset (dx,dy) rounded residual should be between -2 and 2 meters <...> <...> When looking at an xy calibration to adjust the relative location of the GPS <...> to the sonar we try to get the dx and dy values close to zero, bearing <...> in mind that this is a guess. The "signal" should be between 1000 and 5000. <...> Less than 1000 means very little data, more than 5000 means too much change. <...> <...> If values are out of range we make a bulk correction to the entire dataset <...> which should reduce the number of outliers to be edited out. <...> <...> If we make any changes to x or y they should be integers. <...> <...> The command to be run to edit the calibration is listed below, where * <...> is replaced with desired values for the options to be changed and <...> other options removed: <...> <...> quick_adcp.py --steps2rerun rotate:navsteps:calib --rotate_amplitude *.** --rotate_angle *.** --xducer_dx * --xducer_dy * --auto --------------------- ### Run this: (look at the watertrack calibration residuals remaining) catwt.py ### Run this: (look at the bottom track calibration residuals remaining) catbt.py ### Run this: (look at the remaining offsets required) catxy.py --- ### If a large dxdy correction is needed (values greater than 5), do this first: ### Run this: (replace '*' with your values (round to nearest integer)) quick_adcp.py --steps2rerun rotate:apply_edit:navsteps:calib --xducer_dx * --xducer_dy * --auto ### Run this: (look at the watertrack calibration again) catwt.py ### Run this: (look at the offset again) catxy.py ### If a scale factor or phase need to be applied, do that now (replace '*' with your values) ### Run this: quick_adcp.py --steps2rerun rotate:navsteps:calib --rotate_amplitude * --rotate_angle * --auto ### Run this: (look at the watertrack calibration again) catwt.py ### Run this: (look at the offset again) catxy.py <...> How do the calibrations look? See above for median tolerances and <...> repeat steps if the calibrations are still out of median tolerances. ------------------- ### 3. Editing points ------------------- <...> Edit out biased data or artifacts deeper than the range of the sonar <...> in the dataset, or other problems. We are looking for 1 cm/s accuracy <...> in ocean velocities, and any transitions larger than that could be suspect. <...> <...> Use `dataviewer.py -e` to look for problems with the data and <...> flag the bad data as bad. After editing out data we rerun the <...> calibration to see if the changed dataset statistics have <...> changed enough that a new calibration is needed. ------------------- ### Run this: (to go through the dataset and edit out bad values) dataviewer.py -e ### Run this: (to recompute the calibration residuals) quick_adcp.py --steps2rerun navsteps:calib --auto ### Run this: catwt.py ### Run this: catxy.py <...> The calibrations should still be good after editing. -------------------------------------------------- ### 4. Re-check heading correction and other figures -------------------------------------------------- <...> Check all figures from section 1 again to make sure that any problems <...> were addressed and no new problems have appeared after making changes <...> to the dataset. -------------------------------------------------- ### Run this: (Is there anything strange with any of the figures?) figview.py -------------------------------------------------------------- ### 5. Check edited, calibrated dataset against original dataset -------------------------------------------------------------- <...> Compare the edited, calibrated dataset against <...> the original dataset to make sure all problems have been dealt <...> with, and no new problems have appeared.There is likely to be strong <...> colors in the difference plot if big changes were made. <...> Compare sonars against the next nearest frequencies available <...> (Ex. os150 with wh300 and os75, not wh1200 and os38). <...> Compared datasets across different sonars should show a uniform <...> bias, Ex. all faint red or blue, equivalent to roughly 1 cm/s. <...> Look at the compared datasets on the default scale (0.8 days) and a <...> medium term scale (2 to 3 days) to look for longer scale bias. <...> When running dataviewer.py -c, put the finer resolution/smaller bin <...> sized data first to show the diff at the finer resolution. <...> NOTE that changes in bin size or number of bins will look like <...> a major problem. Just choose a different time range so the transition <...> is not in view and it will look better. -------------------------------------------------------------- ### Run this: (Has this corrected the problems in the original?) dataviewer.py -c . ../os75nb.orig ### Run this to compare this sonar with another sonar (after both are finished) dataviewer.py -c ../ANOTHER_SONAR . <...> Use the comparison to determine if anything else needs to be done. ------------------------- ### 6. Make plots and files ------------------------- <...> Create the figures and data files needed to finish processing <...> and make public/submit to a repository (JASADCP, if nowhere else). <...> Make the plots needed for web viewing, matlab files (legacy), <...> and netCDF files, then check that the netCDF files are readable. <...> Plots should be in 3 to 5 day chunks for a cruise longer than one week, <...> or divided by geographic features where it makes sense. <...> <...> If quick_web.py --interactive has been done for a different sonar, do: <...> Run this: (edit for your particular case) <...> mkdir webpy <...> cp ../ANOTHER_SONAR /webpy/sectinfo.txt webpy <...> quick_web.py --redo ------------------------- <...> If it's the first time: ### Run this: quick_web.py --interactive ### Run this to extract matlab files quick_adcp.py --steps2rerun matfiles --auto ### Run this to extract a netCDF file adcp_nc.py adcpdb contour/os75nb CRUISENAME os75nb --ship_name SHIPNAME ### Run this to look at the headers ncdump contour/os75nb.nc -h