2016-04-19 13:25:39,408 INFO root Starting DAS_while_logging.py 2016-04-19 13:25:39,431 INFO root Cruiseid: SP1605 2016-04-19 13:25:39,488 INFO root starting run method 2016-04-19 13:25:39,489 INFO Translate gpsnav gps 1 2016-04-19 13:25:39,489 INFO Translate 2016-04-19 13:25:39,489 INFO Translate gpshead hdg 1 2016-04-19 13:25:39,489 INFO Translate 2016-04-19 13:25:39,489 INFO Translate ashtech adu 1 2016-04-19 13:25:39,489 INFO Translate 2016-04-19 13:25:39,493 INFO root active_procdirnames: ['nb300'] 2016-04-19 13:25:39,493 INFO repeater PeriodicSystem: nb300_lastensq interval is 130 seconds cd /home/adcp/uhdas_tmp; run_lastensq.py -d nb300 --update_pygbin --averages --plotens_mpl --kt_vecprof --pingwarnN 30 --ktsdir 2016-04-19 13:25:39,493 INFO repeater PeriodicSystem: nb300_10pings interval is 140 seconds cd /home/adcp/uhdas_tmp; run_grab_lastpings.py -d nb300 2016-04-19 13:25:39,493 INFO repeater PeriodicSystem: nb300_quick interval is 900 seconds cd /home/adcp/uhdas_tmp; run_quick.py -d nb300 2016-04-19 13:25:39,493 INFO repeater PeriodicSystem: nb300_3dayplots interval is 900 seconds cd /home/adcp/uhdas_tmp; run_3dayplots_mpl.py -d nb300 2016-04-19 13:25:39,493 INFO repeater PeriodicSystem: nb300_tsstats interval is 900 seconds cd /home/adcp/uhdas_tmp; run_plot_tsstats.py nb300 2016-04-19 13:25:39,493 INFO repeater PeriodicSystem: nb300_beamplots interval is 900 seconds cd /home/adcp/uhdas_tmp; plot_beamdiagnostics.py -d nb300 --savefigs --webcopy -p scattering 2016-04-19 13:25:39,493 INFO repeater PeriodicSystem: nb300_beamplots interval is 900 seconds cd /home/adcp/uhdas_tmp; plot_beamdiagnostics.py -d nb300 --savefigs --webcopy -p velocity 2016-04-19 13:25:39,494 INFO uhdas_webgen generating /home/adcp/www/figures/thumbnails.html 2016-04-19 13:25:39,494 INFO uhdas_webgen generating /home/adcp/www/figures/beam_diagnostics.html 2016-04-19 13:25:39,495 INFO uhdas_webgen ptype=scattering 2016-04-19 13:25:39,495 INFO uhdas_webgen title=UNCALIBRATED scattering, dB 2016-04-19 13:25:39,495 INFO uhdas_webgen inst=nb300 2016-04-19 13:25:39,495 INFO uhdas_webgen ptype=velocity 2016-04-19 13:25:39,495 INFO uhdas_webgen title=velocity, m/s 2016-04-19 13:25:39,495 INFO uhdas_webgen inst=nb300 2016-04-19 13:25:39,496 INFO uhdas_webgen generating /home/adcp/www/figures/tsstats.html 2016-04-19 13:25:39,496 INFO uhdas_webgen ptype=tsstats 2016-04-19 13:25:39,496 INFO uhdas_webgen title=(some) ocean velocity errors 2016-04-19 13:25:39,496 INFO uhdas_webgen inst=nb300 2016-04-19 13:25:39,496 INFO uhdas_webgen generating /home/adcp/www/figures/sci.html 2016-04-19 13:25:39,496 INFO uhdas_webgen ptype=ktvecprof 2016-04-19 13:25:39,496 INFO uhdas_webgen title=speed+direction profile 2016-04-19 13:25:39,496 INFO uhdas_webgen inst=nb300 2016-04-19 13:25:39,496 INFO uhdas_webgen adding ashtech to simplefiglinks 2016-04-19 13:25:39,496 INFO uhdas_webgen generating /home/adcp/www/simple_figlinks.html 2016-04-19 13:25:39,497 INFO repeater PeriodicSystem: ashtech_hcorrstats interval is 300 seconds cd /home/adcp/uhdas_tmp; run_hbinhcorrstats.py --hcorr_inst ashtech --plotdh --printstats 2016-04-19 13:25:39,497 INFO root cleanup_figs no_figure 2016-04-19 13:27:49,680 INFO repeater task nb300_lastensq starting 0.2 seconds late 2016-04-19 13:27:52,184 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 13:27:52,191 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 13:27:39 2016-04-19 13:28:02,205 INFO repeater task nb300_10pings starting 2.7 seconds late 2016-04-19 13:28:02,385 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 13:29:52,543 INFO repeater task nb300_lastensq starting 3.5 seconds late 2016-04-19 13:29:55,018 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 13:29:55,018 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 13:29:39 2016-04-19 13:30:20,056 INFO repeater task nb300_10pings starting 0.6 seconds late 2016-04-19 13:30:20,247 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 13:30:40,268 INFO repeater task ashtech_hcorrstats starting 0.8 seconds late 2016-04-19 13:30:41,035 INFO repeater task ashtech_hcorrstats output: could not get ashtech stats Traceback (most recent call last): File "/usr/local/currents/bin/run_hbinhcorrstats.py", line 148, in H.hcorr_stats(winsecs=winsecs) File "/usr/local/lib/python2.7/dist-packages/pycurrents/adcp/hbin_diagnostics.py", line 309, in hcorr_stats self.enspercent = np.round(100.0 * self.ensnumgood/self.ensnumtotal) ZeroDivisionError: float division by zero 2016-04-19 13:30:41,035 INFO repeater task ashtech_hcorrstats ended after 0.8 seconds 2016-04-19 13:31:51,138 INFO repeater task nb300_lastensq starting 2.1 seconds late 2016-04-19 13:31:53,617 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 13:31:53,617 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 13:31:39 2016-04-19 13:32:43,689 INFO repeater task nb300_10pings starting 4.2 seconds late 2016-04-19 13:32:43,867 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 13:33:53,967 INFO repeater task nb300_lastensq starting 5.0 seconds late 2016-04-19 13:33:56,506 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 13:33:56,506 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 13:33:39 2016-04-19 13:35:01,600 INFO repeater task nb300_10pings starting 2.1 seconds late 2016-04-19 13:35:01,781 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 13:35:41,842 INFO repeater task ashtech_hcorrstats starting 2.3 seconds late 2016-04-19 13:35:42,595 INFO repeater task ashtech_hcorrstats ended after 0.8 seconds 2016-04-19 13:35:52,608 INFO repeater task nb300_lastensq starting 3.6 seconds late 2016-04-19 13:35:55,098 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 13:35:55,098 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 13:35:39 2016-04-19 13:37:20,219 INFO repeater task nb300_10pings starting 0.7 seconds late 2016-04-19 13:37:20,394 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 13:37:50,435 INFO repeater task nb300_lastensq starting 1.4 seconds late 2016-04-19 13:37:52,967 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 13:37:52,967 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 13:37:39 2016-04-19 13:39:43,113 INFO repeater task nb300_10pings starting 3.6 seconds late 2016-04-19 13:39:43,292 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 13:39:53,307 INFO repeater task nb300_lastensq starting 4.3 seconds late 2016-04-19 13:39:55,781 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 13:39:55,781 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 13:39:39 2016-04-19 13:40:40,848 INFO repeater task nb300_quick starting 1.4 seconds late 2016-04-19 13:40:43,618 INFO repeater task nb300_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2016-04-19 13:40:43,619 INFO repeater task nb300_quick ended after 2.8 seconds 2016-04-19 13:40:43,619 INFO repeater task nb300_3dayplots starting 4.1 seconds late 2016-04-19 13:40:46,587 INFO repeater task nb300_3dayplots ended after 3.0 seconds 2016-04-19 13:40:46,587 INFO repeater task nb300_tsstats starting 7.1 seconds late 2016-04-19 13:40:47,534 INFO repeater task nb300_tsstats ended after 0.9 seconds 2016-04-19 13:40:47,534 INFO repeater task nb300_beamplots starting 8.0 seconds late 2016-04-19 13:40:49,012 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 13:40:49,012 INFO repeater task nb300_beamplots ended after 1.5 seconds 2016-04-19 13:40:49,012 INFO repeater task nb300_beamplots starting 9.5 seconds late 2016-04-19 13:40:50,500 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 13:40:50,501 INFO repeater task nb300_beamplots ended after 1.5 seconds 2016-04-19 13:40:50,501 INFO repeater task ashtech_hcorrstats starting 11.0 seconds late 2016-04-19 13:40:51,224 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2016-04-19 13:41:51,303 INFO repeater task nb300_lastensq starting 2.3 seconds late 2016-04-19 13:41:53,838 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 13:41:53,839 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 13:41:39 2016-04-19 13:42:03,854 INFO repeater task nb300_10pings starting 4.4 seconds late 2016-04-19 13:42:04,033 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 13:43:49,179 INFO repeater task nb300_lastensq starting 0.2 seconds late 2016-04-19 13:43:51,661 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 13:43:51,662 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 13:43:39 2016-04-19 13:44:21,704 INFO repeater task nb300_10pings starting 2.2 seconds late 2016-04-19 13:44:21,883 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 13:45:42,003 INFO repeater task ashtech_hcorrstats starting 2.5 seconds late 2016-04-19 13:45:42,732 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2016-04-19 13:45:52,745 INFO repeater task nb300_lastensq starting 3.7 seconds late 2016-04-19 13:45:55,288 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 13:45:55,289 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 13:45:39 2016-04-19 13:46:40,349 INFO repeater task nb300_10pings starting 0.9 seconds late 2016-04-19 13:46:40,529 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 13:47:50,625 INFO repeater task nb300_lastensq starting 1.6 seconds late 2016-04-19 13:47:53,099 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 13:47:53,100 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 13:47:39 2016-04-19 13:49:03,183 INFO repeater task nb300_10pings starting 3.7 seconds late 2016-04-19 13:49:03,362 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 13:49:53,437 INFO repeater task nb300_lastensq starting 4.4 seconds late 2016-04-19 13:49:55,973 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 13:49:55,974 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 13:49:39 2016-04-19 13:50:41,043 INFO repeater task ashtech_hcorrstats starting 1.5 seconds late 2016-04-19 13:50:41,792 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2016-04-19 13:51:21,843 INFO repeater task nb300_10pings starting 2.3 seconds late 2016-04-19 13:51:22,025 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 13:51:52,070 INFO repeater task nb300_lastensq starting 3.1 seconds late 2016-04-19 13:51:54,544 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 13:51:54,544 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 13:51:39 2016-04-19 13:53:39,688 INFO repeater task nb300_10pings starting 0.2 seconds late 2016-04-19 13:53:39,870 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 13:53:49,882 INFO repeater task nb300_lastensq starting 0.9 seconds late 2016-04-19 13:53:52,440 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 13:53:52,440 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 13:53:39 2016-04-19 13:55:42,603 INFO repeater task nb300_quick starting 3.1 seconds late 2016-04-19 13:55:45,201 INFO repeater task nb300_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2016-04-19 13:55:45,202 INFO repeater task nb300_quick ended after 2.6 seconds 2016-04-19 13:55:45,202 INFO repeater task nb300_3dayplots starting 5.7 seconds late 2016-04-19 13:55:48,235 INFO repeater task nb300_3dayplots ended after 3.0 seconds 2016-04-19 13:55:48,236 INFO repeater task nb300_tsstats starting 8.7 seconds late 2016-04-19 13:55:49,061 INFO repeater task nb300_tsstats ended after 0.8 seconds 2016-04-19 13:55:49,061 INFO repeater task nb300_beamplots starting 9.6 seconds late 2016-04-19 13:55:50,619 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 13:55:50,619 INFO repeater task nb300_beamplots ended after 1.6 seconds 2016-04-19 13:55:50,619 INFO repeater task nb300_beamplots starting 11.1 seconds late 2016-04-19 13:55:52,229 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 13:55:52,229 INFO repeater task nb300_beamplots ended after 1.6 seconds 2016-04-19 13:55:52,230 INFO repeater task ashtech_hcorrstats starting 12.7 seconds late 2016-04-19 13:55:52,980 INFO repeater task ashtech_hcorrstats ended after 0.8 seconds 2016-04-19 13:55:57,986 INFO repeater task nb300_lastensq starting 9.0 seconds late 2016-04-19 13:56:00,480 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 13:56:00,481 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 13:55:39 2016-04-19 13:56:00,481 INFO repeater task nb300_10pings starting 1.0 seconds late 2016-04-19 13:56:00,656 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 13:57:50,791 INFO repeater task nb300_lastensq starting 1.8 seconds late 2016-04-19 13:57:53,325 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 13:57:53,326 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 13:57:39 2016-04-19 13:58:23,370 INFO repeater task nb300_10pings starting 3.9 seconds late 2016-04-19 13:58:23,550 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 13:59:53,680 INFO repeater task nb300_lastensq starting 4.7 seconds late 2016-04-19 13:59:56,168 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 13:59:56,168 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 13:59:39 2016-04-19 14:00:41,230 INFO repeater task nb300_10pings starting 1.7 seconds late 2016-04-19 14:00:41,407 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 14:00:41,408 INFO repeater task ashtech_hcorrstats starting 1.9 seconds late 2016-04-19 14:00:42,188 INFO repeater task ashtech_hcorrstats ended after 0.8 seconds 2016-04-19 14:01:52,262 INFO repeater task nb300_lastensq starting 3.3 seconds late 2016-04-19 14:01:54,854 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 14:01:54,854 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 14:01:39 2016-04-19 14:02:59,948 INFO repeater task nb300_10pings starting 0.5 seconds late 2016-04-19 14:03:00,132 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 14:03:50,207 INFO repeater task nb300_lastensq starting 1.2 seconds late 2016-04-19 14:03:52,699 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 14:03:52,700 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 14:03:39 2016-04-19 14:05:22,815 INFO repeater task nb300_10pings starting 3.3 seconds late 2016-04-19 14:05:22,997 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 14:05:43,026 INFO repeater task ashtech_hcorrstats starting 3.5 seconds late 2016-04-19 14:05:43,855 INFO repeater task ashtech_hcorrstats ended after 0.8 seconds 2016-04-19 14:05:53,867 INFO repeater task nb300_lastensq starting 4.9 seconds late 2016-04-19 14:05:56,359 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 14:05:56,360 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 14:05:39 2016-04-19 14:07:41,514 INFO repeater task nb300_10pings starting 2.0 seconds late 2016-04-19 14:07:41,693 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 14:07:51,703 INFO repeater task nb300_lastensq starting 2.7 seconds late 2016-04-19 14:07:54,245 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 14:07:54,245 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 14:07:39 2016-04-19 14:09:49,390 INFO repeater task nb300_lastensq starting 0.4 seconds late 2016-04-19 14:09:51,919 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 14:09:51,920 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 14:09:39 2016-04-19 14:10:01,934 INFO repeater task nb300_10pings starting 2.4 seconds late 2016-04-19 14:10:02,116 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 14:10:42,175 INFO repeater task nb300_quick starting 2.7 seconds late 2016-04-19 14:10:44,854 INFO repeater task nb300_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2016-04-19 14:10:44,854 INFO repeater task nb300_quick ended after 2.7 seconds 2016-04-19 14:10:44,855 INFO repeater task nb300_3dayplots starting 5.4 seconds late 2016-04-19 14:10:47,872 INFO repeater task nb300_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2016-04-19 14:10:47,872 INFO repeater task nb300_3dayplots ended after 3.0 seconds 2016-04-19 14:10:47,873 INFO repeater task nb300_tsstats starting 8.4 seconds late 2016-04-19 14:10:48,741 INFO repeater task nb300_tsstats ended after 0.9 seconds 2016-04-19 14:10:48,742 INFO repeater task nb300_beamplots starting 9.2 seconds late 2016-04-19 14:10:50,433 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 14:10:50,433 INFO repeater task nb300_beamplots ended after 1.7 seconds 2016-04-19 14:10:50,433 INFO repeater task nb300_beamplots starting 10.9 seconds late 2016-04-19 14:10:52,078 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 14:10:52,078 INFO repeater task nb300_beamplots ended after 1.6 seconds 2016-04-19 14:10:52,079 INFO repeater task ashtech_hcorrstats starting 12.6 seconds late 2016-04-19 14:10:52,841 INFO repeater task ashtech_hcorrstats ended after 0.8 seconds 2016-04-19 14:11:52,918 INFO repeater task nb300_lastensq starting 3.9 seconds late 2016-04-19 14:11:55,479 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 14:11:55,480 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 14:11:39 2016-04-19 14:12:20,517 INFO repeater task nb300_10pings starting 1.0 seconds late 2016-04-19 14:12:20,694 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 14:13:50,800 INFO repeater task nb300_lastensq starting 1.8 seconds late 2016-04-19 14:13:53,282 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 14:13:53,283 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 14:13:39 2016-04-19 14:14:43,354 INFO repeater task nb300_10pings starting 3.9 seconds late 2016-04-19 14:14:43,534 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 14:15:43,624 INFO repeater task ashtech_hcorrstats starting 4.1 seconds late 2016-04-19 14:15:44,387 INFO repeater task ashtech_hcorrstats ended after 0.8 seconds 2016-04-19 14:15:49,393 INFO repeater task nb300_lastensq starting 0.4 seconds late 2016-04-19 14:15:51,934 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 14:15:51,935 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 14:15:39 2016-04-19 14:17:02,021 INFO repeater task nb300_10pings starting 2.5 seconds late 2016-04-19 14:17:02,203 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 14:17:52,271 INFO repeater task nb300_lastensq starting 3.3 seconds late 2016-04-19 14:17:54,773 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 14:17:54,773 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 14:17:39 2016-04-19 14:19:19,880 INFO repeater task nb300_10pings starting 0.4 seconds late 2016-04-19 14:19:20,060 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 14:19:50,105 INFO repeater task nb300_lastensq starting 1.1 seconds late 2016-04-19 14:19:52,659 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 14:19:52,659 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 14:19:39 2016-04-19 14:20:42,727 INFO repeater task ashtech_hcorrstats starting 3.2 seconds late 2016-04-19 14:20:43,500 INFO repeater task ashtech_hcorrstats ended after 0.8 seconds 2016-04-19 14:21:43,584 INFO repeater task nb300_10pings starting 4.1 seconds late 2016-04-19 14:21:43,762 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 14:21:53,775 INFO repeater task nb300_lastensq starting 4.8 seconds late 2016-04-19 14:21:56,272 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 14:21:56,272 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 14:21:39 2016-04-19 14:23:51,434 INFO repeater task nb300_lastensq starting 2.4 seconds late 2016-04-19 14:23:53,971 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 14:23:53,972 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 14:23:39 2016-04-19 14:24:03,986 INFO repeater task nb300_10pings starting 4.5 seconds late 2016-04-19 14:24:04,166 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 14:25:44,311 INFO repeater task nb300_quick starting 4.8 seconds late 2016-04-19 14:25:46,953 INFO repeater task nb300_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2016-04-19 14:25:46,953 INFO repeater task nb300_quick ended after 2.6 seconds 2016-04-19 14:25:46,953 INFO repeater task nb300_3dayplots starting 7.5 seconds late 2016-04-19 14:25:49,993 INFO repeater task nb300_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2016-04-19 14:25:49,993 INFO repeater task nb300_3dayplots ended after 3.0 seconds 2016-04-19 14:25:49,994 INFO repeater task nb300_tsstats starting 10.5 seconds late 2016-04-19 14:25:50,870 INFO repeater task nb300_tsstats ended after 0.9 seconds 2016-04-19 14:25:50,871 INFO repeater task nb300_beamplots starting 11.4 seconds late 2016-04-19 14:25:52,600 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 14:25:52,600 INFO repeater task nb300_beamplots ended after 1.7 seconds 2016-04-19 14:25:52,600 INFO repeater task nb300_beamplots starting 13.1 seconds late 2016-04-19 14:25:54,415 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 14:25:54,416 INFO repeater task nb300_beamplots ended after 1.8 seconds 2016-04-19 14:25:54,416 INFO repeater task ashtech_hcorrstats starting 14.9 seconds late 2016-04-19 14:25:55,186 INFO repeater task ashtech_hcorrstats ended after 0.8 seconds 2016-04-19 14:26:00,189 INFO repeater task nb300_lastensq starting 11.2 seconds late 2016-04-19 14:26:02,692 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 14:26:02,693 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 14:25:39 2016-04-19 14:26:22,716 INFO repeater task nb300_10pings starting 3.2 seconds late 2016-04-19 14:26:22,902 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 14:27:53,028 INFO repeater task nb300_lastensq starting 4.0 seconds late 2016-04-19 14:27:55,572 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 14:27:55,573 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 14:27:39 2016-04-19 14:28:40,627 INFO repeater task nb300_10pings starting 1.1 seconds late 2016-04-19 14:28:40,805 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 14:29:50,902 INFO repeater task nb300_lastensq starting 1.9 seconds late 2016-04-19 14:29:53,398 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 14:29:53,398 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 14:29:39 2016-04-19 14:30:43,470 INFO repeater task ashtech_hcorrstats starting 4.0 seconds late 2016-04-19 14:30:44,271 INFO repeater task ashtech_hcorrstats ended after 0.8 seconds 2016-04-19 14:31:04,292 INFO repeater task nb300_10pings starting 4.8 seconds late 2016-04-19 14:31:04,470 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 14:31:49,537 INFO repeater task nb300_lastensq starting 0.5 seconds late 2016-04-19 14:31:52,087 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 14:31:52,088 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 14:31:39 2016-04-19 14:33:22,220 INFO repeater task nb300_10pings starting 2.7 seconds late 2016-04-19 14:33:22,400 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 14:33:52,443 INFO repeater task nb300_lastensq starting 3.4 seconds late 2016-04-19 14:33:54,939 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 14:33:54,939 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 14:33:39 2016-04-19 14:35:40,083 INFO repeater task nb300_10pings starting 0.6 seconds late 2016-04-19 14:35:40,266 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 14:35:40,267 INFO repeater task ashtech_hcorrstats starting 0.8 seconds late 2016-04-19 14:35:41,073 INFO repeater task ashtech_hcorrstats ended after 0.8 seconds 2016-04-19 14:35:51,083 INFO repeater task nb300_lastensq starting 2.1 seconds late 2016-04-19 14:35:53,624 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 14:35:53,625 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 14:35:39 2016-04-19 14:37:53,763 INFO repeater task nb300_lastensq starting 4.8 seconds late 2016-04-19 14:37:56,263 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 14:37:56,264 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 14:37:39 2016-04-19 14:38:01,270 INFO repeater task nb300_10pings starting 1.8 seconds late 2016-04-19 14:38:01,447 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 14:39:51,599 INFO repeater task nb300_lastensq starting 2.6 seconds late 2016-04-19 14:39:54,164 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 14:39:54,165 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 14:39:39 2016-04-19 14:40:24,204 INFO repeater task nb300_10pings starting 4.7 seconds late 2016-04-19 14:40:24,380 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 14:40:44,406 INFO repeater task nb300_quick starting 4.9 seconds late 2016-04-19 14:40:47,027 INFO repeater task nb300_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2016-04-19 14:40:47,027 INFO repeater task nb300_quick ended after 2.6 seconds 2016-04-19 14:40:47,028 INFO repeater task nb300_3dayplots starting 7.5 seconds late 2016-04-19 14:40:50,167 INFO repeater task nb300_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2016-04-19 14:40:50,168 INFO repeater task nb300_3dayplots ended after 3.1 seconds 2016-04-19 14:40:50,168 INFO repeater task nb300_tsstats starting 10.7 seconds late 2016-04-19 14:40:51,024 INFO repeater task nb300_tsstats ended after 0.9 seconds 2016-04-19 14:40:51,024 INFO repeater task nb300_beamplots starting 11.5 seconds late 2016-04-19 14:40:52,881 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 14:40:52,882 INFO repeater task nb300_beamplots ended after 1.9 seconds 2016-04-19 14:40:52,882 INFO repeater task nb300_beamplots starting 13.4 seconds late 2016-04-19 14:40:54,723 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 14:40:54,723 INFO repeater task nb300_beamplots ended after 1.8 seconds 2016-04-19 14:40:54,723 INFO repeater task ashtech_hcorrstats starting 15.2 seconds late 2016-04-19 14:40:55,525 INFO repeater task ashtech_hcorrstats ended after 0.8 seconds 2016-04-19 14:41:50,605 INFO repeater task nb300_lastensq starting 1.6 seconds late 2016-04-19 14:41:53,155 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 14:41:53,156 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 14:41:39 2016-04-19 14:42:43,224 INFO repeater task nb300_10pings starting 3.7 seconds late 2016-04-19 14:42:43,404 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 14:42:54,523 INFO Translate translate exiting, adu /home/data/SP1605/rbin/ashtech 2016-04-19 14:42:54,541 INFO Translate translate exiting, hdg /home/data/SP1605/rbin/gpshead 2016-04-19 14:42:54,555 INFO Translate translate exiting, gps /home/data/SP1605/rbin/gpsnav 2016-04-19 14:42:58,425 INFO root cleanup_figs not_logging 2016-04-19 14:42:58,435 INFO root Exiting DAS_while_logging.py 2016-04-19 15:59:52,613 INFO root Starting DAS_while_logging.py 2016-04-19 15:59:52,633 INFO root Cruiseid: SP1605 2016-04-19 15:59:52,690 INFO root starting run method 2016-04-19 15:59:52,690 INFO Translate gpsnav gps 1 2016-04-19 15:59:52,690 INFO Translate 2016-04-19 15:59:52,691 INFO Translate gpshead hdg 1 2016-04-19 15:59:52,691 INFO Translate 2016-04-19 15:59:52,691 INFO Translate ashtech adu 1 2016-04-19 15:59:52,691 INFO Translate 2016-04-19 15:59:52,700 INFO root active_procdirnames: ['nb300'] 2016-04-19 15:59:52,701 INFO repeater PeriodicSystem: nb300_lastensq interval is 130 seconds cd /home/adcp/uhdas_tmp; run_lastensq.py -d nb300 --update_pygbin --averages --plotens_mpl --kt_vecprof --pingwarnN 30 --ktsdir 2016-04-19 15:59:52,701 INFO repeater PeriodicSystem: nb300_10pings interval is 140 seconds cd /home/adcp/uhdas_tmp; run_grab_lastpings.py -d nb300 2016-04-19 15:59:52,701 INFO repeater PeriodicSystem: nb300_quick interval is 900 seconds cd /home/adcp/uhdas_tmp; run_quick.py -d nb300 2016-04-19 15:59:52,701 INFO repeater PeriodicSystem: nb300_3dayplots interval is 900 seconds cd /home/adcp/uhdas_tmp; run_3dayplots_mpl.py -d nb300 2016-04-19 15:59:52,701 INFO repeater PeriodicSystem: nb300_tsstats interval is 900 seconds cd /home/adcp/uhdas_tmp; run_plot_tsstats.py nb300 2016-04-19 15:59:52,701 INFO repeater PeriodicSystem: nb300_beamplots interval is 900 seconds cd /home/adcp/uhdas_tmp; plot_beamdiagnostics.py -d nb300 --savefigs --webcopy -p scattering 2016-04-19 15:59:52,701 INFO repeater PeriodicSystem: nb300_beamplots interval is 900 seconds cd /home/adcp/uhdas_tmp; plot_beamdiagnostics.py -d nb300 --savefigs --webcopy -p velocity 2016-04-19 15:59:52,701 INFO uhdas_webgen generating /home/adcp/www/figures/thumbnails.html 2016-04-19 15:59:52,701 INFO uhdas_webgen generating /home/adcp/www/figures/beam_diagnostics.html 2016-04-19 15:59:52,702 INFO uhdas_webgen ptype=scattering 2016-04-19 15:59:52,702 INFO uhdas_webgen title=UNCALIBRATED scattering, dB 2016-04-19 15:59:52,702 INFO uhdas_webgen inst=nb300 2016-04-19 15:59:52,702 INFO uhdas_webgen ptype=velocity 2016-04-19 15:59:52,702 INFO uhdas_webgen title=velocity, m/s 2016-04-19 15:59:52,702 INFO uhdas_webgen inst=nb300 2016-04-19 15:59:52,702 INFO uhdas_webgen generating /home/adcp/www/figures/tsstats.html 2016-04-19 15:59:52,702 INFO uhdas_webgen ptype=tsstats 2016-04-19 15:59:52,702 INFO uhdas_webgen title=(some) ocean velocity errors 2016-04-19 15:59:52,702 INFO uhdas_webgen inst=nb300 2016-04-19 15:59:52,703 INFO uhdas_webgen generating /home/adcp/www/figures/sci.html 2016-04-19 15:59:52,703 INFO uhdas_webgen ptype=ktvecprof 2016-04-19 15:59:52,703 INFO uhdas_webgen title=speed+direction profile 2016-04-19 15:59:52,703 INFO uhdas_webgen inst=nb300 2016-04-19 15:59:52,703 INFO uhdas_webgen adding ashtech to simplefiglinks 2016-04-19 15:59:52,703 INFO uhdas_webgen generating /home/adcp/www/simple_figlinks.html 2016-04-19 15:59:52,703 INFO repeater PeriodicSystem: ashtech_hcorrstats interval is 300 seconds cd /home/adcp/uhdas_tmp; run_hbinhcorrstats.py --hcorr_inst ashtech --plotdh --printstats 2016-04-19 15:59:52,703 INFO root cleanup_figs no_figure 2016-04-19 16:02:02,869 INFO repeater task nb300_lastensq starting 0.2 seconds late 2016-04-19 16:02:05,496 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 16:02:05,503 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:01:52 2016-04-19 16:02:15,514 INFO repeater task nb300_10pings starting 2.8 seconds late 2016-04-19 16:02:15,699 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 16:04:05,835 INFO repeater task nb300_lastensq starting 3.8 seconds late 2016-04-19 16:04:08,310 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:04:08,310 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:03:52 2016-04-19 16:04:33,344 INFO repeater task nb300_10pings starting 0.6 seconds late 2016-04-19 16:04:33,545 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 16:04:53,568 INFO repeater task ashtech_hcorrstats starting 0.9 seconds late 2016-04-19 16:04:54,458 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 16:06:04,561 INFO repeater task nb300_lastensq starting 2.6 seconds late 2016-04-19 16:06:07,026 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:06:07,026 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:05:52 2016-04-19 16:06:57,097 INFO repeater task nb300_10pings starting 4.4 seconds late 2016-04-19 16:06:57,278 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 16:08:02,372 INFO repeater task nb300_lastensq starting 0.4 seconds late 2016-04-19 16:08:04,893 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:08:04,893 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:07:52 2016-04-19 16:09:14,992 INFO repeater task nb300_10pings starting 2.3 seconds late 2016-04-19 16:09:15,173 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 16:09:55,226 INFO repeater task ashtech_hcorrstats starting 2.5 seconds late 2016-04-19 16:09:56,087 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 16:10:06,100 INFO repeater task nb300_lastensq starting 4.1 seconds late 2016-04-19 16:10:08,569 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:10:08,570 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:09:52 2016-04-19 16:11:33,693 INFO repeater task nb300_10pings starting 1.0 seconds late 2016-04-19 16:11:33,875 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 16:12:03,920 INFO repeater task nb300_lastensq starting 1.9 seconds late 2016-04-19 16:12:06,443 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:12:06,443 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:11:52 2016-04-19 16:13:56,596 INFO repeater task nb300_10pings starting 3.9 seconds late 2016-04-19 16:13:56,778 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 16:14:06,793 INFO repeater task nb300_lastensq starting 4.8 seconds late 2016-04-19 16:14:09,270 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:14:09,270 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:13:52 2016-04-19 16:14:54,333 INFO repeater task nb300_quick starting 1.6 seconds late 2016-04-19 16:14:57,033 INFO repeater task nb300_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2016-04-19 16:14:57,033 INFO repeater task nb300_quick ended after 2.7 seconds 2016-04-19 16:14:57,034 INFO repeater task nb300_3dayplots starting 4.3 seconds late 2016-04-19 16:15:00,161 INFO repeater task nb300_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2016-04-19 16:15:00,161 INFO repeater task nb300_3dayplots ended after 3.1 seconds 2016-04-19 16:15:00,162 INFO repeater task nb300_tsstats starting 7.5 seconds late 2016-04-19 16:15:01,071 INFO repeater task nb300_tsstats ended after 0.9 seconds 2016-04-19 16:15:01,072 INFO repeater task nb300_beamplots starting 8.4 seconds late 2016-04-19 16:15:02,975 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 16:15:02,975 INFO repeater task nb300_beamplots ended after 1.9 seconds 2016-04-19 16:15:02,976 INFO repeater task nb300_beamplots starting 10.3 seconds late 2016-04-19 16:15:04,937 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 16:15:04,938 INFO repeater task nb300_beamplots ended after 2.0 seconds 2016-04-19 16:15:04,938 INFO repeater task ashtech_hcorrstats starting 12.2 seconds late 2016-04-19 16:15:05,850 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 16:16:05,932 INFO repeater task nb300_lastensq starting 3.9 seconds late 2016-04-19 16:16:08,402 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:16:08,403 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:15:52 2016-04-19 16:16:13,410 INFO repeater task nb300_10pings starting 0.7 seconds late 2016-04-19 16:16:13,588 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 16:18:03,750 INFO repeater task nb300_lastensq starting 1.8 seconds late 2016-04-19 16:18:06,270 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:18:06,270 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:17:52 2016-04-19 16:18:36,300 INFO repeater task nb300_10pings starting 3.6 seconds late 2016-04-19 16:18:36,509 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 16:19:56,611 INFO repeater task ashtech_hcorrstats starting 3.9 seconds late 2016-04-19 16:19:57,491 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 16:20:02,493 INFO repeater task nb300_lastensq starting 0.5 seconds late 2016-04-19 16:20:04,949 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:20:04,950 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:19:52 2016-04-19 16:20:55,001 INFO repeater task nb300_10pings starting 2.3 seconds late 2016-04-19 16:20:55,182 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 16:22:05,279 INFO repeater task nb300_lastensq starting 3.3 seconds late 2016-04-19 16:22:07,796 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:22:07,797 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:21:52 2016-04-19 16:23:12,881 INFO repeater task nb300_10pings starting 0.2 seconds late 2016-04-19 16:23:13,063 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 16:24:03,118 INFO repeater task nb300_lastensq starting 1.1 seconds late 2016-04-19 16:24:05,612 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:24:05,613 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:23:52 2016-04-19 16:24:55,681 INFO repeater task ashtech_hcorrstats starting 3.0 seconds late 2016-04-19 16:24:56,550 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 16:25:36,607 INFO repeater task nb300_10pings starting 3.9 seconds late 2016-04-19 16:25:36,787 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 16:26:06,828 INFO repeater task nb300_lastensq starting 4.8 seconds late 2016-04-19 16:26:09,390 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 16:26:09,391 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:25:52 2016-04-19 16:27:54,537 INFO repeater task nb300_10pings starting 1.8 seconds late 2016-04-19 16:27:54,718 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 16:28:04,733 INFO repeater task nb300_lastensq starting 2.7 seconds late 2016-04-19 16:28:07,214 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:28:07,214 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:27:52 2016-04-19 16:29:57,377 INFO repeater task nb300_quick starting 4.7 seconds late 2016-04-19 16:30:00,112 INFO repeater task nb300_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2016-04-19 16:30:00,112 INFO repeater task nb300_quick ended after 2.7 seconds 2016-04-19 16:30:00,112 INFO repeater task nb300_3dayplots starting 7.4 seconds late 2016-04-19 16:30:03,244 INFO repeater task nb300_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2016-04-19 16:30:03,245 INFO repeater task nb300_3dayplots ended after 3.1 seconds 2016-04-19 16:30:03,245 INFO repeater task nb300_tsstats starting 10.5 seconds late 2016-04-19 16:30:04,142 INFO repeater task nb300_tsstats ended after 0.9 seconds 2016-04-19 16:30:04,143 INFO repeater task nb300_beamplots starting 11.4 seconds late 2016-04-19 16:30:06,059 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 16:30:06,059 INFO repeater task nb300_beamplots ended after 1.9 seconds 2016-04-19 16:30:06,059 INFO repeater task nb300_beamplots starting 13.4 seconds late 2016-04-19 16:30:07,933 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 16:30:07,934 INFO repeater task nb300_beamplots ended after 1.9 seconds 2016-04-19 16:30:07,934 INFO repeater task ashtech_hcorrstats starting 15.2 seconds late 2016-04-19 16:30:08,828 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 16:30:13,833 INFO repeater task nb300_lastensq starting 11.8 seconds late 2016-04-19 16:30:16,365 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:30:16,366 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:29:52 2016-04-19 16:30:16,366 INFO repeater task nb300_10pings starting 3.7 seconds late 2016-04-19 16:30:16,541 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 16:32:06,677 INFO repeater task nb300_lastensq starting 4.7 seconds late 2016-04-19 16:32:09,179 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:32:09,179 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:31:52 2016-04-19 16:32:34,210 INFO repeater task nb300_10pings starting 1.5 seconds late 2016-04-19 16:32:34,397 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 16:34:04,513 INFO repeater task nb300_lastensq starting 2.5 seconds late 2016-04-19 16:34:07,050 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:34:07,050 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:33:52 2016-04-19 16:34:57,125 INFO repeater task nb300_10pings starting 4.4 seconds late 2016-04-19 16:34:57,306 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 16:34:57,307 INFO repeater task ashtech_hcorrstats starting 4.6 seconds late 2016-04-19 16:34:58,211 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 16:36:03,299 INFO repeater task nb300_lastensq starting 1.3 seconds late 2016-04-19 16:36:05,816 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:36:05,816 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:35:52 2016-04-19 16:37:15,891 INFO repeater task nb300_10pings starting 3.2 seconds late 2016-04-19 16:37:16,095 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 16:38:06,152 INFO repeater task nb300_lastensq starting 4.2 seconds late 2016-04-19 16:38:08,691 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:38:08,692 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:37:52 2016-04-19 16:39:33,793 INFO repeater task nb300_10pings starting 1.1 seconds late 2016-04-19 16:39:33,973 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 16:39:54,004 INFO repeater task ashtech_hcorrstats starting 1.3 seconds late 2016-04-19 16:39:54,911 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 16:40:04,923 INFO repeater task nb300_lastensq starting 2.9 seconds late 2016-04-19 16:40:07,465 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:40:07,466 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:39:52 2016-04-19 16:41:57,624 INFO repeater task nb300_10pings starting 4.9 seconds late 2016-04-19 16:41:57,804 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 16:42:02,811 INFO repeater task nb300_lastensq starting 0.8 seconds late 2016-04-19 16:42:05,314 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:42:05,315 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:41:52 2016-04-19 16:44:05,481 INFO repeater task nb300_lastensq starting 3.5 seconds late 2016-04-19 16:44:08,030 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:44:08,030 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:43:52 2016-04-19 16:44:13,035 INFO repeater task nb300_10pings starting 0.3 seconds late 2016-04-19 16:44:13,219 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 16:44:53,263 INFO repeater task nb300_quick starting 0.6 seconds late 2016-04-19 16:44:55,924 INFO repeater task nb300_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2016-04-19 16:44:55,924 INFO repeater task nb300_quick ended after 2.7 seconds 2016-04-19 16:44:55,924 INFO repeater task nb300_3dayplots starting 3.2 seconds late 2016-04-19 16:44:59,084 INFO repeater task nb300_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2016-04-19 16:44:59,084 INFO repeater task nb300_3dayplots ended after 3.2 seconds 2016-04-19 16:44:59,084 INFO repeater task nb300_tsstats starting 6.4 seconds late 2016-04-19 16:44:59,960 INFO repeater task nb300_tsstats ended after 0.9 seconds 2016-04-19 16:44:59,961 INFO repeater task nb300_beamplots starting 7.3 seconds late 2016-04-19 16:45:01,857 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 16:45:01,858 INFO repeater task nb300_beamplots ended after 1.9 seconds 2016-04-19 16:45:01,858 INFO repeater task nb300_beamplots starting 9.2 seconds late 2016-04-19 16:45:03,891 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 16:45:03,891 INFO repeater task nb300_beamplots ended after 2.0 seconds 2016-04-19 16:45:03,891 INFO repeater task ashtech_hcorrstats starting 11.2 seconds late 2016-04-19 16:45:04,792 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 16:46:04,872 INFO repeater task nb300_lastensq starting 2.9 seconds late 2016-04-19 16:46:07,386 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:46:07,386 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:45:52 2016-04-19 16:46:37,429 INFO repeater task nb300_10pings starting 4.7 seconds late 2016-04-19 16:46:37,608 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 16:48:02,730 INFO repeater task nb300_lastensq starting 0.7 seconds late 2016-04-19 16:48:05,279 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:48:05,279 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:47:52 2016-04-19 16:48:55,342 INFO repeater task nb300_10pings starting 2.6 seconds late 2016-04-19 16:48:55,523 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 16:49:55,610 INFO repeater task ashtech_hcorrstats starting 2.9 seconds late 2016-04-19 16:49:56,524 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 16:50:06,537 INFO repeater task nb300_lastensq starting 4.5 seconds late 2016-04-19 16:50:09,084 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:50:09,085 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:49:52 2016-04-19 16:51:14,180 INFO repeater task nb300_10pings starting 1.5 seconds late 2016-04-19 16:51:14,364 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 16:52:04,435 INFO repeater task nb300_lastensq starting 2.4 seconds late 2016-04-19 16:52:06,987 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 16:52:06,988 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:51:52 2016-04-19 16:53:37,105 INFO repeater task nb300_10pings starting 4.4 seconds late 2016-04-19 16:53:37,285 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 16:54:02,319 INFO repeater task nb300_lastensq starting 0.3 seconds late 2016-04-19 16:54:04,821 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:54:04,821 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:53:52 2016-04-19 16:54:54,898 INFO repeater task ashtech_hcorrstats starting 2.2 seconds late 2016-04-19 16:54:55,854 INFO repeater task ashtech_hcorrstats ended after 1.0 seconds 2016-04-19 16:55:55,927 INFO repeater task nb300_10pings starting 3.2 seconds late 2016-04-19 16:55:56,132 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 16:56:06,146 INFO repeater task nb300_lastensq starting 4.1 seconds late 2016-04-19 16:56:08,649 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:56:08,649 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:55:52 2016-04-19 16:58:03,817 INFO repeater task nb300_lastensq starting 1.8 seconds late 2016-04-19 16:58:06,329 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 16:58:06,330 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:57:52 2016-04-19 16:58:16,344 INFO repeater task nb300_10pings starting 3.6 seconds late 2016-04-19 16:58:16,564 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 16:59:56,699 INFO repeater task nb300_quick starting 4.0 seconds late 2016-04-19 16:59:59,462 INFO repeater task nb300_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2016-04-19 16:59:59,463 INFO repeater task nb300_quick ended after 2.8 seconds 2016-04-19 16:59:59,463 INFO repeater task nb300_3dayplots starting 6.8 seconds late 2016-04-19 17:00:02,889 INFO repeater task nb300_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2016-04-19 17:00:02,889 INFO repeater task nb300_3dayplots ended after 3.4 seconds 2016-04-19 17:00:02,889 INFO repeater task nb300_tsstats starting 10.2 seconds late 2016-04-19 17:00:03,917 INFO repeater task nb300_tsstats ended after 1.0 seconds 2016-04-19 17:00:03,917 INFO repeater task nb300_beamplots starting 11.2 seconds late 2016-04-19 17:00:05,888 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 17:00:05,888 INFO repeater task nb300_beamplots ended after 2.0 seconds 2016-04-19 17:00:05,888 INFO repeater task nb300_beamplots starting 13.2 seconds late 2016-04-19 17:00:07,934 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 17:00:07,935 INFO repeater task nb300_beamplots ended after 2.0 seconds 2016-04-19 17:00:07,935 INFO repeater task ashtech_hcorrstats starting 15.2 seconds late 2016-04-19 17:00:08,851 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 17:00:13,856 INFO repeater task nb300_lastensq starting 11.9 seconds late 2016-04-19 17:00:16,354 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 17:00:16,354 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 16:59:52 2016-04-19 17:00:36,376 INFO repeater task nb300_10pings starting 3.7 seconds late 2016-04-19 17:00:36,562 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:02:06,697 INFO repeater task nb300_lastensq starting 4.7 seconds late 2016-04-19 17:02:09,271 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 17:02:09,271 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:01:52 2016-04-19 17:02:54,334 INFO repeater task nb300_10pings starting 1.6 seconds late 2016-04-19 17:02:54,512 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:04:04,591 INFO repeater task nb300_lastensq starting 2.6 seconds late 2016-04-19 17:04:07,075 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 17:04:07,076 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:03:52 2016-04-19 17:04:57,149 INFO repeater task ashtech_hcorrstats starting 4.4 seconds late 2016-04-19 17:04:58,072 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 17:05:13,093 INFO repeater task nb300_10pings starting 0.4 seconds late 2016-04-19 17:05:13,305 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:06:03,370 INFO repeater task nb300_lastensq starting 1.4 seconds late 2016-04-19 17:06:05,912 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 17:06:05,913 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:05:52 2016-04-19 17:07:36,036 INFO repeater task nb300_10pings starting 3.3 seconds late 2016-04-19 17:07:36,216 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:08:06,260 INFO repeater task nb300_lastensq starting 4.3 seconds late 2016-04-19 17:08:08,740 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 17:08:08,741 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:07:52 2016-04-19 17:09:53,888 INFO repeater task nb300_10pings starting 1.2 seconds late 2016-04-19 17:09:54,070 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:09:54,071 INFO repeater task ashtech_hcorrstats starting 1.4 seconds late 2016-04-19 17:09:55,044 INFO repeater task ashtech_hcorrstats ended after 1.0 seconds 2016-04-19 17:10:05,057 INFO repeater task nb300_lastensq starting 3.1 seconds late 2016-04-19 17:10:07,541 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 17:10:07,541 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:09:52 2016-04-19 17:12:02,705 INFO repeater task nb300_lastensq starting 0.7 seconds late 2016-04-19 17:12:05,228 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 17:12:05,228 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:11:52 2016-04-19 17:12:15,238 INFO repeater task nb300_10pings starting 2.5 seconds late 2016-04-19 17:12:15,448 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:14:05,579 INFO repeater task nb300_lastensq starting 3.6 seconds late 2016-04-19 17:14:08,126 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 17:14:08,127 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:13:52 2016-04-19 17:14:33,163 INFO repeater task nb300_10pings starting 0.5 seconds late 2016-04-19 17:14:33,342 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:14:53,370 INFO repeater task nb300_quick starting 0.7 seconds late 2016-04-19 17:14:56,159 INFO repeater task nb300_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2016-04-19 17:14:56,160 INFO repeater task nb300_quick ended after 2.8 seconds 2016-04-19 17:14:56,160 INFO repeater task nb300_3dayplots starting 3.5 seconds late 2016-04-19 17:14:59,266 INFO repeater task nb300_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2016-04-19 17:14:59,266 INFO repeater task nb300_3dayplots ended after 3.1 seconds 2016-04-19 17:14:59,267 INFO repeater task nb300_tsstats starting 6.6 seconds late 2016-04-19 17:15:00,146 INFO repeater task nb300_tsstats ended after 0.9 seconds 2016-04-19 17:15:00,147 INFO repeater task nb300_beamplots starting 7.4 seconds late 2016-04-19 17:15:02,079 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 17:15:02,079 INFO repeater task nb300_beamplots ended after 1.9 seconds 2016-04-19 17:15:02,080 INFO repeater task nb300_beamplots starting 9.4 seconds late 2016-04-19 17:15:04,063 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 17:15:04,064 INFO repeater task nb300_beamplots ended after 2.0 seconds 2016-04-19 17:15:04,064 INFO repeater task ashtech_hcorrstats starting 11.4 seconds late 2016-04-19 17:15:04,987 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 17:16:05,065 INFO repeater task nb300_lastensq starting 3.1 seconds late 2016-04-19 17:16:07,593 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 17:16:07,594 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:15:52 2016-04-19 17:16:57,662 INFO repeater task nb300_10pings starting 5.0 seconds late 2016-04-19 17:16:57,845 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:18:02,942 INFO repeater task nb300_lastensq starting 0.9 seconds late 2016-04-19 17:18:05,431 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 17:18:05,432 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:17:52 2016-04-19 17:19:15,537 INFO repeater task nb300_10pings starting 2.8 seconds late 2016-04-19 17:19:15,717 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:19:55,772 INFO repeater task ashtech_hcorrstats starting 3.1 seconds late 2016-04-19 17:19:56,700 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 17:20:06,713 INFO repeater task nb300_lastensq starting 4.7 seconds late 2016-04-19 17:20:09,246 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 17:20:09,247 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:19:52 2016-04-19 17:21:34,370 INFO repeater task nb300_10pings starting 1.7 seconds late 2016-04-19 17:21:34,552 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:22:04,596 INFO repeater task nb300_lastensq starting 2.6 seconds late 2016-04-19 17:22:07,094 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 17:22:07,094 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:21:52 2016-04-19 17:23:57,255 INFO repeater task nb300_10pings starting 4.6 seconds late 2016-04-19 17:23:57,436 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:24:02,442 INFO repeater task nb300_lastensq starting 0.4 seconds late 2016-04-19 17:24:04,997 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 17:24:04,998 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:23:52 2016-04-19 17:24:55,070 INFO repeater task ashtech_hcorrstats starting 2.4 seconds late 2016-04-19 17:24:55,969 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 17:26:06,068 INFO repeater task nb300_lastensq starting 4.1 seconds late 2016-04-19 17:26:08,574 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 17:26:08,574 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:25:52 2016-04-19 17:26:13,580 INFO repeater task nb300_10pings starting 0.9 seconds late 2016-04-19 17:26:13,780 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:28:03,924 INFO repeater task nb300_lastensq starting 1.9 seconds late 2016-04-19 17:28:06,472 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 17:28:06,472 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:27:52 2016-04-19 17:28:36,517 INFO repeater task nb300_10pings starting 3.8 seconds late 2016-04-19 17:28:36,694 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:29:56,796 INFO repeater task nb300_quick starting 4.1 seconds late 2016-04-19 17:29:59,511 INFO repeater task nb300_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2016-04-19 17:29:59,511 INFO repeater task nb300_quick ended after 2.7 seconds 2016-04-19 17:29:59,512 INFO repeater task nb300_3dayplots starting 6.8 seconds late 2016-04-19 17:30:02,678 INFO repeater task nb300_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2016-04-19 17:30:02,678 INFO repeater task nb300_3dayplots ended after 3.2 seconds 2016-04-19 17:30:02,678 INFO repeater task nb300_tsstats starting 10.0 seconds late 2016-04-19 17:30:03,580 INFO repeater task nb300_tsstats ended after 0.9 seconds 2016-04-19 17:30:03,581 INFO repeater task nb300_beamplots starting 10.9 seconds late 2016-04-19 17:30:05,501 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 17:30:05,501 INFO repeater task nb300_beamplots ended after 1.9 seconds 2016-04-19 17:30:05,501 INFO repeater task nb300_beamplots starting 12.8 seconds late 2016-04-19 17:30:07,478 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 17:30:07,478 INFO repeater task nb300_beamplots ended after 2.0 seconds 2016-04-19 17:30:07,478 INFO repeater task ashtech_hcorrstats starting 14.8 seconds late 2016-04-19 17:30:08,382 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 17:30:13,388 INFO repeater task nb300_lastensq starting 11.4 seconds late 2016-04-19 17:30:15,966 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 17:30:15,966 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:29:52 2016-04-19 17:30:56,024 INFO repeater task nb300_10pings starting 3.3 seconds late 2016-04-19 17:30:56,203 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:32:06,300 INFO repeater task nb300_lastensq starting 4.3 seconds late 2016-04-19 17:32:08,808 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 17:32:08,809 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:31:52 2016-04-19 17:33:13,899 INFO repeater task nb300_10pings starting 1.2 seconds late 2016-04-19 17:33:14,080 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:34:04,145 INFO repeater task nb300_lastensq starting 2.1 seconds late 2016-04-19 17:34:06,737 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 17:34:06,737 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:33:52 2016-04-19 17:34:56,813 INFO repeater task ashtech_hcorrstats starting 4.1 seconds late 2016-04-19 17:34:57,714 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 17:35:32,766 INFO repeater task nb300_10pings starting 0.1 seconds late 2016-04-19 17:35:32,945 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:36:02,986 INFO repeater task nb300_lastensq starting 1.0 seconds late 2016-04-19 17:36:05,502 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 17:36:05,502 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:35:52 2016-04-19 17:37:55,638 INFO repeater task nb300_10pings starting 2.9 seconds late 2016-04-19 17:37:55,817 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:38:05,831 INFO repeater task nb300_lastensq starting 3.8 seconds late 2016-04-19 17:38:08,408 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 17:38:08,408 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:37:52 2016-04-19 17:39:53,566 INFO repeater task ashtech_hcorrstats starting 0.9 seconds late 2016-04-19 17:39:54,480 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 17:40:04,493 INFO repeater task nb300_lastensq starting 2.5 seconds late 2016-04-19 17:40:07,021 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 17:40:07,022 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:39:52 2016-04-19 17:40:17,036 INFO repeater task nb300_10pings starting 4.3 seconds late 2016-04-19 17:40:17,215 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:42:02,364 INFO repeater task nb300_lastensq starting 0.4 seconds late 2016-04-19 17:42:04,948 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 17:42:04,948 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:41:52 2016-04-19 17:42:34,989 INFO repeater task nb300_10pings starting 2.3 seconds late 2016-04-19 17:42:35,169 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:44:05,302 INFO repeater task nb300_lastensq starting 3.3 seconds late 2016-04-19 17:44:07,801 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 17:44:07,801 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:43:52 2016-04-19 17:44:52,869 INFO repeater task nb300_10pings starting 0.2 seconds late 2016-04-19 17:44:53,049 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:44:53,049 INFO repeater task nb300_quick starting 0.3 seconds late 2016-04-19 17:44:55,852 INFO repeater task nb300_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2016-04-19 17:44:55,852 INFO repeater task nb300_quick ended after 2.8 seconds 2016-04-19 17:44:55,852 INFO repeater task nb300_3dayplots starting 3.2 seconds late 2016-04-19 17:44:59,026 INFO repeater task nb300_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2016-04-19 17:44:59,026 INFO repeater task nb300_3dayplots ended after 3.2 seconds 2016-04-19 17:44:59,027 INFO repeater task nb300_tsstats starting 6.3 seconds late 2016-04-19 17:44:59,927 INFO repeater task nb300_tsstats ended after 0.9 seconds 2016-04-19 17:44:59,927 INFO repeater task nb300_beamplots starting 7.2 seconds late 2016-04-19 17:45:01,772 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 17:45:01,772 INFO repeater task nb300_beamplots ended after 1.8 seconds 2016-04-19 17:45:01,772 INFO repeater task nb300_beamplots starting 9.1 seconds late 2016-04-19 17:45:03,778 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 17:45:03,779 INFO repeater task nb300_beamplots ended after 2.0 seconds 2016-04-19 17:45:03,779 INFO repeater task ashtech_hcorrstats starting 11.1 seconds late 2016-04-19 17:45:04,713 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 17:46:04,791 INFO repeater task nb300_lastensq starting 2.8 seconds late 2016-04-19 17:46:07,322 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 17:46:07,323 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:45:52 2016-04-19 17:47:17,420 INFO repeater task nb300_10pings starting 4.7 seconds late 2016-04-19 17:47:17,598 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:48:02,656 INFO repeater task nb300_lastensq starting 0.7 seconds late 2016-04-19 17:48:05,252 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 17:48:05,253 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:47:52 2016-04-19 17:49:35,379 INFO repeater task nb300_10pings starting 2.7 seconds late 2016-04-19 17:49:35,561 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:49:55,592 INFO repeater task ashtech_hcorrstats starting 2.9 seconds late 2016-04-19 17:49:56,502 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 17:50:06,515 INFO repeater task nb300_lastensq starting 4.5 seconds late 2016-04-19 17:50:09,024 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 17:50:09,024 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:49:52 2016-04-19 17:51:54,182 INFO repeater task nb300_10pings starting 1.5 seconds late 2016-04-19 17:51:54,367 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:52:04,381 INFO repeater task nb300_lastensq starting 2.4 seconds late 2016-04-19 17:52:06,964 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 17:52:06,964 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:51:52 2016-04-19 17:54:02,132 INFO repeater task nb300_lastensq starting 0.1 seconds late 2016-04-19 17:54:04,640 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 17:54:04,640 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:53:52 2016-04-19 17:54:14,654 INFO repeater task nb300_10pings starting 2.0 seconds late 2016-04-19 17:54:14,834 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:54:54,889 INFO repeater task ashtech_hcorrstats starting 2.2 seconds late 2016-04-19 17:54:55,797 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 17:56:05,895 INFO repeater task nb300_lastensq starting 3.9 seconds late 2016-04-19 17:56:08,457 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 17:56:08,458 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:55:52 2016-04-19 17:56:33,495 INFO repeater task nb300_10pings starting 0.8 seconds late 2016-04-19 17:56:33,677 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:58:03,803 INFO repeater task nb300_lastensq starting 1.8 seconds late 2016-04-19 17:58:06,318 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 17:58:06,319 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:57:52 2016-04-19 17:58:56,386 INFO repeater task nb300_10pings starting 3.7 seconds late 2016-04-19 17:58:56,565 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 17:59:56,653 INFO repeater task nb300_quick starting 4.0 seconds late 2016-04-19 17:59:59,377 INFO repeater task nb300_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2016-04-19 17:59:59,378 INFO repeater task nb300_quick ended after 2.7 seconds 2016-04-19 17:59:59,378 INFO repeater task nb300_3dayplots starting 6.7 seconds late 2016-04-19 18:00:02,534 INFO repeater task nb300_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2016-04-19 18:00:02,534 INFO repeater task nb300_3dayplots ended after 3.2 seconds 2016-04-19 18:00:02,535 INFO repeater task nb300_tsstats starting 9.8 seconds late 2016-04-19 18:00:03,565 INFO repeater task nb300_tsstats ended after 1.0 seconds 2016-04-19 18:00:03,565 INFO repeater task nb300_beamplots starting 10.9 seconds late 2016-04-19 18:00:05,473 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 18:00:05,473 INFO repeater task nb300_beamplots ended after 1.9 seconds 2016-04-19 18:00:05,473 INFO repeater task nb300_beamplots starting 12.8 seconds late 2016-04-19 18:00:07,468 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 18:00:07,469 INFO repeater task nb300_beamplots ended after 2.0 seconds 2016-04-19 18:00:07,469 INFO repeater task ashtech_hcorrstats starting 14.8 seconds late 2016-04-19 18:00:08,453 INFO repeater task ashtech_hcorrstats ended after 1.0 seconds 2016-04-19 18:00:13,459 INFO repeater task nb300_lastensq starting 11.5 seconds late 2016-04-19 18:00:15,990 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 18:00:15,991 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 17:59:52 2016-04-19 18:01:16,079 INFO repeater task nb300_10pings starting 3.4 seconds late 2016-04-19 18:01:16,257 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 18:02:06,312 INFO repeater task nb300_lastensq starting 4.3 seconds late 2016-04-19 18:02:08,891 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 18:02:08,892 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 18:01:52 2016-04-19 18:03:34,013 INFO repeater task nb300_10pings starting 1.3 seconds late 2016-04-19 18:03:34,194 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 18:04:04,238 INFO repeater task nb300_lastensq starting 2.2 seconds late 2016-04-19 18:04:06,730 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 18:04:06,731 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 18:03:52 2016-04-19 18:04:56,807 INFO repeater task ashtech_hcorrstats starting 4.1 seconds late 2016-04-19 18:04:57,719 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 18:05:52,796 INFO repeater task nb300_10pings starting 0.1 seconds late 2016-04-19 18:05:52,976 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 18:06:02,984 INFO repeater task nb300_lastensq starting 1.0 seconds late 2016-04-19 18:06:05,535 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 18:06:05,535 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 18:05:52 2016-04-19 18:08:05,712 INFO repeater task nb300_lastensq starting 3.7 seconds late 2016-04-19 18:08:08,227 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 18:08:08,228 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 18:07:52 2016-04-19 18:08:13,235 INFO repeater task nb300_10pings starting 0.5 seconds late 2016-04-19 18:08:13,415 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 18:09:53,550 INFO repeater task ashtech_hcorrstats starting 0.8 seconds late 2016-04-19 18:09:54,469 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 18:10:04,482 INFO repeater task nb300_lastensq starting 2.5 seconds late 2016-04-19 18:10:07,049 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 18:10:07,050 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 18:09:52 2016-04-19 18:10:37,094 INFO repeater task nb300_10pings starting 4.4 seconds late 2016-04-19 18:10:37,273 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 18:12:02,395 INFO repeater task nb300_lastensq starting 0.4 seconds late 2016-04-19 18:12:04,911 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 18:12:04,912 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 18:11:52 2016-04-19 18:12:54,983 INFO repeater task nb300_10pings starting 2.3 seconds late 2016-04-19 18:12:55,160 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 18:14:05,265 INFO repeater task nb300_lastensq starting 3.3 seconds late 2016-04-19 18:14:07,848 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 18:14:07,848 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 18:13:52 2016-04-19 18:14:52,916 INFO repeater task nb300_quick starting 0.2 seconds late 2016-04-19 18:14:55,620 INFO repeater task nb300_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2016-04-19 18:14:55,620 INFO repeater task nb300_quick ended after 2.7 seconds 2016-04-19 18:14:55,620 INFO repeater task nb300_3dayplots starting 2.9 seconds late 2016-04-19 18:14:58,769 INFO repeater task nb300_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2016-04-19 18:14:58,769 INFO repeater task nb300_3dayplots ended after 3.1 seconds 2016-04-19 18:14:58,769 INFO repeater task nb300_tsstats starting 6.1 seconds late 2016-04-19 18:14:59,619 INFO repeater task nb300_tsstats ended after 0.8 seconds 2016-04-19 18:14:59,619 INFO repeater task nb300_beamplots starting 6.9 seconds late 2016-04-19 18:15:01,527 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 18:15:01,527 INFO repeater task nb300_beamplots ended after 1.9 seconds 2016-04-19 18:15:01,528 INFO repeater task nb300_beamplots starting 8.8 seconds late 2016-04-19 18:15:03,509 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 18:15:03,509 INFO repeater task nb300_beamplots ended after 2.0 seconds 2016-04-19 18:15:03,510 INFO repeater task ashtech_hcorrstats starting 10.8 seconds late 2016-04-19 18:15:04,430 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 18:15:14,441 INFO repeater task nb300_10pings starting 1.7 seconds late 2016-04-19 18:15:14,622 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 18:16:04,692 INFO repeater task nb300_lastensq starting 2.7 seconds late 2016-04-19 18:16:07,275 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 18:16:07,276 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 18:15:52 2016-04-19 18:17:37,383 INFO repeater task nb300_10pings starting 4.7 seconds late 2016-04-19 18:17:37,575 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 18:18:02,605 INFO repeater task nb300_lastensq starting 0.6 seconds late 2016-04-19 18:18:05,100 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 18:18:05,100 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 18:17:52 2016-04-19 18:19:55,233 INFO repeater task nb300_10pings starting 2.5 seconds late 2016-04-19 18:19:55,410 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 18:19:55,411 INFO repeater task ashtech_hcorrstats starting 2.7 seconds late 2016-04-19 18:19:56,320 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 18:20:06,333 INFO repeater task nb300_lastensq starting 4.3 seconds late 2016-04-19 18:20:08,915 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 18:20:08,916 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 18:19:52 2016-04-19 18:22:04,080 INFO repeater task nb300_lastensq starting 2.1 seconds late 2016-04-19 18:22:06,607 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 18:22:06,607 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 18:21:52 2016-04-19 18:22:16,620 INFO repeater task nb300_10pings starting 3.9 seconds late 2016-04-19 18:22:16,798 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 18:24:06,936 INFO repeater task nb300_lastensq starting 4.9 seconds late 2016-04-19 18:24:09,525 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 18:24:09,525 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 18:23:52 2016-04-19 18:24:34,551 INFO repeater task nb300_10pings starting 1.9 seconds late 2016-04-19 18:24:34,732 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 18:24:54,762 INFO repeater task ashtech_hcorrstats starting 2.1 seconds late 2016-04-19 18:24:55,679 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 18:26:05,755 INFO repeater task nb300_lastensq starting 3.8 seconds late 2016-04-19 18:26:08,278 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 18:26:08,278 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 18:25:52 2016-04-19 18:26:53,345 INFO repeater task nb300_10pings starting 0.6 seconds late 2016-04-19 18:26:53,526 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 18:28:03,624 INFO repeater task nb300_lastensq starting 1.6 seconds late 2016-04-19 18:28:06,208 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 18:28:06,208 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 18:27:52 2016-04-19 18:29:16,303 INFO repeater task nb300_10pings starting 3.6 seconds late 2016-04-19 18:29:16,483 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 18:29:56,542 INFO repeater task nb300_quick starting 3.8 seconds late 2016-04-19 18:29:59,224 INFO repeater task nb300_quick ended after 2.7 seconds 2016-04-19 18:29:59,224 INFO repeater task nb300_3dayplots starting 6.5 seconds late 2016-04-19 18:30:02,417 INFO repeater task nb300_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2016-04-19 18:30:02,417 INFO repeater task nb300_3dayplots ended after 3.2 seconds 2016-04-19 18:30:02,418 INFO repeater task nb300_tsstats starting 9.7 seconds late 2016-04-19 18:30:03,274 INFO repeater task nb300_tsstats ended after 0.9 seconds 2016-04-19 18:30:03,275 INFO repeater task nb300_beamplots starting 10.6 seconds late 2016-04-19 18:30:05,164 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 18:30:05,164 INFO repeater task nb300_beamplots ended after 1.9 seconds 2016-04-19 18:30:05,164 INFO repeater task nb300_beamplots starting 12.5 seconds late 2016-04-19 18:30:07,142 INFO repeater task nb300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2016-04-19 18:30:07,142 INFO repeater task nb300_beamplots ended after 2.0 seconds 2016-04-19 18:30:07,143 INFO repeater task ashtech_hcorrstats starting 14.4 seconds late 2016-04-19 18:30:08,053 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 18:30:13,059 INFO repeater task nb300_lastensq starting 11.1 seconds late 2016-04-19 18:30:15,590 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 18:30:15,591 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 18:29:52 2016-04-19 18:31:35,700 INFO repeater task nb300_10pings starting 3.0 seconds late 2016-04-19 18:31:35,895 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 18:32:05,932 INFO repeater task nb300_lastensq starting 3.9 seconds late 2016-04-19 18:32:08,523 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 18:32:08,524 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 18:31:52 2016-04-19 18:33:53,672 INFO repeater task nb300_10pings starting 1.0 seconds late 2016-04-19 18:33:53,850 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 18:34:03,863 INFO repeater task nb300_lastensq starting 1.9 seconds late 2016-04-19 18:34:06,386 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 18:34:06,387 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 18:33:52 2016-04-19 18:34:56,461 INFO repeater task ashtech_hcorrstats starting 3.8 seconds late 2016-04-19 18:34:57,431 INFO repeater task ashtech_hcorrstats ended after 1.0 seconds 2016-04-19 18:36:02,527 INFO repeater task nb300_lastensq starting 0.5 seconds late 2016-04-19 18:36:05,065 INFO repeater task nb300_lastensq ended after 2.5 seconds 2016-04-19 18:36:05,066 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 18:35:52 2016-04-19 18:36:15,080 INFO repeater task nb300_10pings starting 2.4 seconds late 2016-04-19 18:36:15,256 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 18:38:05,410 INFO repeater task nb300_lastensq starting 3.4 seconds late 2016-04-19 18:38:08,005 INFO repeater task nb300_lastensq ended after 2.6 seconds 2016-04-19 18:38:08,005 INFO repeater task nb300_lastensq: Time from /home/data/SP1605/proc/nb300/load/wrote_ens_time: 2016/04/19 18:37:52 2016-04-19 18:38:33,036 INFO repeater task nb300_10pings starting 0.3 seconds late 2016-04-19 18:38:33,214 INFO repeater task nb300_10pings ended after 0.2 seconds 2016-04-19 18:39:53,325 INFO repeater task ashtech_hcorrstats starting 0.6 seconds late 2016-04-19 18:39:54,237 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2016-04-19 18:40:04,251 INFO repeater task nb300_lastensq starting 2.3 seconds late