2018-07-19 00:38:55,318 INFO     repeater     task ashtech_hcorrstats starting 14.7 seconds late
2018-07-19 00:38:56,299 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 00:39:41,365 INFO     repeater     task nb300_10pings starting 0.8 seconds late
2018-07-19 00:39:41,550 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 00:39:51,564 INFO     repeater     task nb300_lastensq starting 1.6 seconds late
2018-07-19 00:39:54,162 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 00:39:54,162 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 00:39:40
2018-07-19 00:41:54,332 INFO     repeater     task nb300_lastensq starting 4.3 seconds late
2018-07-19 00:41:56,991 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 00:41:56,991 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 00:41:40
2018-07-19 00:42:01,996 INFO     repeater     task nb300_10pings starting 1.4 seconds late
2018-07-19 00:42:02,180 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 00:43:42,318 INFO     repeater     task ashtech_hcorrstats starting 1.7 seconds late
2018-07-19 00:43:43,318 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 00:43:53,332 INFO     repeater     task nb300_lastensq starting 3.3 seconds late
2018-07-19 00:43:55,982 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 00:43:55,982 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 00:43:40
2018-07-19 00:44:21,014 INFO     repeater     task nb300_10pings starting 0.4 seconds late
2018-07-19 00:44:21,195 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 00:45:51,313 INFO     repeater     task nb300_lastensq starting 1.3 seconds late
2018-07-19 00:45:53,920 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 00:45:53,920 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 00:45:40
2018-07-19 00:46:43,996 INFO     repeater     task nb300_10pings starting 3.4 seconds late
2018-07-19 00:46:44,180 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 00:47:54,283 INFO     repeater     task nb300_lastensq starting 4.3 seconds late
2018-07-19 00:47:56,929 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 00:47:56,930 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 00:47:40
2018-07-19 00:48:41,991 INFO     repeater     task ashtech_hcorrstats starting 1.4 seconds late
2018-07-19 00:48:43,016 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 00:49:03,043 INFO     repeater     task nb300_10pings starting 2.4 seconds late
2018-07-19 00:49:03,228 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 00:49:53,300 INFO     repeater     task nb300_lastensq starting 3.3 seconds late
2018-07-19 00:49:55,904 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 00:49:55,904 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 00:49:55,904 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 00:49:40
2018-07-19 00:51:21,008 INFO     repeater     task nb300_10pings starting 0.4 seconds late
2018-07-19 00:51:21,190 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 00:51:51,232 INFO     repeater     task nb300_lastensq starting 1.2 seconds late
2018-07-19 00:51:53,838 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 00:51:53,838 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 00:51:40
2018-07-19 00:53:43,996 INFO     repeater     task nb300_10pings starting 3.4 seconds late
2018-07-19 00:53:44,179 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 00:53:44,179 INFO     repeater     task nb300_quick starting 3.6 seconds late
2018-07-19 00:53:46,620 INFO     repeater     task nb300_quick output:
/usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice.
  warnings.warn("Mean of empty slice.", RuntimeWarning)

2018-07-19 00:53:46,620 INFO     repeater     task nb300_quick ended after 2.4 seconds
2018-07-19 00:53:46,620 INFO     repeater     task nb300_3dayplots starting 6.0 seconds late
2018-07-19 00:53:50,259 INFO     repeater     task nb300_3dayplots output:
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered
  warnings.warn("All-NaN slice encountered", RuntimeWarning)
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered
  warnings.warn("All-NaN axis encountered", RuntimeWarning)
zgrid failed
zgrid failed: returning all values masked
zgrid failed
zgrid failed: returning all values masked
zgrid failed
zgrid failed: returning all values masked
zgrid failed
zgrid failed: returning all values masked
zgrid failed
zgrid failed: returning all values masked
zgrid failed
zgrid failed: returning all values masked
zgrid failed
zgrid failed: returning all values masked
zgrid failed
zgrid failed: returning all values masked

2018-07-19 00:53:50,259 INFO     repeater     task nb300_3dayplots ended after 3.6 seconds
2018-07-19 00:53:50,260 INFO     repeater     task nb300_tsstats starting 9.6 seconds late
2018-07-19 00:53:51,149 INFO     repeater     task nb300_tsstats ended after 0.9 seconds
2018-07-19 00:53:51,150 INFO     repeater     task nb300_beamplots starting 10.5 seconds late
2018-07-19 00:53:53,058 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 00:53:53,058 INFO     repeater     task nb300_beamplots ended after 1.9 seconds
2018-07-19 00:53:53,058 INFO     repeater     task nb300_beamplots starting 12.4 seconds late
2018-07-19 00:53:55,024 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 00:53:55,024 INFO     repeater     task nb300_beamplots ended after 2.0 seconds
2018-07-19 00:53:55,025 INFO     repeater     task ashtech_hcorrstats starting 14.4 seconds late
2018-07-19 00:53:56,005 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 00:54:01,011 INFO     repeater     task nb300_lastensq starting 11.0 seconds late
2018-07-19 00:54:03,643 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 00:54:03,643 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 00:53:40
2018-07-19 00:55:53,773 INFO     repeater     task nb300_lastensq starting 3.8 seconds late
2018-07-19 00:55:56,411 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 00:55:56,411 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 00:55:40
2018-07-19 00:56:01,415 INFO     repeater     task nb300_10pings starting 0.8 seconds late
2018-07-19 00:56:01,596 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 00:57:51,758 INFO     repeater     task nb300_lastensq starting 1.8 seconds late
2018-07-19 00:57:54,350 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 00:57:54,351 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 00:57:40
2018-07-19 00:58:24,386 INFO     repeater     task nb300_10pings starting 3.8 seconds late
2018-07-19 00:58:24,593 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 00:58:44,623 INFO     repeater     task ashtech_hcorrstats starting 4.0 seconds late
2018-07-19 00:58:45,633 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 00:59:50,725 INFO     repeater     task nb300_lastensq starting 0.7 seconds late
2018-07-19 00:59:53,315 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 00:59:53,316 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 00:59:40
2018-07-19 01:00:43,388 INFO     repeater     task nb300_10pings starting 2.8 seconds late
2018-07-19 01:00:43,572 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:01:53,678 INFO     repeater     task nb300_lastensq starting 3.7 seconds late
2018-07-19 01:01:56,312 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 01:01:56,312 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:01:40
2018-07-19 01:03:01,405 INFO     repeater     task nb300_10pings starting 0.8 seconds late
2018-07-19 01:03:01,589 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:03:41,647 INFO     repeater     task ashtech_hcorrstats starting 1.0 seconds late
2018-07-19 01:03:42,644 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 01:03:52,657 INFO     repeater     task nb300_lastensq starting 2.7 seconds late
2018-07-19 01:03:55,259 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 01:03:55,259 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:03:40
2018-07-19 01:05:25,387 INFO     repeater     task nb300_10pings starting 4.8 seconds late
2018-07-19 01:05:25,569 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:05:50,606 INFO     repeater     task nb300_lastensq starting 0.6 seconds late
2018-07-19 01:05:53,246 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 01:05:53,246 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:05:40
2018-07-19 01:07:43,398 INFO     repeater     task nb300_10pings starting 2.8 seconds late
2018-07-19 01:07:43,582 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:07:53,596 INFO     repeater     task nb300_lastensq starting 3.6 seconds late
2018-07-19 01:07:56,188 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 01:07:56,188 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:07:40
2018-07-19 01:08:41,247 INFO     repeater     task nb300_quick starting 0.6 seconds late
2018-07-19 01:08:43,745 INFO     repeater     task nb300_quick output:
/usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice.
  warnings.warn("Mean of empty slice.", RuntimeWarning)

2018-07-19 01:08:43,745 INFO     repeater     task nb300_quick ended after 2.5 seconds
2018-07-19 01:08:43,745 INFO     repeater     task nb300_3dayplots starting 3.1 seconds late
2018-07-19 01:08:47,358 INFO     repeater     task nb300_3dayplots output:
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered
  warnings.warn("All-NaN slice encountered", RuntimeWarning)
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered
  warnings.warn("All-NaN axis encountered", RuntimeWarning)
zgrid failed
zgrid failed: returning all values masked
zgrid failed
zgrid failed: returning all values masked
zgrid failed
zgrid failed: returning all values masked
zgrid failed
zgrid failed: returning all values masked

2018-07-19 01:08:47,359 INFO     repeater     task nb300_3dayplots ended after 3.6 seconds
2018-07-19 01:08:47,359 INFO     repeater     task nb300_tsstats starting 6.7 seconds late
2018-07-19 01:08:48,257 INFO     repeater     task nb300_tsstats ended after 0.9 seconds
2018-07-19 01:08:48,258 INFO     repeater     task nb300_beamplots starting 7.6 seconds late
2018-07-19 01:08:50,162 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 01:08:50,162 INFO     repeater     task nb300_beamplots ended after 1.9 seconds
2018-07-19 01:08:50,162 INFO     repeater     task nb300_beamplots starting 9.5 seconds late
2018-07-19 01:08:52,141 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 01:08:52,141 INFO     repeater     task nb300_beamplots ended after 2.0 seconds
2018-07-19 01:08:52,141 INFO     repeater     task ashtech_hcorrstats starting 11.5 seconds late
2018-07-19 01:08:53,169 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 01:09:53,259 INFO     repeater     task nb300_lastensq starting 3.3 seconds late
2018-07-19 01:09:55,850 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 01:09:55,850 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:09:40
2018-07-19 01:10:00,857 INFO     repeater     task nb300_10pings starting 0.2 seconds late
2018-07-19 01:10:01,040 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:11:51,195 INFO     repeater     task nb300_lastensq starting 1.2 seconds late
2018-07-19 01:11:53,850 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 01:11:53,850 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:11:40
2018-07-19 01:12:23,894 INFO     repeater     task nb300_10pings starting 3.3 seconds late
2018-07-19 01:12:24,080 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:13:44,185 INFO     repeater     task ashtech_hcorrstats starting 3.6 seconds late
2018-07-19 01:13:45,186 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 01:13:50,192 INFO     repeater     task nb300_lastensq starting 0.2 seconds late
2018-07-19 01:13:52,806 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 01:13:52,806 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:13:40
2018-07-19 01:14:42,877 INFO     repeater     task nb300_10pings starting 2.3 seconds late
2018-07-19 01:14:43,059 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:15:53,161 INFO     repeater     task nb300_lastensq starting 3.2 seconds late
2018-07-19 01:15:55,808 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 01:15:55,808 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:15:40
2018-07-19 01:17:00,900 INFO     repeater     task nb300_10pings starting 0.3 seconds late
2018-07-19 01:17:01,081 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:17:51,153 INFO     repeater     task nb300_lastensq starting 1.2 seconds late
2018-07-19 01:17:53,752 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 01:17:53,752 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:17:40
2018-07-19 01:18:43,830 INFO     repeater     task ashtech_hcorrstats starting 3.2 seconds late
2018-07-19 01:18:44,856 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 01:19:24,895 INFO     repeater     task nb300_10pings starting 4.3 seconds late
2018-07-19 01:19:25,123 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:19:50,156 INFO     repeater     task nb300_lastensq starting 0.2 seconds late
2018-07-19 01:19:52,766 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 01:19:52,766 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:19:40
2018-07-19 01:21:42,895 INFO     repeater     task nb300_10pings starting 2.3 seconds late
2018-07-19 01:21:43,078 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:21:53,089 INFO     repeater     task nb300_lastensq starting 3.1 seconds late
2018-07-19 01:21:55,725 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 01:21:55,725 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:21:40
2018-07-19 01:23:40,858 INFO     repeater     task nb300_quick starting 0.2 seconds late
2018-07-19 01:23:43,425 INFO     repeater     task nb300_quick output:
/usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice.
  warnings.warn("Mean of empty slice.", RuntimeWarning)

2018-07-19 01:23:43,425 INFO     repeater     task nb300_quick ended after 2.6 seconds
2018-07-19 01:23:43,426 INFO     repeater     task nb300_3dayplots starting 2.8 seconds late
2018-07-19 01:23:47,059 INFO     repeater     task nb300_3dayplots output:
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered
  warnings.warn("All-NaN slice encountered", RuntimeWarning)
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered
  warnings.warn("All-NaN axis encountered", RuntimeWarning)
zgrid failed
zgrid failed: returning all values masked
zgrid failed
zgrid failed: returning all values masked

2018-07-19 01:23:47,059 INFO     repeater     task nb300_3dayplots ended after 3.6 seconds
2018-07-19 01:23:47,059 INFO     repeater     task nb300_tsstats starting 6.4 seconds late
2018-07-19 01:23:47,959 INFO     repeater     task nb300_tsstats ended after 0.9 seconds
2018-07-19 01:23:47,960 INFO     repeater     task nb300_beamplots starting 7.3 seconds late
2018-07-19 01:23:49,873 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 01:23:49,874 INFO     repeater     task nb300_beamplots ended after 1.9 seconds
2018-07-19 01:23:49,874 INFO     repeater     task nb300_beamplots starting 9.3 seconds late
2018-07-19 01:23:51,922 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 01:23:51,922 INFO     repeater     task nb300_beamplots ended after 2.0 seconds
2018-07-19 01:23:51,922 INFO     repeater     task ashtech_hcorrstats starting 11.3 seconds late
2018-07-19 01:23:52,911 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 01:23:57,917 INFO     repeater     task nb300_lastensq starting 7.9 seconds late
2018-07-19 01:24:00,531 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 01:24:00,531 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 01:24:00,531 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:23:40
2018-07-19 01:24:05,538 INFO     repeater     task nb300_10pings starting 4.9 seconds late
2018-07-19 01:24:05,722 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:25:50,848 INFO     repeater     task nb300_lastensq starting 0.8 seconds late
2018-07-19 01:25:53,505 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 01:25:53,506 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 01:25:53,506 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:25:40
2018-07-19 01:26:23,549 INFO     repeater     task nb300_10pings starting 2.9 seconds late
2018-07-19 01:26:23,733 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:27:53,854 INFO     repeater     task nb300_lastensq starting 3.9 seconds late
2018-07-19 01:27:56,470 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 01:27:56,470 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 01:27:56,471 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:27:40
2018-07-19 01:28:41,536 INFO     repeater     task nb300_10pings starting 0.9 seconds late
2018-07-19 01:28:41,724 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:28:41,725 INFO     repeater     task ashtech_hcorrstats starting 1.1 seconds late
2018-07-19 01:28:42,767 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 01:29:52,850 INFO     repeater     task nb300_lastensq starting 2.9 seconds late
2018-07-19 01:29:55,468 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 01:29:55,469 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:29:40
2018-07-19 01:31:05,570 INFO     repeater     task nb300_10pings starting 5.0 seconds late
2018-07-19 01:31:05,754 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:31:50,823 INFO     repeater     task nb300_lastensq starting 0.8 seconds late
2018-07-19 01:31:53,497 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 01:31:53,498 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:31:40
2018-07-19 01:33:23,594 INFO     repeater     task nb300_10pings starting 3.0 seconds late
2018-07-19 01:33:23,806 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:33:43,834 INFO     repeater     task ashtech_hcorrstats starting 3.2 seconds late
2018-07-19 01:33:44,829 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 01:33:54,840 INFO     repeater     task nb300_lastensq starting 4.8 seconds late
2018-07-19 01:33:57,458 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 01:33:57,458 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 01:33:57,458 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:33:40
2018-07-19 01:35:42,601 INFO     repeater     task nb300_10pings starting 2.0 seconds late
2018-07-19 01:35:42,785 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:35:52,800 INFO     repeater     task nb300_lastensq starting 2.8 seconds late
2018-07-19 01:35:55,454 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 01:35:55,454 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 01:35:55,454 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:35:40
2018-07-19 01:37:50,614 INFO     repeater     task nb300_lastensq starting 0.6 seconds late
2018-07-19 01:37:53,233 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 01:37:53,234 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:37:40
2018-07-19 01:38:03,248 INFO     repeater     task nb300_10pings starting 2.6 seconds late
2018-07-19 01:38:03,430 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:38:43,484 INFO     repeater     task nb300_quick starting 2.9 seconds late
2018-07-19 01:38:46,020 INFO     repeater     task nb300_quick output:
/usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice.
  warnings.warn("Mean of empty slice.", RuntimeWarning)

2018-07-19 01:38:46,020 INFO     repeater     task nb300_quick ended after 2.5 seconds
2018-07-19 01:38:46,021 INFO     repeater     task nb300_3dayplots starting 5.4 seconds late
2018-07-19 01:38:49,611 INFO     repeater     task nb300_3dayplots output:
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered
  warnings.warn("All-NaN slice encountered", RuntimeWarning)
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered
  warnings.warn("All-NaN axis encountered", RuntimeWarning)

2018-07-19 01:38:49,611 INFO     repeater     task nb300_3dayplots ended after 3.6 seconds
2018-07-19 01:38:49,611 INFO     repeater     task nb300_tsstats starting 9.0 seconds late
2018-07-19 01:38:50,556 INFO     repeater     task nb300_tsstats ended after 0.9 seconds
2018-07-19 01:38:50,557 INFO     repeater     task nb300_beamplots starting 9.9 seconds late
2018-07-19 01:38:52,489 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 01:38:52,490 INFO     repeater     task nb300_beamplots ended after 1.9 seconds
2018-07-19 01:38:52,490 INFO     repeater     task nb300_beamplots starting 11.9 seconds late
2018-07-19 01:38:54,480 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 01:38:54,480 INFO     repeater     task nb300_beamplots ended after 2.0 seconds
2018-07-19 01:38:54,481 INFO     repeater     task ashtech_hcorrstats starting 13.9 seconds late
2018-07-19 01:38:55,530 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 01:39:50,608 INFO     repeater     task nb300_lastensq starting 0.6 seconds late
2018-07-19 01:39:53,259 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 01:39:53,259 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 01:39:53,259 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:39:40
2018-07-19 01:40:23,296 INFO     repeater     task nb300_10pings starting 2.7 seconds late
2018-07-19 01:40:23,476 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:41:53,598 INFO     repeater     task nb300_lastensq starting 3.6 seconds late
2018-07-19 01:41:56,229 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 01:41:56,229 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:41:40
2018-07-19 01:42:41,294 INFO     repeater     task nb300_10pings starting 0.7 seconds late
2018-07-19 01:42:41,477 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:43:41,565 INFO     repeater     task ashtech_hcorrstats starting 0.9 seconds late
2018-07-19 01:43:42,564 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 01:43:52,578 INFO     repeater     task nb300_lastensq starting 2.6 seconds late
2018-07-19 01:43:55,164 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 01:43:55,164 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:43:40
2018-07-19 01:45:05,258 INFO     repeater     task nb300_10pings starting 4.6 seconds late
2018-07-19 01:45:05,440 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:45:50,508 INFO     repeater     task nb300_lastensq starting 0.5 seconds late
2018-07-19 01:45:53,178 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 01:45:53,178 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 01:45:53,178 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:45:40
2018-07-19 01:47:23,310 INFO     repeater     task nb300_10pings starting 2.7 seconds late
2018-07-19 01:47:23,495 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:47:53,539 INFO     repeater     task nb300_lastensq starting 3.5 seconds late
2018-07-19 01:47:56,163 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 01:47:56,163 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 01:47:56,163 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:47:40
2018-07-19 01:48:41,233 INFO     repeater     task ashtech_hcorrstats starting 0.6 seconds late
2018-07-19 01:48:42,228 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 01:49:42,309 INFO     repeater     task nb300_10pings starting 1.7 seconds late
2018-07-19 01:49:42,493 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:49:52,507 INFO     repeater     task nb300_lastensq starting 2.5 seconds late
2018-07-19 01:49:55,184 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 01:49:55,184 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 01:49:55,184 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:49:40
2018-07-19 01:51:50,328 INFO     repeater     task nb300_lastensq starting 0.3 seconds late
2018-07-19 01:51:52,935 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 01:51:52,935 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:51:40
2018-07-19 01:52:02,950 INFO     repeater     task nb300_10pings starting 2.3 seconds late
2018-07-19 01:52:03,132 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:53:43,270 INFO     repeater     task nb300_quick starting 2.7 seconds late
2018-07-19 01:53:45,838 INFO     repeater     task nb300_quick output:
/usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice.
  warnings.warn("Mean of empty slice.", RuntimeWarning)

2018-07-19 01:53:45,839 INFO     repeater     task nb300_quick ended after 2.6 seconds
2018-07-19 01:53:45,839 INFO     repeater     task nb300_3dayplots starting 5.2 seconds late
2018-07-19 01:53:49,449 INFO     repeater     task nb300_3dayplots output:
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered
  warnings.warn("All-NaN slice encountered", RuntimeWarning)
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered
  warnings.warn("All-NaN axis encountered", RuntimeWarning)

2018-07-19 01:53:49,449 INFO     repeater     task nb300_3dayplots ended after 3.6 seconds
2018-07-19 01:53:49,450 INFO     repeater     task nb300_tsstats starting 8.8 seconds late
2018-07-19 01:53:50,364 INFO     repeater     task nb300_tsstats ended after 0.9 seconds
2018-07-19 01:53:50,364 INFO     repeater     task nb300_beamplots starting 9.8 seconds late
2018-07-19 01:53:52,252 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 01:53:52,252 INFO     repeater     task nb300_beamplots ended after 1.9 seconds
2018-07-19 01:53:52,252 INFO     repeater     task nb300_beamplots starting 11.6 seconds late
2018-07-19 01:53:54,302 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 01:53:54,303 INFO     repeater     task nb300_beamplots ended after 2.1 seconds
2018-07-19 01:53:54,303 INFO     repeater     task ashtech_hcorrstats starting 13.7 seconds late
2018-07-19 01:53:55,300 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 01:54:00,305 INFO     repeater     task nb300_lastensq starting 10.3 seconds late
2018-07-19 01:54:02,918 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 01:54:02,919 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:53:40
2018-07-19 01:54:22,945 INFO     repeater     task nb300_10pings starting 2.3 seconds late
2018-07-19 01:54:23,125 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:55:53,260 INFO     repeater     task nb300_lastensq starting 3.3 seconds late
2018-07-19 01:55:55,935 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 01:55:55,935 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 01:55:55,935 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:55:40
2018-07-19 01:56:41,002 INFO     repeater     task nb300_10pings starting 0.4 seconds late
2018-07-19 01:56:41,183 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:57:51,274 INFO     repeater     task nb300_lastensq starting 1.3 seconds late
2018-07-19 01:57:53,895 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 01:57:53,896 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:57:40
2018-07-19 01:58:43,968 INFO     repeater     task ashtech_hcorrstats starting 3.4 seconds late
2018-07-19 01:58:44,962 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 01:59:04,991 INFO     repeater     task nb300_10pings starting 4.4 seconds late
2018-07-19 01:59:05,185 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 01:59:50,246 INFO     repeater     task nb300_lastensq starting 0.2 seconds late
2018-07-19 01:59:52,900 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 01:59:52,900 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 01:59:52,900 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 01:59:40
2018-07-19 02:01:23,029 INFO     repeater     task nb300_10pings starting 2.4 seconds late
2018-07-19 02:01:23,210 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:01:53,253 INFO     repeater     task nb300_lastensq starting 3.3 seconds late
2018-07-19 02:01:55,873 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:01:55,874 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:01:40
2018-07-19 02:03:40,996 INFO     repeater     task nb300_10pings starting 0.4 seconds late
2018-07-19 02:03:41,179 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:03:41,180 INFO     repeater     task ashtech_hcorrstats starting 0.6 seconds late
2018-07-19 02:03:42,231 INFO     repeater     task ashtech_hcorrstats ended after 1.1 seconds
2018-07-19 02:03:52,244 INFO     repeater     task nb300_lastensq starting 2.2 seconds late
2018-07-19 02:03:54,802 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:03:54,803 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:03:40
2018-07-19 02:05:54,971 INFO     repeater     task nb300_lastensq starting 5.0 seconds late
2018-07-19 02:05:57,603 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:05:57,603 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:05:40
2018-07-19 02:06:02,610 INFO     repeater     task nb300_10pings starting 2.0 seconds late
2018-07-19 02:06:02,793 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:07:52,955 INFO     repeater     task nb300_lastensq starting 3.0 seconds late
2018-07-19 02:07:55,508 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:07:55,508 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:07:40
2018-07-19 02:08:25,546 INFO     repeater     task nb300_10pings starting 4.9 seconds late
2018-07-19 02:08:25,726 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:08:40,745 INFO     repeater     task nb300_quick starting 0.1 seconds late
2018-07-19 02:08:43,307 INFO     repeater     task nb300_quick output:
/usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice.
  warnings.warn("Mean of empty slice.", RuntimeWarning)

2018-07-19 02:08:43,307 INFO     repeater     task nb300_quick ended after 2.6 seconds
2018-07-19 02:08:43,308 INFO     repeater     task nb300_3dayplots starting 2.7 seconds late
2018-07-19 02:08:46,900 INFO     repeater     task nb300_3dayplots output:
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered
  warnings.warn("All-NaN slice encountered", RuntimeWarning)
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered
  warnings.warn("All-NaN axis encountered", RuntimeWarning)

2018-07-19 02:08:46,901 INFO     repeater     task nb300_3dayplots ended after 3.6 seconds
2018-07-19 02:08:46,901 INFO     repeater     task nb300_tsstats starting 6.3 seconds late
2018-07-19 02:08:47,837 INFO     repeater     task nb300_tsstats ended after 0.9 seconds
2018-07-19 02:08:47,838 INFO     repeater     task nb300_beamplots starting 7.2 seconds late
2018-07-19 02:08:49,783 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 02:08:49,784 INFO     repeater     task nb300_beamplots ended after 1.9 seconds
2018-07-19 02:08:49,784 INFO     repeater     task nb300_beamplots starting 9.2 seconds late
2018-07-19 02:08:51,768 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 02:08:51,768 INFO     repeater     task nb300_beamplots ended after 2.0 seconds
2018-07-19 02:08:51,769 INFO     repeater     task ashtech_hcorrstats starting 11.2 seconds late
2018-07-19 02:08:52,785 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 02:09:52,859 INFO     repeater     task nb300_lastensq starting 2.9 seconds late
2018-07-19 02:09:55,458 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:09:55,458 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:09:40
2018-07-19 02:10:45,530 INFO     repeater     task nb300_10pings starting 4.9 seconds late
2018-07-19 02:10:45,716 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:11:50,797 INFO     repeater     task nb300_lastensq starting 0.8 seconds late
2018-07-19 02:11:53,374 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:11:53,374 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:11:40
2018-07-19 02:13:03,471 INFO     repeater     task nb300_10pings starting 2.9 seconds late
2018-07-19 02:13:03,651 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:13:43,707 INFO     repeater     task ashtech_hcorrstats starting 3.1 seconds late
2018-07-19 02:13:44,763 INFO     repeater     task ashtech_hcorrstats ended after 1.1 seconds
2018-07-19 02:13:54,776 INFO     repeater     task nb300_lastensq starting 4.8 seconds late
2018-07-19 02:13:57,363 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:13:57,363 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:13:40
2018-07-19 02:15:22,491 INFO     repeater     task nb300_10pings starting 1.9 seconds late
2018-07-19 02:15:22,674 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:15:52,718 INFO     repeater     task nb300_lastensq starting 2.7 seconds late
2018-07-19 02:15:55,334 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 02:15:55,334 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:15:55,334 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:15:40
2018-07-19 02:17:45,492 INFO     repeater     task nb300_10pings starting 4.9 seconds late
2018-07-19 02:17:45,674 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:17:50,678 INFO     repeater     task nb300_lastensq starting 0.7 seconds late
2018-07-19 02:17:53,249 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:17:53,249 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:17:40
2018-07-19 02:18:43,319 INFO     repeater     task ashtech_hcorrstats starting 2.7 seconds late
2018-07-19 02:18:44,316 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 02:19:54,414 INFO     repeater     task nb300_lastensq starting 4.4 seconds late
2018-07-19 02:19:57,010 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:19:57,010 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:19:40
2018-07-19 02:20:02,017 INFO     repeater     task nb300_10pings starting 1.4 seconds late
2018-07-19 02:20:02,202 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:21:52,342 INFO     repeater     task nb300_lastensq starting 2.3 seconds late
2018-07-19 02:21:54,941 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 02:21:54,941 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:21:54,941 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:21:40
2018-07-19 02:22:24,986 INFO     repeater     task nb300_10pings starting 4.4 seconds late
2018-07-19 02:22:25,172 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:23:45,282 INFO     repeater     task nb300_quick starting 4.7 seconds late
2018-07-19 02:23:47,835 INFO     repeater     task nb300_quick output:
/usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice.
  warnings.warn("Mean of empty slice.", RuntimeWarning)

2018-07-19 02:23:47,835 INFO     repeater     task nb300_quick ended after 2.6 seconds
2018-07-19 02:23:47,836 INFO     repeater     task nb300_3dayplots starting 7.2 seconds late
2018-07-19 02:23:51,446 INFO     repeater     task nb300_3dayplots output:
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered
  warnings.warn("All-NaN slice encountered", RuntimeWarning)
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered
  warnings.warn("All-NaN axis encountered", RuntimeWarning)

2018-07-19 02:23:51,447 INFO     repeater     task nb300_3dayplots ended after 3.6 seconds
2018-07-19 02:23:51,447 INFO     repeater     task nb300_tsstats starting 10.8 seconds late
2018-07-19 02:23:52,402 INFO     repeater     task nb300_tsstats ended after 1.0 seconds
2018-07-19 02:23:52,402 INFO     repeater     task nb300_beamplots starting 11.8 seconds late
2018-07-19 02:23:54,304 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 02:23:54,304 INFO     repeater     task nb300_beamplots ended after 1.9 seconds
2018-07-19 02:23:54,305 INFO     repeater     task nb300_beamplots starting 13.7 seconds late
2018-07-19 02:23:56,227 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 02:23:56,227 INFO     repeater     task nb300_beamplots ended after 1.9 seconds
2018-07-19 02:23:56,227 INFO     repeater     task ashtech_hcorrstats starting 15.6 seconds late
2018-07-19 02:23:57,281 INFO     repeater     task ashtech_hcorrstats ended after 1.1 seconds
2018-07-19 02:24:02,287 INFO     repeater     task nb300_lastensq starting 12.3 seconds late
2018-07-19 02:24:04,880 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:24:04,880 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:23:40
2018-07-19 02:24:44,937 INFO     repeater     task nb300_10pings starting 4.3 seconds late
2018-07-19 02:24:45,122 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:25:50,215 INFO     repeater     task nb300_lastensq starting 0.2 seconds late
2018-07-19 02:25:52,853 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 02:25:52,853 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:25:52,853 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:25:40
2018-07-19 02:27:02,965 INFO     repeater     task nb300_10pings starting 2.4 seconds late
2018-07-19 02:27:03,148 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:27:53,218 INFO     repeater     task nb300_lastensq starting 3.2 seconds late
2018-07-19 02:27:55,776 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 02:27:55,776 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:27:55,776 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:27:40
2018-07-19 02:28:40,837 INFO     repeater     task ashtech_hcorrstats starting 0.2 seconds late
2018-07-19 02:28:41,849 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 02:29:21,901 INFO     repeater     task nb300_10pings starting 1.3 seconds late
2018-07-19 02:29:22,086 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:29:52,123 INFO     repeater     task nb300_lastensq starting 2.1 seconds late
2018-07-19 02:29:54,775 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 02:29:54,776 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 02:29:54,776 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:29:40
2018-07-19 02:31:44,934 INFO     repeater     task nb300_10pings starting 4.3 seconds late
2018-07-19 02:31:45,118 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:31:50,125 INFO     repeater     task nb300_lastensq starting 0.1 seconds late
2018-07-19 02:31:52,728 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 02:31:52,728 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:31:52,728 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:31:40
2018-07-19 02:33:42,886 INFO     repeater     task ashtech_hcorrstats starting 2.3 seconds late
2018-07-19 02:33:43,891 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 02:33:53,904 INFO     repeater     task nb300_lastensq starting 3.9 seconds late
2018-07-19 02:33:56,544 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 02:33:56,544 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:33:56,545 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:33:40
2018-07-19 02:34:01,550 INFO     repeater     task nb300_10pings starting 0.9 seconds late
2018-07-19 02:34:01,731 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:35:51,897 INFO     repeater     task nb300_lastensq starting 1.9 seconds late
2018-07-19 02:35:54,480 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 02:35:54,481 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:35:54,481 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:35:40
2018-07-19 02:36:24,522 INFO     repeater     task nb300_10pings starting 3.9 seconds late
2018-07-19 02:36:24,706 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:37:54,814 INFO     repeater     task nb300_lastensq starting 4.8 seconds late
2018-07-19 02:37:57,447 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 02:37:57,448 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:37:57,448 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:37:40
2018-07-19 02:38:42,508 INFO     repeater     task nb300_10pings starting 1.9 seconds late
2018-07-19 02:38:42,693 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:38:42,693 INFO     repeater     task nb300_quick starting 2.1 seconds late
2018-07-19 02:38:45,220 INFO     repeater     task nb300_quick output:
/usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice.
  warnings.warn("Mean of empty slice.", RuntimeWarning)

2018-07-19 02:38:45,220 INFO     repeater     task nb300_quick ended after 2.5 seconds
2018-07-19 02:38:45,220 INFO     repeater     task nb300_3dayplots starting 4.6 seconds late
2018-07-19 02:38:48,872 INFO     repeater     task nb300_3dayplots output:
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered
  warnings.warn("All-NaN slice encountered", RuntimeWarning)
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered
  warnings.warn("All-NaN axis encountered", RuntimeWarning)

2018-07-19 02:38:48,872 INFO     repeater     task nb300_3dayplots ended after 3.7 seconds
2018-07-19 02:38:48,873 INFO     repeater     task nb300_tsstats starting 8.3 seconds late
2018-07-19 02:38:49,786 INFO     repeater     task nb300_tsstats ended after 0.9 seconds
2018-07-19 02:38:49,787 INFO     repeater     task nb300_beamplots starting 9.2 seconds late
2018-07-19 02:38:51,732 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 02:38:51,732 INFO     repeater     task nb300_beamplots ended after 1.9 seconds
2018-07-19 02:38:51,732 INFO     repeater     task nb300_beamplots starting 11.1 seconds late
2018-07-19 02:38:53,701 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 02:38:53,701 INFO     repeater     task nb300_beamplots ended after 2.0 seconds
2018-07-19 02:38:53,702 INFO     repeater     task ashtech_hcorrstats starting 13.1 seconds late
2018-07-19 02:38:54,706 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 02:39:54,792 INFO     repeater     task nb300_lastensq starting 4.8 seconds late
2018-07-19 02:39:57,436 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 02:39:57,436 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:39:57,436 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:39:40
2018-07-19 02:41:02,535 INFO     repeater     task nb300_10pings starting 1.9 seconds late
2018-07-19 02:41:02,719 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:41:52,794 INFO     repeater     task nb300_lastensq starting 2.8 seconds late
2018-07-19 02:41:55,390 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 02:41:55,390 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:41:55,390 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:41:40
2018-07-19 02:43:25,517 INFO     repeater     task nb300_10pings starting 4.9 seconds late
2018-07-19 02:43:25,698 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:43:40,718 INFO     repeater     task ashtech_hcorrstats starting 0.1 seconds late
2018-07-19 02:43:41,742 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 02:43:51,753 INFO     repeater     task nb300_lastensq starting 1.8 seconds late
2018-07-19 02:43:54,390 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 02:43:54,390 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:43:54,391 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:43:40
2018-07-19 02:45:44,547 INFO     repeater     task nb300_10pings starting 3.9 seconds late
2018-07-19 02:45:44,731 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:45:54,746 INFO     repeater     task nb300_lastensq starting 4.7 seconds late
2018-07-19 02:45:57,368 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 02:45:57,368 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:45:57,369 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:45:40
2018-07-19 02:47:52,523 INFO     repeater     task nb300_lastensq starting 2.5 seconds late
2018-07-19 02:47:55,178 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 02:47:55,178 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 02:47:55,179 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:47:40
2018-07-19 02:48:05,193 INFO     repeater     task nb300_10pings starting 4.6 seconds late
2018-07-19 02:48:05,375 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:48:45,421 INFO     repeater     task ashtech_hcorrstats starting 4.8 seconds late
2018-07-19 02:48:46,424 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 02:49:51,492 INFO     repeater     task nb300_lastensq starting 1.5 seconds late
2018-07-19 02:49:54,138 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 02:49:54,138 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:49:54,138 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:49:40
2018-07-19 02:50:24,180 INFO     repeater     task nb300_10pings starting 3.6 seconds late
2018-07-19 02:50:24,361 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:51:54,470 INFO     repeater     task nb300_lastensq starting 4.5 seconds late
2018-07-19 02:51:57,062 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 02:51:57,062 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:51:57,062 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:51:40
2018-07-19 02:52:42,125 INFO     repeater     task nb300_10pings starting 1.5 seconds late
2018-07-19 02:52:42,310 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:53:42,395 INFO     repeater     task nb300_quick starting 1.8 seconds late
2018-07-19 02:53:44,991 INFO     repeater     task nb300_quick output:
/usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice.
  warnings.warn("Mean of empty slice.", RuntimeWarning)

2018-07-19 02:53:44,991 INFO     repeater     task nb300_quick ended after 2.6 seconds
2018-07-19 02:53:44,992 INFO     repeater     task nb300_3dayplots starting 4.4 seconds late
2018-07-19 02:53:48,595 INFO     repeater     task nb300_3dayplots output:
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered
  warnings.warn("All-NaN slice encountered", RuntimeWarning)
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered
  warnings.warn("All-NaN axis encountered", RuntimeWarning)

2018-07-19 02:53:48,595 INFO     repeater     task nb300_3dayplots ended after 3.6 seconds
2018-07-19 02:53:48,596 INFO     repeater     task nb300_tsstats starting 8.0 seconds late
2018-07-19 02:53:49,542 INFO     repeater     task nb300_tsstats ended after 0.9 seconds
2018-07-19 02:53:49,543 INFO     repeater     task nb300_beamplots starting 8.9 seconds late
2018-07-19 02:53:51,475 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 02:53:51,475 INFO     repeater     task nb300_beamplots ended after 1.9 seconds
2018-07-19 02:53:51,476 INFO     repeater     task nb300_beamplots starting 10.9 seconds late
2018-07-19 02:53:53,411 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 02:53:53,411 INFO     repeater     task nb300_beamplots ended after 1.9 seconds
2018-07-19 02:53:53,412 INFO     repeater     task ashtech_hcorrstats starting 12.8 seconds late
2018-07-19 02:53:54,440 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 02:53:59,443 INFO     repeater     task nb300_lastensq starting 9.4 seconds late
2018-07-19 02:54:02,078 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 02:54:02,078 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:54:02,078 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:53:40
2018-07-19 02:55:02,165 INFO     repeater     task nb300_10pings starting 1.6 seconds late
2018-07-19 02:55:02,350 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:55:52,418 INFO     repeater     task nb300_lastensq starting 2.4 seconds late
2018-07-19 02:55:55,063 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 02:55:55,063 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:55:55,063 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:55:40
2018-07-19 02:57:25,176 INFO     repeater     task nb300_10pings starting 4.6 seconds late
2018-07-19 02:57:25,386 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:57:50,418 INFO     repeater     task nb300_lastensq starting 0.4 seconds late
2018-07-19 02:57:53,070 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 02:57:53,071 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 02:57:53,071 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:57:40
2018-07-19 02:58:43,144 INFO     repeater     task ashtech_hcorrstats starting 2.5 seconds late
2018-07-19 02:58:44,155 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 02:59:44,232 INFO     repeater     task nb300_10pings starting 3.6 seconds late
2018-07-19 02:59:44,413 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 02:59:54,421 INFO     repeater     task nb300_lastensq starting 4.4 seconds late
2018-07-19 02:59:57,059 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 02:59:57,059 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 02:59:57,059 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 02:59:40
2018-07-19 03:01:52,217 INFO     repeater     task nb300_lastensq starting 2.2 seconds late
2018-07-19 03:01:54,823 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:01:54,824 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 03:01:54,824 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:01:40
2018-07-19 03:02:04,837 INFO     repeater     task nb300_10pings starting 4.2 seconds late
2018-07-19 03:02:05,017 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 03:03:45,164 INFO     repeater     task ashtech_hcorrstats starting 4.5 seconds late
2018-07-19 03:03:46,182 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 03:03:51,188 INFO     repeater     task nb300_lastensq starting 1.2 seconds late
2018-07-19 03:03:53,864 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:03:53,864 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 03:03:53,864 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:03:40
2018-07-19 03:04:23,904 INFO     repeater     task nb300_10pings starting 3.3 seconds late
2018-07-19 03:04:24,085 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 03:05:54,206 INFO     repeater     task nb300_lastensq starting 4.2 seconds late
2018-07-19 03:05:56,814 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:05:56,814 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 03:05:56,814 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:05:40
2018-07-19 03:06:41,871 INFO     repeater     task nb300_10pings starting 1.3 seconds late
2018-07-19 03:06:42,049 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 03:07:52,141 INFO     repeater     task nb300_lastensq starting 2.1 seconds late
2018-07-19 03:07:54,805 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:07:54,805 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 03:07:54,805 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:07:40
2018-07-19 03:08:44,880 INFO     repeater     task nb300_quick starting 4.3 seconds late
2018-07-19 03:08:47,419 INFO     repeater     task nb300_quick output:
/usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice.
  warnings.warn("Mean of empty slice.", RuntimeWarning)

2018-07-19 03:08:47,419 INFO     repeater     task nb300_quick ended after 2.5 seconds
2018-07-19 03:08:47,419 INFO     repeater     task nb300_3dayplots starting 6.8 seconds late
2018-07-19 03:08:51,048 INFO     repeater     task nb300_3dayplots output:
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered
  warnings.warn("All-NaN slice encountered", RuntimeWarning)
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered
  warnings.warn("All-NaN axis encountered", RuntimeWarning)

2018-07-19 03:08:51,048 INFO     repeater     task nb300_3dayplots ended after 3.6 seconds
2018-07-19 03:08:51,048 INFO     repeater     task nb300_tsstats starting 10.4 seconds late
2018-07-19 03:08:51,960 INFO     repeater     task nb300_tsstats ended after 0.9 seconds
2018-07-19 03:08:51,960 INFO     repeater     task nb300_beamplots starting 11.3 seconds late
2018-07-19 03:08:53,922 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 03:08:53,923 INFO     repeater     task nb300_beamplots ended after 2.0 seconds
2018-07-19 03:08:53,923 INFO     repeater     task nb300_beamplots starting 13.3 seconds late
2018-07-19 03:08:55,911 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 03:08:55,911 INFO     repeater     task nb300_beamplots ended after 2.0 seconds
2018-07-19 03:08:55,912 INFO     repeater     task ashtech_hcorrstats starting 15.3 seconds late
2018-07-19 03:08:56,936 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 03:09:01,939 INFO     repeater     task nb300_10pings starting 1.3 seconds late
2018-07-19 03:09:02,123 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 03:09:52,196 INFO     repeater     task nb300_lastensq starting 2.2 seconds late
2018-07-19 03:09:54,864 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:09:54,864 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 03:09:54,864 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:09:40
2018-07-19 03:11:24,995 INFO     repeater     task nb300_10pings starting 4.4 seconds late
2018-07-19 03:11:25,175 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 03:11:50,213 INFO     repeater     task nb300_lastensq starting 0.2 seconds late
2018-07-19 03:11:52,831 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:11:52,831 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 03:11:52,832 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:11:40
2018-07-19 03:13:42,994 INFO     repeater     task nb300_10pings starting 2.4 seconds late
2018-07-19 03:13:43,178 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 03:13:43,179 INFO     repeater     task ashtech_hcorrstats starting 2.6 seconds late
2018-07-19 03:13:44,172 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 03:13:54,185 INFO     repeater     task nb300_lastensq starting 4.2 seconds late
2018-07-19 03:13:56,866 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:13:56,867 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 03:13:56,867 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:13:40
2018-07-19 03:15:52,032 INFO     repeater     task nb300_lastensq starting 2.0 seconds late
2018-07-19 03:15:54,642 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:15:54,642 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 03:15:54,642 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:15:40
2018-07-19 03:16:04,656 INFO     repeater     task nb300_10pings starting 4.0 seconds late
2018-07-19 03:16:04,840 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 03:17:54,988 INFO     repeater     task nb300_lastensq starting 5.0 seconds late
2018-07-19 03:17:57,644 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:17:57,644 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 03:17:57,644 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:17:40
2018-07-19 03:18:22,682 INFO     repeater     task nb300_10pings starting 2.1 seconds late
2018-07-19 03:18:22,862 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 03:18:42,893 INFO     repeater     task ashtech_hcorrstats starting 2.3 seconds late
2018-07-19 03:18:43,900 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 03:19:54,001 INFO     repeater     task nb300_lastensq starting 4.0 seconds late
2018-07-19 03:19:56,658 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:19:56,658 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 03:19:56,658 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:19:40
2018-07-19 03:20:41,723 INFO     repeater     task nb300_10pings starting 1.1 seconds late
2018-07-19 03:20:41,903 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 03:21:51,981 INFO     repeater     task nb300_lastensq starting 2.0 seconds late
2018-07-19 03:21:54,595 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:21:54,595 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 03:21:54,595 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:21:40
2018-07-19 03:23:04,699 INFO     repeater     task nb300_10pings starting 4.1 seconds late
2018-07-19 03:23:04,881 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 03:23:44,939 INFO     repeater     task nb300_quick starting 4.3 seconds late
2018-07-19 03:23:47,535 INFO     repeater     task nb300_quick output:
/usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice.
  warnings.warn("Mean of empty slice.", RuntimeWarning)

2018-07-19 03:23:47,535 INFO     repeater     task nb300_quick ended after 2.6 seconds
2018-07-19 03:23:47,536 INFO     repeater     task nb300_3dayplots starting 6.9 seconds late
2018-07-19 03:23:51,140 INFO     repeater     task nb300_3dayplots output:
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered
  warnings.warn("All-NaN slice encountered", RuntimeWarning)
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered
  warnings.warn("All-NaN axis encountered", RuntimeWarning)

2018-07-19 03:23:51,140 INFO     repeater     task nb300_3dayplots ended after 3.6 seconds
2018-07-19 03:23:51,141 INFO     repeater     task nb300_tsstats starting 10.5 seconds late
2018-07-19 03:23:52,052 INFO     repeater     task nb300_tsstats ended after 0.9 seconds
2018-07-19 03:23:52,053 INFO     repeater     task nb300_beamplots starting 11.4 seconds late
2018-07-19 03:23:53,974 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 03:23:53,974 INFO     repeater     task nb300_beamplots ended after 1.9 seconds
2018-07-19 03:23:53,975 INFO     repeater     task nb300_beamplots starting 13.4 seconds late
2018-07-19 03:23:55,965 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 03:23:55,965 INFO     repeater     task nb300_beamplots ended after 2.0 seconds
2018-07-19 03:23:55,965 INFO     repeater     task ashtech_hcorrstats starting 15.3 seconds late
2018-07-19 03:23:56,964 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 03:24:01,965 INFO     repeater     task nb300_lastensq starting 12.0 seconds late
2018-07-19 03:24:04,633 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:24:04,634 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 03:24:04,634 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:23:40
2018-07-19 03:25:24,750 INFO     repeater     task nb300_10pings starting 4.1 seconds late
2018-07-19 03:25:24,932 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 03:25:54,977 INFO     repeater     task nb300_lastensq starting 5.0 seconds late
2018-07-19 03:25:57,599 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:25:57,599 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 03:25:57,600 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:25:40
2018-07-19 03:27:42,753 INFO     repeater     task nb300_10pings starting 2.1 seconds late
2018-07-19 03:27:42,934 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 03:27:52,948 INFO     repeater     task nb300_lastensq starting 2.9 seconds late
2018-07-19 03:27:55,598 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:27:55,598 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 03:27:55,599 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:27:40
2018-07-19 03:28:40,668 INFO     repeater     task ashtech_hcorrstats starting 0.1 seconds late
2018-07-19 03:28:41,667 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 03:29:51,767 INFO     repeater     task nb300_lastensq starting 1.8 seconds late
2018-07-19 03:29:54,442 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:29:54,442 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 03:29:54,442 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:29:40
2018-07-19 03:30:04,455 INFO     repeater     task nb300_10pings starting 3.8 seconds late
2018-07-19 03:30:04,642 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 03:31:54,790 INFO     repeater     task nb300_lastensq starting 4.8 seconds late
2018-07-19 03:31:57,406 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:31:57,406 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 03:31:57,407 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:31:40
2018-07-19 03:32:22,440 INFO     repeater     task nb300_10pings starting 1.8 seconds late
2018-07-19 03:32:22,625 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 03:33:42,735 INFO     repeater     task ashtech_hcorrstats starting 2.1 seconds late
2018-07-19 03:33:43,742 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 03:33:53,755 INFO     repeater     task nb300_lastensq starting 3.8 seconds late
2018-07-19 03:33:56,437 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:33:56,437 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 03:33:56,438 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:33:40
2018-07-19 03:34:41,494 INFO     repeater     task nb300_10pings starting 0.9 seconds late
2018-07-19 03:34:41,675 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 03:35:51,769 INFO     repeater     task nb300_lastensq starting 1.8 seconds late
2018-07-19 03:35:54,375 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:35:54,375 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 03:35:54,375 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:35:40
2018-07-19 03:37:04,477 INFO     repeater     task nb300_10pings starting 3.9 seconds late
2018-07-19 03:37:04,661 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 03:37:54,732 INFO     repeater     task nb300_lastensq starting 4.7 seconds late
2018-07-19 03:37:57,421 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:37:57,421 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 03:37:57,421 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:37:40
2018-07-19 03:38:42,483 INFO     repeater     task nb300_quick starting 1.9 seconds late
2018-07-19 03:38:44,968 INFO     repeater     task nb300_quick output:
/usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice.
  warnings.warn("Mean of empty slice.", RuntimeWarning)

2018-07-19 03:38:44,968 INFO     repeater     task nb300_quick ended after 2.5 seconds
2018-07-19 03:38:44,968 INFO     repeater     task nb300_3dayplots starting 4.4 seconds late
2018-07-19 03:38:48,602 INFO     repeater     task nb300_3dayplots output:
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered
  warnings.warn("All-NaN slice encountered", RuntimeWarning)
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered
  warnings.warn("All-NaN axis encountered", RuntimeWarning)

2018-07-19 03:38:48,602 INFO     repeater     task nb300_3dayplots ended after 3.6 seconds
2018-07-19 03:38:48,603 INFO     repeater     task nb300_tsstats starting 8.0 seconds late
2018-07-19 03:38:49,459 INFO     repeater     task nb300_tsstats ended after 0.9 seconds
2018-07-19 03:38:49,460 INFO     repeater     task nb300_beamplots starting 8.8 seconds late
2018-07-19 03:38:51,424 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 03:38:51,424 INFO     repeater     task nb300_beamplots ended after 2.0 seconds
2018-07-19 03:38:51,425 INFO     repeater     task nb300_beamplots starting 10.8 seconds late
2018-07-19 03:38:53,420 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 03:38:53,420 INFO     repeater     task nb300_beamplots ended after 2.0 seconds
2018-07-19 03:38:53,420 INFO     repeater     task ashtech_hcorrstats starting 12.8 seconds late
2018-07-19 03:38:54,489 INFO     repeater     task ashtech_hcorrstats ended after 1.1 seconds
2018-07-19 03:39:24,533 INFO     repeater     task nb300_10pings starting 3.9 seconds late
2018-07-19 03:39:24,718 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 03:39:54,763 INFO     repeater     task nb300_lastensq starting 4.8 seconds late
2018-07-19 03:39:57,421 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:39:57,421 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 03:39:57,422 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:39:40
2018-07-19 03:41:42,570 INFO     repeater     task nb300_10pings starting 2.0 seconds late
2018-07-19 03:41:42,753 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 03:41:52,767 INFO     repeater     task nb300_lastensq starting 2.8 seconds late
2018-07-19 03:41:55,386 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:41:55,386 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 03:41:55,387 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:41:40
2018-07-19 03:43:45,547 INFO     repeater     task ashtech_hcorrstats starting 4.9 seconds late
2018-07-19 03:43:46,554 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 03:43:51,560 INFO     repeater     task nb300_lastensq starting 1.6 seconds late
2018-07-19 03:43:54,244 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:43:54,244 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 03:43:54,244 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:43:40
2018-07-19 03:44:04,256 INFO     repeater     task nb300_10pings starting 3.6 seconds late
2018-07-19 03:44:04,437 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 03:45:54,604 INFO     repeater     task nb300_lastensq starting 4.6 seconds late
2018-07-19 03:45:57,223 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:45:57,223 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 03:45:57,223 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:45:40
2018-07-19 03:46:22,253 INFO     repeater     task nb300_10pings starting 1.6 seconds late
2018-07-19 03:46:22,438 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 03:47:52,563 INFO     repeater     task nb300_lastensq starting 2.6 seconds late
2018-07-19 03:47:55,261 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:47:55,261 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 03:47:55,262 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:47:40
2018-07-19 03:48:45,337 INFO     repeater     task nb300_10pings starting 4.7 seconds late
2018-07-19 03:48:45,522 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 03:48:45,523 INFO     repeater     task ashtech_hcorrstats starting 4.9 seconds late
2018-07-19 03:48:46,523 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 03:49:51,617 INFO     repeater     task nb300_lastensq starting 1.6 seconds late
2018-07-19 03:49:54,289 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:49:54,290 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 03:49:54,290 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:49:40
2018-07-19 03:51:04,386 INFO     repeater     task nb300_10pings starting 3.8 seconds late
2018-07-19 03:51:04,570 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 03:51:54,639 INFO     repeater     task nb300_lastensq starting 4.6 seconds late
2018-07-19 03:51:57,264 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:51:57,264 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 03:51:57,265 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:51:40
2018-07-19 03:53:22,385 INFO     repeater     task nb300_10pings starting 1.8 seconds late
2018-07-19 03:53:22,567 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 03:53:42,597 INFO     repeater     task nb300_quick starting 2.0 seconds late
2018-07-19 03:53:45,105 INFO     repeater     task nb300_quick output:
/usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice.
  warnings.warn("Mean of empty slice.", RuntimeWarning)

2018-07-19 03:53:45,105 INFO     repeater     task nb300_quick ended after 2.5 seconds
2018-07-19 03:53:45,105 INFO     repeater     task nb300_3dayplots starting 4.5 seconds late
2018-07-19 03:53:48,702 INFO     repeater     task nb300_3dayplots output:
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered
  warnings.warn("All-NaN slice encountered", RuntimeWarning)
/usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered
  warnings.warn("All-NaN axis encountered", RuntimeWarning)

2018-07-19 03:53:48,702 INFO     repeater     task nb300_3dayplots ended after 3.6 seconds
2018-07-19 03:53:48,703 INFO     repeater     task nb300_tsstats starting 8.1 seconds late
2018-07-19 03:53:49,566 INFO     repeater     task nb300_tsstats ended after 0.9 seconds
2018-07-19 03:53:49,566 INFO     repeater     task nb300_beamplots starting 9.0 seconds late
2018-07-19 03:53:51,511 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 03:53:51,512 INFO     repeater     task nb300_beamplots ended after 1.9 seconds
2018-07-19 03:53:51,512 INFO     repeater     task nb300_beamplots starting 10.9 seconds late
2018-07-19 03:53:53,447 INFO     repeater     task nb300_beamplots output:
Starting procsetup
scriptpath is /usr/local/currents/bin

2018-07-19 03:53:53,447 INFO     repeater     task nb300_beamplots ended after 1.9 seconds
2018-07-19 03:53:53,448 INFO     repeater     task ashtech_hcorrstats starting 12.8 seconds late
2018-07-19 03:53:54,454 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 03:53:59,459 INFO     repeater     task nb300_lastensq starting 9.5 seconds late
2018-07-19 03:54:02,164 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:54:02,164 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 03:54:02,164 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:53:40
2018-07-19 03:55:42,301 INFO     repeater     task nb300_10pings starting 1.7 seconds late
2018-07-19 03:55:42,483 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 03:55:52,498 INFO     repeater     task nb300_lastensq starting 2.5 seconds late
2018-07-19 03:55:55,121 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:55:55,121 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 03:55:55,122 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:55:40
2018-07-19 03:57:50,289 INFO     repeater     task nb300_lastensq starting 0.3 seconds late
2018-07-19 03:57:52,981 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:57:52,981 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 03:57:52,981 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:57:40
2018-07-19 03:58:02,996 INFO     repeater     task nb300_10pings starting 2.4 seconds late
2018-07-19 03:58:03,180 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 03:58:43,249 INFO     repeater     task ashtech_hcorrstats starting 2.6 seconds late
2018-07-19 03:58:44,257 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 03:59:54,357 INFO     repeater     task nb300_lastensq starting 4.4 seconds late
2018-07-19 03:59:57,040 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 03:59:57,041 INFO     repeater     task nb300_lastensq ended after 2.7 seconds
2018-07-19 03:59:57,041 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 03:59:40
2018-07-19 04:00:22,078 INFO     repeater     task nb300_10pings starting 1.5 seconds late
2018-07-19 04:00:22,259 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 04:01:52,391 INFO     repeater     task nb300_lastensq starting 2.4 seconds late
2018-07-19 04:01:55,029 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 04:01:55,029 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 04:01:55,030 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 04:01:40
2018-07-19 04:02:45,101 INFO     repeater     task nb300_10pings starting 4.5 seconds late
2018-07-19 04:02:45,285 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 04:03:45,371 INFO     repeater     task ashtech_hcorrstats starting 4.8 seconds late
2018-07-19 04:03:46,375 INFO     repeater     task ashtech_hcorrstats ended after 1.0 seconds
2018-07-19 04:03:51,381 INFO     repeater     task nb300_lastensq starting 1.4 seconds late
2018-07-19 04:03:54,019 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 04:03:54,020 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 04:03:54,020 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 04:03:40
2018-07-19 04:05:04,107 INFO     repeater     task nb300_10pings starting 3.5 seconds late
2018-07-19 04:05:04,289 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 04:05:54,363 INFO     repeater     task nb300_lastensq starting 4.4 seconds late
2018-07-19 04:05:56,939 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 04:05:56,940 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 04:05:56,940 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 04:05:40
2018-07-19 04:07:22,038 INFO     repeater     task nb300_10pings starting 1.4 seconds late
2018-07-19 04:07:22,223 INFO     repeater     task nb300_10pings ended after 0.2 seconds
2018-07-19 04:07:52,267 INFO     repeater     task nb300_lastensq starting 2.3 seconds late
2018-07-19 04:07:54,901 INFO     repeater     task nb300_lastensq output:
/usr/lib/python2.7/dist-packages/numpy/ma/core.py:3847: UserWarning: Warning: converting a masked element to nan.
  warnings.warn("Warning: converting a masked element to nan.")

2018-07-19 04:07:54,902 INFO     repeater     task nb300_lastensq ended after 2.6 seconds
2018-07-19 04:07:54,902 INFO     repeater     task nb300_lastensq: Time from /home/data/SP1819/proc/nb300/load/wrote_ens_time: 2018/07/19 04:07:40
2018-07-19 04:08:44,973 INFO     repeater     task nb300_quick starting 4.4 seconds late