2017-05-13 01:52:00,223 INFO repeater task os75nb_beamplots starting 38.1 seconds late 2017-05-13 01:52:01,019 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-05-13 01:52:01,019 INFO repeater task os75nb_beamplots ended after 0.8 seconds 2017-05-13 01:52:01,019 INFO repeater task phins_hcorrstats starting 38.9 seconds late 2017-05-13 01:52:01,746 INFO repeater task phins_hcorrstats ended after 0.7 seconds 2017-05-13 01:52:01,747 INFO repeater task phins2_hcorrstats starting 39.6 seconds late 2017-05-13 01:52:02,486 INFO repeater task phins2_hcorrstats ended after 0.7 seconds 2017-05-13 01:52:07,492 INFO repeater task wh300_lastensq starting 36.5 seconds late 2017-05-13 01:52:08,996 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 01:52:08,996 INFO repeater task wh300_lastensq ended after 1.5 seconds 2017-05-13 01:52:08,996 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 01:51:21 2017-05-13 01:52:08,996 INFO repeater task wh300_10pings starting 26.8 seconds late 2017-05-13 01:52:09,072 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 01:53:34,292 INFO repeater task wh300_lastensq starting 3.3 seconds late 2017-05-13 01:53:35,708 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 01:53:35,708 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 01:53:35,708 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 01:53:21 2017-05-13 01:54:05,772 INFO repeater task wh300_10pings starting 3.6 seconds late 2017-05-13 01:54:05,855 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 01:55:31,074 INFO repeater task wh300_lastensq starting 0.1 seconds late 2017-05-13 01:55:32,484 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 01:55:32,484 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 01:55:32,484 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 01:55:21 2017-05-13 01:56:02,567 INFO repeater task os75bb_10pings starting 0.4 seconds late 2017-05-13 01:56:02,657 INFO repeater task os75bb_10pings ended after 0.1 seconds 2017-05-13 01:56:02,658 INFO repeater task os75nb_10pings starting 0.5 seconds late 2017-05-13 01:56:02,736 INFO repeater task os75nb_10pings ended after 0.1 seconds 2017-05-13 01:56:22,784 INFO repeater task wh300_10pings starting 0.6 seconds late 2017-05-13 01:56:22,870 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 01:56:22,873 INFO repeater task phins_hcorrstats starting 0.7 seconds late 2017-05-13 01:56:23,609 INFO repeater task phins_hcorrstats ended after 0.7 seconds 2017-05-13 01:56:23,609 INFO repeater task phins2_hcorrstats starting 1.4 seconds late 2017-05-13 01:56:24,350 INFO repeater task phins2_hcorrstats ended after 0.7 seconds 2017-05-13 01:56:34,367 INFO repeater task os75bb_lastensq starting 0.4 seconds late 2017-05-13 01:56:35,349 INFO repeater task os75bb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 01:56:35,349 INFO repeater task os75bb_lastensq ended after 1.0 seconds 2017-05-13 01:56:35,349 INFO repeater task os75bb_lastensq: Time from /home/data/AT37_12/proc/os75bb/load/wrote_ens_time: 2017/05/13 01:56:24 2017-05-13 01:56:35,349 INFO repeater task os75nb_lastensq starting 1.3 seconds late 2017-05-13 01:56:36,314 INFO repeater task os75nb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 01:56:36,314 INFO repeater task os75nb_lastensq ended after 1.0 seconds 2017-05-13 01:56:36,314 INFO repeater task os75nb_lastensq: Time from /home/data/AT37_12/proc/os75nb/load/wrote_ens_time: 2017/05/13 01:56:24 2017-05-13 01:57:31,458 INFO repeater task wh300_lastensq starting 0.5 seconds late 2017-05-13 01:57:32,824 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 01:57:32,824 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 01:57:32,825 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 01:57:21 2017-05-13 01:58:43,016 INFO repeater task wh300_10pings starting 0.9 seconds late 2017-05-13 01:58:43,105 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 01:59:33,239 INFO repeater task wh300_lastensq starting 2.2 seconds late 2017-05-13 01:59:34,626 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 01:59:34,626 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 01:59:34,627 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 01:59:21 2017-05-13 02:01:04,855 INFO repeater task wh300_10pings starting 2.7 seconds late 2017-05-13 02:01:04,933 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:01:24,986 INFO repeater task os75bb_10pings starting 2.8 seconds late 2017-05-13 02:01:25,080 INFO repeater task os75bb_10pings ended after 0.1 seconds 2017-05-13 02:01:25,081 INFO repeater task os75nb_10pings starting 2.9 seconds late 2017-05-13 02:01:25,161 INFO repeater task os75nb_10pings ended after 0.1 seconds 2017-05-13 02:01:25,162 INFO repeater task phins_hcorrstats starting 3.0 seconds late 2017-05-13 02:01:25,908 INFO repeater task phins_hcorrstats ended after 0.7 seconds 2017-05-13 02:01:25,908 INFO repeater task phins2_hcorrstats starting 3.7 seconds late 2017-05-13 02:01:26,627 INFO repeater task phins2_hcorrstats ended after 0.7 seconds 2017-05-13 02:01:31,632 INFO repeater task wh300_lastensq starting 0.6 seconds late 2017-05-13 02:01:33,036 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:01:33,036 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:01:33,036 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:01:21 2017-05-13 02:01:38,047 INFO repeater task os75bb_lastensq starting 4.0 seconds late 2017-05-13 02:01:39,019 INFO repeater task os75bb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:01:39,019 INFO repeater task os75bb_lastensq ended after 1.0 seconds 2017-05-13 02:01:39,019 INFO repeater task os75bb_lastensq: Time from /home/data/AT37_12/proc/os75bb/load/wrote_ens_time: 2017/05/13 02:01:24 2017-05-13 02:01:39,019 INFO repeater task os75nb_lastensq starting 5.0 seconds late 2017-05-13 02:01:39,975 INFO repeater task os75nb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:01:39,976 INFO repeater task os75nb_lastensq ended after 1.0 seconds 2017-05-13 02:01:39,976 INFO repeater task os75nb_lastensq: Time from /home/data/AT37_12/proc/os75nb/load/wrote_ens_time: 2017/05/13 02:01:24 2017-05-13 02:03:25,262 INFO repeater task wh300_10pings starting 3.1 seconds late 2017-05-13 02:03:25,352 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:03:35,373 INFO repeater task wh300_lastensq starting 4.4 seconds late 2017-05-13 02:03:36,795 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:03:36,795 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:03:36,795 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:03:21 2017-05-13 02:05:32,108 INFO repeater task wh300_lastensq starting 1.1 seconds late 2017-05-13 02:05:33,483 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:05:33,483 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:05:33,483 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:05:21 2017-05-13 02:05:43,506 INFO repeater task wh300_10pings starting 1.3 seconds late 2017-05-13 02:05:43,600 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:06:23,708 INFO repeater task wh300_quick starting 1.6 seconds late 2017-05-13 02:06:27,157 INFO repeater task wh300_quick ended after 3.4 seconds 2017-05-13 02:06:27,158 INFO repeater task wh300_3dayplots starting 5.0 seconds late 2017-05-13 02:06:33,164 INFO repeater task wh300_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-05-13 02:06:33,164 INFO repeater task wh300_3dayplots ended after 6.0 seconds 2017-05-13 02:06:33,164 INFO repeater task wh300_tsstats starting 11.0 seconds late 2017-05-13 02:06:33,710 INFO repeater task wh300_tsstats ended after 0.5 seconds 2017-05-13 02:06:33,710 INFO repeater task os75bb_quick starting 11.6 seconds late 2017-05-13 02:06:36,455 INFO repeater task os75bb_quick ended after 2.7 seconds 2017-05-13 02:06:36,455 INFO repeater task os75bb_3dayplots starting 14.3 seconds late 2017-05-13 02:06:42,574 INFO repeater task os75bb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-05-13 02:06:42,574 INFO repeater task os75bb_3dayplots ended after 6.1 seconds 2017-05-13 02:06:42,575 INFO repeater task os75bb_tsstats starting 20.4 seconds late 2017-05-13 02:06:43,065 INFO repeater task os75bb_tsstats ended after 0.5 seconds 2017-05-13 02:06:43,065 INFO repeater task os75nb_lastensq starting 9.1 seconds late 2017-05-13 02:06:44,044 INFO repeater task os75nb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:06:44,044 INFO repeater task os75nb_lastensq ended after 1.0 seconds 2017-05-13 02:06:44,044 INFO repeater task os75nb_lastensq: Time from /home/data/AT37_12/proc/os75nb/load/wrote_ens_time: 2017/05/13 02:06:24 2017-05-13 02:06:44,044 INFO repeater task os75nb_10pings starting 1.9 seconds late 2017-05-13 02:06:44,125 INFO repeater task os75nb_10pings ended after 0.1 seconds 2017-05-13 02:06:44,125 INFO repeater task os75nb_quick starting 22.0 seconds late 2017-05-13 02:06:46,806 INFO repeater task os75nb_quick ended after 2.7 seconds 2017-05-13 02:06:46,807 INFO repeater task os75nb_3dayplots starting 24.6 seconds late 2017-05-13 02:06:52,901 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-05-13 02:06:52,901 INFO repeater task os75nb_3dayplots ended after 6.1 seconds 2017-05-13 02:06:52,901 INFO repeater task os75nb_tsstats starting 30.7 seconds late 2017-05-13 02:06:53,385 INFO repeater task os75nb_tsstats ended after 0.5 seconds 2017-05-13 02:06:53,386 INFO repeater task wh300_beamplots starting 31.2 seconds late 2017-05-13 02:06:54,257 INFO repeater task wh300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-05-13 02:06:54,257 INFO repeater task wh300_beamplots ended after 0.9 seconds 2017-05-13 02:06:54,257 INFO repeater task wh300_beamplots starting 32.1 seconds late 2017-05-13 02:06:55,150 INFO repeater task wh300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-05-13 02:06:55,150 INFO repeater task wh300_beamplots ended after 0.9 seconds 2017-05-13 02:06:55,150 INFO repeater task os75bb_beamplots starting 33.0 seconds late 2017-05-13 02:06:55,920 INFO repeater task os75bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-05-13 02:06:55,920 INFO repeater task os75bb_beamplots ended after 0.8 seconds 2017-05-13 02:06:55,920 INFO repeater task os75bb_beamplots starting 33.8 seconds late 2017-05-13 02:06:56,695 INFO repeater task os75bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-05-13 02:06:56,695 INFO repeater task os75bb_beamplots ended after 0.8 seconds 2017-05-13 02:06:56,695 INFO repeater task os75nb_beamplots starting 34.5 seconds late 2017-05-13 02:06:57,451 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-05-13 02:06:57,452 INFO repeater task os75nb_beamplots ended after 0.8 seconds 2017-05-13 02:06:57,452 INFO repeater task os75nb_beamplots starting 35.3 seconds late 2017-05-13 02:06:58,224 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-05-13 02:06:58,224 INFO repeater task os75nb_beamplots ended after 0.8 seconds 2017-05-13 02:06:58,224 INFO repeater task phins_hcorrstats starting 36.1 seconds late 2017-05-13 02:06:58,957 INFO repeater task phins_hcorrstats ended after 0.7 seconds 2017-05-13 02:06:58,957 INFO repeater task phins2_hcorrstats starting 36.8 seconds late 2017-05-13 02:06:59,673 INFO repeater task phins2_hcorrstats ended after 0.7 seconds 2017-05-13 02:07:04,682 INFO repeater task os75bb_lastensq starting 30.7 seconds late 2017-05-13 02:07:05,677 INFO repeater task os75bb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:07:05,678 INFO repeater task os75bb_lastensq ended after 1.0 seconds 2017-05-13 02:07:05,678 INFO repeater task os75bb_lastensq: Time from /home/data/AT37_12/proc/os75bb/load/wrote_ens_time: 2017/05/13 02:06:24 2017-05-13 02:07:05,678 INFO repeater task os75bb_10pings starting 23.5 seconds late 2017-05-13 02:07:05,762 INFO repeater task os75bb_10pings ended after 0.1 seconds 2017-05-13 02:07:35,841 INFO repeater task wh300_lastensq starting 4.8 seconds late 2017-05-13 02:07:37,228 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:07:37,228 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:07:37,228 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:07:21 2017-05-13 02:08:02,292 INFO repeater task wh300_10pings starting 0.1 seconds late 2017-05-13 02:08:02,384 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:09:32,625 INFO repeater task wh300_lastensq starting 1.6 seconds late 2017-05-13 02:09:34,009 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:09:34,009 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:09:34,009 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:09:21 2017-05-13 02:10:24,137 INFO repeater task wh300_10pings starting 2.0 seconds late 2017-05-13 02:10:24,229 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:11:24,400 INFO repeater task phins_hcorrstats starting 2.2 seconds late 2017-05-13 02:11:25,131 INFO repeater task phins_hcorrstats ended after 0.7 seconds 2017-05-13 02:11:25,131 INFO repeater task phins2_hcorrstats starting 3.0 seconds late 2017-05-13 02:11:25,849 INFO repeater task phins2_hcorrstats ended after 0.7 seconds 2017-05-13 02:11:35,867 INFO repeater task wh300_lastensq starting 4.9 seconds late 2017-05-13 02:11:37,301 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:11:37,301 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:11:37,301 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:11:21 2017-05-13 02:11:37,302 INFO repeater task os75bb_lastensq starting 3.3 seconds late 2017-05-13 02:11:38,263 INFO repeater task os75bb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:11:38,263 INFO repeater task os75bb_lastensq ended after 1.0 seconds 2017-05-13 02:11:38,263 INFO repeater task os75bb_lastensq: Time from /home/data/AT37_12/proc/os75bb/load/wrote_ens_time: 2017/05/13 02:11:24 2017-05-13 02:11:38,264 INFO repeater task os75nb_lastensq starting 4.3 seconds late 2017-05-13 02:11:39,221 INFO repeater task os75nb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:11:39,221 INFO repeater task os75nb_lastensq ended after 1.0 seconds 2017-05-13 02:11:39,221 INFO repeater task os75nb_lastensq: Time from /home/data/AT37_12/proc/os75nb/load/wrote_ens_time: 2017/05/13 02:11:24 2017-05-13 02:12:04,290 INFO repeater task os75bb_10pings starting 2.1 seconds late 2017-05-13 02:12:04,378 INFO repeater task os75bb_10pings ended after 0.1 seconds 2017-05-13 02:12:04,378 INFO repeater task os75nb_10pings starting 2.2 seconds late 2017-05-13 02:12:04,458 INFO repeater task os75nb_10pings ended after 0.1 seconds 2017-05-13 02:12:44,555 INFO repeater task wh300_10pings starting 2.4 seconds late 2017-05-13 02:12:44,647 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:13:34,781 INFO repeater task wh300_lastensq starting 3.8 seconds late 2017-05-13 02:13:36,149 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:13:36,149 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:13:36,149 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:13:21 2017-05-13 02:15:06,403 INFO repeater task wh300_10pings starting 4.2 seconds late 2017-05-13 02:15:06,490 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:15:31,555 INFO repeater task wh300_lastensq starting 0.6 seconds late 2017-05-13 02:15:32,982 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:15:32,982 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:15:32,982 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:15:21 2017-05-13 02:16:23,120 INFO repeater task phins_hcorrstats starting 1.0 seconds late 2017-05-13 02:16:23,850 INFO repeater task phins_hcorrstats ended after 0.7 seconds 2017-05-13 02:16:23,850 INFO repeater task phins2_hcorrstats starting 1.7 seconds late 2017-05-13 02:16:24,568 INFO repeater task phins2_hcorrstats ended after 0.7 seconds 2017-05-13 02:16:34,588 INFO repeater task os75bb_lastensq starting 0.6 seconds late 2017-05-13 02:16:35,589 INFO repeater task os75bb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:16:35,589 INFO repeater task os75bb_lastensq ended after 1.0 seconds 2017-05-13 02:16:35,589 INFO repeater task os75bb_lastensq: Time from /home/data/AT37_12/proc/os75bb/load/wrote_ens_time: 2017/05/13 02:16:24 2017-05-13 02:16:35,590 INFO repeater task os75nb_lastensq starting 1.6 seconds late 2017-05-13 02:16:36,579 INFO repeater task os75nb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:16:36,579 INFO repeater task os75nb_lastensq ended after 1.0 seconds 2017-05-13 02:16:36,579 INFO repeater task os75nb_lastensq: Time from /home/data/AT37_12/proc/os75nb/load/wrote_ens_time: 2017/05/13 02:16:24 2017-05-13 02:17:26,708 INFO repeater task wh300_10pings starting 4.6 seconds late 2017-05-13 02:17:26,802 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:17:26,803 INFO repeater task os75bb_10pings starting 4.6 seconds late 2017-05-13 02:17:26,886 INFO repeater task os75bb_10pings ended after 0.1 seconds 2017-05-13 02:17:26,887 INFO repeater task os75nb_10pings starting 4.7 seconds late 2017-05-13 02:17:26,969 INFO repeater task os75nb_10pings ended after 0.1 seconds 2017-05-13 02:17:31,976 INFO repeater task wh300_lastensq starting 1.0 seconds late 2017-05-13 02:17:33,392 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:17:33,392 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:17:33,393 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:17:21 2017-05-13 02:19:33,656 INFO repeater task wh300_lastensq starting 2.7 seconds late 2017-05-13 02:19:35,047 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:19:35,047 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:19:35,047 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:19:21 2017-05-13 02:19:45,066 INFO repeater task wh300_10pings starting 2.9 seconds late 2017-05-13 02:19:45,153 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:21:25,406 INFO repeater task wh300_quick starting 3.2 seconds late 2017-05-13 02:21:28,849 INFO repeater task wh300_quick ended after 3.4 seconds 2017-05-13 02:21:28,849 INFO repeater task wh300_3dayplots starting 6.7 seconds late 2017-05-13 02:21:35,335 INFO repeater task wh300_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-05-13 02:21:35,335 INFO repeater task wh300_3dayplots ended after 6.5 seconds 2017-05-13 02:21:35,335 INFO repeater task wh300_tsstats starting 13.2 seconds late 2017-05-13 02:21:35,867 INFO repeater task wh300_tsstats ended after 0.5 seconds 2017-05-13 02:21:35,868 INFO repeater task os75bb_lastensq starting 1.9 seconds late 2017-05-13 02:21:36,831 INFO repeater task os75bb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:21:36,831 INFO repeater task os75bb_lastensq ended after 1.0 seconds 2017-05-13 02:21:36,831 INFO repeater task os75bb_lastensq: Time from /home/data/AT37_12/proc/os75bb/load/wrote_ens_time: 2017/05/13 02:21:24 2017-05-13 02:21:36,831 INFO repeater task os75bb_quick starting 14.7 seconds late 2017-05-13 02:21:39,613 INFO repeater task os75bb_quick ended after 2.8 seconds 2017-05-13 02:21:39,613 INFO repeater task os75bb_3dayplots starting 17.5 seconds late 2017-05-13 02:21:45,766 INFO repeater task os75bb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-05-13 02:21:45,766 INFO repeater task os75bb_3dayplots ended after 6.2 seconds 2017-05-13 02:21:45,767 INFO repeater task os75bb_tsstats starting 23.6 seconds late 2017-05-13 02:21:46,262 INFO repeater task os75bb_tsstats ended after 0.5 seconds 2017-05-13 02:21:46,262 INFO repeater task os75nb_lastensq starting 12.3 seconds late 2017-05-13 02:21:47,214 INFO repeater task os75nb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:21:47,214 INFO repeater task os75nb_lastensq ended after 1.0 seconds 2017-05-13 02:21:47,214 INFO repeater task os75nb_lastensq: Time from /home/data/AT37_12/proc/os75nb/load/wrote_ens_time: 2017/05/13 02:21:24 2017-05-13 02:21:47,214 INFO repeater task os75nb_quick starting 25.1 seconds late 2017-05-13 02:21:49,903 INFO repeater task os75nb_quick ended after 2.7 seconds 2017-05-13 02:21:49,904 INFO repeater task os75nb_3dayplots starting 27.7 seconds late 2017-05-13 02:21:56,032 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-05-13 02:21:56,032 INFO repeater task os75nb_3dayplots ended after 6.1 seconds 2017-05-13 02:21:56,032 INFO repeater task os75nb_tsstats starting 33.9 seconds late 2017-05-13 02:21:56,516 INFO repeater task os75nb_tsstats output: /usr/lib/python2.7/dist-packages/numpy/ma/core.py:1073: RuntimeWarning: overflow encountered in true_divide result = self.f(da, db, *args, **kwargs) 2017-05-13 02:21:56,516 INFO repeater task os75nb_tsstats ended after 0.5 seconds 2017-05-13 02:21:56,516 INFO repeater task wh300_beamplots starting 34.4 seconds late 2017-05-13 02:21:57,386 INFO repeater task wh300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-05-13 02:21:57,386 INFO repeater task wh300_beamplots ended after 0.9 seconds 2017-05-13 02:21:57,386 INFO repeater task wh300_beamplots starting 35.2 seconds late 2017-05-13 02:21:58,283 INFO repeater task wh300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-05-13 02:21:58,284 INFO repeater task wh300_beamplots ended after 0.9 seconds 2017-05-13 02:21:58,284 INFO repeater task os75bb_beamplots starting 36.1 seconds late 2017-05-13 02:21:59,086 INFO repeater task os75bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-05-13 02:21:59,086 INFO repeater task os75bb_beamplots ended after 0.8 seconds 2017-05-13 02:21:59,086 INFO repeater task os75bb_beamplots starting 36.9 seconds late 2017-05-13 02:21:59,866 INFO repeater task os75bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-05-13 02:21:59,866 INFO repeater task os75bb_beamplots ended after 0.8 seconds 2017-05-13 02:21:59,866 INFO repeater task os75nb_beamplots starting 37.7 seconds late 2017-05-13 02:22:00,612 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-05-13 02:22:00,612 INFO repeater task os75nb_beamplots ended after 0.7 seconds 2017-05-13 02:22:00,612 INFO repeater task os75nb_beamplots starting 38.5 seconds late 2017-05-13 02:22:01,373 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-05-13 02:22:01,373 INFO repeater task os75nb_beamplots ended after 0.8 seconds 2017-05-13 02:22:01,374 INFO repeater task phins_hcorrstats starting 39.2 seconds late 2017-05-13 02:22:02,099 INFO repeater task phins_hcorrstats ended after 0.7 seconds 2017-05-13 02:22:02,099 INFO repeater task phins2_hcorrstats starting 39.9 seconds late 2017-05-13 02:22:02,826 INFO repeater task phins2_hcorrstats ended after 0.7 seconds 2017-05-13 02:22:07,832 INFO repeater task wh300_lastensq starting 36.8 seconds late 2017-05-13 02:22:09,262 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:22:09,262 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:22:09,262 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:21:21 2017-05-13 02:22:09,262 INFO repeater task wh300_10pings starting 7.1 seconds late 2017-05-13 02:22:09,340 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:22:44,422 INFO repeater task os75bb_10pings starting 2.3 seconds late 2017-05-13 02:22:44,513 INFO repeater task os75bb_10pings ended after 0.1 seconds 2017-05-13 02:22:44,514 INFO repeater task os75nb_10pings starting 2.4 seconds late 2017-05-13 02:22:44,590 INFO repeater task os75nb_10pings ended after 0.1 seconds 2017-05-13 02:23:34,698 INFO repeater task wh300_lastensq starting 3.7 seconds late 2017-05-13 02:23:36,051 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:23:36,051 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:23:36,051 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:23:21 2017-05-13 02:24:26,178 INFO repeater task wh300_10pings starting 4.0 seconds late 2017-05-13 02:24:26,267 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:25:31,438 INFO repeater task wh300_lastensq starting 0.4 seconds late 2017-05-13 02:25:32,847 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:25:32,847 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:25:32,847 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:25:21 2017-05-13 02:26:22,985 INFO repeater task phins_hcorrstats starting 0.8 seconds late 2017-05-13 02:26:23,717 INFO repeater task phins_hcorrstats ended after 0.7 seconds 2017-05-13 02:26:23,717 INFO repeater task phins2_hcorrstats starting 1.6 seconds late 2017-05-13 02:26:24,456 INFO repeater task phins2_hcorrstats ended after 0.7 seconds 2017-05-13 02:26:34,473 INFO repeater task os75bb_lastensq starting 0.5 seconds late 2017-05-13 02:26:35,455 INFO repeater task os75bb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:26:35,455 INFO repeater task os75bb_lastensq ended after 1.0 seconds 2017-05-13 02:26:35,455 INFO repeater task os75bb_lastensq: Time from /home/data/AT37_12/proc/os75bb/load/wrote_ens_time: 2017/05/13 02:26:24 2017-05-13 02:26:35,455 INFO repeater task os75nb_lastensq starting 1.5 seconds late 2017-05-13 02:26:36,421 INFO repeater task os75nb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:26:36,421 INFO repeater task os75nb_lastensq ended after 1.0 seconds 2017-05-13 02:26:36,421 INFO repeater task os75nb_lastensq: Time from /home/data/AT37_12/proc/os75nb/load/wrote_ens_time: 2017/05/13 02:26:24 2017-05-13 02:26:46,439 INFO repeater task wh300_10pings starting 4.3 seconds late 2017-05-13 02:26:46,515 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:27:31,629 INFO repeater task wh300_lastensq starting 0.6 seconds late 2017-05-13 02:27:33,056 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:27:33,056 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:27:33,056 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:27:21 2017-05-13 02:28:03,135 INFO repeater task os75bb_10pings starting 1.0 seconds late 2017-05-13 02:28:03,221 INFO repeater task os75bb_10pings ended after 0.1 seconds 2017-05-13 02:28:03,222 INFO repeater task os75nb_10pings starting 1.1 seconds late 2017-05-13 02:28:03,299 INFO repeater task os75nb_10pings ended after 0.1 seconds 2017-05-13 02:29:03,461 INFO repeater task wh300_10pings starting 1.3 seconds late 2017-05-13 02:29:03,558 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:29:33,633 INFO repeater task wh300_lastensq starting 2.6 seconds late 2017-05-13 02:29:35,033 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:29:35,033 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:29:35,033 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:29:21 2017-05-13 02:31:25,322 INFO repeater task wh300_10pings starting 3.2 seconds late 2017-05-13 02:31:25,409 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:31:25,411 INFO repeater task phins_hcorrstats starting 3.3 seconds late 2017-05-13 02:31:26,160 INFO repeater task phins_hcorrstats ended after 0.7 seconds 2017-05-13 02:31:26,160 INFO repeater task phins2_hcorrstats starting 4.0 seconds late 2017-05-13 02:31:26,874 INFO repeater task phins2_hcorrstats ended after 0.7 seconds 2017-05-13 02:31:31,880 INFO repeater task wh300_lastensq starting 0.9 seconds late 2017-05-13 02:31:33,292 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:31:33,292 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:31:33,292 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:31:21 2017-05-13 02:31:38,303 INFO repeater task os75bb_lastensq starting 4.3 seconds late 2017-05-13 02:31:39,325 INFO repeater task os75bb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:31:39,325 INFO repeater task os75bb_lastensq ended after 1.0 seconds 2017-05-13 02:31:39,325 INFO repeater task os75bb_lastensq: Time from /home/data/AT37_12/proc/os75bb/load/wrote_ens_time: 2017/05/13 02:31:24 2017-05-13 02:31:39,326 INFO repeater task os75nb_lastensq starting 5.3 seconds late 2017-05-13 02:31:40,294 INFO repeater task os75nb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:31:40,294 INFO repeater task os75nb_lastensq ended after 1.0 seconds 2017-05-13 02:31:40,294 INFO repeater task os75nb_lastensq: Time from /home/data/AT37_12/proc/os75nb/load/wrote_ens_time: 2017/05/13 02:31:24 2017-05-13 02:33:25,579 INFO repeater task os75bb_10pings starting 3.4 seconds late 2017-05-13 02:33:25,664 INFO repeater task os75bb_10pings ended after 0.1 seconds 2017-05-13 02:33:25,664 INFO repeater task os75nb_10pings starting 3.5 seconds late 2017-05-13 02:33:25,742 INFO repeater task os75nb_10pings ended after 0.1 seconds 2017-05-13 02:33:35,760 INFO repeater task wh300_lastensq starting 4.8 seconds late 2017-05-13 02:33:37,167 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:33:37,167 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:33:37,167 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:33:21 2017-05-13 02:33:42,176 INFO repeater task wh300_10pings starting 0.0 seconds late 2017-05-13 02:33:42,266 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:35:32,559 INFO repeater task wh300_lastensq starting 1.6 seconds late 2017-05-13 02:35:33,985 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:35:33,985 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:35:33,985 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:35:21 2017-05-13 02:36:04,061 INFO repeater task wh300_10pings starting 1.9 seconds late 2017-05-13 02:36:04,156 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:36:24,193 INFO repeater task wh300_quick starting 2.0 seconds late 2017-05-13 02:36:27,744 INFO repeater task wh300_quick ended after 3.6 seconds 2017-05-13 02:36:27,745 INFO repeater task wh300_3dayplots starting 5.6 seconds late 2017-05-13 02:36:33,705 INFO repeater task wh300_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-05-13 02:36:33,705 INFO repeater task wh300_3dayplots ended after 6.0 seconds 2017-05-13 02:36:33,705 INFO repeater task wh300_tsstats starting 11.5 seconds late 2017-05-13 02:36:34,247 INFO repeater task wh300_tsstats ended after 0.5 seconds 2017-05-13 02:36:34,247 INFO repeater task os75bb_lastensq starting 0.2 seconds late 2017-05-13 02:36:35,208 INFO repeater task os75bb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:36:35,208 INFO repeater task os75bb_lastensq ended after 1.0 seconds 2017-05-13 02:36:35,208 INFO repeater task os75bb_lastensq: Time from /home/data/AT37_12/proc/os75bb/load/wrote_ens_time: 2017/05/13 02:36:24 2017-05-13 02:36:35,209 INFO repeater task os75bb_quick starting 13.1 seconds late 2017-05-13 02:36:37,968 INFO repeater task os75bb_quick ended after 2.8 seconds 2017-05-13 02:36:37,969 INFO repeater task os75bb_3dayplots starting 15.8 seconds late 2017-05-13 02:36:44,010 INFO repeater task os75bb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-05-13 02:36:44,010 INFO repeater task os75bb_3dayplots ended after 6.0 seconds 2017-05-13 02:36:44,010 INFO repeater task os75bb_tsstats starting 21.9 seconds late 2017-05-13 02:36:44,504 INFO repeater task os75bb_tsstats ended after 0.5 seconds 2017-05-13 02:36:44,504 INFO repeater task os75nb_lastensq starting 10.5 seconds late 2017-05-13 02:36:45,475 INFO repeater task os75nb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:36:45,475 INFO repeater task os75nb_lastensq ended after 1.0 seconds 2017-05-13 02:36:45,475 INFO repeater task os75nb_lastensq: Time from /home/data/AT37_12/proc/os75nb/load/wrote_ens_time: 2017/05/13 02:36:24 2017-05-13 02:36:45,475 INFO repeater task os75nb_quick starting 23.3 seconds late 2017-05-13 02:36:48,159 INFO repeater task os75nb_quick ended after 2.7 seconds 2017-05-13 02:36:48,159 INFO repeater task os75nb_3dayplots starting 26.0 seconds late 2017-05-13 02:36:54,358 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-05-13 02:36:54,358 INFO repeater task os75nb_3dayplots ended after 6.2 seconds 2017-05-13 02:36:54,358 INFO repeater task os75nb_tsstats starting 32.2 seconds late 2017-05-13 02:36:54,848 INFO repeater task os75nb_tsstats output: /usr/lib/python2.7/dist-packages/numpy/ma/core.py:1073: RuntimeWarning: overflow encountered in true_divide result = self.f(da, db, *args, **kwargs) 2017-05-13 02:36:54,848 INFO repeater task os75nb_tsstats ended after 0.5 seconds 2017-05-13 02:36:54,849 INFO repeater task wh300_beamplots starting 32.7 seconds late 2017-05-13 02:36:55,718 INFO repeater task wh300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-05-13 02:36:55,719 INFO repeater task wh300_beamplots ended after 0.9 seconds 2017-05-13 02:36:55,719 INFO repeater task wh300_beamplots starting 33.6 seconds late 2017-05-13 02:36:56,622 INFO repeater task wh300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-05-13 02:36:56,622 INFO repeater task wh300_beamplots ended after 0.9 seconds 2017-05-13 02:36:56,622 INFO repeater task os75bb_beamplots starting 34.5 seconds late 2017-05-13 02:36:57,450 INFO repeater task os75bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-05-13 02:36:57,450 INFO repeater task os75bb_beamplots ended after 0.8 seconds 2017-05-13 02:36:57,450 INFO repeater task os75bb_beamplots starting 35.3 seconds late 2017-05-13 02:36:58,207 INFO repeater task os75bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-05-13 02:36:58,207 INFO repeater task os75bb_beamplots ended after 0.8 seconds 2017-05-13 02:36:58,207 INFO repeater task os75nb_beamplots starting 36.0 seconds late 2017-05-13 02:36:58,979 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-05-13 02:36:58,979 INFO repeater task os75nb_beamplots ended after 0.8 seconds 2017-05-13 02:36:58,979 INFO repeater task os75nb_beamplots starting 36.8 seconds late 2017-05-13 02:36:59,765 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-05-13 02:36:59,765 INFO repeater task os75nb_beamplots ended after 0.8 seconds 2017-05-13 02:36:59,766 INFO repeater task phins_hcorrstats starting 37.6 seconds late 2017-05-13 02:37:00,509 INFO repeater task phins_hcorrstats ended after 0.7 seconds 2017-05-13 02:37:00,510 INFO repeater task phins2_hcorrstats starting 38.3 seconds late 2017-05-13 02:37:01,259 INFO repeater task phins2_hcorrstats ended after 0.7 seconds 2017-05-13 02:37:31,337 INFO repeater task wh300_lastensq starting 0.3 seconds late 2017-05-13 02:37:32,741 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:37:32,742 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:37:32,742 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:37:21 2017-05-13 02:38:22,856 INFO repeater task wh300_10pings starting 0.7 seconds late 2017-05-13 02:38:22,941 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:38:42,990 INFO repeater task os75bb_10pings starting 0.8 seconds late 2017-05-13 02:38:43,073 INFO repeater task os75bb_10pings ended after 0.1 seconds 2017-05-13 02:38:43,074 INFO repeater task os75nb_10pings starting 0.9 seconds late 2017-05-13 02:38:43,151 INFO repeater task os75nb_10pings ended after 0.1 seconds 2017-05-13 02:39:33,278 INFO repeater task wh300_lastensq starting 2.3 seconds late 2017-05-13 02:39:34,672 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:39:34,672 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:39:34,672 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:39:21 2017-05-13 02:40:44,860 INFO repeater task wh300_10pings starting 2.7 seconds late 2017-05-13 02:40:44,945 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:41:25,052 INFO repeater task phins_hcorrstats starting 2.9 seconds late 2017-05-13 02:41:25,784 INFO repeater task phins_hcorrstats ended after 0.7 seconds 2017-05-13 02:41:25,785 INFO repeater task phins2_hcorrstats starting 3.6 seconds late 2017-05-13 02:41:26,511 INFO repeater task phins2_hcorrstats ended after 0.7 seconds 2017-05-13 02:41:31,517 INFO repeater task wh300_lastensq starting 0.5 seconds late 2017-05-13 02:41:32,948 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:41:32,948 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:41:32,948 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:41:21 2017-05-13 02:41:37,956 INFO repeater task os75bb_lastensq starting 4.0 seconds late 2017-05-13 02:41:38,937 INFO repeater task os75bb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:41:38,937 INFO repeater task os75bb_lastensq ended after 1.0 seconds 2017-05-13 02:41:38,937 INFO repeater task os75bb_lastensq: Time from /home/data/AT37_12/proc/os75bb/load/wrote_ens_time: 2017/05/13 02:41:24 2017-05-13 02:41:38,937 INFO repeater task os75nb_lastensq starting 4.9 seconds late 2017-05-13 02:41:39,914 INFO repeater task os75nb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:41:39,915 INFO repeater task os75nb_lastensq ended after 1.0 seconds 2017-05-13 02:41:39,915 INFO repeater task os75nb_lastensq: Time from /home/data/AT37_12/proc/os75nb/load/wrote_ens_time: 2017/05/13 02:41:24 2017-05-13 02:43:05,143 INFO repeater task wh300_10pings starting 3.0 seconds late 2017-05-13 02:43:05,232 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:43:35,308 INFO repeater task wh300_lastensq starting 4.3 seconds late 2017-05-13 02:43:36,724 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:43:36,724 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:43:36,724 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:43:21 2017-05-13 02:44:06,794 INFO repeater task os75bb_10pings starting 4.6 seconds late 2017-05-13 02:44:06,887 INFO repeater task os75bb_10pings ended after 0.1 seconds 2017-05-13 02:44:06,888 INFO repeater task os75nb_10pings starting 4.7 seconds late 2017-05-13 02:44:06,969 INFO repeater task os75nb_10pings ended after 0.1 seconds 2017-05-13 02:45:22,164 INFO repeater task wh300_10pings starting 0.0 seconds late 2017-05-13 02:45:22,266 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:45:32,289 INFO repeater task wh300_lastensq starting 1.3 seconds late 2017-05-13 02:45:33,693 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:45:33,693 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:45:33,694 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:45:21 2017-05-13 02:46:23,817 INFO repeater task phins_hcorrstats starting 1.7 seconds late 2017-05-13 02:46:24,542 INFO repeater task phins_hcorrstats ended after 0.7 seconds 2017-05-13 02:46:24,543 INFO repeater task phins2_hcorrstats starting 2.4 seconds late 2017-05-13 02:46:25,274 INFO repeater task phins2_hcorrstats ended after 0.7 seconds 2017-05-13 02:46:35,293 INFO repeater task os75bb_lastensq starting 1.3 seconds late 2017-05-13 02:46:36,291 INFO repeater task os75bb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:46:36,291 INFO repeater task os75bb_lastensq ended after 1.0 seconds 2017-05-13 02:46:36,291 INFO repeater task os75bb_lastensq: Time from /home/data/AT37_12/proc/os75bb/load/wrote_ens_time: 2017/05/13 02:46:24 2017-05-13 02:46:36,292 INFO repeater task os75nb_lastensq starting 2.3 seconds late 2017-05-13 02:46:37,270 INFO repeater task os75nb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:46:37,271 INFO repeater task os75nb_lastensq ended after 1.0 seconds 2017-05-13 02:46:37,271 INFO repeater task os75nb_lastensq: Time from /home/data/AT37_12/proc/os75nb/load/wrote_ens_time: 2017/05/13 02:46:24 2017-05-13 02:47:32,409 INFO repeater task wh300_lastensq starting 1.4 seconds late 2017-05-13 02:47:33,824 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:47:33,824 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:47:33,824 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:47:21 2017-05-13 02:47:43,840 INFO repeater task wh300_10pings starting 1.7 seconds late 2017-05-13 02:47:43,924 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:49:24,094 INFO repeater task os75bb_10pings starting 1.9 seconds late 2017-05-13 02:49:24,192 INFO repeater task os75bb_10pings ended after 0.1 seconds 2017-05-13 02:49:24,192 INFO repeater task os75nb_10pings starting 2.0 seconds late 2017-05-13 02:49:24,277 INFO repeater task os75nb_10pings ended after 0.1 seconds 2017-05-13 02:49:34,298 INFO repeater task wh300_lastensq starting 3.3 seconds late 2017-05-13 02:49:35,716 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:49:35,716 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:49:35,716 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:49:21 2017-05-13 02:50:05,785 INFO repeater task wh300_10pings starting 3.6 seconds late 2017-05-13 02:50:05,885 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:51:26,096 INFO repeater task wh300_quick starting 3.9 seconds late 2017-05-13 02:51:29,535 INFO repeater task wh300_quick ended after 3.4 seconds 2017-05-13 02:51:29,535 INFO repeater task wh300_3dayplots starting 7.4 seconds late 2017-05-13 02:51:35,602 INFO repeater task wh300_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-05-13 02:51:35,603 INFO repeater task wh300_3dayplots ended after 6.1 seconds 2017-05-13 02:51:35,603 INFO repeater task wh300_tsstats starting 13.4 seconds late 2017-05-13 02:51:36,139 INFO repeater task wh300_tsstats ended after 0.5 seconds 2017-05-13 02:51:36,140 INFO repeater task os75bb_lastensq starting 2.1 seconds late 2017-05-13 02:51:37,104 INFO repeater task os75bb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:51:37,104 INFO repeater task os75bb_lastensq ended after 1.0 seconds 2017-05-13 02:51:37,104 INFO repeater task os75bb_lastensq: Time from /home/data/AT37_12/proc/os75bb/load/wrote_ens_time: 2017/05/13 02:51:24 2017-05-13 02:51:37,104 INFO repeater task os75bb_quick starting 14.9 seconds late 2017-05-13 02:51:39,801 INFO repeater task os75bb_quick ended after 2.7 seconds 2017-05-13 02:51:39,801 INFO repeater task os75bb_3dayplots starting 17.6 seconds late 2017-05-13 02:51:45,848 INFO repeater task os75bb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-05-13 02:51:45,848 INFO repeater task os75bb_3dayplots ended after 6.0 seconds 2017-05-13 02:51:45,849 INFO repeater task os75bb_tsstats starting 23.7 seconds late 2017-05-13 02:51:46,329 INFO repeater task os75bb_tsstats ended after 0.5 seconds 2017-05-13 02:51:46,329 INFO repeater task os75nb_lastensq starting 12.3 seconds late 2017-05-13 02:51:47,290 INFO repeater task os75nb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:51:47,290 INFO repeater task os75nb_lastensq ended after 1.0 seconds 2017-05-13 02:51:47,290 INFO repeater task os75nb_lastensq: Time from /home/data/AT37_12/proc/os75nb/load/wrote_ens_time: 2017/05/13 02:51:24 2017-05-13 02:51:47,291 INFO repeater task os75nb_quick starting 25.1 seconds late 2017-05-13 02:51:49,999 INFO repeater task os75nb_quick ended after 2.7 seconds 2017-05-13 02:51:49,999 INFO repeater task os75nb_3dayplots starting 27.8 seconds late 2017-05-13 02:51:56,115 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2017-05-13 02:51:56,115 INFO repeater task os75nb_3dayplots ended after 6.1 seconds 2017-05-13 02:51:56,116 INFO repeater task os75nb_tsstats starting 34.0 seconds late 2017-05-13 02:51:56,587 INFO repeater task os75nb_tsstats ended after 0.5 seconds 2017-05-13 02:51:56,587 INFO repeater task wh300_beamplots starting 34.4 seconds late 2017-05-13 02:51:57,430 INFO repeater task wh300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-05-13 02:51:57,430 INFO repeater task wh300_beamplots ended after 0.8 seconds 2017-05-13 02:51:57,430 INFO repeater task wh300_beamplots starting 35.3 seconds late 2017-05-13 02:51:58,288 INFO repeater task wh300_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-05-13 02:51:58,288 INFO repeater task wh300_beamplots ended after 0.9 seconds 2017-05-13 02:51:58,288 INFO repeater task os75bb_beamplots starting 36.1 seconds late 2017-05-13 02:51:59,075 INFO repeater task os75bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-05-13 02:51:59,075 INFO repeater task os75bb_beamplots ended after 0.8 seconds 2017-05-13 02:51:59,075 INFO repeater task os75bb_beamplots starting 36.9 seconds late 2017-05-13 02:51:59,855 INFO repeater task os75bb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-05-13 02:51:59,855 INFO repeater task os75bb_beamplots ended after 0.8 seconds 2017-05-13 02:51:59,856 INFO repeater task os75nb_beamplots starting 37.7 seconds late 2017-05-13 02:52:00,624 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-05-13 02:52:00,624 INFO repeater task os75nb_beamplots ended after 0.8 seconds 2017-05-13 02:52:00,624 INFO repeater task os75nb_beamplots starting 38.5 seconds late 2017-05-13 02:52:01,430 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /usr/local/currents/bin 2017-05-13 02:52:01,430 INFO repeater task os75nb_beamplots ended after 0.8 seconds 2017-05-13 02:52:01,430 INFO repeater task phins_hcorrstats starting 39.3 seconds late 2017-05-13 02:52:02,152 INFO repeater task phins_hcorrstats ended after 0.7 seconds 2017-05-13 02:52:02,152 INFO repeater task phins2_hcorrstats starting 40.0 seconds late 2017-05-13 02:52:02,882 INFO repeater task phins2_hcorrstats ended after 0.7 seconds 2017-05-13 02:52:07,888 INFO repeater task wh300_lastensq starting 36.9 seconds late 2017-05-13 02:52:09,308 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:52:09,308 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:52:09,308 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:51:21 2017-05-13 02:52:24,346 INFO repeater task wh300_10pings starting 2.2 seconds late 2017-05-13 02:52:24,442 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:53:34,637 INFO repeater task wh300_lastensq starting 3.6 seconds late 2017-05-13 02:53:36,045 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:53:36,046 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:53:36,046 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:53:21 2017-05-13 02:54:46,224 INFO repeater task wh300_10pings starting 4.1 seconds late 2017-05-13 02:54:46,316 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:54:46,317 INFO repeater task os75bb_10pings starting 4.2 seconds late 2017-05-13 02:54:46,394 INFO repeater task os75bb_10pings ended after 0.1 seconds 2017-05-13 02:54:46,394 INFO repeater task os75nb_10pings starting 4.2 seconds late 2017-05-13 02:54:46,471 INFO repeater task os75nb_10pings ended after 0.1 seconds 2017-05-13 02:55:31,574 INFO repeater task wh300_lastensq starting 0.6 seconds late 2017-05-13 02:55:32,969 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:55:32,969 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:55:32,969 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:55:21 2017-05-13 02:56:23,110 INFO repeater task phins_hcorrstats starting 1.0 seconds late 2017-05-13 02:56:23,828 INFO repeater task phins_hcorrstats ended after 0.7 seconds 2017-05-13 02:56:23,828 INFO repeater task phins2_hcorrstats starting 1.7 seconds late 2017-05-13 02:56:24,546 INFO repeater task phins2_hcorrstats ended after 0.7 seconds 2017-05-13 02:56:34,570 INFO repeater task os75bb_lastensq starting 0.6 seconds late 2017-05-13 02:56:35,561 INFO repeater task os75bb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:56:35,561 INFO repeater task os75bb_lastensq ended after 1.0 seconds 2017-05-13 02:56:35,561 INFO repeater task os75bb_lastensq: Time from /home/data/AT37_12/proc/os75bb/load/wrote_ens_time: 2017/05/13 02:56:24 2017-05-13 02:56:35,562 INFO repeater task os75nb_lastensq starting 1.6 seconds late 2017-05-13 02:56:36,536 INFO repeater task os75nb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:56:36,536 INFO repeater task os75nb_lastensq ended after 1.0 seconds 2017-05-13 02:56:36,536 INFO repeater task os75nb_lastensq: Time from /home/data/AT37_12/proc/os75nb/load/wrote_ens_time: 2017/05/13 02:56:24 2017-05-13 02:57:06,610 INFO repeater task wh300_10pings starting 4.5 seconds late 2017-05-13 02:57:06,695 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:57:31,756 INFO repeater task wh300_lastensq starting 0.8 seconds late 2017-05-13 02:57:33,169 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:57:33,169 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:57:33,169 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:57:21 2017-05-13 02:59:23,455 INFO repeater task wh300_10pings starting 1.3 seconds late 2017-05-13 02:59:23,543 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 02:59:33,566 INFO repeater task wh300_lastensq starting 2.6 seconds late 2017-05-13 02:59:34,946 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 02:59:34,946 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 02:59:34,946 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 02:59:21 2017-05-13 03:00:05,017 INFO repeater task os75bb_10pings starting 2.9 seconds late 2017-05-13 03:00:05,111 INFO repeater task os75bb_10pings ended after 0.1 seconds 2017-05-13 03:00:05,112 INFO repeater task os75nb_10pings starting 3.0 seconds late 2017-05-13 03:00:05,191 INFO repeater task os75nb_10pings ended after 0.1 seconds 2017-05-13 03:01:25,408 INFO repeater task phins_hcorrstats starting 3.2 seconds late 2017-05-13 03:01:26,130 INFO repeater task phins_hcorrstats ended after 0.7 seconds 2017-05-13 03:01:26,131 INFO repeater task phins2_hcorrstats starting 4.0 seconds late 2017-05-13 03:01:26,863 INFO repeater task phins2_hcorrstats ended after 0.7 seconds 2017-05-13 03:01:31,869 INFO repeater task wh300_lastensq starting 0.9 seconds late 2017-05-13 03:01:33,292 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 03:01:33,292 INFO repeater task wh300_lastensq ended after 1.4 seconds 2017-05-13 03:01:33,292 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 03:01:21 2017-05-13 03:01:38,302 INFO repeater task os75bb_lastensq starting 4.3 seconds late 2017-05-13 03:01:39,265 INFO repeater task os75bb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 03:01:39,265 INFO repeater task os75bb_lastensq ended after 1.0 seconds 2017-05-13 03:01:39,265 INFO repeater task os75bb_lastensq: Time from /home/data/AT37_12/proc/os75bb/load/wrote_ens_time: 2017/05/13 03:01:24 2017-05-13 03:01:39,266 INFO repeater task os75nb_lastensq starting 5.3 seconds late 2017-05-13 03:01:40,234 INFO repeater task os75nb_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 03:01:40,234 INFO repeater task os75nb_lastensq ended after 1.0 seconds 2017-05-13 03:01:40,234 INFO repeater task os75nb_lastensq: Time from /home/data/AT37_12/proc/os75nb/load/wrote_ens_time: 2017/05/13 03:01:24 2017-05-13 03:01:45,241 INFO repeater task wh300_10pings starting 3.1 seconds late 2017-05-13 03:01:45,335 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 03:03:35,634 INFO repeater task wh300_lastensq starting 4.6 seconds late 2017-05-13 03:03:37,109 INFO repeater task wh300_lastensq output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:79: RuntimeWarning: Degrees of freedom <= 0 for slice warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning) 2017-05-13 03:03:37,109 INFO repeater task wh300_lastensq ended after 1.5 seconds 2017-05-13 03:03:37,109 INFO repeater task wh300_lastensq: Time from /home/data/AT37_12/proc/wh300/load/wrote_ens_time: 2017/05/13 03:03:21 2017-05-13 03:04:02,176 INFO repeater task wh300_10pings starting 0.0 seconds late 2017-05-13 03:04:02,270 INFO repeater task wh300_10pings ended after 0.1 seconds 2017-05-13 03:04:15,774 INFO Translate translate exiting, hdg /home/data/AT37_12/rbin/phins2 2017-05-13 03:04:15,775 INFO Translate translate exiting, gps /home/data/AT37_12/rbin/gpsnav 2017-05-13 03:04:15,777 INFO Translate translate exiting, hdg /home/data/AT37_12/rbin/phins 2017-05-13 03:04:15,790 INFO Translate translate exiting, hdg /home/data/AT37_12/rbin/gyro 2017-05-13 03:04:15,815 INFO Translate translate exiting, gps /home/data/AT37_12/rbin/gpsnav2 2017-05-13 03:04:17,372 INFO root cleanup_figs not_logging 2017-05-13 03:04:17,388 INFO root Exiting DAS_while_logging.py