2015-11-14 22:00:22,211 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-14 22:00:22,211 INFO repeater task phins3_hcorrstats starting 3.6 seconds late 2015-11-14 22:00:22,908 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-14 22:01:33,023 INFO repeater task os150nb_lastensq starting 4.0 seconds late 2015-11-14 22:01:34,634 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 22:01:34,634 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:01:19 2015-11-14 22:02:19,716 INFO repeater task os150nb_10pings starting 1.1 seconds late 2015-11-14 22:02:19,837 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 22:03:29,966 INFO repeater task os150nb_lastensq starting 1.0 seconds late 2015-11-14 22:03:31,552 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 22:03:31,552 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:03:19 2015-11-14 22:04:41,660 INFO repeater task os150nb_10pings starting 3.0 seconds late 2015-11-14 22:04:41,780 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 22:05:21,847 INFO repeater task ashtech_hcorrstats starting 3.2 seconds late 2015-11-14 22:05:22,584 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-14 22:05:22,584 INFO repeater task phins3_hcorrstats starting 4.0 seconds late 2015-11-14 22:05:23,283 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-14 22:05:33,296 INFO repeater task os150nb_lastensq starting 4.3 seconds late 2015-11-14 22:05:34,816 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2015-11-14 22:05:34,816 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:05:19 2015-11-14 22:06:59,968 INFO repeater task os150nb_10pings starting 1.4 seconds late 2015-11-14 22:07:00,091 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 22:07:30,139 INFO repeater task os150nb_lastensq starting 1.1 seconds late 2015-11-14 22:07:31,744 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 22:07:31,744 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:07:19 2015-11-14 22:09:21,926 INFO repeater task os150nb_10pings starting 3.3 seconds late 2015-11-14 22:09:22,045 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 22:09:32,061 INFO repeater task os150nb_lastensq starting 3.1 seconds late 2015-11-14 22:09:33,659 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 22:09:33,659 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:09:19 2015-11-14 22:10:18,742 INFO repeater task os150nb_quick starting 0.1 seconds late 2015-11-14 22:10:20,652 INFO repeater task os150nb_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2015-11-14 22:10:20,653 INFO repeater task os150nb_quick ended after 1.9 seconds 2015-11-14 22:10:20,653 INFO repeater task os150nb_3dayplots starting 2.0 seconds late 2015-11-14 22:10:22,733 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-11-14 22:10:22,733 INFO repeater task os150nb_3dayplots ended after 2.1 seconds 2015-11-14 22:10:22,733 INFO repeater task os150nb_tsstats starting 4.1 seconds late 2015-11-14 22:10:23,305 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2015-11-14 22:10:23,306 INFO repeater task os150nb_beamplots starting 4.7 seconds late 2015-11-14 22:10:24,385 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-14 22:10:24,385 INFO repeater task os150nb_beamplots ended after 1.1 seconds 2015-11-14 22:10:24,385 INFO repeater task os150nb_beamplots starting 5.8 seconds late 2015-11-14 22:10:25,480 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-14 22:10:25,480 INFO repeater task os150nb_beamplots ended after 1.1 seconds 2015-11-14 22:10:25,481 INFO repeater task ashtech_hcorrstats starting 6.9 seconds late 2015-11-14 22:10:26,206 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-14 22:10:26,206 INFO repeater task phins3_hcorrstats starting 7.6 seconds late 2015-11-14 22:10:26,886 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-14 22:11:31,996 INFO repeater task os150nb_lastensq starting 3.0 seconds late 2015-11-14 22:11:33,576 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 22:11:33,577 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:11:19 2015-11-14 22:11:43,594 INFO repeater task os150nb_10pings starting 5.0 seconds late 2015-11-14 22:11:43,710 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 22:13:33,900 INFO repeater task os150nb_lastensq starting 4.9 seconds late 2015-11-14 22:13:35,516 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 22:13:35,516 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:13:19 2015-11-14 22:14:00,561 INFO repeater task os150nb_10pings starting 1.9 seconds late 2015-11-14 22:14:00,685 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 22:15:20,833 INFO repeater task ashtech_hcorrstats starting 2.2 seconds late 2015-11-14 22:15:21,558 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-14 22:15:21,558 INFO repeater task phins3_hcorrstats starting 2.9 seconds late 2015-11-14 22:15:22,234 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-14 22:15:32,249 INFO repeater task os150nb_lastensq starting 3.2 seconds late 2015-11-14 22:15:33,772 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2015-11-14 22:15:33,772 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:15:19 2015-11-14 22:16:18,849 INFO repeater task os150nb_10pings starting 0.2 seconds late 2015-11-14 22:16:18,975 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 22:17:29,083 INFO repeater task os150nb_lastensq starting 0.1 seconds late 2015-11-14 22:17:30,658 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 22:17:30,658 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:17:19 2015-11-14 22:18:40,782 INFO repeater task os150nb_10pings starting 2.2 seconds late 2015-11-14 22:18:40,904 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 22:19:30,989 INFO repeater task os150nb_lastensq starting 2.0 seconds late 2015-11-14 22:19:32,515 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2015-11-14 22:19:32,515 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:19:19 2015-11-14 22:20:22,608 INFO repeater task ashtech_hcorrstats starting 4.0 seconds late 2015-11-14 22:20:23,324 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-14 22:20:23,324 INFO repeater task phins3_hcorrstats starting 4.7 seconds late 2015-11-14 22:20:23,995 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-14 22:20:59,052 INFO repeater task os150nb_10pings starting 0.4 seconds late 2015-11-14 22:20:59,173 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 22:21:29,226 INFO repeater task os150nb_lastensq starting 0.2 seconds late 2015-11-14 22:21:30,831 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 22:21:30,832 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:21:19 2015-11-14 22:23:21,024 INFO repeater task os150nb_10pings starting 2.4 seconds late 2015-11-14 22:23:21,143 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 22:23:31,158 INFO repeater task os150nb_lastensq starting 2.2 seconds late 2015-11-14 22:23:32,748 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 22:23:32,749 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:23:19 2015-11-14 22:25:22,943 INFO repeater task os150nb_quick starting 4.3 seconds late 2015-11-14 22:25:24,844 INFO repeater task os150nb_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2015-11-14 22:25:24,844 INFO repeater task os150nb_quick ended after 1.9 seconds 2015-11-14 22:25:24,844 INFO repeater task os150nb_3dayplots starting 6.2 seconds late 2015-11-14 22:25:26,882 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-11-14 22:25:26,882 INFO repeater task os150nb_3dayplots ended after 2.0 seconds 2015-11-14 22:25:26,882 INFO repeater task os150nb_tsstats starting 8.3 seconds late 2015-11-14 22:25:27,444 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2015-11-14 22:25:27,445 INFO repeater task os150nb_beamplots starting 8.8 seconds late 2015-11-14 22:25:28,508 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-14 22:25:28,509 INFO repeater task os150nb_beamplots ended after 1.1 seconds 2015-11-14 22:25:28,509 INFO repeater task os150nb_beamplots starting 9.9 seconds late 2015-11-14 22:25:29,572 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-14 22:25:29,572 INFO repeater task os150nb_beamplots ended after 1.1 seconds 2015-11-14 22:25:29,572 INFO repeater task ashtech_hcorrstats starting 10.9 seconds late 2015-11-14 22:25:30,289 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-14 22:25:30,289 INFO repeater task phins3_hcorrstats starting 11.7 seconds late 2015-11-14 22:25:30,964 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-14 22:25:35,966 INFO repeater task os150nb_lastensq starting 7.0 seconds late 2015-11-14 22:25:37,563 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 22:25:37,563 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:25:19 2015-11-14 22:25:42,571 INFO repeater task os150nb_10pings starting 4.0 seconds late 2015-11-14 22:25:42,698 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 22:27:32,886 INFO repeater task os150nb_lastensq starting 3.9 seconds late 2015-11-14 22:27:34,432 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2015-11-14 22:27:34,432 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:27:19 2015-11-14 22:27:59,473 INFO repeater task os150nb_10pings starting 0.9 seconds late 2015-11-14 22:27:59,596 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 22:29:29,746 INFO repeater task os150nb_lastensq starting 0.7 seconds late 2015-11-14 22:29:31,353 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 22:29:31,354 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:29:19 2015-11-14 22:30:21,438 INFO repeater task os150nb_10pings starting 2.8 seconds late 2015-11-14 22:30:21,566 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 22:30:21,567 INFO repeater task ashtech_hcorrstats starting 2.9 seconds late 2015-11-14 22:30:22,272 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-14 22:30:22,272 INFO repeater task phins3_hcorrstats starting 3.6 seconds late 2015-11-14 22:30:22,938 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-14 22:31:33,066 INFO repeater task os150nb_lastensq starting 4.1 seconds late 2015-11-14 22:31:34,662 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 22:31:34,662 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:31:19 2015-11-14 22:32:39,777 INFO repeater task os150nb_10pings starting 1.2 seconds late 2015-11-14 22:32:39,901 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 22:33:29,984 INFO repeater task os150nb_lastensq starting 1.0 seconds late 2015-11-14 22:33:31,574 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 22:33:31,575 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:33:19 2015-11-14 22:35:01,728 INFO repeater task os150nb_10pings starting 3.1 seconds late 2015-11-14 22:35:01,848 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 22:35:21,882 INFO repeater task ashtech_hcorrstats starting 3.3 seconds late 2015-11-14 22:35:22,611 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-14 22:35:22,611 INFO repeater task phins3_hcorrstats starting 4.0 seconds late 2015-11-14 22:35:23,299 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-14 22:35:33,315 INFO repeater task os150nb_lastensq starting 4.3 seconds late 2015-11-14 22:35:34,820 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2015-11-14 22:35:34,820 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:35:19 2015-11-14 22:37:20,006 INFO repeater task os150nb_10pings starting 1.4 seconds late 2015-11-14 22:37:20,118 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 22:37:30,130 INFO repeater task os150nb_lastensq starting 1.1 seconds late 2015-11-14 22:37:31,712 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 22:37:31,712 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:37:19 2015-11-14 22:39:31,925 INFO repeater task os150nb_lastensq starting 2.9 seconds late 2015-11-14 22:39:33,534 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 22:39:33,534 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:39:19 2015-11-14 22:39:43,549 INFO repeater task os150nb_10pings starting 4.9 seconds late 2015-11-14 22:39:43,655 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 22:40:18,698 INFO repeater task os150nb_quick starting 0.1 seconds late 2015-11-14 22:40:20,542 INFO repeater task os150nb_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2015-11-14 22:40:20,542 INFO repeater task os150nb_quick ended after 1.8 seconds 2015-11-14 22:40:20,542 INFO repeater task os150nb_3dayplots starting 1.9 seconds late 2015-11-14 22:40:22,534 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-11-14 22:40:22,534 INFO repeater task os150nb_3dayplots ended after 2.0 seconds 2015-11-14 22:40:22,534 INFO repeater task os150nb_tsstats starting 3.9 seconds late 2015-11-14 22:40:23,064 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2015-11-14 22:40:23,064 INFO repeater task os150nb_beamplots starting 4.5 seconds late 2015-11-14 22:40:24,109 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-14 22:40:24,109 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2015-11-14 22:40:24,110 INFO repeater task os150nb_beamplots starting 5.5 seconds late 2015-11-14 22:40:25,177 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-14 22:40:25,178 INFO repeater task os150nb_beamplots ended after 1.1 seconds 2015-11-14 22:40:25,178 INFO repeater task ashtech_hcorrstats starting 6.5 seconds late 2015-11-14 22:40:25,896 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-14 22:40:25,896 INFO repeater task phins3_hcorrstats starting 7.3 seconds late 2015-11-14 22:40:26,582 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-14 22:41:31,693 INFO repeater task os150nb_lastensq starting 2.7 seconds late 2015-11-14 22:41:33,257 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 22:41:33,258 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:41:19 2015-11-14 22:42:03,301 INFO repeater task os150nb_10pings starting 4.7 seconds late 2015-11-14 22:42:03,422 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 22:43:33,579 INFO repeater task os150nb_lastensq starting 4.6 seconds late 2015-11-14 22:43:35,121 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2015-11-14 22:43:35,121 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:43:19 2015-11-14 22:44:20,200 INFO repeater task os150nb_10pings starting 1.6 seconds late 2015-11-14 22:44:20,312 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 22:45:20,420 INFO repeater task ashtech_hcorrstats starting 1.8 seconds late 2015-11-14 22:45:21,135 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-14 22:45:21,136 INFO repeater task phins3_hcorrstats starting 2.5 seconds late 2015-11-14 22:45:21,831 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-14 22:45:31,844 INFO repeater task os150nb_lastensq starting 2.8 seconds late 2015-11-14 22:45:33,422 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 22:45:33,422 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:45:19 2015-11-14 22:46:43,542 INFO repeater task os150nb_10pings starting 4.9 seconds late 2015-11-14 22:46:43,656 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 22:47:33,742 INFO repeater task os150nb_lastensq starting 4.7 seconds late 2015-11-14 22:47:35,325 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 22:47:35,326 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:47:19 2015-11-14 22:49:00,469 INFO repeater task os150nb_10pings starting 1.9 seconds late 2015-11-14 22:49:00,591 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 22:49:30,634 INFO repeater task os150nb_lastensq starting 1.6 seconds late 2015-11-14 22:49:32,226 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 22:49:32,226 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:49:19 2015-11-14 22:50:22,314 INFO repeater task ashtech_hcorrstats starting 3.7 seconds late 2015-11-14 22:50:23,041 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-14 22:50:23,041 INFO repeater task phins3_hcorrstats starting 4.4 seconds late 2015-11-14 22:50:23,718 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-14 22:51:18,820 INFO repeater task os150nb_10pings starting 0.2 seconds late 2015-11-14 22:51:18,936 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 22:51:33,961 INFO repeater task os150nb_lastensq starting 5.0 seconds late 2015-11-14 22:51:35,518 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 22:51:35,518 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:51:19 2015-11-14 22:53:30,721 INFO repeater task os150nb_lastensq starting 1.7 seconds late 2015-11-14 22:53:32,284 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 22:53:32,284 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:53:19 2015-11-14 22:53:42,301 INFO repeater task os150nb_10pings starting 3.7 seconds late 2015-11-14 22:53:42,434 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 22:55:22,607 INFO repeater task os150nb_quick starting 4.0 seconds late 2015-11-14 22:55:24,478 INFO repeater task os150nb_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2015-11-14 22:55:24,478 INFO repeater task os150nb_quick ended after 1.9 seconds 2015-11-14 22:55:24,478 INFO repeater task os150nb_3dayplots starting 5.9 seconds late 2015-11-14 22:55:26,439 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-11-14 22:55:26,439 INFO repeater task os150nb_3dayplots ended after 2.0 seconds 2015-11-14 22:55:26,439 INFO repeater task os150nb_tsstats starting 7.8 seconds late 2015-11-14 22:55:26,971 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2015-11-14 22:55:26,971 INFO repeater task os150nb_beamplots starting 8.4 seconds late 2015-11-14 22:55:28,028 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-14 22:55:28,028 INFO repeater task os150nb_beamplots ended after 1.1 seconds 2015-11-14 22:55:28,029 INFO repeater task os150nb_beamplots starting 9.4 seconds late 2015-11-14 22:55:29,103 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-14 22:55:29,103 INFO repeater task os150nb_beamplots ended after 1.1 seconds 2015-11-14 22:55:29,104 INFO repeater task ashtech_hcorrstats starting 10.5 seconds late 2015-11-14 22:55:29,826 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-14 22:55:29,826 INFO repeater task phins3_hcorrstats starting 11.2 seconds late 2015-11-14 22:55:30,508 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-14 22:55:35,514 INFO repeater task os150nb_lastensq starting 6.5 seconds late 2015-11-14 22:55:37,119 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 22:55:37,119 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:55:19 2015-11-14 22:56:02,155 INFO repeater task os150nb_10pings starting 3.5 seconds late 2015-11-14 22:56:02,280 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 22:57:32,440 INFO repeater task os150nb_lastensq starting 3.4 seconds late 2015-11-14 22:57:34,028 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 22:57:34,028 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:57:19 2015-11-14 22:58:19,109 INFO repeater task os150nb_10pings starting 0.5 seconds late 2015-11-14 22:58:19,233 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 22:59:29,348 INFO repeater task os150nb_lastensq starting 0.3 seconds late 2015-11-14 22:59:30,951 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 22:59:30,952 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 22:59:19 2015-11-14 23:00:21,040 INFO repeater task ashtech_hcorrstats starting 2.4 seconds late 2015-11-14 23:00:21,752 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-14 23:00:21,753 INFO repeater task phins3_hcorrstats starting 3.1 seconds late 2015-11-14 23:00:22,452 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-14 23:00:42,486 INFO repeater task os150nb_10pings starting 3.9 seconds late 2015-11-14 23:00:42,610 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:01:32,701 INFO repeater task os150nb_lastensq starting 3.7 seconds late 2015-11-14 23:01:34,308 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 23:01:34,308 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:01:19 2015-11-14 23:02:59,453 INFO repeater task os150nb_10pings starting 0.8 seconds late 2015-11-14 23:02:59,576 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:03:29,630 INFO repeater task os150nb_lastensq starting 0.6 seconds late 2015-11-14 23:03:31,225 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 23:03:31,226 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:03:19 2015-11-14 23:05:21,423 INFO repeater task os150nb_10pings starting 2.8 seconds late 2015-11-14 23:05:21,552 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:05:21,553 INFO repeater task ashtech_hcorrstats starting 2.9 seconds late 2015-11-14 23:05:22,257 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-14 23:05:22,257 INFO repeater task phins3_hcorrstats starting 3.6 seconds late 2015-11-14 23:05:22,929 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-14 23:05:32,941 INFO repeater task os150nb_lastensq starting 3.9 seconds late 2015-11-14 23:05:34,481 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2015-11-14 23:05:34,482 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:05:19 2015-11-14 23:07:29,674 INFO repeater task os150nb_lastensq starting 0.7 seconds late 2015-11-14 23:07:31,270 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 23:07:31,271 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:07:19 2015-11-14 23:07:41,285 INFO repeater task os150nb_10pings starting 2.7 seconds late 2015-11-14 23:07:41,413 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:09:31,609 INFO repeater task os150nb_lastensq starting 2.6 seconds late 2015-11-14 23:09:33,226 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 23:09:33,226 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:09:19 2015-11-14 23:10:03,275 INFO repeater task os150nb_10pings starting 4.7 seconds late 2015-11-14 23:10:03,382 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:10:23,415 INFO repeater task os150nb_quick starting 4.8 seconds late 2015-11-14 23:10:25,304 INFO repeater task os150nb_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2015-11-14 23:10:25,304 INFO repeater task os150nb_quick ended after 1.9 seconds 2015-11-14 23:10:25,304 INFO repeater task os150nb_3dayplots starting 6.7 seconds late 2015-11-14 23:10:27,386 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-11-14 23:10:27,387 INFO repeater task os150nb_3dayplots ended after 2.1 seconds 2015-11-14 23:10:27,387 INFO repeater task os150nb_tsstats starting 8.8 seconds late 2015-11-14 23:10:27,933 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2015-11-14 23:10:27,933 INFO repeater task os150nb_beamplots starting 9.3 seconds late 2015-11-14 23:10:28,999 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-14 23:10:28,999 INFO repeater task os150nb_beamplots ended after 1.1 seconds 2015-11-14 23:10:29,000 INFO repeater task os150nb_beamplots starting 10.4 seconds late 2015-11-14 23:10:30,058 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-14 23:10:30,059 INFO repeater task os150nb_beamplots ended after 1.1 seconds 2015-11-14 23:10:30,059 INFO repeater task ashtech_hcorrstats starting 11.4 seconds late 2015-11-14 23:10:30,774 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-14 23:10:30,775 INFO repeater task phins3_hcorrstats starting 12.1 seconds late 2015-11-14 23:10:31,465 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-14 23:11:31,567 INFO repeater task os150nb_lastensq starting 2.6 seconds late 2015-11-14 23:11:33,090 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2015-11-14 23:11:33,091 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:11:19 2015-11-14 23:12:23,185 INFO repeater task os150nb_10pings starting 4.6 seconds late 2015-11-14 23:12:23,307 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:13:33,423 INFO repeater task os150nb_lastensq starting 4.4 seconds late 2015-11-14 23:13:35,032 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 23:13:35,032 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:13:19 2015-11-14 23:14:40,142 INFO repeater task os150nb_10pings starting 1.5 seconds late 2015-11-14 23:14:40,261 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:15:20,333 INFO repeater task ashtech_hcorrstats starting 1.7 seconds late 2015-11-14 23:15:21,049 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-14 23:15:21,049 INFO repeater task phins3_hcorrstats starting 2.4 seconds late 2015-11-14 23:15:21,751 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-14 23:15:31,766 INFO repeater task os150nb_lastensq starting 2.8 seconds late 2015-11-14 23:15:33,374 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 23:15:33,374 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:15:19 2015-11-14 23:17:03,531 INFO repeater task os150nb_10pings starting 4.9 seconds late 2015-11-14 23:17:03,656 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:17:33,707 INFO repeater task os150nb_lastensq starting 4.7 seconds late 2015-11-14 23:17:35,256 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2015-11-14 23:17:35,256 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:17:19 2015-11-14 23:19:20,439 INFO repeater task os150nb_10pings starting 1.8 seconds late 2015-11-14 23:19:20,557 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:19:30,573 INFO repeater task os150nb_lastensq starting 1.6 seconds late 2015-11-14 23:19:32,158 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 23:19:32,158 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:19:19 2015-11-14 23:20:22,244 INFO repeater task ashtech_hcorrstats starting 3.6 seconds late 2015-11-14 23:20:22,957 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-14 23:20:22,958 INFO repeater task phins3_hcorrstats starting 4.3 seconds late 2015-11-14 23:20:23,635 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-14 23:21:33,758 INFO repeater task os150nb_lastensq starting 4.8 seconds late 2015-11-14 23:21:35,363 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 23:21:35,365 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:21:19 2015-11-14 23:21:40,372 INFO repeater task os150nb_10pings starting 1.8 seconds late 2015-11-14 23:21:40,497 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:23:30,679 INFO repeater task os150nb_lastensq starting 1.7 seconds late 2015-11-14 23:23:32,300 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 23:23:32,301 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:23:19 2015-11-14 23:24:02,334 INFO repeater task os150nb_10pings starting 3.7 seconds late 2015-11-14 23:24:02,452 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:25:22,561 INFO repeater task os150nb_quick starting 3.9 seconds late 2015-11-14 23:25:24,445 INFO repeater task os150nb_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2015-11-14 23:25:24,445 INFO repeater task os150nb_quick ended after 1.9 seconds 2015-11-14 23:25:24,445 INFO repeater task os150nb_3dayplots starting 5.8 seconds late 2015-11-14 23:25:26,467 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-11-14 23:25:26,467 INFO repeater task os150nb_3dayplots ended after 2.0 seconds 2015-11-14 23:25:26,467 INFO repeater task os150nb_tsstats starting 7.9 seconds late 2015-11-14 23:25:26,999 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2015-11-14 23:25:26,999 INFO repeater task os150nb_beamplots starting 8.4 seconds late 2015-11-14 23:25:28,069 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-14 23:25:28,069 INFO repeater task os150nb_beamplots ended after 1.1 seconds 2015-11-14 23:25:28,070 INFO repeater task os150nb_beamplots starting 9.5 seconds late 2015-11-14 23:25:29,155 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-14 23:25:29,156 INFO repeater task os150nb_beamplots ended after 1.1 seconds 2015-11-14 23:25:29,156 INFO repeater task ashtech_hcorrstats starting 10.5 seconds late 2015-11-14 23:25:29,872 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-14 23:25:29,873 INFO repeater task phins3_hcorrstats starting 11.2 seconds late 2015-11-14 23:25:30,553 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-14 23:25:35,559 INFO repeater task os150nb_lastensq starting 6.6 seconds late 2015-11-14 23:25:37,165 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 23:25:37,166 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:25:19 2015-11-14 23:26:22,236 INFO repeater task os150nb_10pings starting 3.6 seconds late 2015-11-14 23:26:22,360 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:27:32,492 INFO repeater task os150nb_lastensq starting 3.5 seconds late 2015-11-14 23:27:34,100 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 23:27:34,100 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:27:19 2015-11-14 23:28:39,213 INFO repeater task os150nb_10pings starting 0.6 seconds late 2015-11-14 23:28:39,333 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:29:29,425 INFO repeater task os150nb_lastensq starting 0.4 seconds late 2015-11-14 23:29:30,986 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 23:29:30,987 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:29:19 2015-11-14 23:30:21,081 INFO repeater task ashtech_hcorrstats starting 2.5 seconds late 2015-11-14 23:30:21,777 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-14 23:30:21,777 INFO repeater task phins3_hcorrstats starting 3.1 seconds late 2015-11-14 23:30:22,456 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-14 23:31:02,527 INFO repeater task os150nb_10pings starting 3.9 seconds late 2015-11-14 23:31:02,640 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:31:32,686 INFO repeater task os150nb_lastensq starting 3.7 seconds late 2015-11-14 23:31:34,298 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 23:31:34,299 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:31:19 2015-11-14 23:33:19,471 INFO repeater task os150nb_10pings starting 0.9 seconds late 2015-11-14 23:33:19,597 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:33:29,610 INFO repeater task os150nb_lastensq starting 0.6 seconds late 2015-11-14 23:33:31,216 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 23:33:31,217 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:33:19 2015-11-14 23:35:21,426 INFO repeater task ashtech_hcorrstats starting 2.8 seconds late 2015-11-14 23:35:22,136 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-14 23:35:22,136 INFO repeater task phins3_hcorrstats starting 3.5 seconds late 2015-11-14 23:35:22,819 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-14 23:35:32,834 INFO repeater task os150nb_lastensq starting 3.8 seconds late 2015-11-14 23:35:34,381 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2015-11-14 23:35:34,381 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:35:19 2015-11-14 23:35:39,388 INFO repeater task os150nb_10pings starting 0.8 seconds late 2015-11-14 23:35:39,510 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:37:29,708 INFO repeater task os150nb_lastensq starting 0.7 seconds late 2015-11-14 23:37:31,383 INFO repeater task os150nb_lastensq ended after 1.7 seconds 2015-11-14 23:37:31,383 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:37:19 2015-11-14 23:38:01,437 INFO repeater task os150nb_10pings starting 2.8 seconds late 2015-11-14 23:38:01,558 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:39:31,724 INFO repeater task os150nb_lastensq starting 2.7 seconds late 2015-11-14 23:39:33,258 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2015-11-14 23:39:33,258 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:39:19 2015-11-14 23:40:23,342 INFO repeater task os150nb_10pings starting 4.7 seconds late 2015-11-14 23:40:23,461 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:40:23,461 INFO repeater task os150nb_quick starting 4.8 seconds late 2015-11-14 23:40:25,271 INFO repeater task os150nb_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2015-11-14 23:40:25,271 INFO repeater task os150nb_quick ended after 1.8 seconds 2015-11-14 23:40:25,272 INFO repeater task os150nb_3dayplots starting 6.7 seconds late 2015-11-14 23:40:27,318 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-11-14 23:40:27,318 INFO repeater task os150nb_3dayplots ended after 2.0 seconds 2015-11-14 23:40:27,319 INFO repeater task os150nb_tsstats starting 8.7 seconds late 2015-11-14 23:40:27,868 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2015-11-14 23:40:27,869 INFO repeater task os150nb_beamplots starting 9.3 seconds late 2015-11-14 23:40:28,950 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-14 23:40:28,950 INFO repeater task os150nb_beamplots ended after 1.1 seconds 2015-11-14 23:40:28,950 INFO repeater task os150nb_beamplots starting 10.3 seconds late 2015-11-14 23:40:30,053 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-14 23:40:30,053 INFO repeater task os150nb_beamplots ended after 1.1 seconds 2015-11-14 23:40:30,053 INFO repeater task ashtech_hcorrstats starting 11.4 seconds late 2015-11-14 23:40:30,781 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-14 23:40:30,781 INFO repeater task phins3_hcorrstats starting 12.2 seconds late 2015-11-14 23:40:31,480 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-14 23:41:31,588 INFO repeater task os150nb_lastensq starting 2.6 seconds late 2015-11-14 23:41:33,157 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 23:41:33,158 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:41:19 2015-11-14 23:42:43,281 INFO repeater task os150nb_10pings starting 4.7 seconds late 2015-11-14 23:42:43,406 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:43:33,485 INFO repeater task os150nb_lastensq starting 4.5 seconds late 2015-11-14 23:43:35,083 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 23:43:35,083 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:43:19 2015-11-14 23:45:00,241 INFO repeater task os150nb_10pings starting 1.6 seconds late 2015-11-14 23:45:00,363 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:45:20,402 INFO repeater task ashtech_hcorrstats starting 1.8 seconds late 2015-11-14 23:45:21,117 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-14 23:45:21,117 INFO repeater task phins3_hcorrstats starting 2.5 seconds late 2015-11-14 23:45:21,815 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-14 23:45:31,830 INFO repeater task os150nb_lastensq starting 2.8 seconds late 2015-11-14 23:45:33,435 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 23:45:33,435 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:45:19 2015-11-14 23:47:18,615 INFO repeater task os150nb_10pings starting 0.0 seconds late 2015-11-14 23:47:18,733 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:47:33,755 INFO repeater task os150nb_lastensq starting 4.8 seconds late 2015-11-14 23:47:35,317 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 23:47:35,317 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:47:19 2015-11-14 23:49:30,511 INFO repeater task os150nb_lastensq starting 1.5 seconds late 2015-11-14 23:49:32,067 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 23:49:32,068 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:49:19 2015-11-14 23:49:42,084 INFO repeater task os150nb_10pings starting 3.5 seconds late 2015-11-14 23:49:42,206 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:50:22,275 INFO repeater task ashtech_hcorrstats starting 3.6 seconds late 2015-11-14 23:50:22,982 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-14 23:50:22,983 INFO repeater task phins3_hcorrstats starting 4.4 seconds late 2015-11-14 23:50:23,652 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-14 23:51:33,777 INFO repeater task os150nb_lastensq starting 4.8 seconds late 2015-11-14 23:51:35,319 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2015-11-14 23:51:35,319 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:51:19 2015-11-14 23:52:00,360 INFO repeater task os150nb_10pings starting 1.7 seconds late 2015-11-14 23:52:00,482 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:53:30,639 INFO repeater task os150nb_lastensq starting 1.6 seconds late 2015-11-14 23:53:32,180 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2015-11-14 23:53:32,180 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:53:19 2015-11-14 23:54:22,263 INFO repeater task os150nb_10pings starting 3.7 seconds late 2015-11-14 23:54:22,378 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:55:22,490 INFO repeater task os150nb_quick starting 3.9 seconds late 2015-11-14 23:55:24,387 INFO repeater task os150nb_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2015-11-14 23:55:24,387 INFO repeater task os150nb_quick ended after 1.9 seconds 2015-11-14 23:55:24,387 INFO repeater task os150nb_3dayplots starting 5.8 seconds late 2015-11-14 23:55:26,387 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-11-14 23:55:26,388 INFO repeater task os150nb_3dayplots ended after 2.0 seconds 2015-11-14 23:55:26,388 INFO repeater task os150nb_tsstats starting 7.8 seconds late 2015-11-14 23:55:26,928 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2015-11-14 23:55:26,928 INFO repeater task os150nb_beamplots starting 8.3 seconds late 2015-11-14 23:55:27,990 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-14 23:55:27,990 INFO repeater task os150nb_beamplots ended after 1.1 seconds 2015-11-14 23:55:27,991 INFO repeater task os150nb_beamplots starting 9.4 seconds late 2015-11-14 23:55:29,082 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-14 23:55:29,082 INFO repeater task os150nb_beamplots ended after 1.1 seconds 2015-11-14 23:55:29,082 INFO repeater task ashtech_hcorrstats starting 10.5 seconds late 2015-11-14 23:55:29,786 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-14 23:55:29,786 INFO repeater task phins3_hcorrstats starting 11.2 seconds late 2015-11-14 23:55:30,448 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-14 23:55:35,454 INFO repeater task os150nb_lastensq starting 6.5 seconds late 2015-11-14 23:55:37,088 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 23:55:37,089 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:55:19 2015-11-14 23:56:42,199 INFO repeater task os150nb_10pings starting 3.6 seconds late 2015-11-14 23:56:42,320 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:57:32,404 INFO repeater task os150nb_lastensq starting 3.4 seconds late 2015-11-14 23:57:34,045 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 23:57:34,045 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:57:19 2015-11-14 23:58:59,169 INFO repeater task os150nb_10pings starting 0.6 seconds late 2015-11-14 23:58:59,291 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-14 23:59:29,324 INFO repeater task os150nb_lastensq starting 0.3 seconds late 2015-11-14 23:59:30,943 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-14 23:59:30,943 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/14 23:59:19 2015-11-15 00:00:21,031 INFO repeater task ashtech_hcorrstats starting 2.4 seconds late 2015-11-15 00:00:21,733 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-15 00:00:21,734 INFO repeater task phins3_hcorrstats starting 3.1 seconds late 2015-11-15 00:00:22,421 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-15 00:01:22,517 INFO repeater task os150nb_10pings starting 3.9 seconds late 2015-11-15 00:01:22,636 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 00:01:32,652 INFO repeater task os150nb_lastensq starting 3.7 seconds late 2015-11-15 00:01:34,232 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 00:01:34,232 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:01:19 2015-11-15 00:03:29,438 INFO repeater task os150nb_lastensq starting 0.4 seconds late 2015-11-15 00:03:30,998 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 00:03:30,999 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:03:19 2015-11-15 00:03:41,012 INFO repeater task os150nb_10pings starting 2.4 seconds late 2015-11-15 00:03:41,137 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 00:05:21,318 INFO repeater task ashtech_hcorrstats starting 2.7 seconds late 2015-11-15 00:05:22,008 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-15 00:05:22,008 INFO repeater task phins3_hcorrstats starting 3.4 seconds late 2015-11-15 00:05:22,682 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-15 00:05:32,694 INFO repeater task os150nb_lastensq starting 3.7 seconds late 2015-11-15 00:05:34,317 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 00:05:34,317 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:05:19 2015-11-15 00:05:59,361 INFO repeater task os150nb_10pings starting 0.7 seconds late 2015-11-15 00:05:59,486 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 00:07:29,653 INFO repeater task os150nb_lastensq starting 0.7 seconds late 2015-11-15 00:07:31,173 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2015-11-15 00:07:31,173 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:07:19 2015-11-15 00:08:21,241 INFO repeater task os150nb_10pings starting 2.6 seconds late 2015-11-15 00:08:21,353 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 00:09:31,451 INFO repeater task os150nb_lastensq starting 2.5 seconds late 2015-11-15 00:09:32,986 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2015-11-15 00:09:32,986 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:09:19 2015-11-15 00:10:23,068 INFO repeater task os150nb_quick starting 4.5 seconds late 2015-11-15 00:10:24,953 INFO repeater task os150nb_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2015-11-15 00:10:24,953 INFO repeater task os150nb_quick ended after 1.9 seconds 2015-11-15 00:10:24,954 INFO repeater task os150nb_3dayplots starting 6.3 seconds late 2015-11-15 00:10:27,198 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) could not parse 318_os150nb_ddaycont.png could not parse 318_os150nb_latcont.png could not parse 318_os150nb_loncont.png could not parse 318_os150nb_ddaycont.png could not parse 318_os150nb_latcont.png could not parse 318_os150nb_loncont.png 2015-11-15 00:10:27,198 INFO repeater task os150nb_3dayplots ended after 2.2 seconds 2015-11-15 00:10:27,198 INFO repeater task os150nb_tsstats starting 8.6 seconds late 2015-11-15 00:10:27,729 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2015-11-15 00:10:27,730 INFO repeater task os150nb_beamplots starting 9.1 seconds late 2015-11-15 00:10:28,746 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-15 00:10:28,746 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2015-11-15 00:10:28,747 INFO repeater task os150nb_beamplots starting 10.1 seconds late 2015-11-15 00:10:29,780 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-15 00:10:29,781 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2015-11-15 00:10:29,781 INFO repeater task ashtech_hcorrstats starting 11.2 seconds late 2015-11-15 00:10:30,470 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-15 00:10:30,471 INFO repeater task phins3_hcorrstats starting 11.8 seconds late 2015-11-15 00:10:31,138 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-15 00:10:41,154 INFO repeater task os150nb_10pings starting 2.5 seconds late 2015-11-15 00:10:41,283 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 00:11:31,365 INFO repeater task os150nb_lastensq starting 2.4 seconds late 2015-11-15 00:11:32,959 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 00:11:32,959 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:11:19 2015-11-15 00:13:03,113 INFO repeater task os150nb_10pings starting 4.5 seconds late 2015-11-15 00:13:03,231 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 00:13:33,280 INFO repeater task os150nb_lastensq starting 4.3 seconds late 2015-11-15 00:13:34,835 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 00:13:34,835 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:13:19 2015-11-15 00:15:20,016 INFO repeater task os150nb_10pings starting 1.4 seconds late 2015-11-15 00:15:20,135 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 00:15:20,136 INFO repeater task ashtech_hcorrstats starting 1.5 seconds late 2015-11-15 00:15:20,820 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-15 00:15:20,820 INFO repeater task phins3_hcorrstats starting 2.2 seconds late 2015-11-15 00:15:21,491 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-15 00:15:31,506 INFO repeater task os150nb_lastensq starting 2.5 seconds late 2015-11-15 00:15:33,080 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 00:15:33,081 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:15:19 2015-11-15 00:17:33,292 INFO repeater task os150nb_lastensq starting 4.3 seconds late 2015-11-15 00:17:34,879 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 00:17:34,880 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:17:19 2015-11-15 00:17:39,887 INFO repeater task os150nb_10pings starting 1.3 seconds late 2015-11-15 00:17:40,012 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 00:19:30,214 INFO repeater task os150nb_lastensq starting 1.2 seconds late 2015-11-15 00:19:31,803 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 00:19:31,803 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:19:19 2015-11-15 00:20:01,855 INFO repeater task os150nb_10pings starting 3.2 seconds late 2015-11-15 00:20:01,983 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 00:20:22,018 INFO repeater task ashtech_hcorrstats starting 3.4 seconds late 2015-11-15 00:20:22,711 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-15 00:20:22,711 INFO repeater task phins3_hcorrstats starting 4.1 seconds late 2015-11-15 00:20:23,383 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-15 00:21:33,512 INFO repeater task os150nb_lastensq starting 4.5 seconds late 2015-11-15 00:21:35,060 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2015-11-15 00:21:35,060 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:21:19 2015-11-15 00:22:20,138 INFO repeater task os150nb_10pings starting 1.5 seconds late 2015-11-15 00:22:20,253 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 00:23:30,378 INFO repeater task os150nb_lastensq starting 1.4 seconds late 2015-11-15 00:23:31,967 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 00:23:31,968 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:23:19 2015-11-15 00:24:42,080 INFO repeater task os150nb_10pings starting 3.5 seconds late 2015-11-15 00:24:42,201 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 00:25:22,275 INFO repeater task os150nb_quick starting 3.7 seconds late 2015-11-15 00:25:24,124 INFO repeater task os150nb_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2015-11-15 00:25:24,124 INFO repeater task os150nb_quick ended after 1.8 seconds 2015-11-15 00:25:24,124 INFO repeater task os150nb_3dayplots starting 5.5 seconds late 2015-11-15 00:25:26,568 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-11-15 00:25:26,569 INFO repeater task os150nb_3dayplots ended after 2.4 seconds 2015-11-15 00:25:26,569 INFO repeater task os150nb_tsstats starting 8.0 seconds late 2015-11-15 00:25:27,118 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2015-11-15 00:25:27,118 INFO repeater task os150nb_beamplots starting 8.5 seconds late 2015-11-15 00:25:28,134 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-15 00:25:28,135 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2015-11-15 00:25:28,135 INFO repeater task os150nb_beamplots starting 9.5 seconds late 2015-11-15 00:25:29,171 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-15 00:25:29,171 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2015-11-15 00:25:29,171 INFO repeater task ashtech_hcorrstats starting 10.5 seconds late 2015-11-15 00:25:29,867 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-15 00:25:29,867 INFO repeater task phins3_hcorrstats starting 11.2 seconds late 2015-11-15 00:25:30,537 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-15 00:25:35,543 INFO repeater task os150nb_lastensq starting 6.5 seconds late 2015-11-15 00:25:37,121 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 00:25:37,121 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:25:19 2015-11-15 00:27:02,260 INFO repeater task os150nb_10pings starting 3.6 seconds late 2015-11-15 00:27:02,388 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 00:27:32,439 INFO repeater task os150nb_lastensq starting 3.4 seconds late 2015-11-15 00:27:34,018 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 00:27:34,018 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:27:19 2015-11-15 00:29:19,196 INFO repeater task os150nb_10pings starting 0.6 seconds late 2015-11-15 00:29:19,303 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 00:29:29,318 INFO repeater task os150nb_lastensq starting 0.3 seconds late 2015-11-15 00:29:30,870 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 00:29:30,870 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:29:19 2015-11-15 00:30:20,951 INFO repeater task ashtech_hcorrstats starting 2.3 seconds late 2015-11-15 00:30:21,641 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-15 00:30:21,641 INFO repeater task phins3_hcorrstats starting 3.0 seconds late 2015-11-15 00:30:22,327 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-15 00:31:32,436 INFO repeater task os150nb_lastensq starting 3.4 seconds late 2015-11-15 00:31:33,990 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 00:31:33,990 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:31:19 2015-11-15 00:31:38,998 INFO repeater task os150nb_10pings starting 0.4 seconds late 2015-11-15 00:31:39,114 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 00:33:29,306 INFO repeater task os150nb_lastensq starting 0.3 seconds late 2015-11-15 00:33:30,863 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 00:33:30,864 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:33:19 2015-11-15 00:34:00,913 INFO repeater task os150nb_10pings starting 2.3 seconds late 2015-11-15 00:34:01,035 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 00:35:21,172 INFO repeater task ashtech_hcorrstats starting 2.5 seconds late 2015-11-15 00:35:21,868 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-15 00:35:21,868 INFO repeater task phins3_hcorrstats starting 3.2 seconds late 2015-11-15 00:35:22,547 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-15 00:35:32,562 INFO repeater task os150nb_lastensq starting 3.6 seconds late 2015-11-15 00:35:34,112 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2015-11-15 00:35:34,112 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:35:19 2015-11-15 00:36:19,192 INFO repeater task os150nb_10pings starting 0.6 seconds late 2015-11-15 00:36:19,313 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 00:37:29,435 INFO repeater task os150nb_lastensq starting 0.4 seconds late 2015-11-15 00:37:31,045 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 00:37:31,046 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:37:19 2015-11-15 00:38:41,169 INFO repeater task os150nb_10pings starting 2.6 seconds late 2015-11-15 00:38:41,284 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 00:39:31,365 INFO repeater task os150nb_lastensq starting 2.4 seconds late 2015-11-15 00:39:32,928 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 00:39:32,928 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:39:19 2015-11-15 00:40:23,007 INFO repeater task os150nb_quick starting 4.4 seconds late 2015-11-15 00:40:24,920 INFO repeater task os150nb_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2015-11-15 00:40:24,920 INFO repeater task os150nb_quick ended after 1.9 seconds 2015-11-15 00:40:24,920 INFO repeater task os150nb_3dayplots starting 6.3 seconds late 2015-11-15 00:40:27,438 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-11-15 00:40:27,438 INFO repeater task os150nb_3dayplots ended after 2.5 seconds 2015-11-15 00:40:27,439 INFO repeater task os150nb_tsstats starting 8.8 seconds late 2015-11-15 00:40:27,976 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2015-11-15 00:40:27,976 INFO repeater task os150nb_beamplots starting 9.4 seconds late 2015-11-15 00:40:28,999 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-15 00:40:28,999 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2015-11-15 00:40:28,999 INFO repeater task os150nb_beamplots starting 10.4 seconds late 2015-11-15 00:40:30,023 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-15 00:40:30,023 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2015-11-15 00:40:30,023 INFO repeater task ashtech_hcorrstats starting 11.4 seconds late 2015-11-15 00:40:30,729 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-15 00:40:30,730 INFO repeater task phins3_hcorrstats starting 12.1 seconds late 2015-11-15 00:40:31,403 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-15 00:41:01,454 INFO repeater task os150nb_10pings starting 2.8 seconds late 2015-11-15 00:41:01,578 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 00:41:31,627 INFO repeater task os150nb_lastensq starting 2.6 seconds late 2015-11-15 00:41:33,199 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 00:41:33,199 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:41:19 2015-11-15 00:43:23,378 INFO repeater task os150nb_10pings starting 4.8 seconds late 2015-11-15 00:43:23,495 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 00:43:33,513 INFO repeater task os150nb_lastensq starting 4.5 seconds late 2015-11-15 00:43:35,096 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 00:43:35,096 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:43:19 2015-11-15 00:45:20,278 INFO repeater task ashtech_hcorrstats starting 1.6 seconds late 2015-11-15 00:45:20,971 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-15 00:45:20,972 INFO repeater task phins3_hcorrstats starting 2.3 seconds late 2015-11-15 00:45:21,675 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-15 00:45:31,691 INFO repeater task os150nb_lastensq starting 2.7 seconds late 2015-11-15 00:45:33,239 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2015-11-15 00:45:33,239 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:45:19 2015-11-15 00:45:43,253 INFO repeater task os150nb_10pings starting 4.6 seconds late 2015-11-15 00:45:43,358 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 00:47:33,548 INFO repeater task os150nb_lastensq starting 4.5 seconds late 2015-11-15 00:47:35,133 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 00:47:35,134 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:47:19 2015-11-15 00:48:00,169 INFO repeater task os150nb_10pings starting 1.6 seconds late 2015-11-15 00:48:00,288 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 00:49:30,445 INFO repeater task os150nb_lastensq starting 1.4 seconds late 2015-11-15 00:49:32,037 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 00:49:32,038 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:49:19 2015-11-15 00:50:22,123 INFO repeater task os150nb_10pings starting 3.5 seconds late 2015-11-15 00:50:22,250 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 00:50:22,251 INFO repeater task ashtech_hcorrstats starting 3.6 seconds late 2015-11-15 00:50:22,944 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-15 00:50:22,945 INFO repeater task phins3_hcorrstats starting 4.3 seconds late 2015-11-15 00:50:23,635 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-15 00:51:33,758 INFO repeater task os150nb_lastensq starting 4.8 seconds late 2015-11-15 00:51:35,343 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 00:51:35,344 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:51:19 2015-11-15 00:52:40,459 INFO repeater task os150nb_10pings starting 1.8 seconds late 2015-11-15 00:52:40,573 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 00:53:30,665 INFO repeater task os150nb_lastensq starting 1.7 seconds late 2015-11-15 00:53:32,240 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 00:53:32,241 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:53:19 2015-11-15 00:55:02,401 INFO repeater task os150nb_10pings starting 3.8 seconds late 2015-11-15 00:55:02,523 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 00:55:22,559 INFO repeater task os150nb_quick starting 3.9 seconds late 2015-11-15 00:55:24,462 INFO repeater task os150nb_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2015-11-15 00:55:24,462 INFO repeater task os150nb_quick ended after 1.9 seconds 2015-11-15 00:55:24,462 INFO repeater task os150nb_3dayplots starting 5.8 seconds late 2015-11-15 00:55:26,940 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-11-15 00:55:26,940 INFO repeater task os150nb_3dayplots ended after 2.5 seconds 2015-11-15 00:55:26,941 INFO repeater task os150nb_tsstats starting 8.3 seconds late 2015-11-15 00:55:27,484 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2015-11-15 00:55:27,485 INFO repeater task os150nb_beamplots starting 8.9 seconds late 2015-11-15 00:55:28,542 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-15 00:55:28,542 INFO repeater task os150nb_beamplots ended after 1.1 seconds 2015-11-15 00:55:28,542 INFO repeater task os150nb_beamplots starting 9.9 seconds late 2015-11-15 00:55:29,608 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-15 00:55:29,608 INFO repeater task os150nb_beamplots ended after 1.1 seconds 2015-11-15 00:55:29,609 INFO repeater task ashtech_hcorrstats starting 11.0 seconds late 2015-11-15 00:55:30,312 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-15 00:55:30,312 INFO repeater task phins3_hcorrstats starting 11.7 seconds late 2015-11-15 00:55:30,999 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-15 00:55:36,005 INFO repeater task os150nb_lastensq starting 7.0 seconds late 2015-11-15 00:55:37,525 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2015-11-15 00:55:37,525 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:55:19 2015-11-15 00:57:22,709 INFO repeater task os150nb_10pings starting 4.1 seconds late 2015-11-15 00:57:22,828 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 00:57:32,844 INFO repeater task os150nb_lastensq starting 3.8 seconds late 2015-11-15 00:57:34,412 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 00:57:34,412 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:57:19 2015-11-15 00:59:29,615 INFO repeater task os150nb_lastensq starting 0.6 seconds late 2015-11-15 00:59:31,227 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 00:59:31,228 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 00:59:19 2015-11-15 00:59:41,244 INFO repeater task os150nb_10pings starting 2.6 seconds late 2015-11-15 00:59:41,371 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 01:00:21,418 INFO repeater task ashtech_hcorrstats starting 2.8 seconds late 2015-11-15 01:00:22,121 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-15 01:00:22,121 INFO repeater task phins3_hcorrstats starting 3.5 seconds late 2015-11-15 01:00:22,802 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-15 01:01:32,926 INFO repeater task os150nb_lastensq starting 3.9 seconds late 2015-11-15 01:01:34,487 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 01:01:34,487 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 01:01:19 2015-11-15 01:01:59,529 INFO repeater task os150nb_10pings starting 0.9 seconds late 2015-11-15 01:01:59,655 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 01:03:29,814 INFO repeater task os150nb_lastensq starting 0.8 seconds late 2015-11-15 01:03:31,410 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 01:03:31,410 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 01:03:19 2015-11-15 01:04:21,496 INFO repeater task os150nb_10pings starting 2.9 seconds late 2015-11-15 01:04:21,615 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 01:05:21,727 INFO repeater task ashtech_hcorrstats starting 3.1 seconds late 2015-11-15 01:05:22,435 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-15 01:05:22,436 INFO repeater task phins3_hcorrstats starting 3.8 seconds late 2015-11-15 01:05:23,129 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-15 01:05:33,141 INFO repeater task os150nb_lastensq starting 4.1 seconds late 2015-11-15 01:05:34,742 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 01:05:34,742 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 01:05:19 2015-11-15 01:06:39,858 INFO repeater task os150nb_10pings starting 1.2 seconds late 2015-11-15 01:06:39,974 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 01:07:30,061 INFO repeater task os150nb_lastensq starting 1.1 seconds late 2015-11-15 01:07:31,668 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 01:07:31,668 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 01:07:19 2015-11-15 01:09:01,815 INFO repeater task os150nb_10pings starting 3.2 seconds late 2015-11-15 01:09:01,932 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 01:09:31,982 INFO repeater task os150nb_lastensq starting 3.0 seconds late 2015-11-15 01:09:33,524 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2015-11-15 01:09:33,525 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 01:09:19 2015-11-15 01:10:23,596 INFO repeater task os150nb_quick starting 5.0 seconds late 2015-11-15 01:10:25,517 INFO repeater task os150nb_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2015-11-15 01:10:25,518 INFO repeater task os150nb_quick ended after 1.9 seconds 2015-11-15 01:10:25,518 INFO repeater task os150nb_3dayplots starting 6.9 seconds late 2015-11-15 01:10:28,051 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-11-15 01:10:28,051 INFO repeater task os150nb_3dayplots ended after 2.5 seconds 2015-11-15 01:10:28,051 INFO repeater task os150nb_tsstats starting 9.4 seconds late 2015-11-15 01:10:28,596 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2015-11-15 01:10:28,596 INFO repeater task os150nb_beamplots starting 10.0 seconds late 2015-11-15 01:10:29,614 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-15 01:10:29,614 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2015-11-15 01:10:29,614 INFO repeater task os150nb_beamplots starting 11.0 seconds late 2015-11-15 01:10:30,670 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-15 01:10:30,670 INFO repeater task os150nb_beamplots ended after 1.1 seconds 2015-11-15 01:10:30,670 INFO repeater task ashtech_hcorrstats starting 12.0 seconds late 2015-11-15 01:10:31,363 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-15 01:10:31,364 INFO repeater task phins3_hcorrstats starting 12.7 seconds late 2015-11-15 01:10:32,038 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-15 01:11:22,125 INFO repeater task os150nb_10pings starting 3.5 seconds late 2015-11-15 01:11:22,249 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 01:11:32,265 INFO repeater task os150nb_lastensq starting 3.3 seconds late 2015-11-15 01:11:33,866 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 01:11:33,866 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 01:11:19 2015-11-15 01:13:29,024 INFO repeater task os150nb_lastensq starting 0.0 seconds late 2015-11-15 01:13:30,582 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 01:13:30,582 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 01:13:19 2015-11-15 01:13:40,599 INFO repeater task os150nb_10pings starting 2.0 seconds late 2015-11-15 01:13:40,721 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 01:15:20,907 INFO repeater task ashtech_hcorrstats starting 2.3 seconds late 2015-11-15 01:15:21,599 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-15 01:15:21,599 INFO repeater task phins3_hcorrstats starting 3.0 seconds late 2015-11-15 01:15:22,279 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-15 01:15:32,294 INFO repeater task os150nb_lastensq starting 3.3 seconds late 2015-11-15 01:15:33,876 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 01:15:33,877 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 01:15:19 2015-11-15 01:15:58,921 INFO repeater task os150nb_10pings starting 0.3 seconds late 2015-11-15 01:15:59,048 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 01:17:29,211 INFO repeater task os150nb_lastensq starting 0.2 seconds late 2015-11-15 01:17:30,765 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 01:17:30,765 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 01:17:19 2015-11-15 01:18:20,854 INFO repeater task os150nb_10pings starting 2.2 seconds late 2015-11-15 01:18:20,976 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 01:19:31,104 INFO repeater task os150nb_lastensq starting 2.1 seconds late 2015-11-15 01:19:32,647 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2015-11-15 01:19:32,647 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 01:19:19 2015-11-15 01:20:22,733 INFO repeater task ashtech_hcorrstats starting 4.1 seconds late 2015-11-15 01:20:23,458 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-15 01:20:23,459 INFO repeater task phins3_hcorrstats starting 4.8 seconds late 2015-11-15 01:20:24,153 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-15 01:20:39,174 INFO repeater task os150nb_10pings starting 0.6 seconds late 2015-11-15 01:20:39,301 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 01:21:29,392 INFO repeater task os150nb_lastensq starting 0.4 seconds late 2015-11-15 01:21:30,987 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 01:21:30,987 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 01:21:19 2015-11-15 01:23:01,110 INFO repeater task os150nb_10pings starting 2.5 seconds late 2015-11-15 01:23:01,227 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 01:23:31,274 INFO repeater task os150nb_lastensq starting 2.3 seconds late 2015-11-15 01:23:32,901 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 01:23:32,901 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 01:23:19 2015-11-15 01:25:23,081 INFO repeater task os150nb_10pings starting 4.5 seconds late 2015-11-15 01:25:23,203 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 01:25:23,203 INFO repeater task os150nb_quick starting 4.6 seconds late 2015-11-15 01:25:25,100 INFO repeater task os150nb_quick output: /usr/lib/python2.7/dist-packages/numpy/core/_methods.py:55: RuntimeWarning: Mean of empty slice. warnings.warn("Mean of empty slice.", RuntimeWarning) 2015-11-15 01:25:25,100 INFO repeater task os150nb_quick ended after 1.9 seconds 2015-11-15 01:25:25,100 INFO repeater task os150nb_3dayplots starting 6.5 seconds late 2015-11-15 01:25:27,597 INFO repeater task os150nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-11-15 01:25:27,597 INFO repeater task os150nb_3dayplots ended after 2.5 seconds 2015-11-15 01:25:27,598 INFO repeater task os150nb_tsstats starting 9.0 seconds late 2015-11-15 01:25:28,133 INFO repeater task os150nb_tsstats ended after 0.5 seconds 2015-11-15 01:25:28,134 INFO repeater task os150nb_beamplots starting 9.5 seconds late 2015-11-15 01:25:29,149 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-15 01:25:29,149 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2015-11-15 01:25:29,149 INFO repeater task os150nb_beamplots starting 10.5 seconds late 2015-11-15 01:25:30,177 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-11-15 01:25:30,177 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2015-11-15 01:25:30,178 INFO repeater task ashtech_hcorrstats starting 11.5 seconds late 2015-11-15 01:25:30,863 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-15 01:25:30,863 INFO repeater task phins3_hcorrstats starting 12.2 seconds late 2015-11-15 01:25:31,545 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-15 01:25:36,550 INFO repeater task os150nb_lastensq starting 7.6 seconds late 2015-11-15 01:25:38,139 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 01:25:38,140 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 01:25:19 2015-11-15 01:27:33,344 INFO repeater task os150nb_lastensq starting 4.3 seconds late 2015-11-15 01:27:34,953 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 01:27:34,954 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 01:27:19 2015-11-15 01:27:39,961 INFO repeater task os150nb_10pings starting 1.3 seconds late 2015-11-15 01:27:40,083 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 01:29:30,269 INFO repeater task os150nb_lastensq starting 1.3 seconds late 2015-11-15 01:29:31,848 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 01:29:31,848 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 01:29:19 2015-11-15 01:30:01,902 INFO repeater task os150nb_10pings starting 3.3 seconds late 2015-11-15 01:30:02,026 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 01:30:22,059 INFO repeater task ashtech_hcorrstats starting 3.4 seconds late 2015-11-15 01:30:22,786 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-15 01:30:22,787 INFO repeater task phins3_hcorrstats starting 4.2 seconds late 2015-11-15 01:30:23,479 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-15 01:31:33,602 INFO repeater task os150nb_lastensq starting 4.6 seconds late 2015-11-15 01:31:35,212 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 01:31:35,212 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 01:31:19 2015-11-15 01:32:20,295 INFO repeater task os150nb_10pings starting 1.7 seconds late 2015-11-15 01:32:20,417 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 01:33:30,534 INFO repeater task os150nb_lastensq starting 1.5 seconds late 2015-11-15 01:33:32,122 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 01:33:32,123 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 01:33:19 2015-11-15 01:34:42,248 INFO repeater task os150nb_10pings starting 3.6 seconds late 2015-11-15 01:34:42,371 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 01:35:22,444 INFO repeater task ashtech_hcorrstats starting 3.8 seconds late 2015-11-15 01:35:23,144 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2015-11-15 01:35:23,145 INFO repeater task phins3_hcorrstats starting 4.5 seconds late 2015-11-15 01:35:23,827 INFO repeater task phins3_hcorrstats ended after 0.7 seconds 2015-11-15 01:35:33,839 INFO repeater task os150nb_lastensq starting 4.8 seconds late 2015-11-15 01:35:35,434 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 01:35:35,434 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 01:35:19 2015-11-15 01:37:00,590 INFO repeater task os150nb_10pings starting 2.0 seconds late 2015-11-15 01:37:00,707 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 01:37:30,755 INFO repeater task os150nb_lastensq starting 1.8 seconds late 2015-11-15 01:37:32,314 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2015-11-15 01:37:32,315 INFO repeater task os150nb_lastensq: Time from /home/data/RR1516/proc/os150nb/load/wrote_ens_time: 2015/11/15 01:37:19 2015-11-15 01:39:22,520 INFO repeater task os150nb_10pings starting 3.9 seconds late 2015-11-15 01:39:22,642 INFO repeater task os150nb_10pings ended after 0.1 seconds 2015-11-15 01:39:32,658 INFO repeater task os150nb_lastensq starting 3.7 seconds late 2015-11-15 01:39:34,256 INFO repeater task os150nb_lastensq ended after 1.6 seconds