2016-06-28 06:44:16,719 INFO repeater task os75nb_10pings starting 17.5 seconds late 2016-06-28 06:44:16,834 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 06:44:16,834 INFO repeater task os75nb_quick starting 17.6 seconds late 2016-06-28 06:44:19,010 INFO repeater task os75nb_quick ended after 2.2 seconds 2016-06-28 06:44:19,010 INFO repeater task os75nb_3dayplots starting 19.8 seconds late 2016-06-28 06:44:22,052 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2016-06-28 06:44:22,052 INFO repeater task os75nb_3dayplots ended after 3.0 seconds 2016-06-28 06:44:22,052 INFO repeater task os75nb_tsstats starting 22.9 seconds late 2016-06-28 06:44:22,617 INFO repeater task os75nb_tsstats ended after 0.6 seconds 2016-06-28 06:44:22,617 INFO repeater task os150bb_beamplots starting 23.4 seconds late 2016-06-28 06:44:23,574 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 06:44:23,574 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2016-06-28 06:44:23,575 INFO repeater task os150bb_beamplots starting 24.4 seconds late 2016-06-28 06:44:24,562 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 06:44:24,562 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2016-06-28 06:44:24,562 INFO repeater task os150nb_beamplots starting 25.4 seconds late 2016-06-28 06:44:25,499 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 06:44:25,499 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2016-06-28 06:44:25,499 INFO repeater task os150nb_beamplots starting 26.3 seconds late 2016-06-28 06:44:26,442 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 06:44:26,442 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2016-06-28 06:44:26,442 INFO repeater task os75nb_beamplots starting 27.3 seconds late 2016-06-28 06:44:27,454 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 06:44:27,454 INFO repeater task os75nb_beamplots ended after 1.0 seconds 2016-06-28 06:44:27,454 INFO repeater task os75nb_beamplots starting 28.3 seconds late 2016-06-28 06:44:28,437 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 06:44:28,437 INFO repeater task os75nb_beamplots ended after 1.0 seconds 2016-06-28 06:44:28,437 INFO repeater task seapath_hcorrstats starting 29.2 seconds late 2016-06-28 06:44:29,122 INFO repeater task seapath_hcorrstats ended after 0.7 seconds 2016-06-28 06:44:29,122 INFO repeater task ashtech_hcorrstats starting 29.9 seconds late 2016-06-28 06:44:29,799 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2016-06-28 06:44:34,805 INFO repeater task os150bb_lastensq starting 25.8 seconds late 2016-06-28 06:44:36,470 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 06:44:36,470 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 06:43:59 2016-06-28 06:44:36,471 INFO repeater task os150nb_lastensq starting 27.5 seconds late 2016-06-28 06:44:37,954 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2016-06-28 06:44:37,954 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 06:43:59 2016-06-28 06:45:43,143 INFO repeater task os150bb_10pings starting 4.0 seconds late 2016-06-28 06:45:43,269 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 06:45:43,270 INFO repeater task os150nb_10pings starting 4.1 seconds late 2016-06-28 06:45:43,386 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 06:46:13,464 INFO repeater task os150bb_lastensq starting 4.5 seconds late 2016-06-28 06:46:15,186 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 06:46:15,186 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 06:45:59 2016-06-28 06:46:15,187 INFO repeater task os150nb_lastensq starting 6.2 seconds late 2016-06-28 06:46:16,623 INFO repeater task os150nb_lastensq ended after 1.4 seconds 2016-06-28 06:46:16,623 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 06:45:59 2016-06-28 06:47:11,778 INFO repeater task os75nb_lastensq starting 2.8 seconds late 2016-06-28 06:47:13,094 INFO repeater task os75nb_lastensq ended after 1.3 seconds 2016-06-28 06:47:13,094 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 06:46:59 2016-06-28 06:47:23,122 INFO repeater task os75nb_10pings starting 3.9 seconds late 2016-06-28 06:47:23,237 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 06:48:03,350 INFO repeater task os150bb_10pings starting 4.2 seconds late 2016-06-28 06:48:03,475 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 06:48:03,476 INFO repeater task os150nb_10pings starting 4.3 seconds late 2016-06-28 06:48:03,590 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 06:48:13,614 INFO repeater task os150bb_lastensq starting 4.6 seconds late 2016-06-28 06:48:15,452 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2016-06-28 06:48:15,453 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 06:47:59 2016-06-28 06:48:15,454 INFO repeater task os150nb_lastensq starting 6.5 seconds late 2016-06-28 06:48:16,775 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2016-06-28 06:48:16,775 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 06:47:59 2016-06-28 06:49:01,908 INFO repeater task seapath_hcorrstats starting 2.7 seconds late 2016-06-28 06:49:02,575 INFO repeater task seapath_hcorrstats ended after 0.7 seconds 2016-06-28 06:49:02,575 INFO repeater task ashtech_hcorrstats starting 3.4 seconds late 2016-06-28 06:49:03,299 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2016-06-28 06:50:13,490 INFO repeater task os150bb_lastensq starting 4.5 seconds late 2016-06-28 06:50:15,203 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 06:50:15,203 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 06:49:59 2016-06-28 06:50:15,204 INFO repeater task os150nb_lastensq starting 6.2 seconds late 2016-06-28 06:50:16,693 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2016-06-28 06:50:16,693 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 06:49:59 2016-06-28 06:50:16,694 INFO repeater task os75nb_lastensq starting 7.7 seconds late 2016-06-28 06:50:18,029 INFO repeater task os75nb_lastensq ended after 1.3 seconds 2016-06-28 06:50:18,029 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 06:49:59 2016-06-28 06:50:23,037 INFO repeater task os150bb_10pings starting 3.9 seconds late 2016-06-28 06:50:23,161 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 06:50:23,162 INFO repeater task os150nb_10pings starting 4.0 seconds late 2016-06-28 06:50:23,276 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 06:50:43,325 INFO repeater task os75nb_10pings starting 4.1 seconds late 2016-06-28 06:50:43,446 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 06:52:13,837 INFO repeater task os150bb_lastensq starting 4.8 seconds late 2016-06-28 06:52:15,517 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 06:52:15,517 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 06:51:59 2016-06-28 06:52:15,518 INFO repeater task os150nb_lastensq starting 6.5 seconds late 2016-06-28 06:52:16,852 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2016-06-28 06:52:16,852 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 06:51:59 2016-06-28 06:52:41,916 INFO repeater task os150bb_10pings starting 2.7 seconds late 2016-06-28 06:52:42,037 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 06:52:42,038 INFO repeater task os150nb_10pings starting 2.9 seconds late 2016-06-28 06:52:42,145 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 06:53:12,229 INFO repeater task os75nb_lastensq starting 3.2 seconds late 2016-06-28 06:53:13,696 INFO repeater task os75nb_lastensq ended after 1.5 seconds 2016-06-28 06:53:13,697 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 06:52:59 2016-06-28 06:54:03,841 INFO repeater task os75nb_10pings starting 4.7 seconds late 2016-06-28 06:54:03,959 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 06:54:03,960 INFO repeater task seapath_hcorrstats starting 4.8 seconds late 2016-06-28 06:54:04,673 INFO repeater task seapath_hcorrstats ended after 0.7 seconds 2016-06-28 06:54:04,673 INFO repeater task ashtech_hcorrstats starting 5.5 seconds late 2016-06-28 06:54:05,350 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2016-06-28 06:54:10,356 INFO repeater task os150bb_lastensq starting 1.4 seconds late 2016-06-28 06:54:12,071 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 06:54:12,071 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 06:53:59 2016-06-28 06:54:12,072 INFO repeater task os150nb_lastensq starting 3.1 seconds late 2016-06-28 06:54:13,529 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2016-06-28 06:54:13,530 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 06:53:59 2016-06-28 06:55:03,653 INFO repeater task os150bb_10pings starting 4.5 seconds late 2016-06-28 06:55:03,775 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 06:55:03,776 INFO repeater task os150nb_10pings starting 4.6 seconds late 2016-06-28 06:55:03,886 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 06:56:09,070 INFO repeater task os150bb_lastensq starting 0.1 seconds late 2016-06-28 06:56:10,789 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 06:56:10,789 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 06:55:59 2016-06-28 06:56:10,790 INFO repeater task os150nb_lastensq starting 1.8 seconds late 2016-06-28 06:56:12,238 INFO repeater task os150nb_lastensq ended after 1.4 seconds 2016-06-28 06:56:12,238 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 06:55:59 2016-06-28 06:56:12,239 INFO repeater task os75nb_lastensq starting 3.2 seconds late 2016-06-28 06:56:13,580 INFO repeater task os75nb_lastensq ended after 1.3 seconds 2016-06-28 06:56:13,581 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 06:55:59 2016-06-28 06:57:23,775 INFO repeater task os150bb_10pings starting 4.6 seconds late 2016-06-28 06:57:23,898 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 06:57:23,899 INFO repeater task os150nb_10pings starting 4.7 seconds late 2016-06-28 06:57:24,012 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 06:57:24,013 INFO repeater task os75nb_10pings starting 4.8 seconds late 2016-06-28 06:57:24,125 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 06:58:09,241 INFO repeater task os150bb_lastensq starting 0.2 seconds late 2016-06-28 06:58:11,086 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2016-06-28 06:58:11,086 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 06:57:59 2016-06-28 06:58:11,087 INFO repeater task os150nb_lastensq starting 2.1 seconds late 2016-06-28 06:58:12,424 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2016-06-28 06:58:12,424 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 06:57:59 2016-06-28 06:59:02,573 INFO repeater task os150bb_quick starting 3.4 seconds late 2016-06-28 06:59:05,433 INFO repeater task os150bb_quick ended after 2.9 seconds 2016-06-28 06:59:05,433 INFO repeater task os150bb_3dayplots starting 6.2 seconds late 2016-06-28 06:59:08,588 INFO repeater task os150bb_3dayplots ended after 3.2 seconds 2016-06-28 06:59:08,588 INFO repeater task os150bb_tsstats starting 9.4 seconds late 2016-06-28 06:59:09,157 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2016-06-28 06:59:09,158 INFO repeater task os150nb_quick starting 10.0 seconds late 2016-06-28 06:59:11,999 INFO repeater task os150nb_quick ended after 2.8 seconds 2016-06-28 06:59:12,000 INFO repeater task os150nb_3dayplots starting 12.8 seconds late 2016-06-28 06:59:15,235 INFO repeater task os150nb_3dayplots ended after 3.2 seconds 2016-06-28 06:59:15,235 INFO repeater task os150nb_tsstats starting 16.0 seconds late 2016-06-28 06:59:15,810 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2016-06-28 06:59:15,811 INFO repeater task os75nb_lastensq starting 6.8 seconds late 2016-06-28 06:59:17,261 INFO repeater task os75nb_lastensq ended after 1.5 seconds 2016-06-28 06:59:17,262 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 06:58:59 2016-06-28 06:59:17,262 INFO repeater task os75nb_quick starting 18.1 seconds late 2016-06-28 06:59:19,386 INFO repeater task os75nb_quick ended after 2.1 seconds 2016-06-28 06:59:19,386 INFO repeater task os75nb_3dayplots starting 20.2 seconds late 2016-06-28 06:59:22,390 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2016-06-28 06:59:22,390 INFO repeater task os75nb_3dayplots ended after 3.0 seconds 2016-06-28 06:59:22,390 INFO repeater task os75nb_tsstats starting 23.2 seconds late 2016-06-28 06:59:22,947 INFO repeater task os75nb_tsstats ended after 0.6 seconds 2016-06-28 06:59:22,948 INFO repeater task os150bb_beamplots starting 23.8 seconds late 2016-06-28 06:59:23,911 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 06:59:23,911 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2016-06-28 06:59:23,911 INFO repeater task os150bb_beamplots starting 24.7 seconds late 2016-06-28 06:59:24,860 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 06:59:24,860 INFO repeater task os150bb_beamplots ended after 0.9 seconds 2016-06-28 06:59:24,860 INFO repeater task os150nb_beamplots starting 25.7 seconds late 2016-06-28 06:59:25,854 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 06:59:25,854 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2016-06-28 06:59:25,854 INFO repeater task os150nb_beamplots starting 26.7 seconds late 2016-06-28 06:59:26,834 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 06:59:26,834 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2016-06-28 06:59:26,834 INFO repeater task os75nb_beamplots starting 27.6 seconds late 2016-06-28 06:59:27,857 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 06:59:27,858 INFO repeater task os75nb_beamplots ended after 1.0 seconds 2016-06-28 06:59:27,858 INFO repeater task os75nb_beamplots starting 28.7 seconds late 2016-06-28 06:59:28,835 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 06:59:28,836 INFO repeater task os75nb_beamplots ended after 1.0 seconds 2016-06-28 06:59:28,836 INFO repeater task seapath_hcorrstats starting 29.6 seconds late 2016-06-28 06:59:29,501 INFO repeater task seapath_hcorrstats ended after 0.7 seconds 2016-06-28 06:59:29,502 INFO repeater task ashtech_hcorrstats starting 30.3 seconds late 2016-06-28 06:59:30,179 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2016-06-28 06:59:40,193 INFO repeater task os150bb_10pings starting 1.0 seconds late 2016-06-28 06:59:40,322 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 06:59:40,323 INFO repeater task os150nb_10pings starting 1.1 seconds late 2016-06-28 06:59:40,433 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:00:10,658 INFO repeater task os150bb_lastensq starting 1.7 seconds late 2016-06-28 07:00:12,339 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 07:00:12,340 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 06:59:59 2016-06-28 07:00:12,340 INFO repeater task os150nb_lastensq starting 3.3 seconds late 2016-06-28 07:00:13,653 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2016-06-28 07:00:13,653 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 06:59:59 2016-06-28 07:00:43,737 INFO repeater task os75nb_10pings starting 4.5 seconds late 2016-06-28 07:00:43,858 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 07:02:04,085 INFO repeater task os150bb_10pings starting 4.9 seconds late 2016-06-28 07:02:04,207 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 07:02:04,208 INFO repeater task os150nb_10pings starting 5.0 seconds late 2016-06-28 07:02:04,317 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:02:09,325 INFO repeater task os150bb_lastensq starting 0.3 seconds late 2016-06-28 07:02:11,022 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 07:02:11,022 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:01:59 2016-06-28 07:02:11,023 INFO repeater task os150nb_lastensq starting 2.0 seconds late 2016-06-28 07:02:12,333 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2016-06-28 07:02:12,333 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:01:59 2016-06-28 07:02:12,334 INFO repeater task os75nb_lastensq starting 3.3 seconds late 2016-06-28 07:02:13,832 INFO repeater task os75nb_lastensq ended after 1.5 seconds 2016-06-28 07:02:13,832 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 07:01:59 2016-06-28 07:04:04,133 INFO repeater task os75nb_10pings starting 4.9 seconds late 2016-06-28 07:04:04,256 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 07:04:04,257 INFO repeater task seapath_hcorrstats starting 5.1 seconds late 2016-06-28 07:04:04,923 INFO repeater task seapath_hcorrstats ended after 0.7 seconds 2016-06-28 07:04:04,924 INFO repeater task ashtech_hcorrstats starting 5.7 seconds late 2016-06-28 07:04:05,597 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2016-06-28 07:04:10,603 INFO repeater task os150bb_lastensq starting 1.6 seconds late 2016-06-28 07:04:12,254 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 07:04:12,255 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:03:59 2016-06-28 07:04:12,255 INFO repeater task os150nb_lastensq starting 3.3 seconds late 2016-06-28 07:04:13,756 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2016-06-28 07:04:13,757 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:03:59 2016-06-28 07:04:23,778 INFO repeater task os150bb_10pings starting 4.6 seconds late 2016-06-28 07:04:23,897 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 07:04:23,898 INFO repeater task os150nb_10pings starting 4.7 seconds late 2016-06-28 07:04:24,016 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:05:09,150 INFO repeater task os75nb_lastensq starting 0.2 seconds late 2016-06-28 07:05:10,494 INFO repeater task os75nb_lastensq ended after 1.3 seconds 2016-06-28 07:05:10,494 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 07:04:59 2016-06-28 07:06:10,635 INFO repeater task os150bb_lastensq starting 1.6 seconds late 2016-06-28 07:06:12,459 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2016-06-28 07:06:12,460 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:05:59 2016-06-28 07:06:12,460 INFO repeater task os150nb_lastensq starting 3.5 seconds late 2016-06-28 07:06:13,807 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2016-06-28 07:06:13,807 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:05:59 2016-06-28 07:06:43,883 INFO repeater task os150bb_10pings starting 4.7 seconds late 2016-06-28 07:06:44,000 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 07:06:44,001 INFO repeater task os150nb_10pings starting 4.8 seconds late 2016-06-28 07:06:44,114 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:07:19,215 INFO repeater task os75nb_10pings starting 0.0 seconds late 2016-06-28 07:07:19,340 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 07:08:09,481 INFO repeater task os150bb_lastensq starting 0.5 seconds late 2016-06-28 07:08:11,180 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 07:08:11,180 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:07:59 2016-06-28 07:08:11,181 INFO repeater task os150nb_lastensq starting 2.2 seconds late 2016-06-28 07:08:12,738 INFO repeater task os150nb_lastensq ended after 1.6 seconds 2016-06-28 07:08:12,738 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:07:59 2016-06-28 07:08:12,739 INFO repeater task os75nb_lastensq starting 3.7 seconds late 2016-06-28 07:08:14,126 INFO repeater task os75nb_lastensq ended after 1.4 seconds 2016-06-28 07:08:14,127 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 07:07:59 2016-06-28 07:08:59,219 INFO repeater task os150bb_10pings starting 0.0 seconds late 2016-06-28 07:08:59,345 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 07:08:59,346 INFO repeater task os150nb_10pings starting 0.2 seconds late 2016-06-28 07:08:59,450 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:08:59,453 INFO repeater task seapath_hcorrstats starting 0.3 seconds late 2016-06-28 07:09:00,116 INFO repeater task seapath_hcorrstats ended after 0.7 seconds 2016-06-28 07:09:00,116 INFO repeater task ashtech_hcorrstats starting 0.9 seconds late 2016-06-28 07:09:00,832 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2016-06-28 07:10:11,019 INFO repeater task os150bb_lastensq starting 2.0 seconds late 2016-06-28 07:10:12,745 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 07:10:12,745 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:09:59 2016-06-28 07:10:12,746 INFO repeater task os150nb_lastensq starting 3.7 seconds late 2016-06-28 07:10:14,149 INFO repeater task os150nb_lastensq ended after 1.4 seconds 2016-06-28 07:10:14,149 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:09:59 2016-06-28 07:10:39,223 INFO repeater task os75nb_10pings starting 0.0 seconds late 2016-06-28 07:10:39,339 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 07:11:09,402 INFO repeater task os75nb_lastensq starting 0.4 seconds late 2016-06-28 07:11:10,895 INFO repeater task os75nb_lastensq ended after 1.5 seconds 2016-06-28 07:11:10,895 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 07:10:59 2016-06-28 07:11:20,919 INFO repeater task os150bb_10pings starting 1.7 seconds late 2016-06-28 07:11:21,044 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 07:11:21,044 INFO repeater task os150nb_10pings starting 1.9 seconds late 2016-06-28 07:11:21,159 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:12:11,307 INFO repeater task os150bb_lastensq starting 2.3 seconds late 2016-06-28 07:12:13,016 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 07:12:13,016 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:11:59 2016-06-28 07:12:13,017 INFO repeater task os150nb_lastensq starting 4.0 seconds late 2016-06-28 07:12:14,499 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2016-06-28 07:12:14,500 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:11:59 2016-06-28 07:13:39,781 INFO repeater task os150bb_10pings starting 0.6 seconds late 2016-06-28 07:13:39,906 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 07:13:39,907 INFO repeater task os150nb_10pings starting 0.7 seconds late 2016-06-28 07:13:40,022 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:14:00,051 INFO repeater task os150bb_quick starting 0.9 seconds late 2016-06-28 07:14:02,916 INFO repeater task os150bb_quick ended after 2.9 seconds 2016-06-28 07:14:02,916 INFO repeater task os150bb_3dayplots starting 3.7 seconds late 2016-06-28 07:14:06,034 INFO repeater task os150bb_3dayplots ended after 3.1 seconds 2016-06-28 07:14:06,035 INFO repeater task os150bb_tsstats starting 6.8 seconds late 2016-06-28 07:14:06,611 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2016-06-28 07:14:06,612 INFO repeater task os150nb_quick starting 7.4 seconds late 2016-06-28 07:14:09,462 INFO repeater task os150nb_quick ended after 2.8 seconds 2016-06-28 07:14:09,462 INFO repeater task os150nb_3dayplots starting 10.3 seconds late 2016-06-28 07:14:12,701 INFO repeater task os150nb_3dayplots ended after 3.2 seconds 2016-06-28 07:14:12,701 INFO repeater task os150nb_tsstats starting 13.5 seconds late 2016-06-28 07:14:13,285 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2016-06-28 07:14:13,285 INFO repeater task os75nb_lastensq starting 4.3 seconds late 2016-06-28 07:14:14,648 INFO repeater task os75nb_lastensq ended after 1.4 seconds 2016-06-28 07:14:14,648 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 07:13:59 2016-06-28 07:14:14,649 INFO repeater task os75nb_10pings starting 15.5 seconds late 2016-06-28 07:14:14,762 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 07:14:14,762 INFO repeater task os75nb_quick starting 15.6 seconds late 2016-06-28 07:14:16,891 INFO repeater task os75nb_quick ended after 2.1 seconds 2016-06-28 07:14:16,891 INFO repeater task os75nb_3dayplots starting 17.7 seconds late 2016-06-28 07:14:19,944 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2016-06-28 07:14:19,944 INFO repeater task os75nb_3dayplots ended after 3.1 seconds 2016-06-28 07:14:19,944 INFO repeater task os75nb_tsstats starting 20.8 seconds late 2016-06-28 07:14:20,506 INFO repeater task os75nb_tsstats ended after 0.6 seconds 2016-06-28 07:14:20,507 INFO repeater task os150bb_beamplots starting 21.3 seconds late 2016-06-28 07:14:21,465 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 07:14:21,465 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2016-06-28 07:14:21,465 INFO repeater task os150bb_beamplots starting 22.3 seconds late 2016-06-28 07:14:22,439 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 07:14:22,439 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2016-06-28 07:14:22,439 INFO repeater task os150nb_beamplots starting 23.2 seconds late 2016-06-28 07:14:23,370 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 07:14:23,370 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2016-06-28 07:14:23,370 INFO repeater task os150nb_beamplots starting 24.2 seconds late 2016-06-28 07:14:24,310 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 07:14:24,311 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2016-06-28 07:14:24,311 INFO repeater task os75nb_beamplots starting 25.1 seconds late 2016-06-28 07:14:25,290 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 07:14:25,290 INFO repeater task os75nb_beamplots ended after 1.0 seconds 2016-06-28 07:14:25,290 INFO repeater task os75nb_beamplots starting 26.1 seconds late 2016-06-28 07:14:26,261 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 07:14:26,261 INFO repeater task os75nb_beamplots ended after 1.0 seconds 2016-06-28 07:14:26,261 INFO repeater task seapath_hcorrstats starting 27.1 seconds late 2016-06-28 07:14:26,930 INFO repeater task seapath_hcorrstats ended after 0.7 seconds 2016-06-28 07:14:26,930 INFO repeater task ashtech_hcorrstats starting 27.7 seconds late 2016-06-28 07:14:27,599 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2016-06-28 07:14:32,605 INFO repeater task os150bb_lastensq starting 23.6 seconds late 2016-06-28 07:14:34,335 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 07:14:34,335 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:13:59 2016-06-28 07:14:34,336 INFO repeater task os150nb_lastensq starting 25.3 seconds late 2016-06-28 07:14:35,685 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2016-06-28 07:14:35,686 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:13:59 2016-06-28 07:16:00,883 INFO repeater task os150bb_10pings starting 1.7 seconds late 2016-06-28 07:16:01,008 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 07:16:01,009 INFO repeater task os150nb_10pings starting 1.8 seconds late 2016-06-28 07:16:01,123 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:16:11,148 INFO repeater task os150bb_lastensq starting 2.1 seconds late 2016-06-28 07:16:12,949 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2016-06-28 07:16:12,950 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:15:59 2016-06-28 07:16:12,950 INFO repeater task os150nb_lastensq starting 4.0 seconds late 2016-06-28 07:16:14,264 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2016-06-28 07:16:14,264 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:15:59 2016-06-28 07:17:09,415 INFO repeater task os75nb_lastensq starting 0.4 seconds late 2016-06-28 07:17:10,907 INFO repeater task os75nb_lastensq ended after 1.5 seconds 2016-06-28 07:17:10,907 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 07:16:59 2016-06-28 07:17:20,931 INFO repeater task os75nb_10pings starting 1.7 seconds late 2016-06-28 07:17:21,047 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 07:18:11,160 INFO repeater task os150bb_lastensq starting 2.2 seconds late 2016-06-28 07:18:12,837 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 07:18:12,838 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:17:59 2016-06-28 07:18:12,838 INFO repeater task os150nb_lastensq starting 3.8 seconds late 2016-06-28 07:18:14,307 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2016-06-28 07:18:14,307 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:17:59 2016-06-28 07:18:19,315 INFO repeater task os150bb_10pings starting 0.1 seconds late 2016-06-28 07:18:19,435 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 07:18:19,436 INFO repeater task os150nb_10pings starting 0.2 seconds late 2016-06-28 07:18:19,554 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:18:59,666 INFO repeater task seapath_hcorrstats starting 0.5 seconds late 2016-06-28 07:19:00,336 INFO repeater task seapath_hcorrstats ended after 0.7 seconds 2016-06-28 07:19:00,336 INFO repeater task ashtech_hcorrstats starting 1.1 seconds late 2016-06-28 07:19:01,002 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2016-06-28 07:20:11,188 INFO repeater task os150bb_lastensq starting 2.2 seconds late 2016-06-28 07:20:12,876 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 07:20:12,877 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:19:59 2016-06-28 07:20:12,877 INFO repeater task os150nb_lastensq starting 3.9 seconds late 2016-06-28 07:20:14,239 INFO repeater task os150nb_lastensq ended after 1.4 seconds 2016-06-28 07:20:14,240 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:19:59 2016-06-28 07:20:14,241 INFO repeater task os75nb_lastensq starting 5.2 seconds late 2016-06-28 07:20:15,697 INFO repeater task os75nb_lastensq ended after 1.5 seconds 2016-06-28 07:20:15,698 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 07:19:59 2016-06-28 07:20:40,768 INFO repeater task os150bb_10pings starting 1.6 seconds late 2016-06-28 07:20:40,890 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 07:20:40,891 INFO repeater task os150nb_10pings starting 1.7 seconds late 2016-06-28 07:20:41,012 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:20:41,012 INFO repeater task os75nb_10pings starting 1.8 seconds late 2016-06-28 07:20:41,127 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 07:22:11,380 INFO repeater task os150bb_lastensq starting 2.4 seconds late 2016-06-28 07:22:13,153 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2016-06-28 07:22:13,153 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:21:59 2016-06-28 07:22:13,154 INFO repeater task os150nb_lastensq starting 4.2 seconds late 2016-06-28 07:22:14,597 INFO repeater task os150nb_lastensq ended after 1.4 seconds 2016-06-28 07:22:14,597 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:21:59 2016-06-28 07:22:59,725 INFO repeater task os150bb_10pings starting 0.5 seconds late 2016-06-28 07:22:59,847 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 07:22:59,848 INFO repeater task os150nb_10pings starting 0.7 seconds late 2016-06-28 07:22:59,961 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:23:09,990 INFO repeater task os75nb_lastensq starting 1.0 seconds late 2016-06-28 07:23:11,365 INFO repeater task os75nb_lastensq ended after 1.4 seconds 2016-06-28 07:23:11,365 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 07:22:59 2016-06-28 07:24:01,507 INFO repeater task os75nb_10pings starting 2.3 seconds late 2016-06-28 07:24:01,626 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 07:24:01,627 INFO repeater task seapath_hcorrstats starting 2.4 seconds late 2016-06-28 07:24:02,313 INFO repeater task seapath_hcorrstats ended after 0.7 seconds 2016-06-28 07:24:02,313 INFO repeater task ashtech_hcorrstats starting 3.1 seconds late 2016-06-28 07:24:03,001 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2016-06-28 07:24:13,022 INFO repeater task os150bb_lastensq starting 4.0 seconds late 2016-06-28 07:24:14,868 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2016-06-28 07:24:14,868 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:23:59 2016-06-28 07:24:14,869 INFO repeater task os150nb_lastensq starting 5.9 seconds late 2016-06-28 07:24:16,237 INFO repeater task os150nb_lastensq ended after 1.4 seconds 2016-06-28 07:24:16,237 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:23:59 2016-06-28 07:25:21,421 INFO repeater task os150bb_10pings starting 2.2 seconds late 2016-06-28 07:25:21,545 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 07:25:21,546 INFO repeater task os150nb_10pings starting 2.4 seconds late 2016-06-28 07:25:21,662 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:26:11,807 INFO repeater task os150bb_lastensq starting 2.8 seconds late 2016-06-28 07:26:13,637 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2016-06-28 07:26:13,637 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:25:59 2016-06-28 07:26:13,638 INFO repeater task os150nb_lastensq starting 4.6 seconds late 2016-06-28 07:26:14,985 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2016-06-28 07:26:14,985 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:25:59 2016-06-28 07:26:14,986 INFO repeater task os75nb_lastensq starting 6.0 seconds late 2016-06-28 07:26:16,344 INFO repeater task os75nb_lastensq ended after 1.4 seconds 2016-06-28 07:26:16,345 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 07:25:59 2016-06-28 07:27:21,645 INFO repeater task os75nb_10pings starting 2.5 seconds late 2016-06-28 07:27:21,762 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 07:27:41,811 INFO repeater task os150bb_10pings starting 2.6 seconds late 2016-06-28 07:27:41,930 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 07:27:41,931 INFO repeater task os150nb_10pings starting 2.7 seconds late 2016-06-28 07:27:42,042 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:28:12,124 INFO repeater task os150bb_lastensq starting 3.1 seconds late 2016-06-28 07:28:13,825 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 07:28:13,825 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:27:59 2016-06-28 07:28:13,826 INFO repeater task os150nb_lastensq starting 4.8 seconds late 2016-06-28 07:28:15,154 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2016-06-28 07:28:15,154 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:27:59 2016-06-28 07:29:00,415 INFO repeater task os150bb_quick starting 1.2 seconds late 2016-06-28 07:29:03,376 INFO repeater task os150bb_quick ended after 3.0 seconds 2016-06-28 07:29:03,377 INFO repeater task os150bb_3dayplots starting 4.2 seconds late 2016-06-28 07:29:06,516 INFO repeater task os150bb_3dayplots ended after 3.1 seconds 2016-06-28 07:29:06,516 INFO repeater task os150bb_tsstats starting 7.3 seconds late 2016-06-28 07:29:07,091 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2016-06-28 07:29:07,092 INFO repeater task os150nb_quick starting 7.9 seconds late 2016-06-28 07:29:10,001 INFO repeater task os150nb_quick ended after 2.9 seconds 2016-06-28 07:29:10,002 INFO repeater task os150nb_3dayplots starting 10.8 seconds late 2016-06-28 07:29:13,170 INFO repeater task os150nb_3dayplots ended after 3.2 seconds 2016-06-28 07:29:13,170 INFO repeater task os150nb_tsstats starting 14.0 seconds late 2016-06-28 07:29:13,765 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2016-06-28 07:29:13,765 INFO repeater task os75nb_lastensq starting 4.8 seconds late 2016-06-28 07:29:15,122 INFO repeater task os75nb_lastensq ended after 1.4 seconds 2016-06-28 07:29:15,122 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 07:28:59 2016-06-28 07:29:15,123 INFO repeater task os75nb_quick starting 15.9 seconds late 2016-06-28 07:29:17,262 INFO repeater task os75nb_quick ended after 2.1 seconds 2016-06-28 07:29:17,262 INFO repeater task os75nb_3dayplots starting 18.1 seconds late 2016-06-28 07:29:20,515 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2016-06-28 07:29:20,515 INFO repeater task os75nb_3dayplots ended after 3.3 seconds 2016-06-28 07:29:20,515 INFO repeater task os75nb_tsstats starting 21.3 seconds late 2016-06-28 07:29:21,100 INFO repeater task os75nb_tsstats ended after 0.6 seconds 2016-06-28 07:29:21,100 INFO repeater task os150bb_beamplots starting 21.9 seconds late 2016-06-28 07:29:22,084 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 07:29:22,084 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2016-06-28 07:29:22,084 INFO repeater task os150bb_beamplots starting 22.9 seconds late 2016-06-28 07:29:23,089 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 07:29:23,089 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2016-06-28 07:29:23,089 INFO repeater task os150nb_beamplots starting 23.9 seconds late 2016-06-28 07:29:24,091 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 07:29:24,091 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2016-06-28 07:29:24,091 INFO repeater task os150nb_beamplots starting 24.9 seconds late 2016-06-28 07:29:25,083 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 07:29:25,083 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2016-06-28 07:29:25,084 INFO repeater task os75nb_beamplots starting 25.9 seconds late 2016-06-28 07:29:26,113 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 07:29:26,113 INFO repeater task os75nb_beamplots ended after 1.0 seconds 2016-06-28 07:29:26,114 INFO repeater task os75nb_beamplots starting 26.9 seconds late 2016-06-28 07:29:27,090 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 07:29:27,091 INFO repeater task os75nb_beamplots ended after 1.0 seconds 2016-06-28 07:29:27,091 INFO repeater task seapath_hcorrstats starting 27.9 seconds late 2016-06-28 07:29:27,758 INFO repeater task seapath_hcorrstats ended after 0.7 seconds 2016-06-28 07:29:27,759 INFO repeater task ashtech_hcorrstats starting 28.6 seconds late 2016-06-28 07:29:28,445 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2016-06-28 07:30:03,494 INFO repeater task os150bb_10pings starting 4.3 seconds late 2016-06-28 07:30:03,607 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 07:30:03,608 INFO repeater task os150nb_10pings starting 4.4 seconds late 2016-06-28 07:30:03,721 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:30:13,743 INFO repeater task os150bb_lastensq starting 4.7 seconds late 2016-06-28 07:30:15,492 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 07:30:15,492 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:29:59 2016-06-28 07:30:15,493 INFO repeater task os150nb_lastensq starting 6.5 seconds late 2016-06-28 07:30:16,839 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2016-06-28 07:30:16,839 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:29:59 2016-06-28 07:30:41,914 INFO repeater task os75nb_10pings starting 2.7 seconds late 2016-06-28 07:30:42,032 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 07:32:12,274 INFO repeater task os150bb_lastensq starting 3.3 seconds late 2016-06-28 07:32:14,117 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2016-06-28 07:32:14,117 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:31:59 2016-06-28 07:32:14,117 INFO repeater task os150nb_lastensq starting 5.1 seconds late 2016-06-28 07:32:15,440 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2016-06-28 07:32:15,440 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:31:59 2016-06-28 07:32:15,441 INFO repeater task os75nb_lastensq starting 6.4 seconds late 2016-06-28 07:32:16,783 INFO repeater task os75nb_lastensq ended after 1.3 seconds 2016-06-28 07:32:16,783 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 07:31:59 2016-06-28 07:32:21,791 INFO repeater task os150bb_10pings starting 2.6 seconds late 2016-06-28 07:32:21,923 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 07:32:21,923 INFO repeater task os150nb_10pings starting 2.7 seconds late 2016-06-28 07:32:22,034 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:34:02,456 INFO repeater task os75nb_10pings starting 3.3 seconds late 2016-06-28 07:34:02,581 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 07:34:02,582 INFO repeater task seapath_hcorrstats starting 3.4 seconds late 2016-06-28 07:34:03,243 INFO repeater task seapath_hcorrstats ended after 0.7 seconds 2016-06-28 07:34:03,243 INFO repeater task ashtech_hcorrstats starting 4.1 seconds late 2016-06-28 07:34:03,915 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2016-06-28 07:34:13,936 INFO repeater task os150bb_lastensq starting 4.9 seconds late 2016-06-28 07:34:15,650 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 07:34:15,650 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:33:59 2016-06-28 07:34:15,651 INFO repeater task os150nb_lastensq starting 6.7 seconds late 2016-06-28 07:34:16,983 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2016-06-28 07:34:16,983 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:33:59 2016-06-28 07:34:42,051 INFO repeater task os150bb_10pings starting 2.9 seconds late 2016-06-28 07:34:42,170 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 07:34:42,171 INFO repeater task os150nb_10pings starting 3.0 seconds late 2016-06-28 07:34:42,285 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:35:12,353 INFO repeater task os75nb_lastensq starting 3.4 seconds late 2016-06-28 07:35:13,700 INFO repeater task os75nb_lastensq ended after 1.3 seconds 2016-06-28 07:35:13,700 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 07:34:59 2016-06-28 07:36:13,873 INFO repeater task os150bb_lastensq starting 4.9 seconds late 2016-06-28 07:36:15,604 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 07:36:15,604 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:35:59 2016-06-28 07:36:15,605 INFO repeater task os150nb_lastensq starting 6.6 seconds late 2016-06-28 07:36:17,099 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2016-06-28 07:36:17,100 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:35:59 2016-06-28 07:37:02,223 INFO repeater task os150bb_10pings starting 3.0 seconds late 2016-06-28 07:37:02,350 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 07:37:02,351 INFO repeater task os150nb_10pings starting 3.2 seconds late 2016-06-28 07:37:02,459 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:37:22,508 INFO repeater task os75nb_10pings starting 3.3 seconds late 2016-06-28 07:37:22,631 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 07:38:12,770 INFO repeater task os150bb_lastensq starting 3.8 seconds late 2016-06-28 07:38:14,456 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 07:38:14,457 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:37:59 2016-06-28 07:38:14,458 INFO repeater task os150nb_lastensq starting 5.5 seconds late 2016-06-28 07:38:15,958 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2016-06-28 07:38:15,959 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:37:59 2016-06-28 07:38:15,959 INFO repeater task os75nb_lastensq starting 7.0 seconds late 2016-06-28 07:38:17,281 INFO repeater task os75nb_lastensq ended after 1.3 seconds 2016-06-28 07:38:17,281 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 07:37:59 2016-06-28 07:39:02,542 INFO repeater task seapath_hcorrstats starting 3.3 seconds late 2016-06-28 07:39:03,215 INFO repeater task seapath_hcorrstats ended after 0.7 seconds 2016-06-28 07:39:03,215 INFO repeater task ashtech_hcorrstats starting 4.0 seconds late 2016-06-28 07:39:03,886 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2016-06-28 07:39:23,939 INFO repeater task os150bb_10pings starting 4.8 seconds late 2016-06-28 07:39:24,057 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 07:39:24,058 INFO repeater task os150nb_10pings starting 4.9 seconds late 2016-06-28 07:39:24,174 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:40:09,292 INFO repeater task os150bb_lastensq starting 0.3 seconds late 2016-06-28 07:40:11,014 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 07:40:11,014 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:39:59 2016-06-28 07:40:11,015 INFO repeater task os150nb_lastensq starting 2.0 seconds late 2016-06-28 07:40:12,358 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2016-06-28 07:40:12,358 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:39:59 2016-06-28 07:40:42,592 INFO repeater task os75nb_10pings starting 3.4 seconds late 2016-06-28 07:40:42,723 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 07:41:12,810 INFO repeater task os75nb_lastensq starting 3.8 seconds late 2016-06-28 07:41:14,166 INFO repeater task os75nb_lastensq ended after 1.4 seconds 2016-06-28 07:41:14,166 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 07:40:59 2016-06-28 07:41:39,231 INFO repeater task os150bb_10pings starting 0.0 seconds late 2016-06-28 07:41:39,356 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 07:41:39,356 INFO repeater task os150nb_10pings starting 0.2 seconds late 2016-06-28 07:41:39,466 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:42:09,545 INFO repeater task os150bb_lastensq starting 0.5 seconds late 2016-06-28 07:42:11,379 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2016-06-28 07:42:11,379 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:41:59 2016-06-28 07:42:11,380 INFO repeater task os150nb_lastensq starting 2.4 seconds late 2016-06-28 07:42:12,705 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2016-06-28 07:42:12,705 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:41:59 2016-06-28 07:44:03,005 INFO repeater task os150bb_10pings starting 3.8 seconds late 2016-06-28 07:44:03,127 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 07:44:03,127 INFO repeater task os150bb_quick starting 3.9 seconds late 2016-06-28 07:44:05,980 INFO repeater task os150bb_quick ended after 2.9 seconds 2016-06-28 07:44:05,980 INFO repeater task os150bb_3dayplots starting 6.8 seconds late 2016-06-28 07:44:09,173 INFO repeater task os150bb_3dayplots ended after 3.2 seconds 2016-06-28 07:44:09,174 INFO repeater task os150bb_tsstats starting 10.0 seconds late 2016-06-28 07:44:09,744 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2016-06-28 07:44:09,744 INFO repeater task os150nb_lastensq starting 0.7 seconds late 2016-06-28 07:44:11,133 INFO repeater task os150nb_lastensq ended after 1.4 seconds 2016-06-28 07:44:11,133 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:43:59 2016-06-28 07:44:11,133 INFO repeater task os150nb_10pings starting 11.9 seconds late 2016-06-28 07:44:11,245 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:44:11,246 INFO repeater task os150nb_quick starting 12.1 seconds late 2016-06-28 07:44:14,107 INFO repeater task os150nb_quick ended after 2.9 seconds 2016-06-28 07:44:14,107 INFO repeater task os150nb_3dayplots starting 14.9 seconds late 2016-06-28 07:44:17,231 INFO repeater task os150nb_3dayplots ended after 3.1 seconds 2016-06-28 07:44:17,231 INFO repeater task os150nb_tsstats starting 18.0 seconds late 2016-06-28 07:44:17,829 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2016-06-28 07:44:17,830 INFO repeater task os75nb_lastensq starting 8.8 seconds late 2016-06-28 07:44:19,304 INFO repeater task os75nb_lastensq ended after 1.5 seconds 2016-06-28 07:44:19,304 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 07:43:59 2016-06-28 07:44:19,304 INFO repeater task os75nb_10pings starting 20.1 seconds late 2016-06-28 07:44:19,418 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 07:44:19,418 INFO repeater task os75nb_quick starting 20.2 seconds late 2016-06-28 07:44:21,624 INFO repeater task os75nb_quick ended after 2.2 seconds 2016-06-28 07:44:21,625 INFO repeater task os75nb_3dayplots starting 22.4 seconds late 2016-06-28 07:44:24,784 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2016-06-28 07:44:24,784 INFO repeater task os75nb_3dayplots ended after 3.2 seconds 2016-06-28 07:44:24,784 INFO repeater task os75nb_tsstats starting 25.6 seconds late 2016-06-28 07:44:25,380 INFO repeater task os75nb_tsstats ended after 0.6 seconds 2016-06-28 07:44:25,380 INFO repeater task os150bb_beamplots starting 26.2 seconds late 2016-06-28 07:44:26,393 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 07:44:26,394 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2016-06-28 07:44:26,394 INFO repeater task os150bb_beamplots starting 27.2 seconds late 2016-06-28 07:44:27,417 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 07:44:27,417 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2016-06-28 07:44:27,417 INFO repeater task os150nb_beamplots starting 28.2 seconds late 2016-06-28 07:44:28,365 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 07:44:28,365 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2016-06-28 07:44:28,365 INFO repeater task os150nb_beamplots starting 29.2 seconds late 2016-06-28 07:44:29,301 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 07:44:29,301 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2016-06-28 07:44:29,302 INFO repeater task os75nb_beamplots starting 30.1 seconds late 2016-06-28 07:44:30,265 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 07:44:30,265 INFO repeater task os75nb_beamplots ended after 1.0 seconds 2016-06-28 07:44:30,265 INFO repeater task os75nb_beamplots starting 31.1 seconds late 2016-06-28 07:44:31,308 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 07:44:31,308 INFO repeater task os75nb_beamplots ended after 1.0 seconds 2016-06-28 07:44:31,308 INFO repeater task seapath_hcorrstats starting 32.1 seconds late 2016-06-28 07:44:31,989 INFO repeater task seapath_hcorrstats ended after 0.7 seconds 2016-06-28 07:44:31,989 INFO repeater task ashtech_hcorrstats starting 32.8 seconds late 2016-06-28 07:44:32,668 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2016-06-28 07:44:37,674 INFO repeater task os150bb_lastensq starting 28.7 seconds late 2016-06-28 07:44:39,363 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 07:44:39,363 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:43:59 2016-06-28 07:46:09,623 INFO repeater task os150bb_lastensq starting 0.6 seconds late 2016-06-28 07:46:11,447 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2016-06-28 07:46:11,447 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:45:59 2016-06-28 07:46:11,448 INFO repeater task os150nb_lastensq starting 2.4 seconds late 2016-06-28 07:46:12,788 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2016-06-28 07:46:12,788 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:45:59 2016-06-28 07:46:22,810 INFO repeater task os150bb_10pings starting 3.6 seconds late 2016-06-28 07:46:22,929 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 07:46:22,929 INFO repeater task os150nb_10pings starting 3.7 seconds late 2016-06-28 07:46:23,036 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:47:13,183 INFO repeater task os75nb_lastensq starting 4.2 seconds late 2016-06-28 07:47:14,696 INFO repeater task os75nb_lastensq ended after 1.5 seconds 2016-06-28 07:47:14,697 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 07:46:59 2016-06-28 07:47:19,709 INFO repeater task os75nb_10pings starting 0.5 seconds late 2016-06-28 07:47:19,829 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 07:48:09,962 INFO repeater task os150bb_lastensq starting 1.0 seconds late 2016-06-28 07:48:11,649 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 07:48:11,650 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:47:59 2016-06-28 07:48:11,650 INFO repeater task os150nb_lastensq starting 2.7 seconds late 2016-06-28 07:48:13,149 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2016-06-28 07:48:13,149 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:47:59 2016-06-28 07:48:43,228 INFO repeater task os150bb_10pings starting 4.0 seconds late 2016-06-28 07:48:43,352 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 07:48:43,353 INFO repeater task os150nb_10pings starting 4.2 seconds late 2016-06-28 07:48:43,467 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:49:03,528 INFO repeater task seapath_hcorrstats starting 4.3 seconds late 2016-06-28 07:49:04,227 INFO repeater task seapath_hcorrstats ended after 0.7 seconds 2016-06-28 07:49:04,227 INFO repeater task ashtech_hcorrstats starting 5.0 seconds late 2016-06-28 07:49:04,900 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2016-06-28 07:50:10,071 INFO repeater task os150bb_lastensq starting 1.1 seconds late 2016-06-28 07:50:11,770 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 07:50:11,770 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:49:59 2016-06-28 07:50:11,771 INFO repeater task os150nb_lastensq starting 2.8 seconds late 2016-06-28 07:50:13,248 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2016-06-28 07:50:13,248 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:49:59 2016-06-28 07:50:13,249 INFO repeater task os75nb_lastensq starting 4.2 seconds late 2016-06-28 07:50:14,603 INFO repeater task os75nb_lastensq ended after 1.4 seconds 2016-06-28 07:50:14,603 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 07:49:59 2016-06-28 07:50:39,676 INFO repeater task os75nb_10pings starting 0.5 seconds late 2016-06-28 07:50:39,797 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 07:50:59,844 INFO repeater task os150bb_10pings starting 0.7 seconds late 2016-06-28 07:50:59,964 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 07:50:59,965 INFO repeater task os150nb_10pings starting 0.8 seconds late 2016-06-28 07:51:00,076 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:52:10,271 INFO repeater task os150bb_lastensq starting 1.3 seconds late 2016-06-28 07:52:12,111 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2016-06-28 07:52:12,111 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:51:59 2016-06-28 07:52:12,112 INFO repeater task os150nb_lastensq starting 3.1 seconds late 2016-06-28 07:52:13,477 INFO repeater task os150nb_lastensq ended after 1.4 seconds 2016-06-28 07:52:13,478 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:51:59 2016-06-28 07:53:13,643 INFO repeater task os75nb_lastensq starting 4.6 seconds late 2016-06-28 07:53:15,167 INFO repeater task os75nb_lastensq ended after 1.5 seconds 2016-06-28 07:53:15,167 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 07:52:59 2016-06-28 07:53:20,175 INFO repeater task os150bb_10pings starting 1.0 seconds late 2016-06-28 07:53:20,298 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 07:53:20,299 INFO repeater task os150nb_10pings starting 1.1 seconds late 2016-06-28 07:53:20,417 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:54:00,531 INFO repeater task os75nb_10pings starting 1.3 seconds late 2016-06-28 07:54:00,656 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 07:54:00,657 INFO repeater task seapath_hcorrstats starting 1.5 seconds late 2016-06-28 07:54:01,332 INFO repeater task seapath_hcorrstats ended after 0.7 seconds 2016-06-28 07:54:01,333 INFO repeater task ashtech_hcorrstats starting 2.1 seconds late 2016-06-28 07:54:02,006 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2016-06-28 07:54:12,027 INFO repeater task os150bb_lastensq starting 3.0 seconds late 2016-06-28 07:54:13,741 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 07:54:13,741 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:53:59 2016-06-28 07:54:13,741 INFO repeater task os150nb_lastensq starting 4.7 seconds late 2016-06-28 07:54:15,267 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2016-06-28 07:54:15,268 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:53:59 2016-06-28 07:55:40,490 INFO repeater task os150bb_10pings starting 1.3 seconds late 2016-06-28 07:55:40,614 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 07:55:40,615 INFO repeater task os150nb_10pings starting 1.4 seconds late 2016-06-28 07:55:40,726 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:56:10,801 INFO repeater task os150bb_lastensq starting 1.8 seconds late 2016-06-28 07:56:12,514 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 07:56:12,514 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:55:59 2016-06-28 07:56:12,515 INFO repeater task os150nb_lastensq starting 3.5 seconds late 2016-06-28 07:56:13,997 INFO repeater task os150nb_lastensq ended after 1.5 seconds 2016-06-28 07:56:13,997 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:55:59 2016-06-28 07:56:13,997 INFO repeater task os75nb_lastensq starting 5.0 seconds late 2016-06-28 07:56:15,330 INFO repeater task os75nb_lastensq ended after 1.3 seconds 2016-06-28 07:56:15,330 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 07:55:59 2016-06-28 07:57:20,493 INFO repeater task os75nb_10pings starting 1.3 seconds late 2016-06-28 07:57:20,610 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 07:58:00,700 INFO repeater task os150bb_10pings starting 1.5 seconds late 2016-06-28 07:58:00,826 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 07:58:00,827 INFO repeater task os150nb_10pings starting 1.6 seconds late 2016-06-28 07:58:00,943 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 07:58:11,103 INFO repeater task os150bb_lastensq starting 2.1 seconds late 2016-06-28 07:58:12,807 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 07:58:12,807 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:57:59 2016-06-28 07:58:12,808 INFO repeater task os150nb_lastensq starting 3.8 seconds late 2016-06-28 07:58:14,174 INFO repeater task os150nb_lastensq ended after 1.4 seconds 2016-06-28 07:58:14,175 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:57:59 2016-06-28 07:58:59,303 INFO repeater task os150bb_quick starting 0.1 seconds late 2016-06-28 07:59:02,191 INFO repeater task os150bb_quick ended after 2.9 seconds 2016-06-28 07:59:02,191 INFO repeater task os150bb_3dayplots starting 3.0 seconds late 2016-06-28 07:59:05,452 INFO repeater task os150bb_3dayplots ended after 3.3 seconds 2016-06-28 07:59:05,452 INFO repeater task os150bb_tsstats starting 6.3 seconds late 2016-06-28 07:59:06,048 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2016-06-28 07:59:06,049 INFO repeater task os150nb_quick starting 6.9 seconds late 2016-06-28 07:59:08,874 INFO repeater task os150nb_quick ended after 2.8 seconds 2016-06-28 07:59:08,875 INFO repeater task os150nb_3dayplots starting 9.7 seconds late 2016-06-28 07:59:12,034 INFO repeater task os150nb_3dayplots ended after 3.2 seconds 2016-06-28 07:59:12,034 INFO repeater task os150nb_tsstats starting 12.8 seconds late 2016-06-28 07:59:12,592 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2016-06-28 07:59:12,592 INFO repeater task os75nb_lastensq starting 3.6 seconds late 2016-06-28 07:59:13,961 INFO repeater task os75nb_lastensq ended after 1.4 seconds 2016-06-28 07:59:13,961 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 07:58:59 2016-06-28 07:59:13,961 INFO repeater task os75nb_quick starting 14.8 seconds late 2016-06-28 07:59:16,142 INFO repeater task os75nb_quick ended after 2.2 seconds 2016-06-28 07:59:16,143 INFO repeater task os75nb_3dayplots starting 17.0 seconds late 2016-06-28 07:59:19,298 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2016-06-28 07:59:19,298 INFO repeater task os75nb_3dayplots ended after 3.2 seconds 2016-06-28 07:59:19,299 INFO repeater task os75nb_tsstats starting 20.1 seconds late 2016-06-28 07:59:19,887 INFO repeater task os75nb_tsstats ended after 0.6 seconds 2016-06-28 07:59:19,887 INFO repeater task os150bb_beamplots starting 20.7 seconds late 2016-06-28 07:59:20,894 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 07:59:20,895 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2016-06-28 07:59:20,895 INFO repeater task os150bb_beamplots starting 21.7 seconds late 2016-06-28 07:59:21,905 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 07:59:21,905 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2016-06-28 07:59:21,906 INFO repeater task os150nb_beamplots starting 22.7 seconds late 2016-06-28 07:59:22,905 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 07:59:22,905 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2016-06-28 07:59:22,905 INFO repeater task os150nb_beamplots starting 23.7 seconds late 2016-06-28 07:59:23,883 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 07:59:23,884 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2016-06-28 07:59:23,884 INFO repeater task os75nb_beamplots starting 24.7 seconds late 2016-06-28 07:59:24,875 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 07:59:24,875 INFO repeater task os75nb_beamplots ended after 1.0 seconds 2016-06-28 07:59:24,876 INFO repeater task os75nb_beamplots starting 25.7 seconds late 2016-06-28 07:59:25,859 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 07:59:25,859 INFO repeater task os75nb_beamplots ended after 1.0 seconds 2016-06-28 07:59:25,859 INFO repeater task seapath_hcorrstats starting 26.7 seconds late 2016-06-28 07:59:26,527 INFO repeater task seapath_hcorrstats ended after 0.7 seconds 2016-06-28 07:59:26,527 INFO repeater task ashtech_hcorrstats starting 27.3 seconds late 2016-06-28 07:59:27,199 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2016-06-28 08:00:12,316 INFO repeater task os150bb_lastensq starting 3.3 seconds late 2016-06-28 08:00:14,190 INFO repeater task os150bb_lastensq ended after 1.9 seconds 2016-06-28 08:00:14,190 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 07:59:59 2016-06-28 08:00:14,191 INFO repeater task os150nb_lastensq starting 5.2 seconds late 2016-06-28 08:00:15,585 INFO repeater task os150nb_lastensq ended after 1.4 seconds 2016-06-28 08:00:15,585 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 07:59:59 2016-06-28 08:00:20,590 INFO repeater task os150bb_10pings starting 1.4 seconds late 2016-06-28 08:00:20,715 ERROR repeater task os150bb_10pings returncode 8 2016-06-28 08:00:20,715 INFO repeater task os150bb_10pings output: In body of run_grab_lastpings.py Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_grab_lastpings.py", line 69, in data=extract_raw(infile, instname[:2], i0, i1, outfile=outfile, verbose=True) File "/home/currents/programs/pycurrents/adcp/rdiraw.py", line 1473, in extract_raw i0 = ii[i0] * nbytes IndexError: index out of bounds 2016-06-28 08:00:20,715 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 08:00:20,716 INFO repeater task os150nb_10pings starting 1.5 seconds late 2016-06-28 08:00:20,826 ERROR repeater task os150nb_10pings returncode 8 2016-06-28 08:00:20,826 INFO repeater task os150nb_10pings output: In body of run_grab_lastpings.py Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_grab_lastpings.py", line 69, in data=extract_raw(infile, instname[:2], i0, i1, outfile=outfile, verbose=True) File "/home/currents/programs/pycurrents/adcp/rdiraw.py", line 1473, in extract_raw i0 = ii[i0] * nbytes IndexError: index out of bounds 2016-06-28 08:00:20,826 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 08:00:40,880 INFO repeater task os75nb_10pings starting 1.7 seconds late 2016-06-28 08:00:41,006 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 08:02:11,234 INFO repeater task os150bb_lastensq starting 2.2 seconds late 2016-06-28 08:02:12,975 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 08:02:12,976 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 08:01:59 2016-06-28 08:02:12,976 INFO repeater task os150nb_lastensq starting 4.0 seconds late 2016-06-28 08:02:14,287 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2016-06-28 08:02:14,288 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 08:01:59 2016-06-28 08:02:14,288 INFO repeater task os75nb_lastensq starting 5.3 seconds late 2016-06-28 08:02:15,646 INFO repeater task os75nb_lastensq ended after 1.4 seconds 2016-06-28 08:02:15,646 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 08:01:59 2016-06-28 08:02:40,702 INFO repeater task os150bb_10pings starting 1.5 seconds late 2016-06-28 08:02:40,824 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 08:02:40,825 INFO repeater task os150nb_10pings starting 1.6 seconds late 2016-06-28 08:02:40,941 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 08:04:01,304 INFO repeater task os75nb_10pings starting 2.1 seconds late 2016-06-28 08:04:01,422 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 08:04:01,424 INFO repeater task seapath_hcorrstats starting 2.2 seconds late 2016-06-28 08:04:02,112 INFO repeater task seapath_hcorrstats ended after 0.7 seconds 2016-06-28 08:04:02,112 INFO repeater task ashtech_hcorrstats starting 2.9 seconds late 2016-06-28 08:04:02,785 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2016-06-28 08:04:12,806 INFO repeater task os150bb_lastensq starting 3.8 seconds late 2016-06-28 08:04:14,490 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 08:04:14,490 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 08:03:59 2016-06-28 08:04:14,491 INFO repeater task os150nb_lastensq starting 5.5 seconds late 2016-06-28 08:04:15,802 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2016-06-28 08:04:15,802 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 08:03:59 2016-06-28 08:05:00,882 INFO repeater task os150bb_10pings starting 1.7 seconds late 2016-06-28 08:05:00,992 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 08:05:00,992 INFO repeater task os150nb_10pings starting 1.8 seconds late 2016-06-28 08:05:01,101 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 08:05:11,248 INFO repeater task os75nb_lastensq starting 2.2 seconds late 2016-06-28 08:05:12,546 INFO repeater task os75nb_lastensq ended after 1.3 seconds 2016-06-28 08:05:12,547 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 08:04:59 2016-06-28 08:06:12,706 INFO repeater task os150bb_lastensq starting 3.7 seconds late 2016-06-28 08:06:14,371 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 08:06:14,371 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 08:05:59 2016-06-28 08:06:14,372 INFO repeater task os150nb_lastensq starting 5.4 seconds late 2016-06-28 08:06:15,811 INFO repeater task os150nb_lastensq ended after 1.4 seconds 2016-06-28 08:06:15,811 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 08:05:59 2016-06-28 08:07:20,992 INFO repeater task os150bb_10pings starting 1.8 seconds late 2016-06-28 08:07:21,122 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 08:07:21,123 INFO repeater task os150nb_10pings starting 1.9 seconds late 2016-06-28 08:07:21,230 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 08:07:21,231 INFO repeater task os75nb_10pings starting 2.0 seconds late 2016-06-28 08:07:21,346 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 08:08:11,439 INFO repeater task os150bb_lastensq starting 2.4 seconds late 2016-06-28 08:08:13,171 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 08:08:13,171 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 08:07:59 2016-06-28 08:08:13,172 INFO repeater task os150nb_lastensq starting 4.2 seconds late 2016-06-28 08:08:14,458 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2016-06-28 08:08:14,459 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 08:07:59 2016-06-28 08:08:14,588 INFO repeater task os75nb_lastensq starting 5.6 seconds late 2016-06-28 08:08:15,907 INFO repeater task os75nb_lastensq ended after 1.3 seconds 2016-06-28 08:08:15,907 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 08:07:59 2016-06-28 08:09:01,030 INFO repeater task seapath_hcorrstats starting 1.8 seconds late 2016-06-28 08:09:01,703 INFO repeater task seapath_hcorrstats ended after 0.7 seconds 2016-06-28 08:09:01,703 INFO repeater task ashtech_hcorrstats starting 2.5 seconds late 2016-06-28 08:09:02,372 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2016-06-28 08:09:42,460 INFO repeater task os150bb_10pings starting 3.3 seconds late 2016-06-28 08:09:42,583 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 08:09:42,584 INFO repeater task os150nb_10pings starting 3.4 seconds late 2016-06-28 08:09:42,704 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 08:10:12,781 INFO repeater task os150bb_lastensq starting 3.8 seconds late 2016-06-28 08:10:14,510 INFO repeater task os150bb_lastensq ended after 1.7 seconds 2016-06-28 08:10:14,510 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 08:09:59 2016-06-28 08:10:14,511 INFO repeater task os150nb_lastensq starting 5.5 seconds late 2016-06-28 08:10:15,940 INFO repeater task os150nb_lastensq ended after 1.4 seconds 2016-06-28 08:10:15,940 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 08:09:59 2016-06-28 08:10:41,009 INFO repeater task os75nb_10pings starting 1.8 seconds late 2016-06-28 08:10:41,130 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 08:11:11,215 INFO repeater task os75nb_lastensq starting 2.2 seconds late 2016-06-28 08:11:12,540 INFO repeater task os75nb_lastensq ended after 1.3 seconds 2016-06-28 08:11:12,540 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 08:10:59 2016-06-28 08:12:02,658 INFO repeater task os150bb_10pings starting 3.5 seconds late 2016-06-28 08:12:02,780 INFO repeater task os150bb_10pings ended after 0.1 seconds 2016-06-28 08:12:02,781 INFO repeater task os150nb_10pings starting 3.6 seconds late 2016-06-28 08:12:02,895 INFO repeater task os150nb_10pings ended after 0.1 seconds 2016-06-28 08:12:12,916 INFO repeater task os150bb_lastensq starting 3.9 seconds late 2016-06-28 08:12:14,674 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2016-06-28 08:12:14,674 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 08:11:59 2016-06-28 08:12:14,675 INFO repeater task os150nb_lastensq starting 5.7 seconds late 2016-06-28 08:12:15,967 INFO repeater task os150nb_lastensq ended after 1.3 seconds 2016-06-28 08:12:15,968 INFO repeater task os150nb_lastensq: Time from /home/data/RR1608/proc/os150nb/load/wrote_ens_time: 2016/06/28 08:11:59 2016-06-28 08:14:01,239 INFO repeater task os150bb_quick starting 2.1 seconds late 2016-06-28 08:14:04,229 INFO repeater task os150bb_quick ended after 3.0 seconds 2016-06-28 08:14:04,229 INFO repeater task os150bb_3dayplots starting 5.0 seconds late 2016-06-28 08:14:07,373 INFO repeater task os150bb_3dayplots ended after 3.1 seconds 2016-06-28 08:14:07,373 INFO repeater task os150bb_tsstats starting 8.2 seconds late 2016-06-28 08:14:07,957 INFO repeater task os150bb_tsstats ended after 0.6 seconds 2016-06-28 08:14:07,957 INFO repeater task os150nb_quick starting 8.8 seconds late 2016-06-28 08:14:10,780 INFO repeater task os150nb_quick ended after 2.8 seconds 2016-06-28 08:14:10,780 INFO repeater task os150nb_3dayplots starting 11.6 seconds late 2016-06-28 08:14:14,125 INFO repeater task os150nb_3dayplots ended after 3.3 seconds 2016-06-28 08:14:14,125 INFO repeater task os150nb_tsstats starting 14.9 seconds late 2016-06-28 08:14:14,697 INFO repeater task os150nb_tsstats ended after 0.6 seconds 2016-06-28 08:14:14,698 INFO repeater task os75nb_lastensq starting 5.7 seconds late 2016-06-28 08:14:16,008 INFO repeater task os75nb_lastensq ended after 1.3 seconds 2016-06-28 08:14:16,008 INFO repeater task os75nb_lastensq: Time from /home/data/RR1608/proc/os75nb/load/wrote_ens_time: 2016/06/28 08:13:59 2016-06-28 08:14:16,008 INFO repeater task os75nb_10pings starting 16.8 seconds late 2016-06-28 08:14:16,124 INFO repeater task os75nb_10pings ended after 0.1 seconds 2016-06-28 08:14:16,124 INFO repeater task os75nb_quick starting 16.9 seconds late 2016-06-28 08:14:18,292 INFO repeater task os75nb_quick ended after 2.2 seconds 2016-06-28 08:14:18,293 INFO repeater task os75nb_3dayplots starting 19.1 seconds late 2016-06-28 08:14:21,330 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2016-06-28 08:14:21,330 INFO repeater task os75nb_3dayplots ended after 3.0 seconds 2016-06-28 08:14:21,330 INFO repeater task os75nb_tsstats starting 22.1 seconds late 2016-06-28 08:14:21,888 INFO repeater task os75nb_tsstats ended after 0.6 seconds 2016-06-28 08:14:21,888 INFO repeater task os150bb_beamplots starting 22.7 seconds late 2016-06-28 08:14:22,845 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 08:14:22,845 INFO repeater task os150bb_beamplots ended after 1.0 seconds 2016-06-28 08:14:22,845 INFO repeater task os150bb_beamplots starting 23.7 seconds late 2016-06-28 08:14:23,941 INFO repeater task os150bb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 08:14:23,941 INFO repeater task os150bb_beamplots ended after 1.1 seconds 2016-06-28 08:14:23,942 INFO repeater task os150nb_beamplots starting 24.8 seconds late 2016-06-28 08:14:24,920 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 08:14:24,921 INFO repeater task os150nb_beamplots ended after 1.0 seconds 2016-06-28 08:14:24,921 INFO repeater task os150nb_beamplots starting 25.7 seconds late 2016-06-28 08:14:25,850 INFO repeater task os150nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 08:14:25,850 INFO repeater task os150nb_beamplots ended after 0.9 seconds 2016-06-28 08:14:25,850 INFO repeater task os75nb_beamplots starting 26.7 seconds late 2016-06-28 08:14:26,861 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 08:14:26,862 INFO repeater task os75nb_beamplots ended after 1.0 seconds 2016-06-28 08:14:26,862 INFO repeater task os75nb_beamplots starting 27.7 seconds late 2016-06-28 08:14:27,879 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2016-06-28 08:14:27,880 INFO repeater task os75nb_beamplots ended after 1.0 seconds 2016-06-28 08:14:27,880 INFO repeater task seapath_hcorrstats starting 28.7 seconds late 2016-06-28 08:14:28,556 INFO repeater task seapath_hcorrstats ended after 0.7 seconds 2016-06-28 08:14:28,556 INFO repeater task ashtech_hcorrstats starting 29.4 seconds late 2016-06-28 08:14:29,229 INFO repeater task ashtech_hcorrstats ended after 0.7 seconds 2016-06-28 08:14:34,235 INFO repeater task os150bb_lastensq starting 25.2 seconds late 2016-06-28 08:14:36,026 INFO repeater task os150bb_lastensq ended after 1.8 seconds 2016-06-28 08:14:36,026 INFO repeater task os150bb_lastensq: Time from /home/data/RR1608/proc/os150bb/load/wrote_ens_time: 2016/06/28 08:13:59 2016-06-28 08:14:36,026 INFO repeater task os150bb_10pings starting 16.8 seconds late