2015-02-05 18:04:22,584 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 18:04:22,584 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 18:04:22,585 INFO repeater task ashtech_hcorrstats starting 29.5 seconds late 2015-02-05 18:04:23,671 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 18:04:28,677 INFO repeater task os75nb_lastensq starting 30.7 seconds late 2015-02-05 18:04:31,266 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 18:04:31,267 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 18:03:48 2015-02-05 18:08:56,626 INFO repeater task ashtech_hcorrstats starting 3.6 seconds late 2015-02-05 18:08:57,720 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 18:09:02,725 INFO repeater task os75nb_lastensq starting 4.7 seconds late 2015-02-05 18:09:05,287 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 18:09:05,287 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 18:08:48 2015-02-05 18:09:15,302 INFO repeater task os75nb_10pings starting 2.2 seconds late 2015-02-05 18:09:15,487 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 18:13:55,867 INFO repeater task ashtech_hcorrstats starting 2.8 seconds late 2015-02-05 18:13:56,962 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 18:14:01,967 INFO repeater task os75nb_lastensq starting 4.0 seconds late 2015-02-05 18:14:04,510 INFO repeater task os75nb_lastensq ended after 2.5 seconds 2015-02-05 18:14:04,510 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 18:13:48 2015-02-05 18:14:34,555 INFO repeater task os75nb_10pings starting 1.5 seconds late 2015-02-05 18:14:34,740 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 18:18:55,101 INFO repeater task os75nb_quick starting 2.0 seconds late 2015-02-05 18:19:03,302 INFO repeater task os75nb_quick ended after 8.2 seconds 2015-02-05 18:19:03,303 INFO repeater task os75nb_3dayplots starting 10.2 seconds late 2015-02-05 18:19:19,133 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-02-05 18:19:19,133 INFO repeater task os75nb_3dayplots ended after 15.8 seconds 2015-02-05 18:19:19,133 INFO repeater task os75nb_tsstats starting 26.1 seconds late 2015-02-05 18:19:20,316 INFO repeater task os75nb_tsstats ended after 1.2 seconds 2015-02-05 18:19:20,316 INFO repeater task os75nb_beamplots starting 27.3 seconds late 2015-02-05 18:19:21,956 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 18:19:21,956 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 18:19:21,957 INFO repeater task os75nb_beamplots starting 28.9 seconds late 2015-02-05 18:19:23,578 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 18:19:23,578 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 18:19:23,579 INFO repeater task ashtech_hcorrstats starting 30.5 seconds late 2015-02-05 18:19:24,663 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 18:19:29,669 INFO repeater task os75nb_lastensq starting 31.7 seconds late 2015-02-05 18:19:32,238 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 18:19:32,239 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 18:18:48 2015-02-05 18:19:57,276 INFO repeater task os75nb_10pings starting 4.2 seconds late 2015-02-05 18:19:57,462 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 18:23:57,757 INFO repeater task ashtech_hcorrstats starting 4.7 seconds late 2015-02-05 18:23:58,863 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 18:24:03,868 INFO repeater task os75nb_lastensq starting 5.9 seconds late 2015-02-05 18:24:06,441 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 18:24:06,442 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 18:23:48 2015-02-05 18:25:16,539 INFO repeater task os75nb_10pings starting 3.5 seconds late 2015-02-05 18:25:16,723 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 18:28:56,996 INFO repeater task ashtech_hcorrstats starting 3.9 seconds late 2015-02-05 18:28:58,091 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 18:29:03,097 INFO repeater task os75nb_lastensq starting 5.1 seconds late 2015-02-05 18:29:05,665 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 18:29:05,666 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 18:28:48 2015-02-05 18:30:35,787 INFO repeater task os75nb_10pings starting 2.7 seconds late 2015-02-05 18:30:35,970 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 18:33:56,236 INFO repeater task os75nb_quick starting 3.2 seconds late 2015-02-05 18:34:04,499 INFO repeater task os75nb_quick ended after 8.3 seconds 2015-02-05 18:34:04,499 INFO repeater task os75nb_3dayplots starting 11.4 seconds late 2015-02-05 18:34:20,359 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-02-05 18:34:20,359 INFO repeater task os75nb_3dayplots ended after 15.9 seconds 2015-02-05 18:34:20,360 INFO repeater task os75nb_tsstats starting 27.3 seconds late 2015-02-05 18:34:21,551 INFO repeater task os75nb_tsstats ended after 1.2 seconds 2015-02-05 18:34:21,551 INFO repeater task os75nb_beamplots starting 28.5 seconds late 2015-02-05 18:34:23,198 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 18:34:23,198 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 18:34:23,199 INFO repeater task os75nb_beamplots starting 30.1 seconds late 2015-02-05 18:34:24,821 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 18:34:24,821 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 18:34:24,821 INFO repeater task ashtech_hcorrstats starting 31.8 seconds late 2015-02-05 18:34:25,922 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 18:34:30,927 INFO repeater task os75nb_lastensq starting 32.9 seconds late 2015-02-05 18:34:33,469 INFO repeater task os75nb_lastensq ended after 2.5 seconds 2015-02-05 18:34:33,470 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 18:33:48 2015-02-05 18:35:53,572 INFO repeater task os75nb_10pings starting 0.5 seconds late 2015-02-05 18:35:53,754 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 18:38:53,979 INFO repeater task ashtech_hcorrstats starting 0.9 seconds late 2015-02-05 18:38:55,069 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 18:39:00,070 INFO repeater task os75nb_lastensq starting 2.1 seconds late 2015-02-05 18:39:02,625 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 18:39:02,626 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 18:38:48 2015-02-05 18:41:17,823 INFO repeater task os75nb_10pings starting 4.8 seconds late 2015-02-05 18:41:18,009 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 18:43:53,222 INFO repeater task ashtech_hcorrstats starting 0.2 seconds late 2015-02-05 18:43:54,317 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 18:43:59,322 INFO repeater task os75nb_lastensq starting 1.3 seconds late 2015-02-05 18:44:01,890 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 18:44:01,890 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 18:43:48 2015-02-05 18:46:37,105 INFO repeater task os75nb_10pings starting 4.0 seconds late 2015-02-05 18:46:37,290 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 18:48:57,465 INFO repeater task os75nb_quick starting 4.4 seconds late 2015-02-05 18:49:05,695 INFO repeater task os75nb_quick ended after 8.2 seconds 2015-02-05 18:49:05,695 INFO repeater task os75nb_3dayplots starting 12.6 seconds late 2015-02-05 18:49:21,643 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-02-05 18:49:21,643 INFO repeater task os75nb_3dayplots ended after 15.9 seconds 2015-02-05 18:49:21,644 INFO repeater task os75nb_tsstats starting 28.6 seconds late 2015-02-05 18:49:22,852 INFO repeater task os75nb_tsstats ended after 1.2 seconds 2015-02-05 18:49:22,853 INFO repeater task os75nb_beamplots starting 29.8 seconds late 2015-02-05 18:49:24,491 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 18:49:24,491 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 18:49:24,491 INFO repeater task os75nb_beamplots starting 31.4 seconds late 2015-02-05 18:49:26,114 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 18:49:26,114 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 18:49:26,115 INFO repeater task ashtech_hcorrstats starting 33.0 seconds late 2015-02-05 18:49:27,207 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 18:49:32,213 INFO repeater task os75nb_lastensq starting 34.2 seconds late 2015-02-05 18:49:34,796 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 18:49:34,796 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 18:48:48 2015-02-05 18:51:54,986 INFO repeater task os75nb_10pings starting 1.9 seconds late 2015-02-05 18:51:55,177 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 18:53:55,337 INFO repeater task ashtech_hcorrstats starting 2.3 seconds late 2015-02-05 18:53:56,447 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 18:54:01,453 INFO repeater task os75nb_lastensq starting 3.5 seconds late 2015-02-05 18:54:04,005 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 18:54:04,005 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 18:53:48 2015-02-05 18:57:14,234 INFO repeater task os75nb_10pings starting 1.2 seconds late 2015-02-05 18:57:14,418 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 18:58:54,550 INFO repeater task ashtech_hcorrstats starting 1.5 seconds late 2015-02-05 18:58:55,636 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 18:59:00,642 INFO repeater task os75nb_lastensq starting 2.6 seconds late 2015-02-05 18:59:03,239 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 18:59:03,239 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 18:58:48 2015-02-05 19:02:33,498 INFO repeater task os75nb_10pings starting 0.4 seconds late 2015-02-05 19:02:33,684 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 19:03:53,790 INFO repeater task os75nb_quick starting 0.7 seconds late 2015-02-05 19:04:02,026 INFO repeater task os75nb_quick ended after 8.2 seconds 2015-02-05 19:04:02,026 INFO repeater task os75nb_3dayplots starting 9.0 seconds late 2015-02-05 19:04:17,891 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-02-05 19:04:17,892 INFO repeater task os75nb_3dayplots ended after 15.9 seconds 2015-02-05 19:04:17,892 INFO repeater task os75nb_tsstats starting 24.8 seconds late 2015-02-05 19:04:19,102 INFO repeater task os75nb_tsstats ended after 1.2 seconds 2015-02-05 19:04:19,102 INFO repeater task os75nb_beamplots starting 26.0 seconds late 2015-02-05 19:04:20,718 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 19:04:20,718 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 19:04:20,718 INFO repeater task os75nb_beamplots starting 27.7 seconds late 2015-02-05 19:04:22,345 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 19:04:22,345 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 19:04:22,346 INFO repeater task ashtech_hcorrstats starting 29.3 seconds late 2015-02-05 19:04:23,447 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 19:04:28,451 INFO repeater task os75nb_lastensq starting 30.5 seconds late 2015-02-05 19:04:31,030 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 19:04:31,030 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 19:03:48 2015-02-05 19:07:56,268 INFO repeater task os75nb_10pings starting 3.2 seconds late 2015-02-05 19:07:56,454 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 19:08:56,540 INFO repeater task ashtech_hcorrstats starting 3.5 seconds late 2015-02-05 19:08:57,642 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 19:09:02,648 INFO repeater task os75nb_lastensq starting 4.6 seconds late 2015-02-05 19:09:05,212 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 19:09:05,212 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 19:08:48 2015-02-05 19:13:15,532 INFO repeater task os75nb_10pings starting 2.5 seconds late 2015-02-05 19:13:15,718 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 19:13:55,776 INFO repeater task ashtech_hcorrstats starting 2.7 seconds late 2015-02-05 19:13:56,892 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 19:14:01,898 INFO repeater task os75nb_lastensq starting 3.9 seconds late 2015-02-05 19:14:04,449 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 19:14:04,449 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 19:13:48 2015-02-05 19:18:34,796 INFO repeater task os75nb_10pings starting 1.7 seconds late 2015-02-05 19:18:34,983 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 19:18:55,015 INFO repeater task os75nb_quick starting 2.0 seconds late 2015-02-05 19:19:03,278 INFO repeater task os75nb_quick ended after 8.3 seconds 2015-02-05 19:19:03,279 INFO repeater task os75nb_3dayplots starting 10.2 seconds late 2015-02-05 19:19:19,170 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-02-05 19:19:19,171 INFO repeater task os75nb_3dayplots ended after 15.9 seconds 2015-02-05 19:19:19,171 INFO repeater task os75nb_tsstats starting 26.1 seconds late 2015-02-05 19:19:20,374 INFO repeater task os75nb_tsstats ended after 1.2 seconds 2015-02-05 19:19:20,375 INFO repeater task os75nb_beamplots starting 27.3 seconds late 2015-02-05 19:19:22,003 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 19:19:22,004 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 19:19:22,004 INFO repeater task os75nb_beamplots starting 28.9 seconds late 2015-02-05 19:19:23,638 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 19:19:23,638 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 19:19:23,639 INFO repeater task ashtech_hcorrstats starting 30.6 seconds late 2015-02-05 19:19:24,733 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 19:19:29,739 INFO repeater task os75nb_lastensq starting 31.7 seconds late 2015-02-05 19:19:32,291 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 19:19:32,292 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 19:18:48 2015-02-05 19:23:57,671 INFO repeater task os75nb_10pings starting 4.6 seconds late 2015-02-05 19:23:57,857 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 19:23:57,859 INFO repeater task ashtech_hcorrstats starting 4.8 seconds late 2015-02-05 19:23:58,971 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 19:24:03,975 INFO repeater task os75nb_lastensq starting 6.0 seconds late 2015-02-05 19:24:06,561 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 19:24:06,561 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 19:23:48 2015-02-05 19:28:56,909 INFO repeater task ashtech_hcorrstats starting 3.8 seconds late 2015-02-05 19:28:58,011 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 19:29:03,015 INFO repeater task os75nb_lastensq starting 5.0 seconds late 2015-02-05 19:29:05,633 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 19:29:05,633 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 19:28:48 2015-02-05 19:29:15,648 INFO repeater task os75nb_10pings starting 2.6 seconds late 2015-02-05 19:29:15,834 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 19:33:56,249 INFO repeater task os75nb_quick starting 3.2 seconds late 2015-02-05 19:34:04,518 INFO repeater task os75nb_quick ended after 8.3 seconds 2015-02-05 19:34:04,518 INFO repeater task os75nb_3dayplots starting 11.5 seconds late 2015-02-05 19:34:20,427 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-02-05 19:34:20,428 INFO repeater task os75nb_3dayplots ended after 15.9 seconds 2015-02-05 19:34:20,428 INFO repeater task os75nb_tsstats starting 27.4 seconds late 2015-02-05 19:34:21,655 INFO repeater task os75nb_tsstats ended after 1.2 seconds 2015-02-05 19:34:21,656 INFO repeater task os75nb_beamplots starting 28.6 seconds late 2015-02-05 19:34:23,275 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 19:34:23,276 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 19:34:23,276 INFO repeater task os75nb_beamplots starting 30.2 seconds late 2015-02-05 19:34:24,885 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 19:34:24,886 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 19:34:24,886 INFO repeater task ashtech_hcorrstats starting 31.8 seconds late 2015-02-05 19:34:25,987 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 19:34:30,991 INFO repeater task os75nb_lastensq starting 33.0 seconds late 2015-02-05 19:34:33,557 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 19:34:33,557 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 19:33:48 2015-02-05 19:34:33,558 INFO repeater task os75nb_10pings starting 0.5 seconds late 2015-02-05 19:34:33,738 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 19:38:54,101 INFO repeater task ashtech_hcorrstats starting 1.0 seconds late 2015-02-05 19:38:55,203 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 19:39:00,209 INFO repeater task os75nb_lastensq starting 2.2 seconds late 2015-02-05 19:39:02,763 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 19:39:02,763 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 19:38:48 2015-02-05 19:39:57,842 INFO repeater task os75nb_10pings starting 4.8 seconds late 2015-02-05 19:39:58,028 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 19:43:53,336 INFO repeater task ashtech_hcorrstats starting 0.3 seconds late 2015-02-05 19:43:54,457 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 19:43:59,463 INFO repeater task os75nb_lastensq starting 1.5 seconds late 2015-02-05 19:44:02,025 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 19:44:02,025 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 19:43:48 2015-02-05 19:45:17,135 INFO repeater task os75nb_10pings starting 4.1 seconds late 2015-02-05 19:45:17,322 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 19:48:57,582 INFO repeater task os75nb_quick starting 4.5 seconds late 2015-02-05 19:49:05,844 INFO repeater task os75nb_quick ended after 8.3 seconds 2015-02-05 19:49:05,844 INFO repeater task os75nb_3dayplots starting 12.8 seconds late 2015-02-05 19:49:21,710 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-02-05 19:49:21,710 INFO repeater task os75nb_3dayplots ended after 15.9 seconds 2015-02-05 19:49:21,711 INFO repeater task os75nb_tsstats starting 28.6 seconds late 2015-02-05 19:49:22,934 INFO repeater task os75nb_tsstats ended after 1.2 seconds 2015-02-05 19:49:22,934 INFO repeater task os75nb_beamplots starting 29.9 seconds late 2015-02-05 19:49:24,555 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 19:49:24,556 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 19:49:24,556 INFO repeater task os75nb_beamplots starting 31.5 seconds late 2015-02-05 19:49:26,178 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 19:49:26,178 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 19:49:26,179 INFO repeater task ashtech_hcorrstats starting 33.1 seconds late 2015-02-05 19:49:27,285 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 19:49:32,291 INFO repeater task os75nb_lastensq starting 34.3 seconds late 2015-02-05 19:49:34,853 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 19:49:34,854 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 19:48:48 2015-02-05 19:50:34,943 INFO repeater task os75nb_10pings starting 1.9 seconds late 2015-02-05 19:50:35,130 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 19:53:55,417 INFO repeater task ashtech_hcorrstats starting 2.4 seconds late 2015-02-05 19:53:56,531 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 19:54:01,536 INFO repeater task os75nb_lastensq starting 3.5 seconds late 2015-02-05 19:54:04,137 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 19:54:04,137 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 19:53:48 2015-02-05 19:55:54,295 INFO repeater task os75nb_10pings starting 1.2 seconds late 2015-02-05 19:55:54,485 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 19:58:54,747 INFO repeater task ashtech_hcorrstats starting 1.7 seconds late 2015-02-05 19:58:55,879 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 19:59:00,881 INFO repeater task os75nb_lastensq starting 2.9 seconds late 2015-02-05 19:59:03,473 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 19:59:03,473 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 19:58:48 2015-02-05 20:01:13,622 INFO repeater task os75nb_10pings starting 0.6 seconds late 2015-02-05 20:01:13,815 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 20:03:54,015 INFO repeater task os75nb_quick starting 1.0 seconds late 2015-02-05 20:04:02,282 INFO repeater task os75nb_quick ended after 8.3 seconds 2015-02-05 20:04:02,282 INFO repeater task os75nb_3dayplots starting 9.2 seconds late 2015-02-05 20:04:18,171 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-02-05 20:04:18,171 INFO repeater task os75nb_3dayplots ended after 15.9 seconds 2015-02-05 20:04:18,172 INFO repeater task os75nb_tsstats starting 25.1 seconds late 2015-02-05 20:04:19,394 INFO repeater task os75nb_tsstats ended after 1.2 seconds 2015-02-05 20:04:19,394 INFO repeater task os75nb_beamplots starting 26.3 seconds late 2015-02-05 20:04:21,027 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 20:04:21,027 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 20:04:21,028 INFO repeater task os75nb_beamplots starting 28.0 seconds late 2015-02-05 20:04:22,643 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 20:04:22,643 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 20:04:22,643 INFO repeater task ashtech_hcorrstats starting 29.6 seconds late 2015-02-05 20:04:23,758 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 20:04:28,763 INFO repeater task os75nb_lastensq starting 30.8 seconds late 2015-02-05 20:04:31,333 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 20:04:31,334 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 20:03:48 2015-02-05 20:06:36,491 INFO repeater task os75nb_10pings starting 3.4 seconds late 2015-02-05 20:06:36,678 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 20:08:56,853 INFO repeater task ashtech_hcorrstats starting 3.8 seconds late 2015-02-05 20:08:57,967 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 20:09:02,970 INFO repeater task os75nb_lastensq starting 5.0 seconds late 2015-02-05 20:09:05,503 INFO repeater task os75nb_lastensq ended after 2.5 seconds 2015-02-05 20:09:05,503 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 20:08:48 2015-02-05 20:11:55,745 INFO repeater task os75nb_10pings starting 2.7 seconds late 2015-02-05 20:11:55,927 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 20:13:56,093 INFO repeater task ashtech_hcorrstats starting 3.0 seconds late 2015-02-05 20:13:57,225 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 20:14:02,231 INFO repeater task os75nb_lastensq starting 4.2 seconds late 2015-02-05 20:14:04,785 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 20:14:04,785 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 20:13:48 2015-02-05 20:17:15,018 INFO repeater task os75nb_10pings starting 2.0 seconds late 2015-02-05 20:17:15,201 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 20:18:55,333 INFO repeater task os75nb_quick starting 2.3 seconds late 2015-02-05 20:19:03,561 INFO repeater task os75nb_quick ended after 8.2 seconds 2015-02-05 20:19:03,562 INFO repeater task os75nb_3dayplots starting 10.5 seconds late 2015-02-05 20:19:19,483 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-02-05 20:19:19,483 INFO repeater task os75nb_3dayplots ended after 15.9 seconds 2015-02-05 20:19:19,484 INFO repeater task os75nb_tsstats starting 26.4 seconds late 2015-02-05 20:19:20,679 INFO repeater task os75nb_tsstats ended after 1.2 seconds 2015-02-05 20:19:20,680 INFO repeater task os75nb_beamplots starting 27.6 seconds late 2015-02-05 20:19:22,307 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 20:19:22,307 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 20:19:22,307 INFO repeater task os75nb_beamplots starting 29.2 seconds late 2015-02-05 20:19:23,935 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 20:19:23,935 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 20:19:23,935 INFO repeater task ashtech_hcorrstats starting 30.9 seconds late 2015-02-05 20:19:25,042 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 20:19:30,048 INFO repeater task os75nb_lastensq starting 32.0 seconds late 2015-02-05 20:19:32,570 INFO repeater task os75nb_lastensq ended after 2.5 seconds 2015-02-05 20:19:32,571 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 20:18:48 2015-02-05 20:22:37,789 INFO repeater task os75nb_10pings starting 4.7 seconds late 2015-02-05 20:22:37,976 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 20:23:53,076 INFO repeater task ashtech_hcorrstats starting 0.0 seconds late 2015-02-05 20:23:54,186 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 20:23:59,192 INFO repeater task os75nb_lastensq starting 1.2 seconds late 2015-02-05 20:24:01,744 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 20:24:01,744 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 20:23:48 2015-02-05 20:27:57,053 INFO repeater task os75nb_10pings starting 4.0 seconds late 2015-02-05 20:27:57,234 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 20:28:57,323 INFO repeater task ashtech_hcorrstats starting 4.3 seconds late 2015-02-05 20:28:58,433 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 20:29:03,439 INFO repeater task os75nb_lastensq starting 5.4 seconds late 2015-02-05 20:29:05,975 INFO repeater task os75nb_lastensq ended after 2.5 seconds 2015-02-05 20:29:05,975 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 20:28:48 2015-02-05 20:33:16,317 INFO repeater task os75nb_10pings starting 3.3 seconds late 2015-02-05 20:33:16,503 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 20:33:56,564 INFO repeater task os75nb_quick starting 3.5 seconds late 2015-02-05 20:34:04,752 INFO repeater task os75nb_quick ended after 8.2 seconds 2015-02-05 20:34:04,752 INFO repeater task os75nb_3dayplots starting 11.7 seconds late 2015-02-05 20:34:20,642 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-02-05 20:34:20,642 INFO repeater task os75nb_3dayplots ended after 15.9 seconds 2015-02-05 20:34:20,643 INFO repeater task os75nb_tsstats starting 27.6 seconds late 2015-02-05 20:34:21,842 INFO repeater task os75nb_tsstats ended after 1.2 seconds 2015-02-05 20:34:21,843 INFO repeater task os75nb_beamplots starting 28.8 seconds late 2015-02-05 20:34:23,484 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 20:34:23,484 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 20:34:23,485 INFO repeater task os75nb_beamplots starting 30.4 seconds late 2015-02-05 20:34:25,092 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 20:34:25,092 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 20:34:25,093 INFO repeater task ashtech_hcorrstats starting 32.0 seconds late 2015-02-05 20:34:26,194 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 20:34:31,199 INFO repeater task os75nb_lastensq starting 33.2 seconds late 2015-02-05 20:34:33,765 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 20:34:33,765 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 20:33:48 2015-02-05 20:38:34,082 INFO repeater task os75nb_10pings starting 1.0 seconds late 2015-02-05 20:38:34,267 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 20:38:54,296 INFO repeater task ashtech_hcorrstats starting 1.2 seconds late 2015-02-05 20:38:55,399 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 20:39:00,403 INFO repeater task os75nb_lastensq starting 2.4 seconds late 2015-02-05 20:39:02,952 INFO repeater task os75nb_lastensq ended after 2.5 seconds 2015-02-05 20:39:02,953 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 20:38:48 2015-02-05 20:43:53,345 INFO repeater task os75nb_10pings starting 0.3 seconds late 2015-02-05 20:43:53,527 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 20:43:53,528 INFO repeater task ashtech_hcorrstats starting 0.5 seconds late 2015-02-05 20:43:54,648 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 20:43:59,652 INFO repeater task os75nb_lastensq starting 1.7 seconds late 2015-02-05 20:44:02,216 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 20:44:02,216 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 20:43:48 2015-02-05 20:48:57,593 INFO repeater task os75nb_quick starting 4.5 seconds late 2015-02-05 20:49:05,848 INFO repeater task os75nb_quick ended after 8.3 seconds 2015-02-05 20:49:05,848 INFO repeater task os75nb_3dayplots starting 12.8 seconds late 2015-02-05 20:49:21,718 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-02-05 20:49:21,719 INFO repeater task os75nb_3dayplots ended after 15.9 seconds 2015-02-05 20:49:21,719 INFO repeater task os75nb_tsstats starting 28.7 seconds late 2015-02-05 20:49:22,924 INFO repeater task os75nb_tsstats ended after 1.2 seconds 2015-02-05 20:49:22,924 INFO repeater task os75nb_beamplots starting 29.9 seconds late 2015-02-05 20:49:24,553 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 20:49:24,553 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 20:49:24,553 INFO repeater task os75nb_beamplots starting 31.5 seconds late 2015-02-05 20:49:26,181 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 20:49:26,181 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 20:49:26,181 INFO repeater task ashtech_hcorrstats starting 33.1 seconds late 2015-02-05 20:49:27,302 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 20:49:32,308 INFO repeater task os75nb_lastensq starting 34.3 seconds late 2015-02-05 20:49:34,854 INFO repeater task os75nb_lastensq ended after 2.5 seconds 2015-02-05 20:49:34,854 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 20:48:48 2015-02-05 20:49:34,855 INFO repeater task os75nb_10pings starting 21.8 seconds late 2015-02-05 20:49:35,034 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 20:53:55,335 INFO repeater task ashtech_hcorrstats starting 2.3 seconds late 2015-02-05 20:53:56,452 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 20:54:01,458 INFO repeater task os75nb_lastensq starting 3.5 seconds late 2015-02-05 20:54:04,009 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 20:54:04,010 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 20:53:48 2015-02-05 20:54:34,053 INFO repeater task os75nb_10pings starting 1.0 seconds late 2015-02-05 20:54:34,238 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 20:58:54,575 INFO repeater task ashtech_hcorrstats starting 1.5 seconds late 2015-02-05 20:58:55,682 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 20:59:00,688 INFO repeater task os75nb_lastensq starting 2.7 seconds late 2015-02-05 20:59:03,221 INFO repeater task os75nb_lastensq ended after 2.5 seconds 2015-02-05 20:59:03,221 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 20:58:48 2015-02-05 20:59:53,297 INFO repeater task os75nb_10pings starting 0.2 seconds late 2015-02-05 20:59:53,485 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 21:03:53,816 INFO repeater task os75nb_quick starting 0.8 seconds late 2015-02-05 21:04:02,130 INFO repeater task os75nb_quick ended after 8.3 seconds 2015-02-05 21:04:02,131 INFO repeater task os75nb_3dayplots starting 9.1 seconds late 2015-02-05 21:04:17,992 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-02-05 21:04:17,992 INFO repeater task os75nb_3dayplots ended after 15.9 seconds 2015-02-05 21:04:17,992 INFO repeater task os75nb_tsstats starting 24.9 seconds late 2015-02-05 21:04:19,190 INFO repeater task os75nb_tsstats ended after 1.2 seconds 2015-02-05 21:04:19,191 INFO repeater task os75nb_beamplots starting 26.1 seconds late 2015-02-05 21:04:20,788 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 21:04:20,788 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 21:04:20,788 INFO repeater task os75nb_beamplots starting 27.7 seconds late 2015-02-05 21:04:22,390 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 21:04:22,390 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 21:04:22,390 INFO repeater task ashtech_hcorrstats starting 29.3 seconds late 2015-02-05 21:04:23,492 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 21:04:28,498 INFO repeater task os75nb_lastensq starting 30.5 seconds late 2015-02-05 21:04:31,054 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 21:04:31,055 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 21:03:48 2015-02-05 21:05:16,107 INFO repeater task os75nb_10pings starting 3.0 seconds late 2015-02-05 21:05:16,295 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 21:08:56,565 INFO repeater task ashtech_hcorrstats starting 3.5 seconds late 2015-02-05 21:08:57,692 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 21:09:02,697 INFO repeater task os75nb_lastensq starting 4.7 seconds late 2015-02-05 21:09:05,260 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 21:09:05,260 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 21:08:48 2015-02-05 21:10:35,376 INFO repeater task os75nb_10pings starting 2.3 seconds late 2015-02-05 21:10:35,561 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 21:13:55,816 INFO repeater task ashtech_hcorrstats starting 2.7 seconds late 2015-02-05 21:13:56,925 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 21:14:01,930 INFO repeater task os75nb_lastensq starting 3.9 seconds late 2015-02-05 21:14:04,487 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 21:14:04,487 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 21:13:48 2015-02-05 21:15:54,633 INFO repeater task os75nb_10pings starting 1.6 seconds late 2015-02-05 21:15:54,813 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 21:18:55,047 INFO repeater task os75nb_quick starting 2.0 seconds late 2015-02-05 21:19:03,253 INFO repeater task os75nb_quick ended after 8.2 seconds 2015-02-05 21:19:03,253 INFO repeater task os75nb_3dayplots starting 10.2 seconds late 2015-02-05 21:19:19,172 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-02-05 21:19:19,172 INFO repeater task os75nb_3dayplots ended after 15.9 seconds 2015-02-05 21:19:19,172 INFO repeater task os75nb_tsstats starting 26.1 seconds late 2015-02-05 21:19:20,379 INFO repeater task os75nb_tsstats ended after 1.2 seconds 2015-02-05 21:19:20,380 INFO repeater task os75nb_beamplots starting 27.3 seconds late 2015-02-05 21:19:21,971 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 21:19:21,971 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 21:19:21,972 INFO repeater task os75nb_beamplots starting 28.9 seconds late 2015-02-05 21:19:23,549 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 21:19:23,550 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 21:19:23,550 INFO repeater task ashtech_hcorrstats starting 30.5 seconds late 2015-02-05 21:19:24,653 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 21:19:29,659 INFO repeater task os75nb_lastensq starting 31.7 seconds late 2015-02-05 21:19:32,203 INFO repeater task os75nb_lastensq ended after 2.5 seconds 2015-02-05 21:19:32,204 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 21:18:48 2015-02-05 21:21:17,342 INFO repeater task os75nb_10pings starting 4.3 seconds late 2015-02-05 21:21:17,523 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 21:23:57,750 INFO repeater task ashtech_hcorrstats starting 4.7 seconds late 2015-02-05 21:23:58,877 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 21:24:03,883 INFO repeater task os75nb_lastensq starting 5.9 seconds late 2015-02-05 21:24:06,471 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 21:24:06,471 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 21:23:48 2015-02-05 21:26:36,668 INFO repeater task os75nb_10pings starting 3.6 seconds late 2015-02-05 21:26:36,855 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 21:28:57,037 INFO repeater task ashtech_hcorrstats starting 4.0 seconds late 2015-02-05 21:28:58,157 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 21:29:03,162 INFO repeater task os75nb_lastensq starting 5.2 seconds late 2015-02-05 21:29:05,701 INFO repeater task os75nb_lastensq ended after 2.5 seconds 2015-02-05 21:29:05,701 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 21:28:48 2015-02-05 21:31:55,939 INFO repeater task os75nb_10pings starting 2.9 seconds late 2015-02-05 21:31:56,122 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 21:33:56,276 INFO repeater task os75nb_quick starting 3.2 seconds late 2015-02-05 21:34:04,656 INFO repeater task os75nb_quick ended after 8.4 seconds 2015-02-05 21:34:04,656 INFO repeater task os75nb_3dayplots starting 11.6 seconds late 2015-02-05 21:34:20,681 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-02-05 21:34:20,681 INFO repeater task os75nb_3dayplots ended after 16.0 seconds 2015-02-05 21:34:20,682 INFO repeater task os75nb_tsstats starting 27.6 seconds late 2015-02-05 21:34:21,879 INFO repeater task os75nb_tsstats ended after 1.2 seconds 2015-02-05 21:34:21,879 INFO repeater task os75nb_beamplots starting 28.8 seconds late 2015-02-05 21:34:23,461 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 21:34:23,462 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 21:34:23,462 INFO repeater task os75nb_beamplots starting 30.4 seconds late 2015-02-05 21:34:25,054 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 21:34:25,054 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 21:34:25,054 INFO repeater task ashtech_hcorrstats starting 32.0 seconds late 2015-02-05 21:34:26,169 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 21:34:31,175 INFO repeater task os75nb_lastensq starting 33.2 seconds late 2015-02-05 21:34:33,730 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 21:34:33,730 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 21:33:48 2015-02-05 21:37:13,962 INFO repeater task os75nb_10pings starting 0.9 seconds late 2015-02-05 21:37:14,149 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 21:38:54,297 INFO repeater task ashtech_hcorrstats starting 1.2 seconds late 2015-02-05 21:38:55,410 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 21:39:00,416 INFO repeater task os75nb_lastensq starting 2.4 seconds late 2015-02-05 21:39:02,985 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 21:39:02,986 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 21:38:48 2015-02-05 21:42:33,268 INFO repeater task os75nb_10pings starting 0.2 seconds late 2015-02-05 21:42:33,451 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 21:43:53,561 INFO repeater task ashtech_hcorrstats starting 0.5 seconds late 2015-02-05 21:43:54,669 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 21:43:59,674 INFO repeater task os75nb_lastensq starting 1.7 seconds late 2015-02-05 21:44:02,266 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 21:44:02,267 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 21:43:48 2015-02-05 21:47:57,567 INFO repeater task os75nb_10pings starting 4.5 seconds late 2015-02-05 21:47:57,750 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 21:48:57,838 INFO repeater task os75nb_quick starting 4.8 seconds late 2015-02-05 21:49:06,337 INFO repeater task os75nb_quick ended after 8.5 seconds 2015-02-05 21:49:06,337 INFO repeater task os75nb_3dayplots starting 13.3 seconds late 2015-02-05 21:49:22,216 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-02-05 21:49:22,216 INFO repeater task os75nb_3dayplots ended after 15.9 seconds 2015-02-05 21:49:22,217 INFO repeater task os75nb_tsstats starting 29.2 seconds late 2015-02-05 21:49:23,407 INFO repeater task os75nb_tsstats ended after 1.2 seconds 2015-02-05 21:49:23,407 INFO repeater task os75nb_beamplots starting 30.3 seconds late 2015-02-05 21:49:25,000 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 21:49:25,000 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 21:49:25,000 INFO repeater task os75nb_beamplots starting 31.9 seconds late 2015-02-05 21:49:26,573 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 21:49:26,574 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 21:49:26,574 INFO repeater task ashtech_hcorrstats starting 33.5 seconds late 2015-02-05 21:49:27,688 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 21:49:32,694 INFO repeater task os75nb_lastensq starting 34.7 seconds late 2015-02-05 21:49:35,270 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 21:49:35,270 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 21:48:48 2015-02-05 21:53:15,603 INFO repeater task os75nb_10pings starting 2.5 seconds late 2015-02-05 21:53:15,787 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 21:53:55,847 INFO repeater task ashtech_hcorrstats starting 2.8 seconds late 2015-02-05 21:53:56,962 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 21:54:01,966 INFO repeater task os75nb_lastensq starting 4.0 seconds late 2015-02-05 21:54:04,536 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 21:54:04,537 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 21:53:48 2015-02-05 21:58:34,866 INFO repeater task os75nb_10pings starting 1.8 seconds late 2015-02-05 21:58:35,052 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 21:58:55,081 INFO repeater task ashtech_hcorrstats starting 2.0 seconds late 2015-02-05 21:58:56,204 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 21:59:01,210 INFO repeater task os75nb_lastensq starting 3.2 seconds late 2015-02-05 21:59:03,773 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 21:59:03,773 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 21:58:48 2015-02-05 22:03:54,192 INFO repeater task os75nb_10pings starting 1.1 seconds late 2015-02-05 22:03:54,379 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 22:03:54,380 INFO repeater task os75nb_quick starting 1.3 seconds late 2015-02-05 22:04:02,581 INFO repeater task os75nb_quick ended after 8.2 seconds 2015-02-05 22:04:02,582 INFO repeater task os75nb_3dayplots starting 9.5 seconds late 2015-02-05 22:04:18,477 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-02-05 22:04:18,477 INFO repeater task os75nb_3dayplots ended after 15.9 seconds 2015-02-05 22:04:18,477 INFO repeater task os75nb_tsstats starting 25.4 seconds late 2015-02-05 22:04:19,676 INFO repeater task os75nb_tsstats ended after 1.2 seconds 2015-02-05 22:04:19,676 INFO repeater task os75nb_beamplots starting 26.6 seconds late 2015-02-05 22:04:21,267 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 22:04:21,267 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 22:04:21,268 INFO repeater task os75nb_beamplots starting 28.2 seconds late 2015-02-05 22:04:22,865 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 22:04:22,865 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 22:04:22,866 INFO repeater task ashtech_hcorrstats starting 29.8 seconds late 2015-02-05 22:04:23,973 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 22:04:28,979 INFO repeater task os75nb_lastensq starting 31.0 seconds late 2015-02-05 22:04:31,549 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 22:04:31,549 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 22:03:48 2015-02-05 22:08:56,882 INFO repeater task ashtech_hcorrstats starting 3.8 seconds late 2015-02-05 22:08:58,002 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 22:09:03,008 INFO repeater task os75nb_lastensq starting 5.0 seconds late 2015-02-05 22:09:05,534 INFO repeater task os75nb_lastensq ended after 2.5 seconds 2015-02-05 22:09:05,534 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 22:08:48 2015-02-05 22:09:15,548 INFO repeater task os75nb_10pings starting 2.5 seconds late 2015-02-05 22:09:15,734 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 22:13:56,121 INFO repeater task ashtech_hcorrstats starting 3.1 seconds late 2015-02-05 22:13:57,229 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 22:14:02,235 INFO repeater task os75nb_lastensq starting 4.2 seconds late 2015-02-05 22:14:04,766 INFO repeater task os75nb_lastensq ended after 2.5 seconds 2015-02-05 22:14:04,766 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 22:13:48 2015-02-05 22:14:34,811 INFO repeater task os75nb_10pings starting 1.8 seconds late 2015-02-05 22:14:34,997 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 22:18:55,392 INFO repeater task os75nb_quick starting 2.3 seconds late 2015-02-05 22:19:03,663 INFO repeater task os75nb_quick ended after 8.3 seconds 2015-02-05 22:19:03,664 INFO repeater task os75nb_3dayplots starting 10.6 seconds late 2015-02-05 22:19:19,548 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-02-05 22:19:19,548 INFO repeater task os75nb_3dayplots ended after 15.9 seconds 2015-02-05 22:19:19,548 INFO repeater task os75nb_tsstats starting 26.5 seconds late 2015-02-05 22:19:20,754 INFO repeater task os75nb_tsstats ended after 1.2 seconds 2015-02-05 22:19:20,754 INFO repeater task os75nb_beamplots starting 27.7 seconds late 2015-02-05 22:19:22,355 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 22:19:22,355 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 22:19:22,356 INFO repeater task os75nb_beamplots starting 29.3 seconds late 2015-02-05 22:19:23,928 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 22:19:23,929 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 22:19:23,929 INFO repeater task ashtech_hcorrstats starting 30.9 seconds late 2015-02-05 22:19:25,031 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 22:19:30,037 INFO repeater task os75nb_lastensq starting 32.0 seconds late 2015-02-05 22:19:32,582 INFO repeater task os75nb_lastensq ended after 2.5 seconds 2015-02-05 22:19:32,582 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 22:18:48 2015-02-05 22:19:57,618 INFO repeater task os75nb_10pings starting 4.6 seconds late 2015-02-05 22:19:57,805 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 22:23:53,104 INFO repeater task ashtech_hcorrstats starting 0.0 seconds late 2015-02-05 22:23:54,224 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 22:23:59,227 INFO repeater task os75nb_lastensq starting 1.2 seconds late 2015-02-05 22:24:01,799 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 22:24:01,800 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 22:23:48 2015-02-05 22:25:16,898 INFO repeater task os75nb_10pings starting 3.8 seconds late 2015-02-05 22:25:17,082 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 22:28:57,351 INFO repeater task ashtech_hcorrstats starting 4.3 seconds late 2015-02-05 22:28:58,474 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 22:29:03,479 INFO repeater task os75nb_lastensq starting 5.5 seconds late 2015-02-05 22:29:06,038 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 22:29:06,039 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 22:28:48 2015-02-05 22:30:36,162 INFO repeater task os75nb_10pings starting 3.1 seconds late 2015-02-05 22:30:36,344 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 22:33:56,591 INFO repeater task os75nb_quick starting 3.5 seconds late 2015-02-05 22:34:04,845 INFO repeater task os75nb_quick ended after 8.3 seconds 2015-02-05 22:34:04,846 INFO repeater task os75nb_3dayplots starting 11.8 seconds late 2015-02-05 22:34:20,745 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-02-05 22:34:20,746 INFO repeater task os75nb_3dayplots ended after 15.9 seconds 2015-02-05 22:34:20,746 INFO repeater task os75nb_tsstats starting 27.7 seconds late 2015-02-05 22:34:21,937 INFO repeater task os75nb_tsstats ended after 1.2 seconds 2015-02-05 22:34:21,937 INFO repeater task os75nb_beamplots starting 28.9 seconds late 2015-02-05 22:34:23,556 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 22:34:23,556 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 22:34:23,556 INFO repeater task os75nb_beamplots starting 30.5 seconds late 2015-02-05 22:34:25,171 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 22:34:25,171 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 22:34:25,172 INFO repeater task ashtech_hcorrstats starting 32.1 seconds late 2015-02-05 22:34:26,299 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 22:34:31,305 INFO repeater task os75nb_lastensq starting 33.3 seconds late 2015-02-05 22:34:33,868 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 22:34:33,869 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 22:33:48 2015-02-05 22:35:53,987 INFO repeater task os75nb_10pings starting 0.9 seconds late 2015-02-05 22:35:54,175 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 22:38:54,422 INFO repeater task ashtech_hcorrstats starting 1.4 seconds late 2015-02-05 22:38:55,550 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 22:39:00,555 INFO repeater task os75nb_lastensq starting 2.6 seconds late 2015-02-05 22:39:03,121 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 22:39:03,122 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 22:38:48 2015-02-05 22:41:13,290 INFO repeater task os75nb_10pings starting 0.2 seconds late 2015-02-05 22:41:13,477 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 22:43:53,675 INFO repeater task ashtech_hcorrstats starting 0.6 seconds late 2015-02-05 22:43:54,790 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 22:43:59,795 INFO repeater task os75nb_lastensq starting 1.8 seconds late 2015-02-05 22:44:02,337 INFO repeater task os75nb_lastensq ended after 2.5 seconds 2015-02-05 22:44:02,338 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 22:43:48 2015-02-05 22:46:37,559 INFO repeater task os75nb_10pings starting 4.5 seconds late 2015-02-05 22:46:37,746 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 22:48:57,920 INFO repeater task os75nb_quick starting 4.9 seconds late 2015-02-05 22:49:06,210 INFO repeater task os75nb_quick ended after 8.3 seconds 2015-02-05 22:49:06,210 INFO repeater task os75nb_3dayplots starting 13.1 seconds late 2015-02-05 22:49:22,080 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-02-05 22:49:22,080 INFO repeater task os75nb_3dayplots ended after 15.9 seconds 2015-02-05 22:49:22,080 INFO repeater task os75nb_tsstats starting 29.0 seconds late 2015-02-05 22:49:23,274 INFO repeater task os75nb_tsstats ended after 1.2 seconds 2015-02-05 22:49:23,275 INFO repeater task os75nb_beamplots starting 30.2 seconds late 2015-02-05 22:49:24,842 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 22:49:24,843 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 22:49:24,843 INFO repeater task os75nb_beamplots starting 31.8 seconds late 2015-02-05 22:49:26,401 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 22:49:26,401 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 22:49:26,402 INFO repeater task ashtech_hcorrstats starting 33.3 seconds late 2015-02-05 22:49:27,508 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 22:49:32,514 INFO repeater task os75nb_lastensq starting 34.5 seconds late 2015-02-05 22:49:35,102 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 22:49:35,102 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 22:48:48 2015-02-05 22:51:55,307 INFO repeater task os75nb_10pings starting 2.2 seconds late 2015-02-05 22:51:55,494 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 22:53:55,656 INFO repeater task ashtech_hcorrstats starting 2.6 seconds late 2015-02-05 22:53:56,785 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 22:54:01,790 INFO repeater task os75nb_lastensq starting 3.8 seconds late 2015-02-05 22:54:04,354 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 22:54:04,355 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 22:53:48 2015-02-05 22:57:14,589 INFO repeater task os75nb_10pings starting 1.5 seconds late 2015-02-05 22:57:14,777 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 22:58:54,905 INFO repeater task ashtech_hcorrstats starting 1.8 seconds late 2015-02-05 22:58:56,025 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 22:59:01,031 INFO repeater task os75nb_lastensq starting 3.0 seconds late 2015-02-05 22:59:03,589 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 22:59:03,589 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 22:58:48 2015-02-05 23:02:33,854 INFO repeater task os75nb_10pings starting 0.8 seconds late 2015-02-05 23:02:34,038 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 23:03:54,145 INFO repeater task os75nb_quick starting 1.1 seconds late 2015-02-05 23:04:02,416 INFO repeater task os75nb_quick ended after 8.3 seconds 2015-02-05 23:04:02,417 INFO repeater task os75nb_3dayplots starting 9.4 seconds late 2015-02-05 23:04:18,343 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-02-05 23:04:18,344 INFO repeater task os75nb_3dayplots ended after 15.9 seconds 2015-02-05 23:04:18,344 INFO repeater task os75nb_tsstats starting 25.3 seconds late 2015-02-05 23:04:19,547 INFO repeater task os75nb_tsstats ended after 1.2 seconds 2015-02-05 23:04:19,548 INFO repeater task os75nb_beamplots starting 26.5 seconds late 2015-02-05 23:04:21,161 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 23:04:21,161 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 23:04:21,161 INFO repeater task os75nb_beamplots starting 28.1 seconds late 2015-02-05 23:04:22,741 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 23:04:22,742 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 23:04:22,742 INFO repeater task ashtech_hcorrstats starting 29.7 seconds late 2015-02-05 23:04:23,863 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 23:04:28,869 INFO repeater task os75nb_lastensq starting 30.9 seconds late 2015-02-05 23:04:31,435 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 23:04:31,435 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 23:03:48 2015-02-05 23:07:56,723 INFO repeater task os75nb_10pings starting 3.7 seconds late 2015-02-05 23:07:56,911 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 23:08:56,994 INFO repeater task ashtech_hcorrstats starting 3.9 seconds late 2015-02-05 23:08:58,076 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 23:09:03,082 INFO repeater task os75nb_lastensq starting 5.1 seconds late 2015-02-05 23:09:05,666 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 23:09:05,667 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 23:08:48 2015-02-05 23:13:15,986 INFO repeater task os75nb_10pings starting 2.9 seconds late 2015-02-05 23:13:16,182 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 23:13:56,233 INFO repeater task ashtech_hcorrstats starting 3.2 seconds late 2015-02-05 23:13:57,349 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 23:14:02,351 INFO repeater task os75nb_lastensq starting 4.4 seconds late 2015-02-05 23:14:04,916 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 23:14:04,917 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 23:13:48 2015-02-05 23:18:35,252 INFO repeater task os75nb_10pings starting 2.2 seconds late 2015-02-05 23:18:35,438 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 23:18:55,468 INFO repeater task os75nb_quick starting 2.4 seconds late 2015-02-05 23:19:03,738 INFO repeater task os75nb_quick ended after 8.3 seconds 2015-02-05 23:19:03,739 INFO repeater task os75nb_3dayplots starting 10.7 seconds late 2015-02-05 23:19:19,610 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-02-05 23:19:19,611 INFO repeater task os75nb_3dayplots ended after 15.9 seconds 2015-02-05 23:19:19,611 INFO repeater task os75nb_tsstats starting 26.5 seconds late 2015-02-05 23:19:20,796 INFO repeater task os75nb_tsstats ended after 1.2 seconds 2015-02-05 23:19:20,797 INFO repeater task os75nb_beamplots starting 27.7 seconds late 2015-02-05 23:19:22,371 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 23:19:22,372 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 23:19:22,372 INFO repeater task os75nb_beamplots starting 29.3 seconds late 2015-02-05 23:19:23,939 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 23:19:23,940 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 23:19:23,940 INFO repeater task ashtech_hcorrstats starting 30.9 seconds late 2015-02-05 23:19:25,043 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 23:19:30,049 INFO repeater task os75nb_lastensq starting 32.0 seconds late 2015-02-05 23:19:32,620 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 23:19:32,620 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 23:18:48 2015-02-05 23:23:57,927 INFO repeater task os75nb_10pings starting 4.9 seconds late 2015-02-05 23:23:58,111 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 23:23:58,112 INFO repeater task ashtech_hcorrstats starting 5.0 seconds late 2015-02-05 23:23:59,244 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 23:24:04,250 INFO repeater task os75nb_lastensq starting 6.3 seconds late 2015-02-05 23:24:06,805 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 23:24:06,806 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 23:23:48 2015-02-05 23:28:57,169 INFO repeater task ashtech_hcorrstats starting 4.1 seconds late 2015-02-05 23:28:58,267 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 23:29:03,272 INFO repeater task os75nb_lastensq starting 5.3 seconds late 2015-02-05 23:29:05,841 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 23:29:05,841 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 23:28:48 2015-02-05 23:29:15,855 INFO repeater task os75nb_10pings starting 2.8 seconds late 2015-02-05 23:29:16,037 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 23:33:56,411 INFO repeater task os75nb_quick starting 3.4 seconds late 2015-02-05 23:34:04,730 INFO repeater task os75nb_quick ended after 8.3 seconds 2015-02-05 23:34:04,730 INFO repeater task os75nb_3dayplots starting 11.7 seconds late 2015-02-05 23:34:20,661 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-02-05 23:34:20,661 INFO repeater task os75nb_3dayplots ended after 15.9 seconds 2015-02-05 23:34:20,661 INFO repeater task os75nb_tsstats starting 27.6 seconds late 2015-02-05 23:34:21,850 INFO repeater task os75nb_tsstats ended after 1.2 seconds 2015-02-05 23:34:21,851 INFO repeater task os75nb_beamplots starting 28.8 seconds late 2015-02-05 23:34:23,429 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 23:34:23,430 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 23:34:23,430 INFO repeater task os75nb_beamplots starting 30.4 seconds late 2015-02-05 23:34:24,992 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 23:34:24,992 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 23:34:24,993 INFO repeater task ashtech_hcorrstats starting 31.9 seconds late 2015-02-05 23:34:26,112 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 23:34:31,117 INFO repeater task os75nb_lastensq starting 33.1 seconds late 2015-02-05 23:34:33,698 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 23:34:33,698 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 23:33:48 2015-02-05 23:34:33,698 INFO repeater task os75nb_10pings starting 0.6 seconds late 2015-02-05 23:34:33,880 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 23:38:54,253 INFO repeater task ashtech_hcorrstats starting 1.2 seconds late 2015-02-05 23:38:55,375 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 23:39:00,381 INFO repeater task os75nb_lastensq starting 2.4 seconds late 2015-02-05 23:39:02,935 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 23:39:02,936 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 23:38:48 2015-02-05 23:39:58,016 INFO repeater task os75nb_10pings starting 5.0 seconds late 2015-02-05 23:39:58,201 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 23:43:53,493 INFO repeater task ashtech_hcorrstats starting 0.4 seconds late 2015-02-05 23:43:54,608 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 23:43:59,614 INFO repeater task os75nb_lastensq starting 1.6 seconds late 2015-02-05 23:44:02,182 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 23:44:02,182 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 23:43:48 2015-02-05 23:45:17,287 INFO repeater task os75nb_10pings starting 4.2 seconds late 2015-02-05 23:45:17,471 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 23:48:57,739 INFO repeater task os75nb_quick starting 4.7 seconds late 2015-02-05 23:49:06,015 INFO repeater task os75nb_quick ended after 8.3 seconds 2015-02-05 23:49:06,015 INFO repeater task os75nb_3dayplots starting 13.0 seconds late 2015-02-05 23:49:21,889 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-02-05 23:49:21,889 INFO repeater task os75nb_3dayplots ended after 15.9 seconds 2015-02-05 23:49:21,890 INFO repeater task os75nb_tsstats starting 28.8 seconds late 2015-02-05 23:49:23,087 INFO repeater task os75nb_tsstats ended after 1.2 seconds 2015-02-05 23:49:23,087 INFO repeater task os75nb_beamplots starting 30.0 seconds late 2015-02-05 23:49:24,670 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 23:49:24,671 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 23:49:24,671 INFO repeater task os75nb_beamplots starting 31.6 seconds late 2015-02-05 23:49:26,256 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-05 23:49:26,256 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-05 23:49:26,256 INFO repeater task ashtech_hcorrstats starting 33.2 seconds late 2015-02-05 23:49:27,367 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 23:49:32,372 INFO repeater task os75nb_lastensq starting 34.4 seconds late 2015-02-05 23:49:34,955 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 23:49:34,956 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 23:48:48 2015-02-05 23:50:35,046 INFO repeater task os75nb_10pings starting 2.0 seconds late 2015-02-05 23:50:35,229 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 23:53:55,499 INFO repeater task ashtech_hcorrstats starting 2.4 seconds late 2015-02-05 23:53:56,614 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 23:54:01,620 INFO repeater task os75nb_lastensq starting 3.6 seconds late 2015-02-05 23:54:04,192 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 23:54:04,193 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 23:53:48 2015-02-05 23:55:54,316 INFO repeater task os75nb_10pings starting 1.3 seconds late 2015-02-05 23:55:54,499 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-05 23:58:54,723 INFO repeater task ashtech_hcorrstats starting 1.7 seconds late 2015-02-05 23:58:55,840 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-05 23:59:00,846 INFO repeater task os75nb_lastensq starting 2.8 seconds late 2015-02-05 23:59:03,417 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-05 23:59:03,417 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/05 23:58:48 2015-02-06 00:01:13,580 INFO repeater task os75nb_10pings starting 0.5 seconds late 2015-02-06 00:01:13,763 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-06 00:03:53,963 INFO repeater task os75nb_quick starting 0.9 seconds late 2015-02-06 00:04:02,233 INFO repeater task os75nb_quick ended after 8.3 seconds 2015-02-06 00:04:02,234 INFO repeater task os75nb_3dayplots starting 9.2 seconds late 2015-02-06 00:04:18,078 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-02-06 00:04:18,078 INFO repeater task os75nb_3dayplots ended after 15.8 seconds 2015-02-06 00:04:18,079 INFO repeater task os75nb_tsstats starting 25.0 seconds late 2015-02-06 00:04:19,271 INFO repeater task os75nb_tsstats ended after 1.2 seconds 2015-02-06 00:04:19,281 INFO repeater task os75nb_beamplots starting 26.2 seconds late 2015-02-06 00:04:20,863 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-06 00:04:20,864 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-06 00:04:20,864 INFO repeater task os75nb_beamplots starting 27.8 seconds late 2015-02-06 00:04:22,429 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-06 00:04:22,430 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-06 00:04:22,430 INFO repeater task ashtech_hcorrstats starting 29.4 seconds late 2015-02-06 00:04:23,552 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-06 00:04:28,558 INFO repeater task os75nb_lastensq starting 30.6 seconds late 2015-02-06 00:04:31,177 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-06 00:04:31,177 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/06 00:03:48 2015-02-06 00:06:36,350 INFO repeater task os75nb_10pings starting 3.3 seconds late 2015-02-06 00:06:36,534 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-06 00:08:56,712 INFO repeater task ashtech_hcorrstats starting 3.6 seconds late 2015-02-06 00:08:57,828 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-06 00:09:02,833 INFO repeater task os75nb_lastensq starting 4.8 seconds late 2015-02-06 00:09:05,358 INFO repeater task os75nb_lastensq ended after 2.5 seconds 2015-02-06 00:09:05,358 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/06 00:08:48 2015-02-06 00:11:55,601 INFO repeater task os75nb_10pings starting 2.5 seconds late 2015-02-06 00:11:55,787 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-06 00:13:55,953 INFO repeater task ashtech_hcorrstats starting 2.9 seconds late 2015-02-06 00:13:57,072 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-06 00:14:02,077 INFO repeater task os75nb_lastensq starting 4.1 seconds late 2015-02-06 00:14:04,665 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-06 00:14:04,665 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/06 00:13:48 2015-02-06 00:17:14,878 INFO repeater task os75nb_10pings starting 1.8 seconds late 2015-02-06 00:17:15,064 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-06 00:18:55,193 INFO repeater task os75nb_quick starting 2.1 seconds late 2015-02-06 00:19:03,604 INFO repeater task os75nb_quick ended after 8.4 seconds 2015-02-06 00:19:03,604 INFO repeater task os75nb_3dayplots starting 10.5 seconds late 2015-02-06 00:19:19,984 INFO repeater task os75nb_3dayplots output: /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:319: RuntimeWarning: All-NaN slice encountered warnings.warn("All-NaN slice encountered", RuntimeWarning) /usr/lib/python2.7/dist-packages/numpy/lib/nanfunctions.py:220: RuntimeWarning: All-NaN axis encountered warnings.warn("All-NaN axis encountered", RuntimeWarning) 2015-02-06 00:19:19,985 INFO repeater task os75nb_3dayplots ended after 16.4 seconds 2015-02-06 00:19:19,985 INFO repeater task os75nb_tsstats starting 26.9 seconds late 2015-02-06 00:19:21,185 INFO repeater task os75nb_tsstats ended after 1.2 seconds 2015-02-06 00:19:21,186 INFO repeater task os75nb_beamplots starting 28.1 seconds late 2015-02-06 00:19:22,816 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-06 00:19:22,816 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-06 00:19:22,817 INFO repeater task os75nb_beamplots starting 29.8 seconds late 2015-02-06 00:19:24,452 INFO repeater task os75nb_beamplots output: Starting procsetup scriptpath is /home/currents/programs/uhdas/scripts CURRENTS_REPO_PATH not found; assuming in-place installation programs_path is /home/currents/programs 2015-02-06 00:19:24,452 INFO repeater task os75nb_beamplots ended after 1.6 seconds 2015-02-06 00:19:24,452 INFO repeater task ashtech_hcorrstats starting 31.4 seconds late 2015-02-06 00:19:25,564 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-06 00:19:30,570 INFO repeater task os75nb_lastensq starting 32.6 seconds late 2015-02-06 00:19:33,129 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-06 00:19:33,130 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/06 00:18:48 2015-02-06 00:22:33,361 INFO repeater task os75nb_10pings starting 0.3 seconds late 2015-02-06 00:22:33,545 INFO repeater task os75nb_10pings ended after 0.2 seconds 2015-02-06 00:23:53,637 INFO repeater task ashtech_hcorrstats starting 0.6 seconds late 2015-02-06 00:23:54,752 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2015-02-06 00:23:59,758 INFO repeater task os75nb_lastensq starting 1.8 seconds late 2015-02-06 00:24:02,329 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2015-02-06 00:24:02,330 INFO repeater task os75nb_lastensq: Time from /home/data/CALCOFI1501NH/proc/os75nb/load/wrote_ens_time: 2015/02/06 00:23:48