2012-07-14 22:00:24,630 INFO repeater PeriodicSystem: os75nb_quick interval is 900 seconds cd /home/adcp/uhdas_tmp; run_quick.py -d os75nb 2012-07-14 22:00:24,631 INFO repeater PeriodicSystem: os75nb_3dayplots interval is 900 seconds cd /home/adcp/uhdas_tmp; run_3dayplots_mpl.py -d os75nb 2012-07-14 22:00:24,632 INFO repeater PeriodicSystem: ashtech_hcorrstats interval is 300 seconds cd /home/adcp/uhdas_tmp; run_hbinhcorrstats.py --hcorr_inst ashtech --plotdh --printstats 2012-07-14 22:00:24,632 INFO repeater PeriodicSystem: phins_hcorrstats interval is 300 seconds cd /home/adcp/uhdas_tmp; run_hbinhcorrstats.py --hcorr_inst phins --plotdh --printstats 2012-07-14 22:00:24,632 INFO root cleanup_figs no_figure 2012-07-14 22:05:25,252 INFO repeater task ashtech_hcorrstats starting 0.6 seconds late 2012-07-14 22:05:27,233 INFO repeater task ashtech_hcorrstats ended after 2.0 seconds 2012-07-14 22:05:27,233 INFO repeater task phins_hcorrstats starting 2.6 seconds late 2012-07-14 22:05:28,170 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-14 22:05:28,171 INFO repeater task phins_hcorrstats ended after 0.9 seconds 2012-07-14 22:05:38,173 INFO repeater task nb150_lastensq starting 3.5 seconds late 2012-07-14 22:05:41,960 INFO repeater task nb150_lastensq ended after 3.8 seconds 2012-07-14 22:05:41,965 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/14 22:05:23 2012-07-14 22:05:41,990 INFO repeater task os75nb_lastensq starting 7.4 seconds late 2012-07-14 22:05:44,076 INFO repeater task os75nb_lastensq ended after 2.1 seconds 2012-07-14 22:05:44,076 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/14 22:05:24 2012-07-14 22:10:24,961 INFO repeater task ashtech_hcorrstats starting 0.3 seconds late 2012-07-14 22:10:26,367 INFO repeater task ashtech_hcorrstats ended after 1.4 seconds 2012-07-14 22:10:26,372 INFO repeater task phins_hcorrstats starting 1.7 seconds late 2012-07-14 22:10:27,819 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-14 22:10:27,819 INFO repeater task phins_hcorrstats ended after 1.4 seconds 2012-07-14 22:10:37,831 INFO repeater task nb150_lastensq starting 4.8 seconds late 2012-07-14 22:10:40,543 INFO repeater task nb150_lastensq ended after 2.7 seconds 2012-07-14 22:10:40,543 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/14 22:10:23 2012-07-14 22:10:40,544 INFO repeater task os75nb_lastensq starting 6.5 seconds late 2012-07-14 22:10:42,710 INFO repeater task os75nb_lastensq ended after 2.2 seconds 2012-07-14 22:10:42,710 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/14 22:10:24 2012-07-14 22:15:29,251 INFO repeater task nb150_quick starting 4.6 seconds late 2012-07-14 22:15:40,206 INFO repeater task nb150_quick output: no data available 2012-07-14 22:15:40,206 INFO repeater task nb150_quick ended after 11.0 seconds 2012-07-14 22:15:40,207 INFO repeater task nb150_3dayplots starting 15.6 seconds late 2012-07-14 22:15:56,042 INFO repeater task nb150_3dayplots ended after 15.8 seconds 2012-07-14 22:15:56,043 INFO repeater task os75nb_lastensq starting 22.0 seconds late 2012-07-14 22:15:58,201 INFO repeater task os75nb_lastensq ended after 2.2 seconds 2012-07-14 22:15:58,202 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/14 22:15:24 2012-07-14 22:15:58,206 INFO repeater task os75nb_quick starting 33.6 seconds late 2012-07-14 22:16:08,117 INFO repeater task os75nb_quick output: no data available 2012-07-14 22:16:08,118 INFO repeater task os75nb_quick ended after 9.9 seconds 2012-07-14 22:16:08,122 INFO repeater task os75nb_3dayplots starting 43.5 seconds late 2012-07-14 22:16:25,982 INFO repeater task os75nb_3dayplots ended after 17.9 seconds 2012-07-14 22:16:25,983 INFO repeater task ashtech_hcorrstats starting 61.4 seconds late 2012-07-14 22:16:28,094 INFO repeater task ashtech_hcorrstats ended after 2.1 seconds 2012-07-14 22:16:28,094 INFO repeater task phins_hcorrstats starting 63.5 seconds late 2012-07-14 22:16:29,257 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-14 22:16:29,257 INFO repeater task phins_hcorrstats ended after 1.2 seconds 2012-07-14 22:16:34,271 INFO repeater task nb150_lastensq starting 61.3 seconds late 2012-07-14 22:16:39,299 INFO repeater task nb150_lastensq ended after 5.0 seconds 2012-07-14 22:16:39,300 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/14 22:15:23 2012-07-14 22:20:25,213 INFO repeater task ashtech_hcorrstats starting 0.6 seconds late 2012-07-14 22:20:27,300 INFO repeater task ashtech_hcorrstats ended after 2.1 seconds 2012-07-14 22:20:27,306 INFO repeater task phins_hcorrstats starting 2.7 seconds late 2012-07-14 22:20:29,163 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-14 22:20:29,163 INFO repeater task phins_hcorrstats ended after 1.9 seconds 2012-07-14 22:20:34,168 INFO repeater task nb150_lastensq starting 1.2 seconds late 2012-07-14 22:20:37,424 INFO repeater task nb150_lastensq ended after 3.3 seconds 2012-07-14 22:20:37,425 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/14 22:20:23 2012-07-14 22:20:37,429 INFO repeater task os75nb_lastensq starting 3.4 seconds late 2012-07-14 22:20:38,833 INFO repeater task os75nb_lastensq ended after 1.4 seconds 2012-07-14 22:20:38,833 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/14 22:20:24 2012-07-14 22:25:25,005 INFO repeater task ashtech_hcorrstats starting 0.4 seconds late 2012-07-14 22:25:25,964 INFO repeater task ashtech_hcorrstats ended after 1.0 seconds 2012-07-14 22:25:25,965 INFO repeater task phins_hcorrstats starting 1.3 seconds late 2012-07-14 22:25:26,850 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-14 22:25:26,851 INFO repeater task phins_hcorrstats ended after 0.9 seconds 2012-07-14 22:25:36,901 INFO repeater task nb150_lastensq starting 3.9 seconds late 2012-07-14 22:25:40,382 INFO repeater task nb150_lastensq ended after 3.5 seconds 2012-07-14 22:25:40,382 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/14 22:25:23 2012-07-14 22:25:40,383 INFO repeater task os75nb_lastensq starting 6.4 seconds late 2012-07-14 22:25:41,933 INFO repeater task os75nb_lastensq ended after 1.5 seconds 2012-07-14 22:25:41,933 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/14 22:25:24 2012-07-14 22:30:27,661 INFO repeater task nb150_quick starting 3.0 seconds late 2012-07-14 22:30:40,209 INFO repeater task nb150_quick output: no data available 2012-07-14 22:30:40,209 INFO repeater task nb150_quick ended after 12.5 seconds 2012-07-14 22:30:40,210 INFO repeater task nb150_3dayplots starting 15.6 seconds late 2012-07-14 22:30:56,775 INFO repeater task nb150_3dayplots ended after 16.6 seconds 2012-07-14 22:30:56,775 INFO repeater task os75nb_lastensq starting 22.8 seconds late 2012-07-14 22:30:58,032 INFO repeater task os75nb_lastensq ended after 1.3 seconds 2012-07-14 22:30:58,032 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/14 22:30:24 2012-07-14 22:30:58,033 INFO repeater task os75nb_quick starting 33.4 seconds late 2012-07-14 22:31:09,621 INFO repeater task os75nb_quick output: no data available 2012-07-14 22:31:09,621 INFO repeater task os75nb_quick ended after 11.6 seconds 2012-07-14 22:31:09,622 INFO repeater task os75nb_3dayplots starting 45.0 seconds late 2012-07-14 22:31:27,750 INFO repeater task os75nb_3dayplots ended after 18.1 seconds 2012-07-14 22:31:27,757 INFO repeater task ashtech_hcorrstats starting 63.1 seconds late 2012-07-14 22:31:30,457 INFO repeater task ashtech_hcorrstats ended after 2.7 seconds 2012-07-14 22:31:30,458 INFO repeater task phins_hcorrstats starting 65.8 seconds late 2012-07-14 22:31:31,611 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-14 22:31:31,611 INFO repeater task phins_hcorrstats ended after 1.2 seconds 2012-07-14 22:31:36,618 INFO repeater task nb150_lastensq starting 63.6 seconds late 2012-07-14 22:31:39,699 INFO repeater task nb150_lastensq ended after 3.1 seconds 2012-07-14 22:31:39,700 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/14 22:30:23 2012-07-14 22:35:25,859 INFO repeater task ashtech_hcorrstats starting 1.2 seconds late 2012-07-14 22:35:27,777 INFO repeater task ashtech_hcorrstats ended after 1.9 seconds 2012-07-14 22:35:27,783 INFO repeater task phins_hcorrstats starting 3.2 seconds late 2012-07-14 22:35:29,182 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-14 22:35:29,182 INFO repeater task phins_hcorrstats ended after 1.4 seconds 2012-07-14 22:35:34,189 INFO repeater task nb150_lastensq starting 1.2 seconds late 2012-07-14 22:35:38,092 INFO repeater task nb150_lastensq ended after 3.9 seconds 2012-07-14 22:35:38,093 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/14 22:35:23 2012-07-14 22:35:38,112 INFO repeater task os75nb_lastensq starting 4.1 seconds late 2012-07-14 22:35:39,483 INFO repeater task os75nb_lastensq ended after 1.4 seconds 2012-07-14 22:35:39,484 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/14 22:35:24 2012-07-14 22:40:26,802 INFO repeater task ashtech_hcorrstats starting 2.2 seconds late 2012-07-14 22:40:28,824 INFO repeater task ashtech_hcorrstats ended after 2.0 seconds 2012-07-14 22:40:28,825 INFO repeater task phins_hcorrstats starting 4.2 seconds late 2012-07-14 22:40:29,884 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-14 22:40:29,884 INFO repeater task phins_hcorrstats ended after 1.1 seconds 2012-07-14 22:40:34,888 INFO repeater task nb150_lastensq starting 1.9 seconds late 2012-07-14 22:40:38,998 INFO repeater task nb150_lastensq ended after 4.1 seconds 2012-07-14 22:40:38,999 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/14 22:40:23 2012-07-14 22:40:39,000 INFO repeater task os75nb_lastensq starting 5.0 seconds late 2012-07-14 22:40:40,061 INFO repeater task os75nb_lastensq ended after 1.1 seconds 2012-07-14 22:40:40,061 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/14 22:40:24 2012-07-14 22:45:25,635 INFO repeater task nb150_quick starting 1.0 seconds late 2012-07-14 22:45:36,600 INFO repeater task nb150_quick output: no data available 2012-07-14 22:45:36,600 INFO repeater task nb150_quick ended after 11.0 seconds 2012-07-14 22:45:36,606 INFO repeater task nb150_3dayplots starting 12.0 seconds late 2012-07-14 22:45:53,105 INFO repeater task nb150_3dayplots ended after 16.5 seconds 2012-07-14 22:45:53,106 INFO repeater task os75nb_lastensq starting 19.1 seconds late 2012-07-14 22:45:54,235 INFO repeater task os75nb_lastensq ended after 1.1 seconds 2012-07-14 22:45:54,235 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/14 22:45:24 2012-07-14 22:45:54,235 INFO repeater task os75nb_quick starting 29.6 seconds late 2012-07-14 22:46:06,029 INFO repeater task os75nb_quick output: no data available 2012-07-14 22:46:06,029 INFO repeater task os75nb_quick ended after 11.8 seconds 2012-07-14 22:46:06,030 INFO repeater task os75nb_3dayplots starting 41.4 seconds late 2012-07-14 22:46:19,993 INFO repeater task os75nb_3dayplots ended after 14.0 seconds 2012-07-14 22:46:19,995 INFO repeater task ashtech_hcorrstats starting 55.4 seconds late 2012-07-14 22:46:21,784 INFO repeater task ashtech_hcorrstats ended after 1.8 seconds 2012-07-14 22:46:21,790 INFO repeater task phins_hcorrstats starting 57.2 seconds late 2012-07-14 22:46:22,983 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-14 22:46:22,983 INFO repeater task phins_hcorrstats ended after 1.2 seconds 2012-07-14 22:46:27,988 INFO repeater task nb150_lastensq starting 55.0 seconds late 2012-07-14 22:46:31,980 INFO repeater task nb150_lastensq ended after 4.0 seconds 2012-07-14 22:46:31,980 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/14 22:45:23 2012-07-14 22:50:28,865 INFO repeater task ashtech_hcorrstats starting 4.2 seconds late 2012-07-14 22:50:30,722 INFO repeater task ashtech_hcorrstats ended after 1.9 seconds 2012-07-14 22:50:30,722 INFO repeater task phins_hcorrstats starting 6.1 seconds late 2012-07-14 22:50:32,180 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-14 22:50:32,183 INFO repeater task phins_hcorrstats ended after 1.5 seconds 2012-07-14 22:50:37,193 INFO repeater task nb150_lastensq starting 4.2 seconds late 2012-07-14 22:50:40,247 INFO repeater task nb150_lastensq ended after 3.1 seconds 2012-07-14 22:50:40,248 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/14 22:50:23 2012-07-14 22:50:40,255 INFO repeater task os75nb_lastensq starting 6.3 seconds late 2012-07-14 22:50:41,423 INFO repeater task os75nb_lastensq ended after 1.2 seconds 2012-07-14 22:50:41,423 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/14 22:50:24 2012-07-14 22:55:29,011 INFO repeater task ashtech_hcorrstats starting 4.4 seconds late 2012-07-14 22:55:30,325 INFO repeater task ashtech_hcorrstats ended after 1.3 seconds 2012-07-14 22:55:30,326 INFO repeater task phins_hcorrstats starting 5.7 seconds late 2012-07-14 22:55:31,297 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-14 22:55:31,297 INFO repeater task phins_hcorrstats ended after 1.0 seconds 2012-07-14 22:55:36,301 INFO repeater task nb150_lastensq starting 3.3 seconds late 2012-07-14 22:55:39,766 INFO repeater task nb150_lastensq ended after 3.5 seconds 2012-07-14 22:55:39,766 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/14 22:55:23 2012-07-14 22:55:39,767 INFO repeater task os75nb_lastensq starting 5.8 seconds late 2012-07-14 22:55:40,874 INFO repeater task os75nb_lastensq ended after 1.1 seconds 2012-07-14 22:55:40,875 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/14 22:55:24 2012-07-14 23:00:28,161 INFO repeater task nb150_quick starting 3.5 seconds late 2012-07-14 23:00:43,521 INFO repeater task nb150_quick output: no data available 2012-07-14 23:00:43,521 INFO repeater task nb150_quick ended after 15.4 seconds 2012-07-14 23:00:43,525 INFO repeater task nb150_3dayplots starting 18.9 seconds late 2012-07-14 23:01:00,674 INFO repeater task nb150_3dayplots ended after 17.1 seconds 2012-07-14 23:01:00,675 INFO repeater task os75nb_lastensq starting 26.7 seconds late 2012-07-14 23:01:01,774 INFO repeater task os75nb_lastensq ended after 1.1 seconds 2012-07-14 23:01:01,774 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/14 23:00:24 2012-07-14 23:01:01,779 INFO repeater task os75nb_quick starting 37.1 seconds late 2012-07-14 23:01:12,784 INFO repeater task os75nb_quick output: no data available 2012-07-14 23:01:12,785 INFO repeater task os75nb_quick ended after 11.0 seconds 2012-07-14 23:01:12,788 INFO repeater task os75nb_3dayplots starting 48.2 seconds late 2012-07-14 23:01:30,674 INFO repeater task os75nb_3dayplots ended after 17.9 seconds 2012-07-14 23:01:30,674 INFO repeater task ashtech_hcorrstats starting 66.0 seconds late 2012-07-14 23:01:32,792 INFO repeater task ashtech_hcorrstats ended after 2.1 seconds 2012-07-14 23:01:32,805 INFO repeater task phins_hcorrstats starting 68.2 seconds late 2012-07-14 23:01:35,763 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-14 23:01:35,763 INFO repeater task phins_hcorrstats ended after 3.0 seconds 2012-07-14 23:01:40,764 INFO repeater task nb150_lastensq starting 67.8 seconds late 2012-07-14 23:01:44,645 INFO repeater task nb150_lastensq ended after 3.9 seconds 2012-07-14 23:01:44,646 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/14 23:00:23 2012-07-14 23:05:26,770 INFO repeater task ashtech_hcorrstats starting 2.1 seconds late 2012-07-14 23:05:28,720 INFO repeater task ashtech_hcorrstats ended after 2.0 seconds 2012-07-14 23:05:28,724 INFO repeater task phins_hcorrstats starting 4.1 seconds late 2012-07-14 23:05:29,712 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-14 23:05:29,712 INFO repeater task phins_hcorrstats ended after 1.0 seconds 2012-07-14 23:05:34,713 INFO repeater task nb150_lastensq starting 1.7 seconds late 2012-07-14 23:05:37,220 INFO repeater task nb150_lastensq ended after 2.5 seconds 2012-07-14 23:05:37,221 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/14 23:05:23 2012-07-14 23:05:37,250 INFO repeater task os75nb_lastensq starting 3.3 seconds late 2012-07-14 23:05:39,519 INFO repeater task os75nb_lastensq ended after 2.3 seconds 2012-07-14 23:05:39,520 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/14 23:05:24 2012-07-14 23:10:25,103 INFO repeater task ashtech_hcorrstats starting 0.5 seconds late 2012-07-14 23:10:26,066 INFO repeater task ashtech_hcorrstats ended after 1.0 seconds 2012-07-14 23:10:26,067 INFO repeater task phins_hcorrstats starting 1.4 seconds late 2012-07-14 23:10:27,177 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-14 23:10:27,177 INFO repeater task phins_hcorrstats ended after 1.1 seconds 2012-07-14 23:10:37,228 INFO repeater task nb150_lastensq starting 4.2 seconds late 2012-07-14 23:10:40,142 INFO repeater task nb150_lastensq ended after 2.9 seconds 2012-07-14 23:10:40,143 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/14 23:10:23 2012-07-14 23:10:40,144 INFO repeater task os75nb_lastensq starting 6.1 seconds late 2012-07-14 23:10:41,687 INFO repeater task os75nb_lastensq ended after 1.5 seconds 2012-07-14 23:10:41,688 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/14 23:10:24 2012-07-14 23:15:29,154 INFO repeater task nb150_quick starting 4.5 seconds late 2012-07-14 23:15:39,631 INFO repeater task nb150_quick output: no data available 2012-07-14 23:15:39,631 INFO repeater task nb150_quick ended after 10.5 seconds 2012-07-14 23:15:39,631 INFO repeater task nb150_3dayplots starting 15.0 seconds late 2012-07-14 23:15:52,748 INFO repeater task nb150_3dayplots ended after 13.1 seconds 2012-07-14 23:15:52,752 INFO repeater task os75nb_lastensq starting 18.8 seconds late 2012-07-14 23:15:54,371 INFO repeater task os75nb_lastensq ended after 1.6 seconds 2012-07-14 23:15:54,372 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/14 23:15:24 2012-07-14 23:15:54,372 INFO repeater task os75nb_quick starting 29.7 seconds late 2012-07-14 23:16:07,177 INFO repeater task os75nb_quick output: no data available 2012-07-14 23:16:07,177 INFO repeater task os75nb_quick ended after 12.8 seconds 2012-07-14 23:16:07,177 INFO repeater task os75nb_3dayplots starting 42.5 seconds late 2012-07-14 23:16:21,475 INFO repeater task os75nb_3dayplots ended after 14.3 seconds 2012-07-14 23:16:21,483 INFO repeater task ashtech_hcorrstats starting 56.9 seconds late 2012-07-14 23:16:23,682 INFO repeater task ashtech_hcorrstats ended after 2.2 seconds 2012-07-14 23:16:23,690 INFO repeater task phins_hcorrstats starting 59.1 seconds late 2012-07-14 23:16:25,435 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-14 23:16:25,435 INFO repeater task phins_hcorrstats ended after 1.7 seconds 2012-07-14 23:16:30,445 INFO repeater task nb150_lastensq starting 57.4 seconds late 2012-07-14 23:16:32,801 INFO repeater task nb150_lastensq ended after 2.4 seconds 2012-07-14 23:16:32,801 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/14 23:15:23 2012-07-14 23:20:29,425 INFO repeater task ashtech_hcorrstats starting 4.8 seconds late 2012-07-14 23:20:30,390 INFO repeater task ashtech_hcorrstats ended after 1.0 seconds 2012-07-14 23:20:30,391 INFO repeater task phins_hcorrstats starting 5.8 seconds late 2012-07-14 23:20:31,870 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-14 23:20:31,871 INFO repeater task phins_hcorrstats ended after 1.5 seconds 2012-07-14 23:20:36,875 INFO repeater task nb150_lastensq starting 3.9 seconds late 2012-07-14 23:20:40,074 INFO repeater task nb150_lastensq ended after 3.2 seconds 2012-07-14 23:20:40,075 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/14 23:20:23 2012-07-14 23:20:40,083 INFO repeater task os75nb_lastensq starting 6.1 seconds late 2012-07-14 23:20:41,309 INFO repeater task os75nb_lastensq ended after 1.2 seconds 2012-07-14 23:20:41,309 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/14 23:20:24 2012-07-14 23:25:28,317 INFO repeater task ashtech_hcorrstats starting 3.7 seconds late 2012-07-14 23:25:29,632 INFO repeater task ashtech_hcorrstats ended after 1.3 seconds 2012-07-14 23:25:29,633 INFO repeater task phins_hcorrstats starting 5.0 seconds late 2012-07-14 23:25:30,743 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-14 23:25:30,743 INFO repeater task phins_hcorrstats ended after 1.1 seconds 2012-07-14 23:25:35,744 INFO repeater task nb150_lastensq starting 2.7 seconds late 2012-07-14 23:25:38,402 INFO repeater task nb150_lastensq ended after 2.7 seconds 2012-07-14 23:25:38,403 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/14 23:25:23 2012-07-14 23:25:38,403 INFO repeater task os75nb_lastensq starting 4.4 seconds late 2012-07-14 23:25:39,531 INFO repeater task os75nb_lastensq ended after 1.1 seconds 2012-07-14 23:25:39,532 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/14 23:25:24 2012-07-14 23:30:25,784 INFO repeater task nb150_quick starting 1.2 seconds late 2012-07-14 23:30:39,445 INFO repeater task nb150_quick output: no data available 2012-07-14 23:30:39,445 INFO repeater task nb150_quick ended after 13.7 seconds 2012-07-14 23:30:39,446 INFO repeater task nb150_3dayplots starting 14.8 seconds late 2012-07-14 23:30:52,054 INFO repeater task nb150_3dayplots ended after 12.6 seconds 2012-07-14 23:30:52,055 INFO repeater task os75nb_lastensq starting 18.1 seconds late 2012-07-14 23:30:53,950 INFO repeater task os75nb_lastensq ended after 1.9 seconds 2012-07-14 23:30:53,950 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/14 23:30:24 2012-07-14 23:30:53,951 INFO repeater task os75nb_quick starting 29.3 seconds late 2012-07-14 23:31:06,698 INFO repeater task os75nb_quick output: no data available 2012-07-14 23:31:06,699 INFO repeater task os75nb_quick ended after 12.7 seconds 2012-07-14 23:31:06,699 INFO repeater task os75nb_3dayplots starting 42.1 seconds late 2012-07-14 23:31:23,027 INFO repeater task os75nb_3dayplots ended after 16.3 seconds 2012-07-14 23:31:23,028 INFO repeater task ashtech_hcorrstats starting 58.4 seconds late 2012-07-14 23:31:24,118 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2012-07-14 23:31:24,120 INFO repeater task phins_hcorrstats starting 59.5 seconds late 2012-07-14 23:31:25,695 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-14 23:31:25,695 INFO repeater task phins_hcorrstats ended after 1.6 seconds 2012-07-14 23:31:30,701 INFO repeater task nb150_lastensq starting 57.7 seconds late 2012-07-14 23:31:33,896 INFO repeater task nb150_lastensq ended after 3.2 seconds 2012-07-14 23:31:33,897 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/14 23:30:23 2012-07-14 23:35:29,385 INFO repeater task ashtech_hcorrstats starting 4.8 seconds late 2012-07-14 23:35:30,343 INFO repeater task ashtech_hcorrstats ended after 1.0 seconds 2012-07-14 23:35:30,343 INFO repeater task phins_hcorrstats starting 5.7 seconds late 2012-07-14 23:35:31,559 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-14 23:35:31,559 INFO repeater task phins_hcorrstats ended after 1.2 seconds 2012-07-14 23:35:36,565 INFO repeater task nb150_lastensq starting 3.6 seconds late 2012-07-14 23:35:39,830 INFO repeater task nb150_lastensq ended after 3.3 seconds 2012-07-14 23:35:39,831 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/14 23:35:23 2012-07-14 23:35:39,833 INFO repeater task os75nb_lastensq starting 5.8 seconds late 2012-07-14 23:35:41,711 INFO repeater task os75nb_lastensq ended after 1.9 seconds 2012-07-14 23:35:41,712 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/14 23:35:24 2012-07-14 23:40:28,529 INFO repeater task ashtech_hcorrstats starting 3.9 seconds late 2012-07-14 23:40:30,430 INFO repeater task ashtech_hcorrstats ended after 1.9 seconds 2012-07-14 23:40:30,431 INFO repeater task phins_hcorrstats starting 5.8 seconds late 2012-07-14 23:40:31,321 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-14 23:40:31,322 INFO repeater task phins_hcorrstats ended after 0.9 seconds 2012-07-14 23:40:36,328 INFO repeater task nb150_lastensq starting 3.3 seconds late 2012-07-14 23:40:39,888 INFO repeater task nb150_lastensq ended after 3.6 seconds 2012-07-14 23:40:39,889 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/14 23:40:23 2012-07-14 23:40:39,890 INFO repeater task os75nb_lastensq starting 5.9 seconds late 2012-07-14 23:40:40,975 INFO repeater task os75nb_lastensq ended after 1.1 seconds 2012-07-14 23:40:40,976 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/14 23:40:24 2012-07-14 23:45:28,587 INFO repeater task nb150_quick starting 4.0 seconds late 2012-07-14 23:45:40,656 INFO repeater task nb150_quick output: no data available 2012-07-14 23:45:40,656 INFO repeater task nb150_quick ended after 12.1 seconds 2012-07-14 23:45:40,657 INFO repeater task nb150_3dayplots starting 16.0 seconds late 2012-07-14 23:45:55,344 INFO repeater task nb150_3dayplots ended after 14.7 seconds 2012-07-14 23:45:55,345 INFO repeater task os75nb_lastensq starting 21.3 seconds late 2012-07-14 23:45:57,215 INFO repeater task os75nb_lastensq ended after 1.9 seconds 2012-07-14 23:45:57,215 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/14 23:45:24 2012-07-14 23:45:57,221 INFO repeater task os75nb_quick starting 32.6 seconds late 2012-07-14 23:46:11,527 INFO repeater task os75nb_quick output: no data available 2012-07-14 23:46:11,527 INFO repeater task os75nb_quick ended after 14.3 seconds 2012-07-14 23:46:11,532 INFO repeater task os75nb_3dayplots starting 46.9 seconds late 2012-07-14 23:46:26,838 INFO repeater task os75nb_3dayplots ended after 15.3 seconds 2012-07-14 23:46:26,839 INFO repeater task ashtech_hcorrstats starting 62.2 seconds late 2012-07-14 23:46:29,428 INFO repeater task ashtech_hcorrstats ended after 2.6 seconds 2012-07-14 23:46:29,431 INFO repeater task phins_hcorrstats starting 64.8 seconds late 2012-07-14 23:46:30,898 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-14 23:46:30,898 INFO repeater task phins_hcorrstats ended after 1.5 seconds 2012-07-14 23:46:35,899 INFO repeater task nb150_lastensq starting 62.9 seconds late 2012-07-14 23:46:39,547 INFO repeater task nb150_lastensq ended after 3.6 seconds 2012-07-14 23:46:39,548 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/14 23:45:23 2012-07-14 23:50:25,211 INFO repeater task ashtech_hcorrstats starting 0.6 seconds late 2012-07-14 23:50:28,640 INFO repeater task ashtech_hcorrstats ended after 3.4 seconds 2012-07-14 23:50:28,645 INFO repeater task phins_hcorrstats starting 4.0 seconds late 2012-07-14 23:50:30,877 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-14 23:50:30,877 INFO repeater task phins_hcorrstats ended after 2.2 seconds 2012-07-14 23:50:35,882 INFO repeater task nb150_lastensq starting 2.9 seconds late 2012-07-14 23:50:40,296 INFO repeater task nb150_lastensq ended after 4.4 seconds 2012-07-14 23:50:40,296 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/14 23:50:23 2012-07-14 23:50:40,298 INFO repeater task os75nb_lastensq starting 6.3 seconds late 2012-07-14 23:50:42,215 INFO repeater task os75nb_lastensq ended after 1.9 seconds 2012-07-14 23:50:42,216 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/14 23:50:24 2012-07-14 23:55:24,767 INFO repeater task ashtech_hcorrstats starting 0.1 seconds late 2012-07-14 23:55:25,749 INFO repeater task ashtech_hcorrstats ended after 1.0 seconds 2012-07-14 23:55:25,750 INFO repeater task phins_hcorrstats starting 1.1 seconds late 2012-07-14 23:55:27,872 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-14 23:55:27,873 INFO repeater task phins_hcorrstats ended after 2.1 seconds 2012-07-14 23:55:37,930 INFO repeater task nb150_lastensq starting 4.9 seconds late 2012-07-14 23:55:40,908 INFO repeater task nb150_lastensq ended after 3.0 seconds 2012-07-14 23:55:40,908 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/14 23:55:23 2012-07-14 23:55:40,925 INFO repeater task os75nb_lastensq starting 6.9 seconds late 2012-07-14 23:55:43,314 INFO repeater task os75nb_lastensq ended after 2.4 seconds 2012-07-14 23:55:43,314 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/14 23:55:24 2012-07-15 00:00:25,431 INFO repeater task nb150_quick starting 0.8 seconds late 2012-07-15 00:00:39,279 INFO repeater task nb150_quick output: no data available 2012-07-15 00:00:39,279 INFO repeater task nb150_quick ended after 13.8 seconds 2012-07-15 00:00:39,280 INFO repeater task nb150_3dayplots starting 14.7 seconds late 2012-07-15 00:00:51,397 INFO repeater task nb150_3dayplots ended after 12.1 seconds 2012-07-15 00:00:51,402 INFO repeater task os75nb_lastensq starting 17.4 seconds late 2012-07-15 00:00:53,004 INFO repeater task os75nb_lastensq ended after 1.6 seconds 2012-07-15 00:00:53,004 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 00:00:24 2012-07-15 00:00:53,005 INFO repeater task os75nb_quick starting 28.4 seconds late 2012-07-15 00:01:06,271 INFO repeater task os75nb_quick output: no data available 2012-07-15 00:01:06,271 INFO repeater task os75nb_quick ended after 13.3 seconds 2012-07-15 00:01:06,276 INFO repeater task os75nb_3dayplots starting 41.6 seconds late 2012-07-15 00:01:25,234 INFO repeater task os75nb_3dayplots output: could not parse 196_os75nb_ddaycont.png could not parse 196_os75nb_latcont.png could not parse 196_os75nb_loncont.png could not parse 196_os75nb_ddaycont.png could not parse 196_os75nb_latcont.png could not parse 196_os75nb_loncont.png 2012-07-15 00:01:25,234 INFO repeater task os75nb_3dayplots ended after 19.0 seconds 2012-07-15 00:01:25,241 INFO repeater task ashtech_hcorrstats starting 60.6 seconds late 2012-07-15 00:01:27,516 INFO repeater task ashtech_hcorrstats ended after 2.3 seconds 2012-07-15 00:01:27,517 INFO repeater task phins_hcorrstats starting 62.9 seconds late 2012-07-15 00:01:28,407 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 00:01:28,407 INFO repeater task phins_hcorrstats ended after 0.9 seconds 2012-07-15 00:01:33,415 INFO repeater task nb150_lastensq starting 60.4 seconds late 2012-07-15 00:01:37,366 INFO repeater task nb150_lastensq ended after 4.0 seconds 2012-07-15 00:01:37,367 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 00:00:23 2012-07-15 00:05:28,075 INFO repeater task ashtech_hcorrstats starting 3.4 seconds late 2012-07-15 00:05:29,659 INFO repeater task ashtech_hcorrstats ended after 1.6 seconds 2012-07-15 00:05:29,660 INFO repeater task phins_hcorrstats starting 5.0 seconds late 2012-07-15 00:05:31,732 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 00:05:31,733 INFO repeater task phins_hcorrstats ended after 2.1 seconds 2012-07-15 00:05:36,739 INFO repeater task nb150_lastensq starting 3.7 seconds late 2012-07-15 00:05:39,262 INFO repeater task nb150_lastensq ended after 2.5 seconds 2012-07-15 00:05:39,263 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 00:05:23 2012-07-15 00:05:39,267 INFO repeater task os75nb_lastensq starting 5.3 seconds late 2012-07-15 00:05:40,289 INFO repeater task os75nb_lastensq ended after 1.0 seconds 2012-07-15 00:05:40,290 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 00:05:24 2012-07-15 00:10:27,427 INFO repeater task ashtech_hcorrstats starting 2.8 seconds late 2012-07-15 00:10:29,508 INFO repeater task ashtech_hcorrstats ended after 2.1 seconds 2012-07-15 00:10:29,522 INFO repeater task phins_hcorrstats starting 4.9 seconds late 2012-07-15 00:10:30,572 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 00:10:30,572 INFO repeater task phins_hcorrstats ended after 1.1 seconds 2012-07-15 00:10:35,577 INFO repeater task nb150_lastensq starting 2.6 seconds late 2012-07-15 00:10:38,989 INFO repeater task nb150_lastensq ended after 3.4 seconds 2012-07-15 00:10:38,990 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 00:10:23 2012-07-15 00:10:39,012 INFO repeater task os75nb_lastensq starting 5.0 seconds late 2012-07-15 00:10:40,364 INFO repeater task os75nb_lastensq ended after 1.4 seconds 2012-07-15 00:10:40,365 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 00:10:24 2012-07-15 00:15:27,175 INFO repeater task nb150_quick starting 2.5 seconds late 2012-07-15 00:15:39,570 INFO repeater task nb150_quick output: no data available 2012-07-15 00:15:39,570 INFO repeater task nb150_quick ended after 12.4 seconds 2012-07-15 00:15:39,574 INFO repeater task nb150_3dayplots starting 14.9 seconds late 2012-07-15 00:15:56,419 INFO repeater task nb150_3dayplots output: could not parse 196_nb150_ddaycont.png could not parse 196_nb150_latcont.png could not parse 196_nb150_loncont.png could not parse 196_nb150_ddaycont.png could not parse 196_nb150_latcont.png could not parse 196_nb150_loncont.png 2012-07-15 00:15:56,419 INFO repeater task nb150_3dayplots ended after 16.8 seconds 2012-07-15 00:15:56,430 INFO repeater task os75nb_lastensq starting 22.4 seconds late 2012-07-15 00:15:57,979 INFO repeater task os75nb_lastensq ended after 1.5 seconds 2012-07-15 00:15:57,979 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 00:15:24 2012-07-15 00:15:57,981 INFO repeater task os75nb_quick starting 33.4 seconds late 2012-07-15 00:16:12,130 INFO repeater task os75nb_quick output: no data available 2012-07-15 00:16:12,131 INFO repeater task os75nb_quick ended after 14.1 seconds 2012-07-15 00:16:12,131 INFO repeater task os75nb_3dayplots starting 47.5 seconds late 2012-07-15 00:16:31,219 INFO repeater task os75nb_3dayplots ended after 19.1 seconds 2012-07-15 00:16:31,223 INFO repeater task ashtech_hcorrstats starting 66.6 seconds late 2012-07-15 00:16:32,721 INFO repeater task ashtech_hcorrstats ended after 1.5 seconds 2012-07-15 00:16:32,722 INFO repeater task phins_hcorrstats starting 68.1 seconds late 2012-07-15 00:16:33,789 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 00:16:33,789 INFO repeater task phins_hcorrstats ended after 1.1 seconds 2012-07-15 00:16:38,798 INFO repeater task nb150_lastensq starting 65.8 seconds late 2012-07-15 00:16:42,467 INFO repeater task nb150_lastensq ended after 3.7 seconds 2012-07-15 00:16:42,468 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 00:15:23 2012-07-15 00:20:28,911 INFO repeater task ashtech_hcorrstats starting 4.3 seconds late 2012-07-15 00:20:30,118 INFO repeater task ashtech_hcorrstats ended after 1.2 seconds 2012-07-15 00:20:30,120 INFO repeater task phins_hcorrstats starting 5.5 seconds late 2012-07-15 00:20:32,241 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 00:20:32,241 INFO repeater task phins_hcorrstats ended after 2.1 seconds 2012-07-15 00:20:37,242 INFO repeater task nb150_lastensq starting 4.2 seconds late 2012-07-15 00:20:39,652 INFO repeater task nb150_lastensq ended after 2.4 seconds 2012-07-15 00:20:39,652 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 00:20:23 2012-07-15 00:20:39,662 INFO repeater task os75nb_lastensq starting 5.7 seconds late 2012-07-15 00:20:42,333 INFO repeater task os75nb_lastensq ended after 2.7 seconds 2012-07-15 00:20:42,334 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 00:20:24 2012-07-15 00:25:29,273 INFO repeater task ashtech_hcorrstats starting 4.6 seconds late 2012-07-15 00:25:30,528 INFO repeater task ashtech_hcorrstats ended after 1.3 seconds 2012-07-15 00:25:30,546 INFO repeater task phins_hcorrstats starting 5.9 seconds late 2012-07-15 00:25:32,477 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 00:25:32,478 INFO repeater task phins_hcorrstats ended after 1.9 seconds 2012-07-15 00:25:37,485 INFO repeater task nb150_lastensq starting 4.5 seconds late 2012-07-15 00:25:40,998 INFO repeater task nb150_lastensq ended after 3.5 seconds 2012-07-15 00:25:40,998 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 00:25:23 2012-07-15 00:25:41,022 INFO repeater task os75nb_lastensq starting 7.0 seconds late 2012-07-15 00:25:42,857 INFO repeater task os75nb_lastensq ended after 1.8 seconds 2012-07-15 00:25:42,857 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 00:25:24 2012-07-15 00:30:24,766 INFO repeater task nb150_quick starting 0.1 seconds late 2012-07-15 00:30:40,768 INFO repeater task nb150_quick output: no data available 2012-07-15 00:30:40,769 INFO repeater task nb150_quick ended after 16.0 seconds 2012-07-15 00:30:40,773 INFO repeater task nb150_3dayplots starting 16.1 seconds late 2012-07-15 00:30:58,331 INFO repeater task nb150_3dayplots ended after 17.6 seconds 2012-07-15 00:30:58,332 INFO repeater task os75nb_lastensq starting 24.3 seconds late 2012-07-15 00:30:59,652 INFO repeater task os75nb_lastensq ended after 1.3 seconds 2012-07-15 00:30:59,653 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 00:30:24 2012-07-15 00:30:59,653 INFO repeater task os75nb_quick starting 35.0 seconds late 2012-07-15 00:31:14,878 INFO repeater task os75nb_quick output: no data available 2012-07-15 00:31:14,878 INFO repeater task os75nb_quick ended after 15.2 seconds 2012-07-15 00:31:14,878 INFO repeater task os75nb_3dayplots starting 50.2 seconds late 2012-07-15 00:31:34,954 INFO repeater task os75nb_3dayplots ended after 20.1 seconds 2012-07-15 00:31:34,954 INFO repeater task ashtech_hcorrstats starting 70.3 seconds late 2012-07-15 00:31:35,940 INFO repeater task ashtech_hcorrstats ended after 1.0 seconds 2012-07-15 00:31:35,941 INFO repeater task phins_hcorrstats starting 71.3 seconds late 2012-07-15 00:31:38,260 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 00:31:38,260 INFO repeater task phins_hcorrstats ended after 2.3 seconds 2012-07-15 00:31:43,267 INFO repeater task nb150_lastensq starting 70.3 seconds late 2012-07-15 00:31:47,707 INFO repeater task nb150_lastensq ended after 4.4 seconds 2012-07-15 00:31:47,708 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 00:30:23 2012-07-15 00:35:29,265 INFO repeater task ashtech_hcorrstats starting 4.6 seconds late 2012-07-15 00:35:31,044 INFO repeater task ashtech_hcorrstats ended after 1.8 seconds 2012-07-15 00:35:31,044 INFO repeater task phins_hcorrstats starting 6.4 seconds late 2012-07-15 00:35:32,381 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 00:35:32,382 INFO repeater task phins_hcorrstats ended after 1.3 seconds 2012-07-15 00:35:37,388 INFO repeater task nb150_lastensq starting 4.4 seconds late 2012-07-15 00:35:42,410 INFO repeater task nb150_lastensq ended after 5.0 seconds 2012-07-15 00:35:42,411 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 00:35:23 2012-07-15 00:35:42,414 INFO repeater task os75nb_lastensq starting 8.4 seconds late 2012-07-15 00:35:44,066 INFO repeater task os75nb_lastensq ended after 1.7 seconds 2012-07-15 00:35:44,067 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 00:35:24 2012-07-15 00:40:24,981 INFO repeater task ashtech_hcorrstats starting 0.3 seconds late 2012-07-15 00:40:27,392 INFO repeater task ashtech_hcorrstats ended after 2.4 seconds 2012-07-15 00:40:27,394 INFO repeater task phins_hcorrstats starting 2.8 seconds late 2012-07-15 00:40:28,288 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 00:40:28,288 INFO repeater task phins_hcorrstats ended after 0.9 seconds 2012-07-15 00:40:33,295 INFO repeater task nb150_lastensq starting 0.3 seconds late 2012-07-15 00:40:39,122 INFO repeater task nb150_lastensq ended after 5.8 seconds 2012-07-15 00:40:39,123 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 00:40:23 2012-07-15 00:40:39,123 INFO repeater task os75nb_lastensq starting 5.1 seconds late 2012-07-15 00:40:40,811 INFO repeater task os75nb_lastensq ended after 1.7 seconds 2012-07-15 00:40:40,812 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 00:40:24 2012-07-15 00:45:27,605 INFO repeater task nb150_quick starting 3.0 seconds late 2012-07-15 00:45:40,523 INFO repeater task nb150_quick output: no data available 2012-07-15 00:45:40,523 INFO repeater task nb150_quick ended after 12.9 seconds 2012-07-15 00:45:40,528 INFO repeater task nb150_3dayplots starting 15.9 seconds late 2012-07-15 00:45:56,315 INFO repeater task nb150_3dayplots ended after 15.8 seconds 2012-07-15 00:45:56,316 INFO repeater task os75nb_lastensq starting 22.3 seconds late 2012-07-15 00:45:57,535 INFO repeater task os75nb_lastensq ended after 1.2 seconds 2012-07-15 00:45:57,535 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 00:45:24 2012-07-15 00:45:57,536 INFO repeater task os75nb_quick starting 32.9 seconds late 2012-07-15 00:46:06,623 INFO repeater task os75nb_quick output: no data available 2012-07-15 00:46:06,624 INFO repeater task os75nb_quick ended after 9.1 seconds 2012-07-15 00:46:06,624 INFO repeater task os75nb_3dayplots starting 42.0 seconds late 2012-07-15 00:46:22,343 INFO repeater task os75nb_3dayplots ended after 15.7 seconds 2012-07-15 00:46:22,344 INFO repeater task ashtech_hcorrstats starting 57.7 seconds late 2012-07-15 00:46:23,470 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2012-07-15 00:46:23,471 INFO repeater task phins_hcorrstats starting 58.8 seconds late 2012-07-15 00:46:25,430 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 00:46:25,431 INFO repeater task phins_hcorrstats ended after 2.0 seconds 2012-07-15 00:46:30,433 INFO repeater task nb150_lastensq starting 57.4 seconds late 2012-07-15 00:46:35,409 INFO repeater task nb150_lastensq ended after 5.0 seconds 2012-07-15 00:46:35,410 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 00:45:23 2012-07-15 00:50:25,989 INFO repeater task ashtech_hcorrstats starting 1.4 seconds late 2012-07-15 00:50:26,975 INFO repeater task ashtech_hcorrstats ended after 1.0 seconds 2012-07-15 00:50:26,975 INFO repeater task phins_hcorrstats starting 2.3 seconds late 2012-07-15 00:50:28,256 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 00:50:28,256 INFO repeater task phins_hcorrstats ended after 1.3 seconds 2012-07-15 00:50:33,260 INFO repeater task nb150_lastensq starting 0.3 seconds late 2012-07-15 00:50:39,277 INFO repeater task nb150_lastensq ended after 6.0 seconds 2012-07-15 00:50:39,277 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 00:50:23 2012-07-15 00:50:39,294 INFO repeater task os75nb_lastensq starting 5.3 seconds late 2012-07-15 00:50:41,381 INFO repeater task os75nb_lastensq ended after 2.1 seconds 2012-07-15 00:50:41,381 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 00:50:24 2012-07-15 00:55:28,848 INFO repeater task ashtech_hcorrstats starting 4.2 seconds late 2012-07-15 00:55:29,802 INFO repeater task ashtech_hcorrstats ended after 1.0 seconds 2012-07-15 00:55:29,810 INFO repeater task phins_hcorrstats starting 5.2 seconds late 2012-07-15 00:55:32,176 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 00:55:32,176 INFO repeater task phins_hcorrstats ended after 2.4 seconds 2012-07-15 00:55:37,181 INFO repeater task nb150_lastensq starting 4.2 seconds late 2012-07-15 00:55:40,850 INFO repeater task nb150_lastensq ended after 3.7 seconds 2012-07-15 00:55:40,850 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 00:55:23 2012-07-15 00:55:40,865 INFO repeater task os75nb_lastensq starting 6.9 seconds late 2012-07-15 00:55:44,445 INFO repeater task os75nb_lastensq ended after 3.6 seconds 2012-07-15 00:55:44,446 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 00:55:24 2012-07-15 01:00:25,731 INFO repeater task nb150_quick starting 1.1 seconds late 2012-07-15 01:00:39,834 INFO repeater task nb150_quick output: no data available 2012-07-15 01:00:39,834 INFO repeater task nb150_quick ended after 14.1 seconds 2012-07-15 01:00:39,839 INFO repeater task nb150_3dayplots starting 15.2 seconds late 2012-07-15 01:01:04,848 INFO repeater task nb150_3dayplots ended after 25.0 seconds 2012-07-15 01:01:04,852 INFO repeater task os75nb_lastensq starting 30.9 seconds late 2012-07-15 01:01:06,432 INFO repeater task os75nb_lastensq ended after 1.6 seconds 2012-07-15 01:01:06,433 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 01:00:24 2012-07-15 01:01:06,437 INFO repeater task os75nb_quick starting 41.8 seconds late 2012-07-15 01:01:20,544 INFO repeater task os75nb_quick output: no data available 2012-07-15 01:01:20,545 INFO repeater task os75nb_quick ended after 14.1 seconds 2012-07-15 01:01:20,548 INFO repeater task os75nb_3dayplots starting 55.9 seconds late 2012-07-15 01:01:43,692 INFO repeater task os75nb_3dayplots ended after 23.1 seconds 2012-07-15 01:01:43,697 INFO repeater task ashtech_hcorrstats starting 79.1 seconds late 2012-07-15 01:01:46,049 INFO repeater task ashtech_hcorrstats ended after 2.4 seconds 2012-07-15 01:01:46,049 INFO repeater task phins_hcorrstats starting 81.4 seconds late 2012-07-15 01:01:48,041 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 01:01:48,041 INFO repeater task phins_hcorrstats ended after 2.0 seconds 2012-07-15 01:01:53,050 INFO repeater task nb150_lastensq starting 80.1 seconds late 2012-07-15 01:01:55,348 INFO repeater task nb150_lastensq ended after 2.3 seconds 2012-07-15 01:01:55,348 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 01:00:23 2012-07-15 01:05:25,947 INFO repeater task ashtech_hcorrstats starting 1.3 seconds late 2012-07-15 01:05:27,971 INFO repeater task ashtech_hcorrstats ended after 2.0 seconds 2012-07-15 01:05:27,979 INFO repeater task phins_hcorrstats starting 3.3 seconds late 2012-07-15 01:05:29,011 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 01:05:29,011 INFO repeater task phins_hcorrstats ended after 1.0 seconds 2012-07-15 01:05:34,014 INFO repeater task nb150_lastensq starting 1.0 seconds late 2012-07-15 01:05:37,238 INFO repeater task nb150_lastensq ended after 3.2 seconds 2012-07-15 01:05:37,238 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 01:05:23 2012-07-15 01:05:37,268 INFO repeater task os75nb_lastensq starting 3.3 seconds late 2012-07-15 01:05:38,957 INFO repeater task os75nb_lastensq ended after 1.7 seconds 2012-07-15 01:05:38,958 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 01:05:24 2012-07-15 01:10:25,721 INFO repeater task ashtech_hcorrstats starting 1.1 seconds late 2012-07-15 01:10:27,532 INFO repeater task ashtech_hcorrstats ended after 1.8 seconds 2012-07-15 01:10:27,536 INFO repeater task phins_hcorrstats starting 2.9 seconds late 2012-07-15 01:10:29,281 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 01:10:29,281 INFO repeater task phins_hcorrstats ended after 1.7 seconds 2012-07-15 01:10:34,289 INFO repeater task nb150_lastensq starting 1.3 seconds late 2012-07-15 01:10:39,676 INFO repeater task nb150_lastensq ended after 5.4 seconds 2012-07-15 01:10:39,676 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 01:10:23 2012-07-15 01:10:39,693 INFO repeater task os75nb_lastensq starting 5.7 seconds late 2012-07-15 01:10:40,992 INFO repeater task os75nb_lastensq ended after 1.3 seconds 2012-07-15 01:10:40,992 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 01:10:24 2012-07-15 01:15:28,425 INFO repeater task nb150_quick starting 3.8 seconds late 2012-07-15 01:15:39,118 INFO repeater task nb150_quick output: no data available 2012-07-15 01:15:39,119 INFO repeater task nb150_quick ended after 10.7 seconds 2012-07-15 01:15:39,119 INFO repeater task nb150_3dayplots starting 14.5 seconds late 2012-07-15 01:15:55,535 INFO repeater task nb150_3dayplots ended after 16.4 seconds 2012-07-15 01:15:55,536 INFO repeater task os75nb_lastensq starting 21.5 seconds late 2012-07-15 01:15:57,219 INFO repeater task os75nb_lastensq ended after 1.7 seconds 2012-07-15 01:15:57,220 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 01:15:24 2012-07-15 01:15:57,221 INFO repeater task os75nb_quick starting 32.6 seconds late 2012-07-15 01:16:07,129 INFO repeater task os75nb_quick output: no data available 2012-07-15 01:16:07,129 INFO repeater task os75nb_quick ended after 9.9 seconds 2012-07-15 01:16:07,130 INFO repeater task os75nb_3dayplots starting 42.5 seconds late 2012-07-15 01:16:20,427 INFO repeater task os75nb_3dayplots ended after 13.3 seconds 2012-07-15 01:16:20,427 INFO repeater task ashtech_hcorrstats starting 55.8 seconds late 2012-07-15 01:16:21,410 INFO repeater task ashtech_hcorrstats ended after 1.0 seconds 2012-07-15 01:16:21,414 INFO repeater task phins_hcorrstats starting 56.8 seconds late 2012-07-15 01:16:22,726 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 01:16:22,727 INFO repeater task phins_hcorrstats ended after 1.3 seconds 2012-07-15 01:16:27,728 INFO repeater task nb150_lastensq starting 54.7 seconds late 2012-07-15 01:16:29,944 INFO repeater task nb150_lastensq ended after 2.2 seconds 2012-07-15 01:16:29,945 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 01:15:23 2012-07-15 01:20:25,684 INFO repeater task ashtech_hcorrstats starting 1.1 seconds late 2012-07-15 01:20:27,259 INFO repeater task ashtech_hcorrstats ended after 1.6 seconds 2012-07-15 01:20:27,260 INFO repeater task phins_hcorrstats starting 2.6 seconds late 2012-07-15 01:20:28,666 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 01:20:28,666 INFO repeater task phins_hcorrstats ended after 1.4 seconds 2012-07-15 01:20:33,667 INFO repeater task nb150_lastensq starting 0.7 seconds late 2012-07-15 01:20:36,214 INFO repeater task nb150_lastensq ended after 2.5 seconds 2012-07-15 01:20:36,214 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 01:20:23 2012-07-15 01:20:36,248 INFO repeater task os75nb_lastensq starting 2.2 seconds late 2012-07-15 01:20:38,459 INFO repeater task os75nb_lastensq ended after 2.2 seconds 2012-07-15 01:20:38,459 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 01:20:24 2012-07-15 01:25:29,636 INFO repeater task ashtech_hcorrstats starting 5.0 seconds late 2012-07-15 01:25:32,391 INFO repeater task ashtech_hcorrstats ended after 2.8 seconds 2012-07-15 01:25:32,392 INFO repeater task phins_hcorrstats starting 7.8 seconds late 2012-07-15 01:25:33,446 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 01:25:33,446 INFO repeater task phins_hcorrstats ended after 1.1 seconds 2012-07-15 01:25:38,446 INFO repeater task nb150_lastensq starting 5.4 seconds late 2012-07-15 01:25:41,540 INFO repeater task nb150_lastensq ended after 3.1 seconds 2012-07-15 01:25:41,541 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 01:25:23 2012-07-15 01:25:41,559 INFO repeater task os75nb_lastensq starting 7.6 seconds late 2012-07-15 01:25:43,039 INFO repeater task os75nb_lastensq ended after 1.5 seconds 2012-07-15 01:25:43,039 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 01:25:24 2012-07-15 01:30:29,033 INFO repeater task nb150_quick starting 4.4 seconds late 2012-07-15 01:30:42,370 INFO repeater task nb150_quick output: no data available 2012-07-15 01:30:42,370 INFO repeater task nb150_quick ended after 13.3 seconds 2012-07-15 01:30:42,371 INFO repeater task nb150_3dayplots starting 17.7 seconds late 2012-07-15 01:31:00,701 INFO repeater task nb150_3dayplots ended after 18.3 seconds 2012-07-15 01:31:00,702 INFO repeater task os75nb_lastensq starting 26.7 seconds late 2012-07-15 01:31:02,618 INFO repeater task os75nb_lastensq ended after 1.9 seconds 2012-07-15 01:31:02,618 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 01:30:24 2012-07-15 01:31:02,619 INFO repeater task os75nb_quick starting 38.0 seconds late 2012-07-15 01:31:11,976 INFO repeater task os75nb_quick output: no data available 2012-07-15 01:31:11,977 INFO repeater task os75nb_quick ended after 9.4 seconds 2012-07-15 01:31:11,981 INFO repeater task os75nb_3dayplots starting 47.4 seconds late 2012-07-15 01:31:29,031 INFO repeater task os75nb_3dayplots ended after 17.0 seconds 2012-07-15 01:31:29,032 INFO repeater task ashtech_hcorrstats starting 64.4 seconds late 2012-07-15 01:31:29,977 INFO repeater task ashtech_hcorrstats ended after 0.9 seconds 2012-07-15 01:31:29,978 INFO repeater task phins_hcorrstats starting 65.3 seconds late 2012-07-15 01:31:30,979 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 01:31:30,980 INFO repeater task phins_hcorrstats ended after 1.0 seconds 2012-07-15 01:31:35,983 INFO repeater task nb150_lastensq starting 63.0 seconds late 2012-07-15 01:31:38,734 INFO repeater task nb150_lastensq ended after 2.8 seconds 2012-07-15 01:31:38,734 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 01:30:23 2012-07-15 01:35:25,328 INFO repeater task ashtech_hcorrstats starting 0.7 seconds late 2012-07-15 01:35:26,564 INFO repeater task ashtech_hcorrstats ended after 1.2 seconds 2012-07-15 01:35:26,567 INFO repeater task phins_hcorrstats starting 1.9 seconds late 2012-07-15 01:35:28,410 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 01:35:28,410 INFO repeater task phins_hcorrstats ended after 1.8 seconds 2012-07-15 01:35:33,413 INFO repeater task nb150_lastensq starting 0.4 seconds late 2012-07-15 01:35:37,212 INFO repeater task nb150_lastensq ended after 3.8 seconds 2012-07-15 01:35:37,212 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 01:35:23 2012-07-15 01:35:37,234 INFO repeater task os75nb_lastensq starting 3.2 seconds late 2012-07-15 01:35:38,962 INFO repeater task os75nb_lastensq ended after 1.7 seconds 2012-07-15 01:35:38,963 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 01:35:24 2012-07-15 01:40:25,608 INFO repeater task ashtech_hcorrstats starting 1.0 seconds late 2012-07-15 01:40:26,600 INFO repeater task ashtech_hcorrstats ended after 1.0 seconds 2012-07-15 01:40:26,605 INFO repeater task phins_hcorrstats starting 2.0 seconds late 2012-07-15 01:40:27,849 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 01:40:27,849 INFO repeater task phins_hcorrstats ended after 1.2 seconds 2012-07-15 01:40:37,864 INFO repeater task nb150_lastensq starting 4.9 seconds late 2012-07-15 01:40:41,032 INFO repeater task nb150_lastensq ended after 3.2 seconds 2012-07-15 01:40:41,032 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 01:40:23 2012-07-15 01:40:41,034 INFO repeater task os75nb_lastensq starting 7.0 seconds late 2012-07-15 01:40:43,623 INFO repeater task os75nb_lastensq ended after 2.6 seconds 2012-07-15 01:40:43,624 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 01:40:24 2012-07-15 01:45:25,334 INFO repeater task nb150_quick starting 0.7 seconds late 2012-07-15 01:45:35,134 INFO repeater task nb150_quick output: no data available 2012-07-15 01:45:35,134 INFO repeater task nb150_quick ended after 9.8 seconds 2012-07-15 01:45:35,142 INFO repeater task nb150_3dayplots starting 10.5 seconds late 2012-07-15 01:45:50,486 INFO repeater task nb150_3dayplots ended after 15.3 seconds 2012-07-15 01:45:50,486 INFO repeater task os75nb_lastensq starting 16.5 seconds late 2012-07-15 01:45:51,648 INFO repeater task os75nb_lastensq ended after 1.2 seconds 2012-07-15 01:45:51,648 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 01:45:24 2012-07-15 01:45:51,652 INFO repeater task os75nb_quick starting 27.0 seconds late 2012-07-15 01:46:02,964 INFO repeater task os75nb_quick output: no data available 2012-07-15 01:46:02,965 INFO repeater task os75nb_quick ended after 11.3 seconds 2012-07-15 01:46:02,965 INFO repeater task os75nb_3dayplots starting 38.3 seconds late 2012-07-15 01:46:18,826 INFO repeater task os75nb_3dayplots ended after 15.9 seconds 2012-07-15 01:46:18,826 INFO repeater task ashtech_hcorrstats starting 54.2 seconds late 2012-07-15 01:46:20,878 INFO repeater task ashtech_hcorrstats ended after 2.1 seconds 2012-07-15 01:46:20,878 INFO repeater task phins_hcorrstats starting 56.2 seconds late 2012-07-15 01:46:21,891 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 01:46:21,891 INFO repeater task phins_hcorrstats ended after 1.0 seconds 2012-07-15 01:46:26,897 INFO repeater task nb150_lastensq starting 53.9 seconds late 2012-07-15 01:46:30,180 INFO repeater task nb150_lastensq ended after 3.3 seconds 2012-07-15 01:46:30,180 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 01:45:23 2012-07-15 01:50:26,159 INFO repeater task ashtech_hcorrstats starting 1.5 seconds late 2012-07-15 01:50:28,032 INFO repeater task ashtech_hcorrstats ended after 1.9 seconds 2012-07-15 01:50:28,033 INFO repeater task phins_hcorrstats starting 3.4 seconds late 2012-07-15 01:50:29,046 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 01:50:29,047 INFO repeater task phins_hcorrstats ended after 1.0 seconds 2012-07-15 01:50:34,055 INFO repeater task nb150_lastensq starting 1.1 seconds late 2012-07-15 01:50:38,276 INFO repeater task nb150_lastensq ended after 4.2 seconds 2012-07-15 01:50:38,277 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 01:50:23 2012-07-15 01:50:38,278 INFO repeater task os75nb_lastensq starting 4.3 seconds late 2012-07-15 01:50:40,162 INFO repeater task os75nb_lastensq ended after 1.9 seconds 2012-07-15 01:50:40,162 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 01:50:24 2012-07-15 01:55:27,486 INFO repeater task ashtech_hcorrstats starting 2.9 seconds late 2012-07-15 01:55:28,442 INFO repeater task ashtech_hcorrstats ended after 1.0 seconds 2012-07-15 01:55:28,442 INFO repeater task phins_hcorrstats starting 3.8 seconds late 2012-07-15 01:55:29,918 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 01:55:29,918 INFO repeater task phins_hcorrstats ended after 1.5 seconds 2012-07-15 01:55:34,928 INFO repeater task nb150_lastensq starting 1.9 seconds late 2012-07-15 01:55:38,677 INFO repeater task nb150_lastensq ended after 3.7 seconds 2012-07-15 01:55:38,677 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 01:55:23 2012-07-15 01:55:38,678 INFO repeater task os75nb_lastensq starting 4.7 seconds late 2012-07-15 01:55:39,737 INFO repeater task os75nb_lastensq ended after 1.1 seconds 2012-07-15 01:55:39,737 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 01:55:24 2012-07-15 02:00:25,500 INFO repeater task nb150_quick starting 0.9 seconds late 2012-07-15 02:00:36,614 INFO repeater task nb150_quick output: no data available 2012-07-15 02:00:36,614 INFO repeater task nb150_quick ended after 11.1 seconds 2012-07-15 02:00:36,618 INFO repeater task nb150_3dayplots starting 12.0 seconds late 2012-07-15 02:00:57,226 INFO repeater task nb150_3dayplots ended after 20.6 seconds 2012-07-15 02:00:57,227 INFO repeater task os75nb_lastensq starting 23.2 seconds late 2012-07-15 02:00:59,284 INFO repeater task os75nb_lastensq ended after 2.1 seconds 2012-07-15 02:00:59,284 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 02:00:24 2012-07-15 02:00:59,292 INFO repeater task os75nb_quick starting 34.7 seconds late 2012-07-15 02:01:12,158 INFO repeater task os75nb_quick output: no data available 2012-07-15 02:01:12,158 INFO repeater task os75nb_quick ended after 12.9 seconds 2012-07-15 02:01:12,173 INFO repeater task os75nb_3dayplots starting 47.5 seconds late 2012-07-15 02:01:30,607 INFO repeater task os75nb_3dayplots ended after 18.4 seconds 2012-07-15 02:01:30,608 INFO repeater task ashtech_hcorrstats starting 66.0 seconds late 2012-07-15 02:01:32,079 INFO repeater task ashtech_hcorrstats ended after 1.5 seconds 2012-07-15 02:01:32,084 INFO repeater task phins_hcorrstats starting 67.5 seconds late 2012-07-15 02:01:33,598 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 02:01:33,599 INFO repeater task phins_hcorrstats ended after 1.5 seconds 2012-07-15 02:01:38,601 INFO repeater task nb150_lastensq starting 65.6 seconds late 2012-07-15 02:01:41,913 INFO repeater task nb150_lastensq ended after 3.3 seconds 2012-07-15 02:01:41,913 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 02:00:23 2012-07-15 02:05:28,796 INFO repeater task ashtech_hcorrstats starting 4.2 seconds late 2012-07-15 02:05:30,373 INFO repeater task ashtech_hcorrstats ended after 1.6 seconds 2012-07-15 02:05:30,374 INFO repeater task phins_hcorrstats starting 5.7 seconds late 2012-07-15 02:05:31,416 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 02:05:31,416 INFO repeater task phins_hcorrstats ended after 1.0 seconds 2012-07-15 02:05:36,421 INFO repeater task nb150_lastensq starting 3.4 seconds late 2012-07-15 02:05:39,251 INFO repeater task nb150_lastensq ended after 2.8 seconds 2012-07-15 02:05:39,251 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 02:05:23 2012-07-15 02:05:39,271 INFO repeater task os75nb_lastensq starting 5.3 seconds late 2012-07-15 02:05:40,522 INFO repeater task os75nb_lastensq ended after 1.3 seconds 2012-07-15 02:05:40,522 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 02:05:24 2012-07-15 02:10:26,912 INFO repeater task ashtech_hcorrstats starting 2.3 seconds late 2012-07-15 02:10:28,195 INFO repeater task ashtech_hcorrstats ended after 1.3 seconds 2012-07-15 02:10:28,197 INFO repeater task phins_hcorrstats starting 3.6 seconds late 2012-07-15 02:10:29,639 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 02:10:29,640 INFO repeater task phins_hcorrstats ended after 1.4 seconds 2012-07-15 02:10:34,647 INFO repeater task nb150_lastensq starting 1.6 seconds late 2012-07-15 02:10:37,102 INFO repeater task nb150_lastensq ended after 2.5 seconds 2012-07-15 02:10:37,103 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 02:10:23 2012-07-15 02:10:37,107 INFO repeater task os75nb_lastensq starting 3.1 seconds late 2012-07-15 02:10:38,139 INFO repeater task os75nb_lastensq ended after 1.0 seconds 2012-07-15 02:10:38,140 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 02:10:24 2012-07-15 02:15:29,090 INFO repeater task nb150_quick starting 4.5 seconds late 2012-07-15 02:15:40,037 INFO repeater task nb150_quick output: no data available 2012-07-15 02:15:40,037 INFO repeater task nb150_quick ended after 10.9 seconds 2012-07-15 02:15:40,042 INFO repeater task nb150_3dayplots starting 15.4 seconds late 2012-07-15 02:15:56,676 INFO repeater task nb150_3dayplots ended after 16.6 seconds 2012-07-15 02:15:56,681 INFO repeater task os75nb_lastensq starting 22.7 seconds late 2012-07-15 02:15:59,429 INFO repeater task os75nb_lastensq ended after 2.7 seconds 2012-07-15 02:15:59,429 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 02:15:24 2012-07-15 02:15:59,431 INFO repeater task os75nb_quick starting 34.8 seconds late 2012-07-15 02:16:09,299 INFO repeater task os75nb_quick output: no data available 2012-07-15 02:16:09,299 INFO repeater task os75nb_quick ended after 9.9 seconds 2012-07-15 02:16:09,304 INFO repeater task os75nb_3dayplots starting 44.7 seconds late 2012-07-15 02:16:28,389 INFO repeater task os75nb_3dayplots ended after 19.1 seconds 2012-07-15 02:16:28,390 INFO repeater task ashtech_hcorrstats starting 63.8 seconds late 2012-07-15 02:16:29,448 INFO repeater task ashtech_hcorrstats ended after 1.1 seconds 2012-07-15 02:16:29,449 INFO repeater task phins_hcorrstats starting 64.8 seconds late 2012-07-15 02:16:30,337 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 02:16:30,337 INFO repeater task phins_hcorrstats ended after 0.9 seconds 2012-07-15 02:16:35,341 INFO repeater task nb150_lastensq starting 62.3 seconds late 2012-07-15 02:16:38,158 INFO repeater task nb150_lastensq ended after 2.8 seconds 2012-07-15 02:16:38,158 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 02:15:23 2012-07-15 02:20:24,716 INFO repeater task ashtech_hcorrstats starting 0.1 seconds late 2012-07-15 02:20:27,624 INFO repeater task ashtech_hcorrstats ended after 2.9 seconds 2012-07-15 02:20:27,629 INFO repeater task phins_hcorrstats starting 3.0 seconds late 2012-07-15 02:20:29,694 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 02:20:29,694 INFO repeater task phins_hcorrstats ended after 2.1 seconds 2012-07-15 02:20:34,700 INFO repeater task nb150_lastensq starting 1.7 seconds late 2012-07-15 02:20:39,540 INFO repeater task nb150_lastensq ended after 4.8 seconds 2012-07-15 02:20:39,541 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 02:20:23 2012-07-15 02:20:39,542 INFO repeater task os75nb_lastensq starting 5.5 seconds late 2012-07-15 02:20:41,578 INFO repeater task os75nb_lastensq ended after 2.0 seconds 2012-07-15 02:20:41,579 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 02:20:24 2012-07-15 02:25:27,821 INFO repeater task ashtech_hcorrstats starting 3.2 seconds late 2012-07-15 02:25:30,118 INFO repeater task ashtech_hcorrstats ended after 2.3 seconds 2012-07-15 02:25:30,119 INFO repeater task phins_hcorrstats starting 5.5 seconds late 2012-07-15 02:25:30,978 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 02:25:30,978 INFO repeater task phins_hcorrstats ended after 0.9 seconds 2012-07-15 02:25:35,989 INFO repeater task nb150_lastensq starting 3.0 seconds late 2012-07-15 02:25:40,190 INFO repeater task nb150_lastensq ended after 4.2 seconds 2012-07-15 02:25:40,191 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 02:25:23 2012-07-15 02:25:40,194 INFO repeater task os75nb_lastensq starting 6.2 seconds late 2012-07-15 02:25:41,874 INFO repeater task os75nb_lastensq ended after 1.7 seconds 2012-07-15 02:25:41,874 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 02:25:24 2012-07-15 02:30:27,962 INFO repeater task nb150_quick starting 3.3 seconds late 2012-07-15 02:30:37,720 INFO repeater task nb150_quick output: no data available 2012-07-15 02:30:37,720 INFO repeater task nb150_quick ended after 9.8 seconds 2012-07-15 02:30:37,733 INFO repeater task nb150_3dayplots starting 13.1 seconds late 2012-07-15 02:30:50,675 INFO repeater task nb150_3dayplots ended after 12.9 seconds 2012-07-15 02:30:50,681 INFO repeater task os75nb_lastensq starting 16.7 seconds late 2012-07-15 02:30:52,002 INFO repeater task os75nb_lastensq ended after 1.3 seconds 2012-07-15 02:30:52,002 INFO repeater task os75nb_lastensq: Time from /home/data/RR1208/proc/os75nb/load/wrote_ens_time: 2012/07/15 02:30:24 2012-07-15 02:30:52,006 INFO repeater task os75nb_quick starting 27.4 seconds late 2012-07-15 02:31:06,288 INFO repeater task os75nb_quick output: no data available 2012-07-15 02:31:06,289 INFO repeater task os75nb_quick ended after 14.3 seconds 2012-07-15 02:31:06,291 INFO repeater task os75nb_3dayplots starting 41.7 seconds late 2012-07-15 02:31:25,086 INFO repeater task os75nb_3dayplots ended after 18.8 seconds 2012-07-15 02:31:25,087 INFO repeater task ashtech_hcorrstats starting 60.5 seconds late 2012-07-15 02:31:26,073 INFO repeater task ashtech_hcorrstats ended after 1.0 seconds 2012-07-15 02:31:26,081 INFO repeater task phins_hcorrstats starting 61.4 seconds late 2012-07-15 02:31:27,181 INFO repeater task phins_hcorrstats output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") could not get phins stats Traceback (most recent call last): File "/home/currents/programs/uhdas/scripts/run_hbinhcorrstats.py", line 148, in np.round(winsecs))] NameError: name 'winsecs' is not defined 2012-07-15 02:31:27,181 INFO repeater task phins_hcorrstats ended after 1.1 seconds 2012-07-15 02:31:32,188 INFO repeater task nb150_lastensq starting 59.2 seconds late 2012-07-15 02:31:36,141 INFO repeater task nb150_lastensq ended after 4.0 seconds 2012-07-15 02:31:36,141 INFO repeater task nb150_lastensq: Time from /home/data/RR1208/proc/nb150/load/wrote_ens_time: 2012/07/15 02:30:23 2012-07-15 02:35:28,816 INFO repeater task ashtech_hcorrstats starting 4.2 seconds late