2010-11-04 03:45:16,679 INFO repeater task ashpaq2_hcorrstats starting 4.4 seconds late 2010-11-04 03:45:22,000 INFO repeater task ashpaq2_hcorrstats ended after 5.3 seconds 2010-11-04 03:45:22,001 INFO repeater task ashpaq5_hcorrstats starting 9.7 seconds late 2010-11-04 03:45:27,296 INFO repeater task ashpaq5_hcorrstats ended after 5.3 seconds 2010-11-04 03:45:52,319 INFO repeater task os75bb_lastensq starting 0.3 seconds late 2010-11-04 03:46:04,064 INFO repeater task os75bb_lastensq ended after 11.7 seconds 2010-11-04 03:46:04,065 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 03:45:45 2010-11-04 03:50:14,322 INFO repeater task ashpaq2_hcorrstats starting 2.1 seconds late 2010-11-04 03:50:19,709 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 03:50:19,709 INFO repeater task ashpaq5_hcorrstats starting 7.4 seconds late 2010-11-04 03:50:25,060 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 03:50:55,095 INFO repeater task os75bb_lastensq starting 0.1 seconds late 2010-11-04 03:51:07,045 INFO repeater task os75bb_lastensq ended after 11.9 seconds 2010-11-04 03:51:07,045 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 03:50:48 2010-11-04 03:55:12,293 INFO repeater task os75bb_quick starting 0.0 seconds late 2010-11-04 03:55:21,858 INFO repeater task os75bb_quick ended after 9.6 seconds 2010-11-04 03:55:21,859 INFO repeater task os75bb_3dayplots starting 9.6 seconds late 2010-11-04 03:55:26,481 INFO repeater task os75bb_3dayplots ended after 4.6 seconds 2010-11-04 03:55:26,482 INFO repeater task os75bb_beamplots starting 14.2 seconds late 2010-11-04 03:55:27,738 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 03:55:27,738 INFO repeater task os75bb_beamhistory starting 15.5 seconds late 2010-11-04 03:55:50,623 INFO repeater task os75bb_beamhistory output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") saving file to os75bb_bin_history.png 2010-11-04 03:55:50,623 INFO repeater task os75bb_beamhistory ended after 22.9 seconds 2010-11-04 03:55:50,623 INFO repeater task ashpaq2_hcorrstats starting 38.4 seconds late 2010-11-04 03:55:55,921 INFO repeater task ashpaq2_hcorrstats ended after 5.3 seconds 2010-11-04 03:55:55,922 INFO repeater task ashpaq5_hcorrstats starting 43.7 seconds late 2010-11-04 03:56:01,176 INFO repeater task ashpaq5_hcorrstats ended after 5.3 seconds 2010-11-04 03:56:06,180 INFO repeater task os75bb_lastensq starting 8.2 seconds late 2010-11-04 03:56:18,068 INFO repeater task os75bb_lastensq ended after 11.9 seconds 2010-11-04 03:56:18,068 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 03:55:51 2010-11-04 04:00:13,320 INFO repeater task ashpaq2_hcorrstats starting 1.1 seconds late 2010-11-04 04:00:18,717 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 04:00:18,718 INFO repeater task ashpaq5_hcorrstats starting 6.5 seconds late 2010-11-04 04:00:24,157 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 04:01:04,195 INFO repeater task os75bb_lastensq starting 3.2 seconds late 2010-11-04 04:01:17,403 INFO repeater task os75bb_lastensq ended after 13.2 seconds 2010-11-04 04:01:17,403 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 04:00:55 2010-11-04 04:05:12,600 INFO repeater task os75bb_beamplots starting 0.3 seconds late 2010-11-04 04:05:14,024 INFO repeater task os75bb_beamplots ended after 1.4 seconds 2010-11-04 04:05:14,025 INFO repeater task ashpaq2_hcorrstats starting 1.8 seconds late 2010-11-04 04:05:19,325 INFO repeater task ashpaq2_hcorrstats ended after 5.3 seconds 2010-11-04 04:05:19,325 INFO repeater task ashpaq5_hcorrstats starting 7.1 seconds late 2010-11-04 04:05:24,560 INFO repeater task ashpaq5_hcorrstats ended after 5.2 seconds 2010-11-04 04:06:09,606 INFO repeater task os75bb_lastensq starting 4.6 seconds late 2010-11-04 04:06:20,701 INFO repeater task os75bb_lastensq ended after 11.1 seconds 2010-11-04 04:06:20,702 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 04:05:58 2010-11-04 04:10:15,948 INFO repeater task ashpaq2_hcorrstats starting 3.7 seconds late 2010-11-04 04:10:21,217 INFO repeater task ashpaq2_hcorrstats ended after 5.3 seconds 2010-11-04 04:10:21,218 INFO repeater task ashpaq5_hcorrstats starting 9.0 seconds late 2010-11-04 04:10:26,524 INFO repeater task ashpaq5_hcorrstats ended after 5.3 seconds 2010-11-04 04:11:11,574 INFO repeater task os75bb_lastensq starting 3.6 seconds late 2010-11-04 04:11:22,810 INFO repeater task os75bb_lastensq ended after 11.2 seconds 2010-11-04 04:11:22,811 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 04:11:01 2010-11-04 04:15:13,060 INFO repeater task os75bb_beamplots starting 0.8 seconds late 2010-11-04 04:15:14,340 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 04:15:14,341 INFO repeater task ashpaq2_hcorrstats starting 2.1 seconds late 2010-11-04 04:15:19,772 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 04:15:19,773 INFO repeater task ashpaq5_hcorrstats starting 7.5 seconds late 2010-11-04 04:15:25,131 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 04:16:15,183 INFO repeater task os75bb_lastensq starting 4.2 seconds late 2010-11-04 04:16:26,440 INFO repeater task os75bb_lastensq ended after 11.3 seconds 2010-11-04 04:16:26,440 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 04:16:04 2010-11-04 04:20:16,682 INFO repeater task ashpaq2_hcorrstats starting 4.4 seconds late 2010-11-04 04:20:22,120 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 04:20:22,121 INFO repeater task ashpaq5_hcorrstats starting 9.9 seconds late 2010-11-04 04:20:27,490 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 04:21:17,546 INFO repeater task os75bb_lastensq starting 3.5 seconds late 2010-11-04 04:21:29,025 INFO repeater task os75bb_lastensq ended after 11.5 seconds 2010-11-04 04:21:29,026 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 04:21:08 2010-11-04 04:25:14,242 INFO repeater task os75bb_quick starting 2.0 seconds late 2010-11-04 04:25:23,734 INFO repeater task os75bb_quick ended after 9.5 seconds 2010-11-04 04:25:23,735 INFO repeater task os75bb_3dayplots starting 11.5 seconds late 2010-11-04 04:25:28,337 INFO repeater task os75bb_3dayplots ended after 4.6 seconds 2010-11-04 04:25:28,339 INFO repeater task os75bb_beamplots starting 16.1 seconds late 2010-11-04 04:25:29,624 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 04:25:29,624 INFO repeater task ashpaq2_hcorrstats starting 17.4 seconds late 2010-11-04 04:25:35,069 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 04:25:35,069 INFO repeater task ashpaq5_hcorrstats starting 22.8 seconds late 2010-11-04 04:25:40,459 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 04:26:20,492 INFO repeater task os75bb_lastensq starting 2.5 seconds late 2010-11-04 04:26:32,090 INFO repeater task os75bb_lastensq ended after 11.6 seconds 2010-11-04 04:26:32,091 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 04:26:10 2010-11-04 04:30:12,292 INFO repeater task ashpaq2_hcorrstats starting 0.0 seconds late 2010-11-04 04:30:17,685 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 04:30:17,686 INFO repeater task ashpaq5_hcorrstats starting 5.4 seconds late 2010-11-04 04:30:22,902 INFO repeater task ashpaq5_hcorrstats ended after 5.2 seconds 2010-11-04 04:31:22,951 INFO repeater task os75bb_lastensq starting 3.0 seconds late 2010-11-04 04:31:34,575 INFO repeater task os75bb_lastensq ended after 11.6 seconds 2010-11-04 04:31:34,575 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 04:31:14 2010-11-04 04:35:14,832 INFO repeater task os75bb_beamplots starting 2.6 seconds late 2010-11-04 04:35:16,118 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 04:35:16,119 INFO repeater task ashpaq2_hcorrstats starting 3.9 seconds late 2010-11-04 04:35:21,500 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 04:35:21,500 INFO repeater task ashpaq5_hcorrstats starting 9.2 seconds late 2010-11-04 04:35:26,822 INFO repeater task ashpaq5_hcorrstats ended after 5.3 seconds 2010-11-04 04:36:26,883 INFO repeater task os75bb_lastensq starting 2.9 seconds late 2010-11-04 04:36:38,626 INFO repeater task os75bb_lastensq ended after 11.7 seconds 2010-11-04 04:36:38,626 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 04:36:17 2010-11-04 04:40:13,870 INFO repeater task ashpaq2_hcorrstats starting 1.6 seconds late 2010-11-04 04:40:19,305 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 04:40:19,305 INFO repeater task ashpaq5_hcorrstats starting 7.0 seconds late 2010-11-04 04:40:24,625 INFO repeater task ashpaq5_hcorrstats ended after 5.3 seconds 2010-11-04 04:41:29,693 INFO repeater task os75bb_lastensq starting 2.7 seconds late 2010-11-04 04:41:41,533 INFO repeater task os75bb_lastensq ended after 11.8 seconds 2010-11-04 04:41:41,534 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 04:41:20 2010-11-04 04:45:16,732 INFO repeater task os75bb_beamplots starting 4.5 seconds late 2010-11-04 04:45:18,097 INFO repeater task os75bb_beamplots ended after 1.4 seconds 2010-11-04 04:45:18,098 INFO repeater task ashpaq2_hcorrstats starting 5.8 seconds late 2010-11-04 04:45:23,477 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 04:45:23,478 INFO repeater task ashpaq5_hcorrstats starting 11.2 seconds late 2010-11-04 04:45:28,756 INFO repeater task ashpaq5_hcorrstats ended after 5.3 seconds 2010-11-04 04:46:33,816 INFO repeater task os75bb_lastensq starting 3.8 seconds late 2010-11-04 04:46:45,615 INFO repeater task os75bb_lastensq ended after 11.8 seconds 2010-11-04 04:46:45,616 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 04:46:23 2010-11-04 04:50:15,844 INFO repeater task ashpaq2_hcorrstats starting 3.6 seconds late 2010-11-04 04:50:21,114 INFO repeater task ashpaq2_hcorrstats ended after 5.3 seconds 2010-11-04 04:50:21,115 INFO repeater task ashpaq5_hcorrstats starting 8.8 seconds late 2010-11-04 04:50:26,358 INFO repeater task ashpaq5_hcorrstats ended after 5.2 seconds 2010-11-04 04:51:36,428 INFO repeater task os75bb_lastensq starting 3.4 seconds late 2010-11-04 04:51:48,381 INFO repeater task os75bb_lastensq ended after 12.0 seconds 2010-11-04 04:51:48,382 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 04:51:27 2010-11-04 04:55:13,600 INFO repeater task os75bb_quick starting 1.3 seconds late 2010-11-04 04:55:22,966 INFO repeater task os75bb_quick ended after 9.4 seconds 2010-11-04 04:55:22,966 INFO repeater task os75bb_3dayplots starting 10.7 seconds late 2010-11-04 04:55:27,493 INFO repeater task os75bb_3dayplots ended after 4.5 seconds 2010-11-04 04:55:27,494 INFO repeater task os75bb_beamplots starting 15.2 seconds late 2010-11-04 04:55:28,780 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 04:55:28,780 INFO repeater task os75bb_beamhistory starting 16.5 seconds late 2010-11-04 04:55:51,001 INFO repeater task os75bb_beamhistory output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") saving file to os75bb_bin_history.png 2010-11-04 04:55:51,001 INFO repeater task os75bb_beamhistory ended after 22.2 seconds 2010-11-04 04:55:51,001 INFO repeater task ashpaq2_hcorrstats starting 38.7 seconds late 2010-11-04 04:55:56,482 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 04:55:56,483 INFO repeater task ashpaq5_hcorrstats starting 44.2 seconds late 2010-11-04 04:56:01,889 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 04:56:41,920 INFO repeater task os75bb_lastensq starting 4.9 seconds late 2010-11-04 04:56:54,035 INFO repeater task os75bb_lastensq ended after 12.1 seconds 2010-11-04 04:56:54,036 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 04:56:30 2010-11-04 05:00:14,255 INFO repeater task ashpaq2_hcorrstats starting 2.0 seconds late 2010-11-04 05:00:19,517 INFO repeater task ashpaq2_hcorrstats ended after 5.3 seconds 2010-11-04 05:00:19,518 INFO repeater task ashpaq5_hcorrstats starting 7.3 seconds late 2010-11-04 05:00:24,864 INFO repeater task ashpaq5_hcorrstats ended after 5.3 seconds 2010-11-04 05:01:44,947 INFO repeater task os75bb_lastensq starting 4.9 seconds late 2010-11-04 05:01:57,058 INFO repeater task os75bb_lastensq ended after 12.1 seconds 2010-11-04 05:01:57,058 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 05:01:33 2010-11-04 05:05:17,267 INFO repeater task os75bb_beamplots starting 5.0 seconds late 2010-11-04 05:05:18,551 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 05:05:18,551 INFO repeater task ashpaq2_hcorrstats starting 6.3 seconds late 2010-11-04 05:05:24,004 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 05:05:24,004 INFO repeater task ashpaq5_hcorrstats starting 11.7 seconds late 2010-11-04 05:05:29,405 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 05:06:44,483 INFO repeater task os75bb_lastensq starting 1.5 seconds late 2010-11-04 05:06:56,839 INFO repeater task os75bb_lastensq ended after 12.4 seconds 2010-11-04 05:06:56,840 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 05:06:36 2010-11-04 05:10:17,025 INFO repeater task ashpaq2_hcorrstats starting 4.8 seconds late 2010-11-04 05:10:22,375 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 05:10:22,376 INFO repeater task ashpaq5_hcorrstats starting 10.1 seconds late 2010-11-04 05:10:27,722 INFO repeater task ashpaq5_hcorrstats ended after 5.3 seconds 2010-11-04 05:11:47,791 INFO repeater task os75bb_lastensq starting 1.8 seconds late 2010-11-04 05:12:00,104 INFO repeater task os75bb_lastensq ended after 12.3 seconds 2010-11-04 05:12:00,105 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 05:11:40 2010-11-04 05:15:15,308 INFO repeater task os75bb_beamplots starting 3.0 seconds late 2010-11-04 05:15:16,597 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 05:15:16,598 INFO repeater task ashpaq2_hcorrstats starting 4.3 seconds late 2010-11-04 05:15:22,074 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 05:15:22,074 INFO repeater task ashpaq5_hcorrstats starting 9.8 seconds late 2010-11-04 05:15:27,435 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 05:16:52,523 INFO repeater task os75bb_lastensq starting 2.5 seconds late 2010-11-04 05:17:05,108 INFO repeater task os75bb_lastensq ended after 12.6 seconds 2010-11-04 05:17:05,109 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 05:16:42 2010-11-04 05:20:15,272 INFO repeater task ashpaq2_hcorrstats starting 3.0 seconds late 2010-11-04 05:20:20,695 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 05:20:20,696 INFO repeater task ashpaq5_hcorrstats starting 8.4 seconds late 2010-11-04 05:20:26,031 INFO repeater task ashpaq5_hcorrstats ended after 5.3 seconds 2010-11-04 05:21:56,106 INFO repeater task os75bb_lastensq starting 4.1 seconds late 2010-11-04 05:22:08,737 INFO repeater task os75bb_lastensq ended after 12.6 seconds 2010-11-04 05:22:08,737 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 05:21:46 2010-11-04 05:25:13,946 INFO repeater task os75bb_quick starting 1.7 seconds late 2010-11-04 05:25:23,595 INFO repeater task os75bb_quick ended after 9.6 seconds 2010-11-04 05:25:23,595 INFO repeater task os75bb_3dayplots starting 11.3 seconds late 2010-11-04 05:25:28,257 INFO repeater task os75bb_3dayplots ended after 4.7 seconds 2010-11-04 05:25:28,258 INFO repeater task os75bb_beamplots starting 16.0 seconds late 2010-11-04 05:25:29,565 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 05:25:29,565 INFO repeater task ashpaq2_hcorrstats starting 17.3 seconds late 2010-11-04 05:25:34,954 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 05:25:34,955 INFO repeater task ashpaq5_hcorrstats starting 22.7 seconds late 2010-11-04 05:25:40,368 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 05:27:00,463 INFO repeater task os75bb_lastensq starting 4.5 seconds late 2010-11-04 05:27:13,154 INFO repeater task os75bb_lastensq ended after 12.7 seconds 2010-11-04 05:27:13,155 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 05:26:49 2010-11-04 05:30:13,319 INFO repeater task ashpaq2_hcorrstats starting 1.1 seconds late 2010-11-04 05:30:18,757 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 05:30:18,757 INFO repeater task ashpaq5_hcorrstats starting 6.5 seconds late 2010-11-04 05:30:24,223 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 05:31:59,312 INFO repeater task os75bb_lastensq starting 0.3 seconds late 2010-11-04 05:32:12,120 INFO repeater task os75bb_lastensq ended after 12.8 seconds 2010-11-04 05:32:12,121 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 05:31:52 2010-11-04 05:35:12,313 INFO repeater task os75bb_beamplots starting 0.1 seconds late 2010-11-04 05:35:13,609 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 05:35:13,610 INFO repeater task ashpaq2_hcorrstats starting 1.3 seconds late 2010-11-04 05:35:19,018 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 05:35:19,018 INFO repeater task ashpaq5_hcorrstats starting 6.8 seconds late 2010-11-04 05:35:24,432 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 05:37:04,538 INFO repeater task os75bb_lastensq starting 2.5 seconds late 2010-11-04 05:37:17,404 INFO repeater task os75bb_lastensq ended after 12.9 seconds 2010-11-04 05:37:17,405 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 05:36:55 2010-11-04 05:40:12,557 INFO repeater task ashpaq2_hcorrstats starting 0.3 seconds late 2010-11-04 05:40:18,024 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 05:40:18,025 INFO repeater task ashpaq5_hcorrstats starting 5.8 seconds late 2010-11-04 05:40:23,477 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 05:42:08,564 INFO repeater task os75bb_lastensq starting 3.6 seconds late 2010-11-04 05:42:21,539 INFO repeater task os75bb_lastensq ended after 13.0 seconds 2010-11-04 05:42:21,540 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 05:41:59 2010-11-04 05:45:16,724 INFO repeater task os75bb_beamplots starting 4.5 seconds late 2010-11-04 05:45:18,017 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 05:45:18,017 INFO repeater task ashpaq2_hcorrstats starting 5.8 seconds late 2010-11-04 05:45:23,449 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 05:45:23,450 INFO repeater task ashpaq5_hcorrstats starting 11.2 seconds late 2010-11-04 05:45:28,891 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 05:47:09,001 INFO repeater task os75bb_lastensq starting 0.0 seconds late 2010-11-04 05:47:22,058 INFO repeater task os75bb_lastensq ended after 13.1 seconds 2010-11-04 05:47:22,059 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 05:47:02 2010-11-04 05:50:17,240 INFO repeater task ashpaq2_hcorrstats starting 5.0 seconds late 2010-11-04 05:50:22,558 INFO repeater task ashpaq2_hcorrstats ended after 5.3 seconds 2010-11-04 05:50:22,559 INFO repeater task ashpaq5_hcorrstats starting 10.3 seconds late 2010-11-04 05:50:27,813 INFO repeater task ashpaq5_hcorrstats ended after 5.3 seconds 2010-11-04 05:52:12,934 INFO repeater task os75bb_lastensq starting 0.9 seconds late 2010-11-04 05:52:26,074 INFO repeater task os75bb_lastensq ended after 13.1 seconds 2010-11-04 05:52:26,074 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 05:52:05 2010-11-04 05:55:16,254 INFO repeater task os75bb_quick starting 4.0 seconds late 2010-11-04 05:55:25,652 INFO repeater task os75bb_quick ended after 9.4 seconds 2010-11-04 05:55:25,653 INFO repeater task os75bb_3dayplots starting 13.4 seconds late 2010-11-04 05:55:30,315 INFO repeater task os75bb_3dayplots ended after 4.7 seconds 2010-11-04 05:55:30,316 INFO repeater task os75bb_beamplots starting 18.1 seconds late 2010-11-04 05:55:31,641 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 05:55:31,641 INFO repeater task os75bb_beamhistory starting 19.4 seconds late 2010-11-04 05:55:54,624 INFO repeater task os75bb_beamhistory output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") saving file to os75bb_bin_history.png 2010-11-04 05:55:54,624 INFO repeater task os75bb_beamhistory ended after 23.0 seconds 2010-11-04 05:55:54,625 INFO repeater task ashpaq2_hcorrstats starting 42.4 seconds late 2010-11-04 05:55:59,934 INFO repeater task ashpaq2_hcorrstats ended after 5.3 seconds 2010-11-04 05:55:59,934 INFO repeater task ashpaq5_hcorrstats starting 47.7 seconds late 2010-11-04 05:56:05,325 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 05:57:15,400 INFO repeater task os75bb_lastensq starting 0.4 seconds late 2010-11-04 05:57:28,650 INFO repeater task os75bb_lastensq ended after 13.3 seconds 2010-11-04 05:57:28,651 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 05:57:08 2010-11-04 06:00:13,824 INFO repeater task ashpaq2_hcorrstats starting 1.6 seconds late 2010-11-04 06:00:19,269 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 06:00:19,269 INFO repeater task ashpaq5_hcorrstats starting 7.0 seconds late 2010-11-04 06:00:24,735 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 06:02:19,863 INFO repeater task os75bb_lastensq starting 1.9 seconds late 2010-11-04 06:02:34,043 INFO repeater task os75bb_lastensq ended after 14.2 seconds 2010-11-04 06:02:34,043 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 06:02:12 2010-11-04 06:05:14,208 INFO repeater task os75bb_beamplots starting 1.9 seconds late 2010-11-04 06:05:15,498 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 06:05:15,499 INFO repeater task ashpaq2_hcorrstats starting 3.2 seconds late 2010-11-04 06:05:20,949 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 06:05:20,949 INFO repeater task ashpaq5_hcorrstats starting 8.7 seconds late 2010-11-04 06:05:26,379 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 06:07:26,506 INFO repeater task os75bb_lastensq starting 4.5 seconds late 2010-11-04 06:07:38,049 INFO repeater task os75bb_lastensq ended after 11.5 seconds 2010-11-04 06:07:38,049 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 06:07:15 2010-11-04 06:10:13,188 INFO repeater task ashpaq2_hcorrstats starting 0.9 seconds late 2010-11-04 06:10:18,677 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 06:10:18,677 INFO repeater task ashpaq5_hcorrstats starting 6.4 seconds late 2010-11-04 06:10:24,118 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 06:12:29,216 INFO repeater task os75bb_lastensq starting 4.2 seconds late 2010-11-04 06:12:40,989 INFO repeater task os75bb_lastensq ended after 11.8 seconds 2010-11-04 06:12:40,989 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 06:12:18 2010-11-04 06:15:16,149 INFO repeater task os75bb_beamplots starting 3.9 seconds late 2010-11-04 06:15:17,461 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 06:15:17,462 INFO repeater task ashpaq2_hcorrstats starting 5.2 seconds late 2010-11-04 06:15:22,801 INFO repeater task ashpaq2_hcorrstats ended after 5.3 seconds 2010-11-04 06:15:22,802 INFO repeater task ashpaq5_hcorrstats starting 10.5 seconds late 2010-11-04 06:15:28,159 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 06:17:28,287 INFO repeater task os75bb_lastensq starting 0.3 seconds late 2010-11-04 06:17:40,065 INFO repeater task os75bb_lastensq ended after 11.8 seconds 2010-11-04 06:17:40,066 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 06:17:21 2010-11-04 06:20:15,239 INFO repeater task ashpaq2_hcorrstats starting 3.0 seconds late 2010-11-04 06:20:20,681 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 06:20:20,681 INFO repeater task ashpaq5_hcorrstats starting 8.4 seconds late 2010-11-04 06:20:26,045 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 06:22:31,183 INFO repeater task os75bb_lastensq starting 0.2 seconds late 2010-11-04 06:22:43,006 INFO repeater task os75bb_lastensq ended after 11.8 seconds 2010-11-04 06:22:43,007 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 06:22:25 2010-11-04 06:25:13,158 INFO repeater task os75bb_quick starting 0.9 seconds late 2010-11-04 06:25:22,680 INFO repeater task os75bb_quick ended after 9.5 seconds 2010-11-04 06:25:22,681 INFO repeater task os75bb_3dayplots starting 10.4 seconds late 2010-11-04 06:25:27,720 INFO repeater task os75bb_3dayplots ended after 5.0 seconds 2010-11-04 06:25:27,721 INFO repeater task os75bb_beamplots starting 15.5 seconds late 2010-11-04 06:25:29,021 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 06:25:29,022 INFO repeater task ashpaq2_hcorrstats starting 16.8 seconds late 2010-11-04 06:25:34,507 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 06:25:34,508 INFO repeater task ashpaq5_hcorrstats starting 22.2 seconds late 2010-11-04 06:25:39,942 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 06:27:35,044 INFO repeater task os75bb_lastensq starting 0.0 seconds late 2010-11-04 06:27:47,034 INFO repeater task os75bb_lastensq ended after 12.0 seconds 2010-11-04 06:27:47,035 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 06:27:27 2010-11-04 06:30:17,193 INFO repeater task ashpaq2_hcorrstats starting 4.9 seconds late 2010-11-04 06:30:22,587 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 06:30:22,587 INFO repeater task ashpaq5_hcorrstats starting 10.3 seconds late 2010-11-04 06:30:27,916 INFO repeater task ashpaq5_hcorrstats ended after 5.3 seconds 2010-11-04 06:32:38,048 INFO repeater task os75bb_lastensq starting 1.0 seconds late 2010-11-04 06:32:50,110 INFO repeater task os75bb_lastensq ended after 12.1 seconds 2010-11-04 06:32:50,111 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 06:32:31 2010-11-04 06:35:15,268 INFO repeater task os75bb_beamplots starting 3.0 seconds late 2010-11-04 06:35:16,559 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 06:35:16,559 INFO repeater task ashpaq2_hcorrstats starting 4.3 seconds late 2010-11-04 06:35:22,017 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 06:35:22,017 INFO repeater task ashpaq5_hcorrstats starting 9.8 seconds late 2010-11-04 06:35:27,431 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 06:37:42,577 INFO repeater task os75bb_lastensq starting 1.6 seconds late 2010-11-04 06:37:54,689 INFO repeater task os75bb_lastensq ended after 12.1 seconds 2010-11-04 06:37:54,689 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 06:37:34 2010-11-04 06:40:14,812 INFO repeater task ashpaq2_hcorrstats starting 2.5 seconds late 2010-11-04 06:40:20,161 INFO repeater task ashpaq2_hcorrstats ended after 5.3 seconds 2010-11-04 06:40:20,162 INFO repeater task ashpaq5_hcorrstats starting 7.9 seconds late 2010-11-04 06:40:25,600 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 06:42:45,719 INFO repeater task os75bb_lastensq starting 1.7 seconds late 2010-11-04 06:42:57,940 INFO repeater task os75bb_lastensq ended after 12.2 seconds 2010-11-04 06:42:57,941 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 06:42:37 2010-11-04 06:45:13,084 INFO repeater task os75bb_beamplots starting 0.8 seconds late 2010-11-04 06:45:14,373 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 06:45:14,374 INFO repeater task ashpaq2_hcorrstats starting 2.1 seconds late 2010-11-04 06:45:19,818 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 06:45:19,819 INFO repeater task ashpaq5_hcorrstats starting 7.6 seconds late 2010-11-04 06:45:25,239 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 06:47:50,399 INFO repeater task os75bb_lastensq starting 3.4 seconds late 2010-11-04 06:48:02,631 INFO repeater task os75bb_lastensq ended after 12.2 seconds 2010-11-04 06:48:02,632 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 06:47:40 2010-11-04 06:50:12,744 INFO repeater task ashpaq2_hcorrstats starting 0.5 seconds late 2010-11-04 06:50:18,229 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 06:50:18,230 INFO repeater task ashpaq5_hcorrstats starting 6.0 seconds late 2010-11-04 06:50:23,764 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 06:52:53,888 INFO repeater task os75bb_lastensq starting 3.9 seconds late 2010-11-04 06:53:06,324 INFO repeater task os75bb_lastensq ended after 12.4 seconds 2010-11-04 06:53:06,324 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 06:52:44 2010-11-04 06:55:16,476 INFO repeater task os75bb_quick starting 4.2 seconds late 2010-11-04 06:55:26,039 INFO repeater task os75bb_quick ended after 9.6 seconds 2010-11-04 06:55:26,040 INFO repeater task os75bb_3dayplots starting 13.8 seconds late 2010-11-04 06:55:30,650 INFO repeater task os75bb_3dayplots ended after 4.6 seconds 2010-11-04 06:55:30,659 INFO repeater task os75bb_beamplots starting 18.4 seconds late 2010-11-04 06:55:32,200 INFO repeater task os75bb_beamplots ended after 1.5 seconds 2010-11-04 06:55:32,200 INFO repeater task os75bb_beamhistory starting 19.9 seconds late 2010-11-04 06:55:54,077 INFO repeater task os75bb_beamhistory output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") saving file to os75bb_bin_history.png 2010-11-04 06:55:54,077 INFO repeater task os75bb_beamhistory ended after 21.9 seconds 2010-11-04 06:55:54,078 INFO repeater task ashpaq2_hcorrstats starting 41.8 seconds late 2010-11-04 06:55:59,676 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 06:55:59,676 INFO repeater task ashpaq5_hcorrstats starting 47.4 seconds late 2010-11-04 06:56:05,158 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 06:57:55,275 INFO repeater task os75bb_lastensq starting 1.3 seconds late 2010-11-04 06:58:07,862 INFO repeater task os75bb_lastensq ended after 12.6 seconds 2010-11-04 06:58:07,862 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 06:57:47 2010-11-04 07:00:12,989 INFO repeater task ashpaq2_hcorrstats starting 0.7 seconds late 2010-11-04 07:00:18,403 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 07:00:18,404 INFO repeater task ashpaq5_hcorrstats starting 6.1 seconds late 2010-11-04 07:00:23,805 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 07:02:58,975 INFO repeater task os75bb_lastensq starting 2.0 seconds late 2010-11-04 07:03:11,470 INFO repeater task os75bb_lastensq ended after 12.5 seconds 2010-11-04 07:03:11,470 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 07:02:50 2010-11-04 07:05:16,607 INFO repeater task os75bb_beamplots starting 4.3 seconds late 2010-11-04 07:05:17,891 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 07:05:17,892 INFO repeater task ashpaq2_hcorrstats starting 5.6 seconds late 2010-11-04 07:05:23,388 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 07:05:23,389 INFO repeater task ashpaq5_hcorrstats starting 11.1 seconds late 2010-11-04 07:05:28,987 INFO repeater task ashpaq5_hcorrstats ended after 5.6 seconds 2010-11-04 07:08:04,134 INFO repeater task os75bb_lastensq starting 4.1 seconds late 2010-11-04 07:08:16,904 INFO repeater task os75bb_lastensq ended after 12.8 seconds 2010-11-04 07:08:16,905 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 07:07:53 2010-11-04 07:10:17,050 INFO repeater task ashpaq2_hcorrstats starting 4.8 seconds late 2010-11-04 07:10:22,438 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 07:10:22,439 INFO repeater task ashpaq5_hcorrstats starting 10.2 seconds late 2010-11-04 07:10:27,861 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 07:13:03,035 INFO repeater task os75bb_lastensq starting 0.0 seconds late 2010-11-04 07:13:15,792 INFO repeater task os75bb_lastensq ended after 12.8 seconds 2010-11-04 07:13:15,792 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 07:12:57 2010-11-04 07:15:15,922 INFO repeater task os75bb_beamplots starting 3.7 seconds late 2010-11-04 07:15:17,213 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 07:15:17,214 INFO repeater task ashpaq2_hcorrstats starting 4.9 seconds late 2010-11-04 07:15:22,806 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 07:15:22,806 INFO repeater task ashpaq5_hcorrstats starting 10.5 seconds late 2010-11-04 07:15:28,308 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 07:18:08,439 INFO repeater task os75bb_lastensq starting 1.4 seconds late 2010-11-04 07:18:21,395 INFO repeater task os75bb_lastensq ended after 13.0 seconds 2010-11-04 07:18:21,395 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 07:18:00 2010-11-04 07:20:16,513 INFO repeater task ashpaq2_hcorrstats starting 4.2 seconds late 2010-11-04 07:20:21,970 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 07:20:21,970 INFO repeater task ashpaq5_hcorrstats starting 9.7 seconds late 2010-11-04 07:20:27,512 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 07:23:12,677 INFO repeater task os75bb_lastensq starting 2.7 seconds late 2010-11-04 07:23:25,653 INFO repeater task os75bb_lastensq ended after 13.0 seconds 2010-11-04 07:23:25,653 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 07:23:03 2010-11-04 07:25:15,756 INFO repeater task os75bb_quick starting 3.5 seconds late 2010-11-04 07:25:25,213 INFO repeater task os75bb_quick ended after 9.5 seconds 2010-11-04 07:25:25,214 INFO repeater task os75bb_3dayplots starting 13.0 seconds late 2010-11-04 07:25:29,919 INFO repeater task os75bb_3dayplots ended after 4.7 seconds 2010-11-04 07:25:29,919 INFO repeater task os75bb_beamplots starting 17.7 seconds late 2010-11-04 07:25:31,246 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 07:25:31,247 INFO repeater task ashpaq2_hcorrstats starting 19.0 seconds late 2010-11-04 07:25:36,753 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 07:25:36,754 INFO repeater task ashpaq5_hcorrstats starting 24.5 seconds late 2010-11-04 07:25:42,232 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 07:28:17,364 INFO repeater task os75bb_lastensq starting 4.4 seconds late 2010-11-04 07:28:30,434 INFO repeater task os75bb_lastensq ended after 13.1 seconds 2010-11-04 07:28:30,435 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 07:28:06 2010-11-04 07:30:15,547 INFO repeater task ashpaq2_hcorrstats starting 3.3 seconds late 2010-11-04 07:30:21,109 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 07:30:21,110 INFO repeater task ashpaq5_hcorrstats starting 8.8 seconds late 2010-11-04 07:30:26,654 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 07:33:16,828 INFO repeater task os75bb_lastensq starting 0.8 seconds late 2010-11-04 07:33:30,062 INFO repeater task os75bb_lastensq ended after 13.2 seconds 2010-11-04 07:33:30,063 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 07:33:09 2010-11-04 07:35:15,152 INFO repeater task os75bb_beamplots starting 2.9 seconds late 2010-11-04 07:35:16,480 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 07:35:16,481 INFO repeater task ashpaq2_hcorrstats starting 4.2 seconds late 2010-11-04 07:35:21,977 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 07:35:21,978 INFO repeater task ashpaq5_hcorrstats starting 9.7 seconds late 2010-11-04 07:35:27,524 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 07:38:22,714 INFO repeater task os75bb_lastensq starting 3.7 seconds late 2010-11-04 07:38:36,022 INFO repeater task os75bb_lastensq ended after 13.3 seconds 2010-11-04 07:38:36,023 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 07:38:12 2010-11-04 07:40:16,125 INFO repeater task ashpaq2_hcorrstats starting 3.9 seconds late 2010-11-04 07:40:21,735 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 07:40:21,736 INFO repeater task ashpaq5_hcorrstats starting 9.5 seconds late 2010-11-04 07:40:27,308 INFO repeater task ashpaq5_hcorrstats ended after 5.6 seconds 2010-11-04 07:43:22,448 INFO repeater task os75bb_lastensq starting 0.4 seconds late 2010-11-04 07:43:35,910 INFO repeater task os75bb_lastensq ended after 13.5 seconds 2010-11-04 07:43:35,911 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 07:43:16 2010-11-04 07:45:16,019 INFO repeater task os75bb_beamplots starting 3.8 seconds late 2010-11-04 07:45:17,298 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 07:45:17,298 INFO repeater task ashpaq2_hcorrstats starting 5.0 seconds late 2010-11-04 07:45:22,810 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 07:45:22,810 INFO repeater task ashpaq5_hcorrstats starting 10.5 seconds late 2010-11-04 07:45:28,318 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 07:48:28,536 INFO repeater task os75bb_lastensq starting 2.5 seconds late 2010-11-04 07:48:41,968 INFO repeater task os75bb_lastensq ended after 13.4 seconds 2010-11-04 07:48:41,968 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 07:48:19 2010-11-04 07:50:17,075 INFO repeater task ashpaq2_hcorrstats starting 4.8 seconds late 2010-11-04 07:50:22,515 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 07:50:22,516 INFO repeater task ashpaq5_hcorrstats starting 10.2 seconds late 2010-11-04 07:50:28,064 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 07:53:33,250 INFO repeater task os75bb_lastensq starting 4.3 seconds late 2010-11-04 07:53:46,986 INFO repeater task os75bb_lastensq ended after 13.7 seconds 2010-11-04 07:53:46,987 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 07:53:22 2010-11-04 07:55:17,068 INFO repeater task os75bb_quick starting 4.8 seconds late 2010-11-04 07:55:27,279 INFO repeater task os75bb_quick ended after 10.2 seconds 2010-11-04 07:55:27,279 INFO repeater task os75bb_3dayplots starting 15.0 seconds late 2010-11-04 07:55:31,904 INFO repeater task os75bb_3dayplots ended after 4.6 seconds 2010-11-04 07:55:31,904 INFO repeater task os75bb_beamplots starting 19.6 seconds late 2010-11-04 07:55:33,309 INFO repeater task os75bb_beamplots ended after 1.4 seconds 2010-11-04 07:55:33,310 INFO repeater task os75bb_beamhistory starting 21.0 seconds late 2010-11-04 07:55:55,966 INFO repeater task os75bb_beamhistory output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") saving file to os75bb_bin_history.png 2010-11-04 07:55:55,966 INFO repeater task os75bb_beamhistory ended after 22.7 seconds 2010-11-04 07:55:55,967 INFO repeater task ashpaq2_hcorrstats starting 43.7 seconds late 2010-11-04 07:56:01,499 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 07:56:01,500 INFO repeater task ashpaq5_hcorrstats starting 49.2 seconds late 2010-11-04 07:56:07,065 INFO repeater task ashpaq5_hcorrstats ended after 5.6 seconds 2010-11-04 07:58:32,199 INFO repeater task os75bb_lastensq starting 0.2 seconds late 2010-11-04 07:58:45,928 INFO repeater task os75bb_lastensq ended after 13.7 seconds 2010-11-04 07:58:45,929 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 07:58:25 2010-11-04 08:00:16,027 INFO repeater task ashpaq2_hcorrstats starting 3.8 seconds late 2010-11-04 08:00:21,566 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 08:00:21,566 INFO repeater task ashpaq5_hcorrstats starting 9.3 seconds late 2010-11-04 08:00:27,064 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 08:03:37,258 INFO repeater task os75bb_lastensq starting 2.3 seconds late 2010-11-04 08:03:51,847 INFO repeater task os75bb_lastensq ended after 14.6 seconds 2010-11-04 08:03:51,848 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 08:03:29 2010-11-04 08:05:16,936 INFO repeater task os75bb_beamplots starting 4.7 seconds late 2010-11-04 08:05:18,224 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 08:05:18,224 INFO repeater task ashpaq2_hcorrstats starting 6.0 seconds late 2010-11-04 08:05:23,751 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 08:05:23,752 INFO repeater task ashpaq5_hcorrstats starting 11.5 seconds late 2010-11-04 08:05:29,284 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 08:08:39,495 INFO repeater task os75bb_lastensq starting 0.5 seconds late 2010-11-04 08:08:51,063 INFO repeater task os75bb_lastensq ended after 11.6 seconds 2010-11-04 08:08:51,064 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 08:08:32 2010-11-04 08:10:16,151 INFO repeater task ashpaq2_hcorrstats starting 3.9 seconds late 2010-11-04 08:10:21,702 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 08:10:21,703 INFO repeater task ashpaq5_hcorrstats starting 9.4 seconds late 2010-11-04 08:10:27,216 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 08:13:42,431 INFO repeater task os75bb_lastensq starting 0.4 seconds late 2010-11-04 08:13:54,110 INFO repeater task os75bb_lastensq ended after 11.7 seconds 2010-11-04 08:13:54,110 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 08:13:35 2010-11-04 08:15:14,178 INFO repeater task os75bb_beamplots starting 1.9 seconds late 2010-11-04 08:15:15,591 INFO repeater task os75bb_beamplots ended after 1.4 seconds 2010-11-04 08:15:15,592 INFO repeater task ashpaq2_hcorrstats starting 3.3 seconds late 2010-11-04 08:15:21,018 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 08:15:21,019 INFO repeater task ashpaq5_hcorrstats starting 8.8 seconds late 2010-11-04 08:15:26,446 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 08:18:46,656 INFO repeater task os75bb_lastensq starting 1.7 seconds late 2010-11-04 08:18:58,360 INFO repeater task os75bb_lastensq ended after 11.7 seconds 2010-11-04 08:18:58,360 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 08:18:38 2010-11-04 08:20:13,434 INFO repeater task ashpaq2_hcorrstats starting 1.2 seconds late 2010-11-04 08:20:19,035 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 08:20:19,036 INFO repeater task ashpaq5_hcorrstats starting 6.8 seconds late 2010-11-04 08:20:24,604 INFO repeater task ashpaq5_hcorrstats ended after 5.6 seconds 2010-11-04 08:23:49,839 INFO repeater task os75bb_lastensq starting 1.8 seconds late 2010-11-04 08:24:01,771 INFO repeater task os75bb_lastensq ended after 11.9 seconds 2010-11-04 08:24:01,772 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 08:23:42 2010-11-04 08:25:16,848 INFO repeater task os75bb_quick starting 4.6 seconds late 2010-11-04 08:25:26,357 INFO repeater task os75bb_quick ended after 9.5 seconds 2010-11-04 08:25:26,358 INFO repeater task os75bb_3dayplots starting 14.1 seconds late 2010-11-04 08:25:30,996 INFO repeater task os75bb_3dayplots ended after 4.6 seconds 2010-11-04 08:25:31,153 INFO repeater task os75bb_beamplots starting 18.9 seconds late 2010-11-04 08:25:32,455 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 08:25:32,456 INFO repeater task ashpaq2_hcorrstats starting 20.2 seconds late 2010-11-04 08:25:37,964 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 08:25:37,965 INFO repeater task ashpaq5_hcorrstats starting 25.7 seconds late 2010-11-04 08:25:43,475 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 08:28:53,672 INFO repeater task os75bb_lastensq starting 1.7 seconds late 2010-11-04 08:29:05,653 INFO repeater task os75bb_lastensq ended after 12.0 seconds 2010-11-04 08:29:05,653 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 08:28:44 2010-11-04 08:30:15,712 INFO repeater task ashpaq2_hcorrstats starting 3.4 seconds late 2010-11-04 08:30:21,128 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 08:30:21,128 INFO repeater task ashpaq5_hcorrstats starting 8.9 seconds late 2010-11-04 08:30:26,531 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 08:33:56,723 INFO repeater task os75bb_lastensq starting 2.7 seconds late 2010-11-04 08:34:08,760 INFO repeater task os75bb_lastensq ended after 12.0 seconds 2010-11-04 08:34:08,761 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 08:33:48 2010-11-04 08:35:13,834 INFO repeater task os75bb_beamplots starting 1.6 seconds late 2010-11-04 08:35:15,114 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 08:35:15,115 INFO repeater task ashpaq2_hcorrstats starting 2.8 seconds late 2010-11-04 08:35:20,618 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 08:35:20,619 INFO repeater task ashpaq5_hcorrstats starting 8.4 seconds late 2010-11-04 08:35:26,155 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 08:39:01,371 INFO repeater task os75bb_lastensq starting 3.4 seconds late 2010-11-04 08:39:13,545 INFO repeater task os75bb_lastensq ended after 12.2 seconds 2010-11-04 08:39:13,545 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 08:38:51 2010-11-04 08:40:13,597 INFO repeater task ashpaq2_hcorrstats starting 1.3 seconds late 2010-11-04 08:40:19,138 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 08:40:19,138 INFO repeater task ashpaq5_hcorrstats starting 6.9 seconds late 2010-11-04 08:40:24,632 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 08:44:04,811 INFO repeater task os75bb_lastensq starting 3.8 seconds late 2010-11-04 08:44:17,174 INFO repeater task os75bb_lastensq ended after 12.4 seconds 2010-11-04 08:44:17,175 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 08:43:54 2010-11-04 08:45:17,234 INFO repeater task os75bb_beamplots starting 5.0 seconds late 2010-11-04 08:45:18,511 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 08:45:18,511 INFO repeater task ashpaq2_hcorrstats starting 6.2 seconds late 2010-11-04 08:45:24,075 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 08:45:24,076 INFO repeater task ashpaq5_hcorrstats starting 11.8 seconds late 2010-11-04 08:45:29,625 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 08:49:04,855 INFO repeater task os75bb_lastensq starting 0.9 seconds late 2010-11-04 08:49:17,093 INFO repeater task os75bb_lastensq ended after 12.2 seconds 2010-11-04 08:49:17,094 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 08:48:57 2010-11-04 08:50:17,145 INFO repeater task ashpaq2_hcorrstats starting 4.9 seconds late 2010-11-04 08:50:22,694 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 08:50:22,694 INFO repeater task ashpaq5_hcorrstats starting 10.4 seconds late 2010-11-04 08:50:28,205 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 08:54:08,406 INFO repeater task os75bb_lastensq starting 1.4 seconds late 2010-11-04 08:54:20,958 INFO repeater task os75bb_lastensq ended after 12.6 seconds 2010-11-04 08:54:20,958 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 08:54:01 2010-11-04 08:55:16,014 INFO repeater task os75bb_quick starting 3.8 seconds late 2010-11-04 08:55:26,371 INFO repeater task os75bb_quick ended after 10.4 seconds 2010-11-04 08:55:26,371 INFO repeater task os75bb_3dayplots starting 14.1 seconds late 2010-11-04 08:55:31,065 INFO repeater task os75bb_3dayplots ended after 4.7 seconds 2010-11-04 08:55:31,066 INFO repeater task os75bb_beamplots starting 18.8 seconds late 2010-11-04 08:55:32,348 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 08:55:32,348 INFO repeater task os75bb_beamhistory starting 20.1 seconds late 2010-11-04 08:55:54,148 INFO repeater task os75bb_beamhistory output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") saving file to os75bb_bin_history.png 2010-11-04 08:55:54,149 INFO repeater task os75bb_beamhistory ended after 21.8 seconds 2010-11-04 08:55:54,149 INFO repeater task ashpaq2_hcorrstats starting 41.9 seconds late 2010-11-04 08:55:59,700 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 08:55:59,701 INFO repeater task ashpaq5_hcorrstats starting 47.4 seconds late 2010-11-04 08:56:05,271 INFO repeater task ashpaq5_hcorrstats ended after 5.6 seconds 2010-11-04 08:59:15,427 INFO repeater task os75bb_lastensq starting 4.4 seconds late 2010-11-04 08:59:28,027 INFO repeater task os75bb_lastensq ended after 12.6 seconds 2010-11-04 08:59:28,027 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 08:59:04 2010-11-04 09:00:13,076 INFO repeater task ashpaq2_hcorrstats starting 0.8 seconds late 2010-11-04 09:00:18,577 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 09:00:18,577 INFO repeater task ashpaq5_hcorrstats starting 6.3 seconds late 2010-11-04 09:00:24,105 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 09:04:14,350 INFO repeater task os75bb_lastensq starting 0.4 seconds late 2010-11-04 09:04:26,915 INFO repeater task os75bb_lastensq ended after 12.6 seconds 2010-11-04 09:04:26,915 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 09:04:07 2010-11-04 09:05:16,967 INFO repeater task os75bb_beamplots starting 4.7 seconds late 2010-11-04 09:05:18,315 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 09:05:18,316 INFO repeater task ashpaq2_hcorrstats starting 6.1 seconds late 2010-11-04 09:05:23,753 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 09:05:23,754 INFO repeater task ashpaq5_hcorrstats starting 11.5 seconds late 2010-11-04 09:05:29,223 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 09:09:19,419 INFO repeater task os75bb_lastensq starting 2.4 seconds late 2010-11-04 09:09:32,146 INFO repeater task os75bb_lastensq ended after 12.7 seconds 2010-11-04 09:09:32,146 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 09:09:10 2010-11-04 09:10:17,196 INFO repeater task ashpaq2_hcorrstats starting 4.9 seconds late 2010-11-04 09:10:22,730 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 09:10:22,731 INFO repeater task ashpaq5_hcorrstats starting 10.5 seconds late 2010-11-04 09:10:28,294 INFO repeater task ashpaq5_hcorrstats ended after 5.6 seconds 2010-11-04 09:14:23,550 INFO repeater task os75bb_lastensq starting 3.6 seconds late 2010-11-04 09:14:36,345 INFO repeater task os75bb_lastensq ended after 12.8 seconds 2010-11-04 09:14:36,345 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 09:14:14 2010-11-04 09:15:16,380 INFO repeater task os75bb_beamplots starting 4.1 seconds late 2010-11-04 09:15:17,772 INFO repeater task os75bb_beamplots ended after 1.4 seconds 2010-11-04 09:15:17,772 INFO repeater task ashpaq2_hcorrstats starting 5.5 seconds late 2010-11-04 09:15:23,172 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 09:15:23,173 INFO repeater task ashpaq5_hcorrstats starting 10.9 seconds late 2010-11-04 09:15:28,634 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 09:19:28,880 INFO repeater task os75bb_lastensq starting 4.9 seconds late 2010-11-04 09:19:41,766 INFO repeater task os75bb_lastensq ended after 12.9 seconds 2010-11-04 09:19:41,767 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 09:19:17 2010-11-04 09:20:16,801 INFO repeater task ashpaq2_hcorrstats starting 4.5 seconds late 2010-11-04 09:20:22,323 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 09:20:22,324 INFO repeater task ashpaq5_hcorrstats starting 10.1 seconds late 2010-11-04 09:20:27,854 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 09:24:28,111 INFO repeater task os75bb_lastensq starting 1.1 seconds late 2010-11-04 09:24:41,106 INFO repeater task os75bb_lastensq ended after 13.0 seconds 2010-11-04 09:24:41,106 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 09:24:20 2010-11-04 09:25:16,140 INFO repeater task os75bb_quick starting 3.9 seconds late 2010-11-04 09:25:27,119 INFO repeater task os75bb_quick ended after 11.0 seconds 2010-11-04 09:25:27,120 INFO repeater task os75bb_3dayplots starting 14.9 seconds late 2010-11-04 09:25:31,847 INFO repeater task os75bb_3dayplots ended after 4.7 seconds 2010-11-04 09:25:31,847 INFO repeater task os75bb_beamplots starting 19.6 seconds late 2010-11-04 09:25:33,125 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 09:25:33,126 INFO repeater task ashpaq2_hcorrstats starting 20.9 seconds late 2010-11-04 09:25:38,704 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 09:25:38,705 INFO repeater task ashpaq5_hcorrstats starting 26.4 seconds late 2010-11-04 09:25:44,279 INFO repeater task ashpaq5_hcorrstats ended after 5.6 seconds 2010-11-04 09:29:34,525 INFO repeater task os75bb_lastensq starting 4.5 seconds late 2010-11-04 09:29:47,664 INFO repeater task os75bb_lastensq ended after 13.1 seconds 2010-11-04 09:29:47,665 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 09:29:23 2010-11-04 09:30:12,689 INFO repeater task ashpaq2_hcorrstats starting 0.4 seconds late 2010-11-04 09:30:18,185 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 09:30:18,186 INFO repeater task ashpaq5_hcorrstats starting 5.9 seconds late 2010-11-04 09:30:23,682 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 09:34:33,964 INFO repeater task os75bb_lastensq starting 1.0 seconds late 2010-11-04 09:34:47,265 INFO repeater task os75bb_lastensq ended after 13.3 seconds 2010-11-04 09:34:47,265 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 09:34:27 2010-11-04 09:35:12,294 INFO repeater task os75bb_beamplots starting 0.0 seconds late 2010-11-04 09:35:13,566 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 09:35:13,567 INFO repeater task ashpaq2_hcorrstats starting 1.3 seconds late 2010-11-04 09:35:19,164 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 09:35:19,164 INFO repeater task ashpaq5_hcorrstats starting 6.9 seconds late 2010-11-04 09:35:24,681 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 09:39:39,908 INFO repeater task os75bb_lastensq starting 2.9 seconds late 2010-11-04 09:39:53,252 INFO repeater task os75bb_lastensq ended after 13.3 seconds 2010-11-04 09:39:53,252 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 09:39:29 2010-11-04 09:40:13,272 INFO repeater task ashpaq2_hcorrstats starting 1.0 seconds late 2010-11-04 09:40:18,838 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 09:40:18,838 INFO repeater task ashpaq5_hcorrstats starting 6.6 seconds late 2010-11-04 09:40:24,335 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 09:44:39,597 INFO repeater task os75bb_lastensq starting 0.6 seconds late 2010-11-04 09:44:53,148 INFO repeater task os75bb_lastensq ended after 13.6 seconds 2010-11-04 09:44:53,148 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 09:44:33 2010-11-04 09:45:13,168 INFO repeater task os75bb_beamplots starting 0.9 seconds late 2010-11-04 09:45:14,497 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 09:45:14,498 INFO repeater task ashpaq2_hcorrstats starting 2.2 seconds late 2010-11-04 09:45:20,027 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 09:45:20,028 INFO repeater task ashpaq5_hcorrstats starting 7.8 seconds late 2010-11-04 09:45:25,577 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 09:49:45,851 INFO repeater task os75bb_lastensq starting 2.9 seconds late 2010-11-04 09:49:59,356 INFO repeater task os75bb_lastensq ended after 13.5 seconds 2010-11-04 09:49:59,357 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 09:49:36 2010-11-04 09:50:14,369 INFO repeater task ashpaq2_hcorrstats starting 2.1 seconds late 2010-11-04 09:50:19,969 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 09:50:19,969 INFO repeater task ashpaq5_hcorrstats starting 7.7 seconds late 2010-11-04 09:50:25,549 INFO repeater task ashpaq5_hcorrstats ended after 5.6 seconds 2010-11-04 09:54:50,780 INFO repeater task os75bb_lastensq starting 4.8 seconds late 2010-11-04 09:55:04,478 INFO repeater task os75bb_lastensq ended after 13.7 seconds 2010-11-04 09:55:04,478 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 09:54:39 2010-11-04 09:55:14,490 INFO repeater task os75bb_quick starting 2.2 seconds late 2010-11-04 09:55:24,820 INFO repeater task os75bb_quick ended after 10.3 seconds 2010-11-04 09:55:24,820 INFO repeater task os75bb_3dayplots starting 12.6 seconds late 2010-11-04 09:55:29,452 INFO repeater task os75bb_3dayplots ended after 4.6 seconds 2010-11-04 09:55:29,452 INFO repeater task os75bb_beamplots starting 17.2 seconds late 2010-11-04 09:55:30,735 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 09:55:30,736 INFO repeater task os75bb_beamhistory starting 18.5 seconds late 2010-11-04 09:55:53,243 INFO repeater task os75bb_beamhistory output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") saving file to os75bb_bin_history.png 2010-11-04 09:55:53,244 INFO repeater task os75bb_beamhistory ended after 22.5 seconds 2010-11-04 09:55:53,244 INFO repeater task ashpaq2_hcorrstats starting 41.0 seconds late 2010-11-04 09:55:58,737 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 09:55:58,738 INFO repeater task ashpaq5_hcorrstats starting 46.5 seconds late 2010-11-04 09:56:04,259 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 09:59:49,498 INFO repeater task os75bb_lastensq starting 0.5 seconds late 2010-11-04 10:00:03,142 INFO repeater task os75bb_lastensq ended after 13.6 seconds 2010-11-04 10:00:03,143 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 09:59:42 2010-11-04 10:00:13,152 INFO repeater task ashpaq2_hcorrstats starting 0.9 seconds late 2010-11-04 10:00:18,789 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 10:00:18,790 INFO repeater task ashpaq5_hcorrstats starting 6.5 seconds late 2010-11-04 10:00:24,312 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 10:04:54,530 INFO repeater task os75bb_lastensq starting 2.5 seconds late 2010-11-04 10:05:09,093 INFO repeater task os75bb_lastensq ended after 14.6 seconds 2010-11-04 10:05:09,094 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 10:04:45 2010-11-04 10:05:14,100 INFO repeater task os75bb_beamplots starting 1.8 seconds late 2010-11-04 10:05:15,441 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 10:05:15,442 INFO repeater task ashpaq2_hcorrstats starting 3.2 seconds late 2010-11-04 10:05:20,908 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 10:05:20,909 INFO repeater task ashpaq5_hcorrstats starting 8.6 seconds late 2010-11-04 10:05:26,404 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 10:09:56,652 INFO repeater task os75bb_lastensq starting 1.7 seconds late 2010-11-04 10:10:08,233 INFO repeater task os75bb_lastensq ended after 11.6 seconds 2010-11-04 10:10:08,233 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 10:09:48 2010-11-04 10:10:13,241 INFO repeater task ashpaq2_hcorrstats starting 1.0 seconds late 2010-11-04 10:10:18,650 INFO repeater task ashpaq2_hcorrstats ended after 5.4 seconds 2010-11-04 10:10:18,650 INFO repeater task ashpaq5_hcorrstats starting 6.4 seconds late 2010-11-04 10:10:24,072 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 10:14:59,343 INFO repeater task os75bb_lastensq starting 1.3 seconds late 2010-11-04 10:15:10,995 INFO repeater task os75bb_lastensq ended after 11.7 seconds 2010-11-04 10:15:10,995 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 10:14:50 2010-11-04 10:15:16,000 INFO repeater task os75bb_beamplots starting 3.7 seconds late 2010-11-04 10:15:17,270 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 10:15:17,270 INFO repeater task ashpaq2_hcorrstats starting 5.0 seconds late 2010-11-04 10:15:22,834 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 10:15:22,834 INFO repeater task ashpaq5_hcorrstats starting 10.6 seconds late 2010-11-04 10:15:28,347 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 10:20:03,654 INFO repeater task os75bb_lastensq starting 3.7 seconds late 2010-11-04 10:20:15,484 INFO repeater task os75bb_lastensq ended after 11.8 seconds 2010-11-04 10:20:15,484 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 10:19:53 2010-11-04 10:20:15,485 INFO repeater task ashpaq2_hcorrstats starting 3.2 seconds late 2010-11-04 10:20:21,098 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 10:20:21,099 INFO repeater task ashpaq5_hcorrstats starting 8.8 seconds late 2010-11-04 10:20:26,662 INFO repeater task ashpaq5_hcorrstats ended after 5.6 seconds 2010-11-04 10:25:06,898 INFO repeater task os75bb_lastensq starting 3.9 seconds late 2010-11-04 10:25:18,930 INFO repeater task os75bb_lastensq ended after 12.0 seconds 2010-11-04 10:25:18,931 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 10:24:57 2010-11-04 10:25:18,931 INFO repeater task os75bb_quick starting 6.7 seconds late 2010-11-04 10:25:30,593 INFO repeater task os75bb_quick ended after 11.7 seconds 2010-11-04 10:25:30,593 INFO repeater task os75bb_3dayplots starting 18.3 seconds late 2010-11-04 10:25:34,967 INFO repeater task os75bb_3dayplots ended after 4.4 seconds 2010-11-04 10:25:34,968 INFO repeater task os75bb_beamplots starting 22.7 seconds late 2010-11-04 10:25:36,249 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 10:25:36,250 INFO repeater task ashpaq2_hcorrstats starting 24.0 seconds late 2010-11-04 10:25:41,853 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 10:25:41,854 INFO repeater task ashpaq5_hcorrstats starting 29.6 seconds late 2010-11-04 10:25:47,491 INFO repeater task ashpaq5_hcorrstats ended after 5.6 seconds 2010-11-04 10:30:07,755 INFO repeater task os75bb_lastensq starting 0.8 seconds late 2010-11-04 10:30:19,777 INFO repeater task os75bb_lastensq ended after 12.0 seconds 2010-11-04 10:30:19,778 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 10:30:00 2010-11-04 10:30:19,780 INFO repeater task ashpaq2_hcorrstats starting 7.5 seconds late 2010-11-04 10:30:25,321 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 10:30:25,321 INFO repeater task ashpaq5_hcorrstats starting 13.1 seconds late 2010-11-04 10:30:30,813 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 10:35:11,087 INFO repeater task os75bb_lastensq starting 1.1 seconds late 2010-11-04 10:35:23,192 INFO repeater task os75bb_lastensq ended after 12.1 seconds 2010-11-04 10:35:23,192 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 10:35:03 2010-11-04 10:35:23,193 INFO repeater task os75bb_beamplots starting 10.9 seconds late 2010-11-04 10:35:24,468 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 10:35:24,468 INFO repeater task ashpaq2_hcorrstats starting 12.2 seconds late 2010-11-04 10:35:30,037 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 10:35:30,038 INFO repeater task ashpaq5_hcorrstats starting 17.8 seconds late 2010-11-04 10:35:35,653 INFO repeater task ashpaq5_hcorrstats ended after 5.6 seconds 2010-11-04 10:40:15,954 INFO repeater task os75bb_lastensq starting 3.0 seconds late 2010-11-04 10:40:28,098 INFO repeater task os75bb_lastensq ended after 12.1 seconds 2010-11-04 10:40:28,099 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 10:40:06 2010-11-04 10:40:28,100 INFO repeater task ashpaq2_hcorrstats starting 15.8 seconds late 2010-11-04 10:40:33,679 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 10:40:33,679 INFO repeater task ashpaq5_hcorrstats starting 21.4 seconds late 2010-11-04 10:40:39,229 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 10:45:14,472 INFO repeater task os75bb_beamplots starting 2.2 seconds late 2010-11-04 10:45:15,871 INFO repeater task os75bb_beamplots ended after 1.4 seconds 2010-11-04 10:45:15,872 INFO repeater task ashpaq2_hcorrstats starting 3.6 seconds late 2010-11-04 10:45:21,367 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 10:45:21,368 INFO repeater task ashpaq5_hcorrstats starting 9.1 seconds late 2010-11-04 10:45:26,849 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 10:45:31,855 INFO repeater task os75bb_lastensq starting 15.9 seconds late 2010-11-04 10:45:44,069 INFO repeater task os75bb_lastensq ended after 12.2 seconds 2010-11-04 10:45:44,070 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 10:45:10 2010-11-04 10:50:14,368 INFO repeater task ashpaq2_hcorrstats starting 2.1 seconds late 2010-11-04 10:50:19,958 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 10:50:19,958 INFO repeater task ashpaq5_hcorrstats starting 7.7 seconds late 2010-11-04 10:50:25,491 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 10:50:30,493 INFO repeater task os75bb_lastensq starting 10.5 seconds late 2010-11-04 10:50:42,891 INFO repeater task os75bb_lastensq ended after 12.4 seconds 2010-11-04 10:50:42,891 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 10:50:12 2010-11-04 10:55:13,124 INFO repeater task os75bb_quick starting 0.9 seconds late 2010-11-04 10:55:23,474 INFO repeater task os75bb_quick ended after 10.3 seconds 2010-11-04 10:55:23,474 INFO repeater task os75bb_3dayplots starting 11.2 seconds late 2010-11-04 10:55:27,725 INFO repeater task os75bb_3dayplots ended after 4.3 seconds 2010-11-04 10:55:27,726 INFO repeater task os75bb_beamplots starting 15.5 seconds late 2010-11-04 10:55:29,103 INFO repeater task os75bb_beamplots ended after 1.4 seconds 2010-11-04 10:55:29,104 INFO repeater task os75bb_beamhistory starting 16.8 seconds late 2010-11-04 10:55:50,788 INFO repeater task os75bb_beamhistory output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") saving file to os75bb_bin_history.png 2010-11-04 10:55:50,788 INFO repeater task os75bb_beamhistory ended after 21.7 seconds 2010-11-04 10:55:50,788 INFO repeater task ashpaq2_hcorrstats starting 38.5 seconds late 2010-11-04 10:55:56,366 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 10:55:56,366 INFO repeater task ashpaq5_hcorrstats starting 44.1 seconds late 2010-11-04 10:56:01,982 INFO repeater task ashpaq5_hcorrstats ended after 5.6 seconds 2010-11-04 10:56:06,986 INFO repeater task os75bb_lastensq starting 45.0 seconds late 2010-11-04 10:56:19,549 INFO repeater task os75bb_lastensq ended after 12.6 seconds 2010-11-04 10:56:19,550 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 10:55:16 2010-11-04 11:00:14,751 INFO repeater task ashpaq2_hcorrstats starting 2.5 seconds late 2010-11-04 11:00:20,380 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 11:00:20,380 INFO repeater task ashpaq5_hcorrstats starting 8.1 seconds late 2010-11-04 11:00:26,004 INFO repeater task ashpaq5_hcorrstats ended after 5.6 seconds 2010-11-04 11:00:31,008 INFO repeater task os75bb_lastensq starting 5.0 seconds late 2010-11-04 11:00:43,500 INFO repeater task os75bb_lastensq ended after 12.5 seconds 2010-11-04 11:00:43,501 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 11:00:19 2010-11-04 11:05:13,793 INFO repeater task os75bb_beamplots starting 1.5 seconds late 2010-11-04 11:05:15,083 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 11:05:15,084 INFO repeater task ashpaq2_hcorrstats starting 2.8 seconds late 2010-11-04 11:05:20,640 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 11:05:20,640 INFO repeater task ashpaq5_hcorrstats starting 8.4 seconds late 2010-11-04 11:05:26,282 INFO repeater task ashpaq5_hcorrstats ended after 5.6 seconds 2010-11-04 11:05:31,288 INFO repeater task os75bb_lastensq starting 2.3 seconds late 2010-11-04 11:05:43,985 INFO repeater task os75bb_lastensq ended after 12.7 seconds 2010-11-04 11:05:43,986 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 11:05:22 2010-11-04 11:10:14,248 INFO repeater task ashpaq2_hcorrstats starting 2.0 seconds late 2010-11-04 11:10:19,745 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 11:10:19,745 INFO repeater task ashpaq5_hcorrstats starting 7.5 seconds late 2010-11-04 11:10:25,180 INFO repeater task ashpaq5_hcorrstats ended after 5.4 seconds 2010-11-04 11:10:35,187 INFO repeater task os75bb_lastensq starting 3.2 seconds late 2010-11-04 11:10:47,884 INFO repeater task os75bb_lastensq ended after 12.7 seconds 2010-11-04 11:10:47,884 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 11:10:25 2010-11-04 11:15:13,152 INFO repeater task os75bb_beamplots starting 0.9 seconds late 2010-11-04 11:15:14,430 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 11:15:14,431 INFO repeater task ashpaq2_hcorrstats starting 2.2 seconds late 2010-11-04 11:15:20,016 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 11:15:20,017 INFO repeater task ashpaq5_hcorrstats starting 7.8 seconds late 2010-11-04 11:15:25,604 INFO repeater task ashpaq5_hcorrstats ended after 5.6 seconds 2010-11-04 11:15:35,612 INFO repeater task os75bb_lastensq starting 0.6 seconds late 2010-11-04 11:15:48,572 INFO repeater task os75bb_lastensq ended after 13.0 seconds 2010-11-04 11:15:48,573 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 11:15:29 2010-11-04 11:20:13,856 INFO repeater task ashpaq2_hcorrstats starting 1.6 seconds late 2010-11-04 11:20:19,312 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 11:20:19,313 INFO repeater task ashpaq5_hcorrstats starting 7.0 seconds late 2010-11-04 11:20:24,829 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 11:20:39,842 INFO repeater task os75bb_lastensq starting 0.8 seconds late 2010-11-04 11:20:52,870 INFO repeater task os75bb_lastensq ended after 13.0 seconds 2010-11-04 11:20:52,871 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 11:20:32 2010-11-04 11:25:13,117 INFO repeater task os75bb_quick starting 0.9 seconds late 2010-11-04 11:25:23,474 INFO repeater task os75bb_quick ended after 10.4 seconds 2010-11-04 11:25:23,474 INFO repeater task os75bb_3dayplots starting 11.2 seconds late 2010-11-04 11:25:28,109 INFO repeater task os75bb_3dayplots ended after 4.6 seconds 2010-11-04 11:25:28,109 INFO repeater task os75bb_beamplots starting 15.8 seconds late 2010-11-04 11:25:29,436 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 11:25:29,436 INFO repeater task ashpaq2_hcorrstats starting 17.2 seconds late 2010-11-04 11:25:35,035 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 11:25:35,036 INFO repeater task ashpaq5_hcorrstats starting 22.8 seconds late 2010-11-04 11:25:40,589 INFO repeater task ashpaq5_hcorrstats ended after 5.6 seconds 2010-11-04 11:25:45,592 INFO repeater task os75bb_lastensq starting 3.6 seconds late 2010-11-04 11:25:58,664 INFO repeater task os75bb_lastensq ended after 13.1 seconds 2010-11-04 11:25:58,664 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 11:25:35 2010-11-04 11:30:14,128 INFO repeater task ashpaq2_hcorrstats starting 1.9 seconds late 2010-11-04 11:30:19,730 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 11:30:19,731 INFO repeater task ashpaq5_hcorrstats starting 7.5 seconds late 2010-11-04 11:30:25,320 INFO repeater task ashpaq5_hcorrstats ended after 5.6 seconds 2010-11-04 11:30:45,343 INFO repeater task os75bb_lastensq starting 0.3 seconds late 2010-11-04 11:30:58,491 INFO repeater task os75bb_lastensq ended after 13.1 seconds 2010-11-04 11:30:58,492 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 11:30:38 2010-11-04 11:35:13,752 INFO repeater task os75bb_beamplots starting 1.5 seconds late 2010-11-04 11:35:15,049 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 11:35:15,050 INFO repeater task ashpaq2_hcorrstats starting 2.8 seconds late 2010-11-04 11:35:20,546 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 11:35:20,546 INFO repeater task ashpaq5_hcorrstats starting 8.3 seconds late 2010-11-04 11:35:26,155 INFO repeater task ashpaq5_hcorrstats ended after 5.6 seconds 2010-11-04 11:35:51,183 INFO repeater task os75bb_lastensq starting 3.2 seconds late 2010-11-04 11:36:04,406 INFO repeater task os75bb_lastensq ended after 13.2 seconds 2010-11-04 11:36:04,406 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 11:35:42 2010-11-04 11:40:14,617 INFO repeater task ashpaq2_hcorrstats starting 2.4 seconds late 2010-11-04 11:40:20,286 INFO repeater task ashpaq2_hcorrstats ended after 5.7 seconds 2010-11-04 11:40:20,286 INFO repeater task ashpaq5_hcorrstats starting 8.0 seconds late 2010-11-04 11:40:25,832 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 11:40:55,859 INFO repeater task os75bb_lastensq starting 3.9 seconds late 2010-11-04 11:41:09,084 INFO repeater task os75bb_lastensq ended after 13.2 seconds 2010-11-04 11:41:09,084 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 11:40:44 2010-11-04 11:45:14,345 INFO repeater task os75bb_beamplots starting 2.1 seconds late 2010-11-04 11:45:15,636 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 11:45:15,636 INFO repeater task ashpaq2_hcorrstats starting 3.4 seconds late 2010-11-04 11:45:21,208 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 11:45:21,208 INFO repeater task ashpaq5_hcorrstats starting 8.9 seconds late 2010-11-04 11:45:26,784 INFO repeater task ashpaq5_hcorrstats ended after 5.6 seconds 2010-11-04 11:45:56,807 INFO repeater task os75bb_lastensq starting 2.8 seconds late 2010-11-04 11:46:10,272 INFO repeater task os75bb_lastensq ended after 13.5 seconds 2010-11-04 11:46:10,273 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 11:45:48 2010-11-04 11:50:15,532 INFO repeater task ashpaq2_hcorrstats starting 3.3 seconds late 2010-11-04 11:50:21,113 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 11:50:21,114 INFO repeater task ashpaq5_hcorrstats starting 8.8 seconds late 2010-11-04 11:50:26,701 INFO repeater task ashpaq5_hcorrstats ended after 5.6 seconds 2010-11-04 11:51:01,742 INFO repeater task os75bb_lastensq starting 3.7 seconds late 2010-11-04 11:51:15,247 INFO repeater task os75bb_lastensq ended after 13.5 seconds 2010-11-04 11:51:15,248 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 11:50:51 2010-11-04 11:55:15,456 INFO repeater task os75bb_quick starting 3.2 seconds late 2010-11-04 11:55:25,864 INFO repeater task os75bb_quick ended after 10.4 seconds 2010-11-04 11:55:25,864 INFO repeater task os75bb_3dayplots starting 13.6 seconds late 2010-11-04 11:55:30,120 INFO repeater task os75bb_3dayplots ended after 4.3 seconds 2010-11-04 11:55:30,121 INFO repeater task os75bb_beamplots starting 17.9 seconds late 2010-11-04 11:55:31,539 INFO repeater task os75bb_beamplots ended after 1.4 seconds 2010-11-04 11:55:31,539 INFO repeater task os75bb_beamhistory starting 19.3 seconds late 2010-11-04 11:55:54,009 INFO repeater task os75bb_beamhistory output: /usr/local/lib/python2.6/dist-packages/numpy/ma/core.py:3808: UserWarning: Warning: converting a masked element to nan. warnings.warn("Warning: converting a masked element to nan.") saving file to os75bb_bin_history.png 2010-11-04 11:55:54,010 INFO repeater task os75bb_beamhistory ended after 22.5 seconds 2010-11-04 11:55:54,010 INFO repeater task ashpaq2_hcorrstats starting 41.7 seconds late 2010-11-04 11:55:59,555 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 11:55:59,556 INFO repeater task ashpaq5_hcorrstats starting 47.3 seconds late 2010-11-04 11:56:05,152 INFO repeater task ashpaq5_hcorrstats ended after 5.6 seconds 2010-11-04 11:56:10,155 INFO repeater task os75bb_lastensq starting 9.2 seconds late 2010-11-04 11:56:23,825 INFO repeater task os75bb_lastensq ended after 13.7 seconds 2010-11-04 11:56:23,826 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 11:55:54 2010-11-04 12:00:14,080 INFO repeater task ashpaq2_hcorrstats starting 1.8 seconds late 2010-11-04 12:00:19,626 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 12:00:19,627 INFO repeater task ashpaq5_hcorrstats starting 7.4 seconds late 2010-11-04 12:00:25,303 INFO repeater task ashpaq5_hcorrstats ended after 5.7 seconds 2010-11-04 12:01:05,347 INFO repeater task os75bb_lastensq starting 1.3 seconds late 2010-11-04 12:01:19,788 INFO repeater task os75bb_lastensq ended after 14.4 seconds 2010-11-04 12:01:19,788 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 12:00:57 2010-11-04 12:05:15,037 INFO repeater task os75bb_beamplots starting 2.8 seconds late 2010-11-04 12:05:16,335 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 12:05:16,335 INFO repeater task ashpaq2_hcorrstats starting 4.1 seconds late 2010-11-04 12:05:21,936 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 12:05:21,937 INFO repeater task ashpaq5_hcorrstats starting 9.7 seconds late 2010-11-04 12:05:27,572 INFO repeater task ashpaq5_hcorrstats ended after 5.6 seconds 2010-11-04 12:06:07,614 INFO repeater task os75bb_lastensq starting 0.6 seconds late 2010-11-04 12:06:19,103 INFO repeater task os75bb_lastensq ended after 11.5 seconds 2010-11-04 12:06:19,103 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 12:06:01 2010-11-04 12:10:14,340 INFO repeater task ashpaq2_hcorrstats starting 2.1 seconds late 2010-11-04 12:10:19,948 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 12:10:19,948 INFO repeater task ashpaq5_hcorrstats starting 7.7 seconds late 2010-11-04 12:10:25,519 INFO repeater task ashpaq5_hcorrstats ended after 5.6 seconds 2010-11-04 12:11:15,575 INFO repeater task os75bb_lastensq starting 4.6 seconds late 2010-11-04 12:11:27,222 INFO repeater task os75bb_lastensq ended after 11.6 seconds 2010-11-04 12:11:27,223 INFO repeater task os75bb_lastensq: Time from /home/data/EN487/proc/os75bb/load/wrote_ens_time: 2010/11/04 12:11:04 2010-11-04 12:15:12,409 INFO repeater task os75bb_beamplots starting 0.1 seconds late 2010-11-04 12:15:13,813 INFO repeater task os75bb_beamplots ended after 1.4 seconds 2010-11-04 12:15:13,814 INFO repeater task ashpaq2_hcorrstats starting 1.5 seconds late 2010-11-04 12:15:19,297 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 12:15:19,298 INFO repeater task ashpaq5_hcorrstats starting 7.0 seconds late 2010-11-04 12:15:24,835 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 12:16:14,887 INFO repeater task os75bb_lastensq starting 0.9 seconds late 2010-11-04 12:16:24,848 INFO repeater task os75bb_lastensq ended after 10.0 seconds 2010-11-04 12:16:24,849 WARNING repeater task os75bb_lastensq: Expected update of /home/data/EN487/proc/os75bb/load/wrote_ens_time did not occur, dt = -289.887 2010-11-04 12:20:15,129 INFO repeater task ashpaq2_hcorrstats starting 2.9 seconds late 2010-11-04 12:20:20,642 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 12:20:20,642 INFO repeater task ashpaq5_hcorrstats starting 8.4 seconds late 2010-11-04 12:20:26,150 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 12:21:26,215 INFO repeater task os75bb_lastensq starting 2.2 seconds late 2010-11-04 12:21:36,168 INFO repeater task os75bb_lastensq ended after 10.0 seconds 2010-11-04 12:21:36,168 WARNING repeater task os75bb_lastensq: Expected update of /home/data/EN487/proc/os75bb/load/wrote_ens_time did not occur, dt = -601.216 2010-11-04 12:25:16,364 INFO repeater task os75bb_quick starting 4.1 seconds late 2010-11-04 12:25:26,788 INFO repeater task os75bb_quick ended after 10.4 seconds 2010-11-04 12:25:26,789 INFO repeater task os75bb_3dayplots starting 14.5 seconds late 2010-11-04 12:25:31,512 INFO repeater task os75bb_3dayplots ended after 4.7 seconds 2010-11-04 12:25:31,515 INFO repeater task os75bb_beamplots starting 19.3 seconds late 2010-11-04 12:25:32,914 INFO repeater task os75bb_beamplots ended after 1.4 seconds 2010-11-04 12:25:32,915 INFO repeater task ashpaq2_hcorrstats starting 20.6 seconds late 2010-11-04 12:25:38,447 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 12:25:38,448 INFO repeater task ashpaq5_hcorrstats starting 26.2 seconds late 2010-11-04 12:25:43,986 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 12:26:34,038 INFO repeater task os75bb_lastensq starting 0.0 seconds late 2010-11-04 12:26:44,005 INFO repeater task os75bb_lastensq ended after 10.0 seconds 2010-11-04 12:26:44,006 WARNING repeater task os75bb_lastensq: Expected update of /home/data/EN487/proc/os75bb/load/wrote_ens_time did not occur, dt = -909.038 2010-11-04 12:30:14,256 INFO repeater task ashpaq2_hcorrstats starting 2.0 seconds late 2010-11-04 12:30:19,734 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 12:30:19,734 INFO repeater task ashpaq5_hcorrstats starting 7.5 seconds late 2010-11-04 12:30:25,270 INFO repeater task ashpaq5_hcorrstats ended after 5.5 seconds 2010-11-04 12:31:45,343 INFO repeater task os75bb_lastensq starting 1.3 seconds late 2010-11-04 12:31:55,315 INFO repeater task os75bb_lastensq ended after 10.0 seconds 2010-11-04 12:31:55,316 WARNING repeater task os75bb_lastensq: Expected update of /home/data/EN487/proc/os75bb/load/wrote_ens_time did not occur, dt = -1220.344 2010-11-04 12:35:15,532 INFO repeater task os75bb_beamplots starting 3.3 seconds late 2010-11-04 12:35:16,849 INFO repeater task os75bb_beamplots ended after 1.3 seconds 2010-11-04 12:35:16,850 INFO repeater task ashpaq2_hcorrstats starting 4.6 seconds late 2010-11-04 12:35:22,428 INFO repeater task ashpaq2_hcorrstats ended after 5.6 seconds 2010-11-04 12:35:22,428 INFO repeater task ashpaq5_hcorrstats starting 10.2 seconds late 2010-11-04 12:35:28,051 INFO repeater task ashpaq5_hcorrstats ended after 5.6 seconds 2010-11-04 12:36:58,145 INFO repeater task os75bb_lastensq starting 4.1 seconds late 2010-11-04 12:37:08,072 INFO repeater task os75bb_lastensq ended after 9.9 seconds 2010-11-04 12:37:08,072 WARNING repeater task os75bb_lastensq: Expected update of /home/data/EN487/proc/os75bb/load/wrote_ens_time did not occur, dt = -1533.145 2010-11-04 12:40:13,252 INFO repeater task ashpaq2_hcorrstats starting 1.0 seconds late 2010-11-04 12:40:18,750 INFO repeater task ashpaq2_hcorrstats ended after 5.5 seconds 2010-11-04 12:40:18,750 INFO repeater task ashpaq5_hcorrstats starting 6.5 seconds late 2010-11-04 12:40:24,321 INFO repeater task ashpaq5_hcorrstats ended after 5.6 seconds 2010-11-04 12:42:04,407 INFO repeater task os75bb_lastensq starting 0.4 seconds late 2010-11-04 12:42:14,406 INFO repeater task os75bb_lastensq ended after 10.0 seconds 2010-11-04 12:42:14,406 WARNING repeater task os75bb_lastensq: Expected update of /home/data/EN487/proc/os75bb/load/wrote_ens_time did not occur, dt = -1839.408 2010-11-04 12:42:41,361 INFO Translate translate exiting, gps /home/data/EN487/rbin/gpsnav 2010-11-04 12:42:41,362 INFO Translate translate exiting, gps /home/data/EN487/rbin/gpsnav1 2010-11-04 12:42:41,363 INFO Translate translate exiting, gps /home/data/EN487/rbin/ashpaq5 2010-11-04 12:42:41,363 INFO Translate translate exiting, paq /home/data/EN487/rbin/ashpaq5 2010-11-04 12:42:41,363 INFO Translate translate exiting, paq /home/data/EN487/rbin/ashpaq2 2010-11-04 12:42:41,364 INFO Translate translate exiting, gps /home/data/EN487/rbin/ashpaq2 2010-11-04 12:42:41,364 INFO Translate translate exiting, hdg /home/data/EN487/rbin/gyro 2010-11-04 12:42:44,442 INFO root cleanup_figs not_logging 2010-11-04 12:42:44,452 INFO root Exiting DAS_while_logging.py 2010-11-04 12:44:00,477 INFO root Starting DAS_while_logging.py 2010-11-04 12:44:00,497 DEBUG procsetup Starting procsetup 2010-11-04 12:44:00,506 INFO root Cruiseid: EN487 2010-11-04 12:44:00,530 INFO Translate gyro hdg 1 2010-11-04 12:44:00,531 INFO Translate 2010-11-04 12:44:00,531 INFO Translate ashpaq2 gps 1 2010-11-04 12:44:00,531 INFO Translate 2010-11-04 12:44:00,531 INFO Translate ashpaq2 paq 0 2010-11-04 12:44:00,531 INFO Translate 2010-11-04 12:44:00,532 INFO Translate ashpaq5 gps 1 2010-11-04 12:44:00,532 INFO Translate 2010-11-04 12:44:00,532 INFO Translate ashpaq5 paq 0 2010-11-04 12:44:00,532 INFO Translate 2010-11-04 12:44:00,532 INFO Translate gpsnav gps 1 2010-11-04 12:44:00,532 INFO Translate 2010-11-04 12:44:00,533 INFO Translate gpsnav1 gps 1 2010-11-04 12:44:00,533 INFO Translate 2010-11-04 12:44:00,566 INFO root active_procdirnames: ['wh300', 'os75bb']