2012-05-14 00:01:56,283 INFO repeater task os75nb_3dayplots ended after 55.4 seconds 2012-05-14 00:01:56,337 INFO repeater task wh300_beamplots starting 602.3 seconds late 2012-05-14 00:02:07,132 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 00:02:07,133 INFO repeater task wh300_beamplots ended after 10.8 seconds 2012-05-14 00:02:07,133 INFO repeater task wh300_beamplots: added 1 extra increments to time_to_run 2012-05-14 00:02:07,134 INFO repeater task wh300_beamplots starting 613.1 seconds late 2012-05-14 00:02:20,561 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 00:02:20,562 INFO repeater task wh300_beamplots ended after 13.4 seconds 2012-05-14 00:02:20,562 INFO repeater task wh300_beamplots: added 1 extra increments to time_to_run 2012-05-14 00:02:20,563 INFO repeater task os75bb_beamplots starting 626.5 seconds late 2012-05-14 00:02:29,021 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 00:02:29,021 INFO repeater task os75bb_beamplots ended after 8.5 seconds 2012-05-14 00:02:29,021 INFO repeater task os75bb_beamplots: added 1 extra increments to time_to_run 2012-05-14 00:02:29,024 INFO repeater task os75bb_beamplots starting 635.0 seconds late 2012-05-14 00:02:37,752 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 00:02:37,753 INFO repeater task os75bb_beamplots ended after 8.7 seconds 2012-05-14 00:02:37,753 INFO repeater task os75bb_beamplots: added 1 extra increments to time_to_run 2012-05-14 00:02:37,755 INFO repeater task os75nb_beamplots starting 643.7 seconds late 2012-05-14 00:02:44,431 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 00:02:44,432 INFO repeater task os75nb_beamplots ended after 6.7 seconds 2012-05-14 00:02:44,432 INFO repeater task os75nb_beamplots: added 1 extra increments to time_to_run 2012-05-14 00:02:44,435 INFO repeater task os75nb_beamplots starting 650.4 seconds late 2012-05-14 00:02:50,000 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 00:02:50,001 INFO repeater task os75nb_beamplots ended after 5.6 seconds 2012-05-14 00:02:50,001 INFO repeater task os75nb_beamplots: added 1 extra increments to time_to_run 2012-05-14 00:02:50,002 INFO repeater task phins_hcorrstats starting 656.0 seconds late 2012-05-14 00:02:56,092 INFO repeater task phins_hcorrstats ended after 6.1 seconds 2012-05-14 00:02:56,093 INFO repeater task phins_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 00:02:56,093 INFO repeater task phins2_hcorrstats starting 662.1 seconds late 2012-05-14 00:03:01,268 INFO repeater task phins2_hcorrstats ended after 5.2 seconds 2012-05-14 00:03:01,269 INFO repeater task phins2_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 00:03:06,273 INFO repeater task wh300_lastensq starting 621.3 seconds late 2012-05-14 00:03:15,811 INFO repeater task wh300_lastensq ended after 9.5 seconds 2012-05-14 00:03:15,812 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 00:02:35 2012-05-14 00:03:15,816 INFO repeater task os75bb_lastensq starting 210.8 seconds late 2012-05-14 00:03:25,682 INFO repeater task os75bb_lastensq ended after 9.9 seconds 2012-05-14 00:03:25,683 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/13 23:59:35 2012-05-14 00:03:25,684 INFO repeater task os75nb_lastensq starting 220.7 seconds late 2012-05-14 00:03:30,277 INFO repeater task os75nb_lastensq ended after 4.6 seconds 2012-05-14 00:03:30,278 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/13 23:59:35 2012-05-14 00:04:45,913 INFO repeater task wh300_lastensq starting 0.9 seconds late 2012-05-14 00:04:50,774 INFO repeater task wh300_lastensq ended after 4.9 seconds 2012-05-14 00:04:50,775 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 00:04:35 2012-05-14 00:04:50,781 INFO repeater task os75bb_lastensq starting 5.8 seconds late 2012-05-14 00:04:59,412 INFO repeater task os75bb_lastensq ended after 8.6 seconds 2012-05-14 00:04:59,413 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 00:04:35 2012-05-14 00:04:59,415 INFO repeater task os75nb_lastensq starting 14.4 seconds late 2012-05-14 00:05:04,219 INFO repeater task os75nb_lastensq ended after 4.8 seconds 2012-05-14 00:05:04,220 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 00:04:35 2012-05-14 00:06:49,509 INFO repeater task wh300_lastensq starting 4.5 seconds late 2012-05-14 00:06:54,544 INFO repeater task wh300_lastensq ended after 5.0 seconds 2012-05-14 00:06:54,545 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 00:06:35 2012-05-14 00:06:54,546 INFO repeater task wh300_quick starting 0.5 seconds late 2012-05-14 00:09:47,446 INFO repeater task wh300_quick ended after 172.9 seconds 2012-05-14 00:09:47,448 INFO repeater task wh300_3dayplots starting 173.4 seconds late 2012-05-14 00:11:04,705 INFO repeater task wh300_3dayplots output: could not parse 134_wh300_ddaycont.png could not parse 134_wh300_latcont.png could not parse 134_wh300_loncont.png 2012-05-14 00:11:04,706 INFO repeater task wh300_3dayplots ended after 77.3 seconds 2012-05-14 00:11:04,706 INFO repeater task os75bb_lastensq starting 79.7 seconds late 2012-05-14 00:11:15,859 INFO repeater task os75bb_lastensq ended after 11.2 seconds 2012-05-14 00:11:15,860 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 00:09:35 2012-05-14 00:11:15,861 INFO repeater task os75bb_quick starting 261.9 seconds late 2012-05-14 00:13:12,573 INFO repeater task os75bb_quick ended after 116.7 seconds 2012-05-14 00:13:12,574 INFO repeater task os75bb_3dayplots starting 378.6 seconds late 2012-05-14 00:14:08,647 INFO repeater task os75bb_3dayplots output: could not parse 134_os75bb_ddaycont.png could not parse 134_os75bb_latcont.png could not parse 134_os75bb_loncont.png 2012-05-14 00:14:08,647 INFO repeater task os75bb_3dayplots ended after 56.1 seconds 2012-05-14 00:14:08,648 INFO repeater task os75nb_lastensq starting 263.6 seconds late 2012-05-14 00:14:17,595 INFO repeater task os75nb_lastensq ended after 8.9 seconds 2012-05-14 00:14:17,650 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 00:09:35 2012-05-14 00:14:17,651 INFO repeater task os75nb_quick starting 443.6 seconds late 2012-05-14 00:16:05,034 INFO repeater task os75nb_quick ended after 107.4 seconds 2012-05-14 00:16:05,035 INFO repeater task os75nb_3dayplots starting 551.0 seconds late 2012-05-14 00:17:12,604 INFO repeater task os75nb_3dayplots output: could not parse 134_os75nb_ddaycont.png could not parse 134_os75nb_latcont.png could not parse 134_os75nb_loncont.png 2012-05-14 00:17:12,604 INFO repeater task os75nb_3dayplots ended after 67.6 seconds 2012-05-14 00:17:12,606 INFO repeater task wh300_beamplots starting 318.6 seconds late 2012-05-14 00:17:23,120 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 00:17:23,120 INFO repeater task wh300_beamplots ended after 10.5 seconds 2012-05-14 00:17:23,121 INFO repeater task wh300_beamplots starting 329.1 seconds late 2012-05-14 00:17:33,261 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 00:17:33,262 INFO repeater task wh300_beamplots ended after 10.1 seconds 2012-05-14 00:17:33,263 INFO repeater task os75bb_beamplots starting 339.2 seconds late 2012-05-14 00:17:38,555 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 00:17:38,555 INFO repeater task os75bb_beamplots ended after 5.3 seconds 2012-05-14 00:17:38,557 INFO repeater task os75bb_beamplots starting 344.5 seconds late 2012-05-14 00:17:43,692 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 00:17:43,692 INFO repeater task os75bb_beamplots ended after 5.1 seconds 2012-05-14 00:17:43,693 INFO repeater task os75nb_beamplots starting 349.7 seconds late 2012-05-14 00:17:48,676 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 00:17:48,677 INFO repeater task os75nb_beamplots ended after 5.0 seconds 2012-05-14 00:17:48,677 INFO repeater task os75nb_beamplots starting 354.7 seconds late 2012-05-14 00:17:53,623 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 00:17:53,624 INFO repeater task os75nb_beamplots ended after 4.9 seconds 2012-05-14 00:17:53,624 INFO repeater task phins_hcorrstats starting 659.6 seconds late 2012-05-14 00:17:59,647 INFO repeater task phins_hcorrstats ended after 6.0 seconds 2012-05-14 00:17:59,648 INFO repeater task phins_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 00:17:59,653 INFO repeater task phins2_hcorrstats starting 665.6 seconds late 2012-05-14 00:18:05,087 INFO repeater task phins2_hcorrstats ended after 5.4 seconds 2012-05-14 00:18:05,088 INFO repeater task phins2_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 00:18:10,097 INFO repeater task wh300_lastensq starting 565.1 seconds late 2012-05-14 00:18:18,135 INFO repeater task wh300_lastensq ended after 8.0 seconds 2012-05-14 00:18:18,136 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 00:16:35 2012-05-14 00:18:18,139 INFO repeater task os75bb_lastensq starting 213.1 seconds late 2012-05-14 00:18:27,324 INFO repeater task os75bb_lastensq ended after 9.2 seconds 2012-05-14 00:18:27,325 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 00:14:35 2012-05-14 00:18:27,326 INFO repeater task os75nb_lastensq starting 222.3 seconds late 2012-05-14 00:18:32,079 INFO repeater task os75nb_lastensq ended after 4.8 seconds 2012-05-14 00:18:32,079 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 00:14:35 2012-05-14 00:18:47,129 INFO repeater task wh300_lastensq starting 2.1 seconds late 2012-05-14 00:18:52,373 INFO repeater task wh300_lastensq ended after 5.2 seconds 2012-05-14 00:18:52,373 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 00:18:35 2012-05-14 00:19:47,570 INFO repeater task os75bb_lastensq starting 2.6 seconds late 2012-05-14 00:19:55,830 INFO repeater task os75bb_lastensq ended after 8.3 seconds 2012-05-14 00:19:55,831 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 00:19:35 2012-05-14 00:19:55,833 INFO repeater task os75nb_lastensq starting 10.8 seconds late 2012-05-14 00:20:00,564 INFO repeater task os75nb_lastensq ended after 4.7 seconds 2012-05-14 00:20:00,565 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 00:19:35 2012-05-14 00:20:45,682 INFO repeater task wh300_lastensq starting 0.7 seconds late 2012-05-14 00:20:50,762 INFO repeater task wh300_lastensq ended after 5.1 seconds 2012-05-14 00:20:50,763 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 00:20:35 2012-05-14 00:21:55,909 INFO repeater task wh300_quick starting 1.9 seconds late 2012-05-14 00:24:47,673 INFO repeater task wh300_quick ended after 171.8 seconds 2012-05-14 00:24:47,678 INFO repeater task wh300_3dayplots starting 173.7 seconds late 2012-05-14 00:25:44,542 INFO repeater task wh300_3dayplots ended after 56.9 seconds 2012-05-14 00:25:44,543 INFO repeater task os75bb_lastensq starting 59.5 seconds late 2012-05-14 00:25:55,482 INFO repeater task os75bb_lastensq ended after 10.9 seconds 2012-05-14 00:25:55,483 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 00:24:35 2012-05-14 00:25:55,483 INFO repeater task os75bb_quick starting 241.5 seconds late 2012-05-14 00:27:59,749 INFO repeater task os75bb_quick ended after 124.3 seconds 2012-05-14 00:27:59,750 INFO repeater task os75bb_3dayplots starting 365.7 seconds late 2012-05-14 00:29:14,564 INFO repeater task os75bb_3dayplots ended after 74.8 seconds 2012-05-14 00:29:14,565 INFO repeater task os75nb_lastensq starting 269.6 seconds late 2012-05-14 00:29:21,229 INFO repeater task os75nb_lastensq ended after 6.7 seconds 2012-05-14 00:29:21,230 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 00:24:35 2012-05-14 00:29:21,233 INFO repeater task os75nb_quick starting 447.2 seconds late 2012-05-14 00:31:06,218 INFO repeater task os75nb_quick ended after 105.0 seconds 2012-05-14 00:31:06,219 INFO repeater task os75nb_3dayplots starting 552.2 seconds late 2012-05-14 00:31:58,720 INFO repeater task os75nb_3dayplots ended after 52.5 seconds 2012-05-14 00:31:58,722 INFO repeater task wh300_beamplots starting 604.7 seconds late 2012-05-14 00:32:11,375 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 00:32:11,375 INFO repeater task wh300_beamplots ended after 12.7 seconds 2012-05-14 00:32:11,375 INFO repeater task wh300_beamplots: added 1 extra increments to time_to_run 2012-05-14 00:32:11,380 INFO repeater task wh300_beamplots starting 617.4 seconds late 2012-05-14 00:32:24,559 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 00:32:24,559 INFO repeater task wh300_beamplots ended after 13.2 seconds 2012-05-14 00:32:24,559 INFO repeater task wh300_beamplots: added 1 extra increments to time_to_run 2012-05-14 00:32:24,560 INFO repeater task os75bb_beamplots starting 630.5 seconds late 2012-05-14 00:32:34,136 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 00:32:34,137 INFO repeater task os75bb_beamplots ended after 9.6 seconds 2012-05-14 00:32:34,137 INFO repeater task os75bb_beamplots: added 1 extra increments to time_to_run 2012-05-14 00:32:34,144 INFO repeater task os75bb_beamplots starting 640.1 seconds late 2012-05-14 00:32:41,392 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 00:32:41,392 INFO repeater task os75bb_beamplots ended after 7.2 seconds 2012-05-14 00:32:41,393 INFO repeater task os75bb_beamplots: added 1 extra increments to time_to_run 2012-05-14 00:32:41,398 INFO repeater task os75nb_beamplots starting 647.4 seconds late 2012-05-14 00:32:47,519 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 00:32:47,520 INFO repeater task os75nb_beamplots ended after 6.1 seconds 2012-05-14 00:32:47,520 INFO repeater task os75nb_beamplots: added 1 extra increments to time_to_run 2012-05-14 00:32:47,521 INFO repeater task os75nb_beamplots starting 653.5 seconds late 2012-05-14 00:32:52,849 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 00:32:52,849 INFO repeater task os75nb_beamplots ended after 5.3 seconds 2012-05-14 00:32:52,849 INFO repeater task os75nb_beamplots: added 1 extra increments to time_to_run 2012-05-14 00:32:52,855 INFO repeater task phins_hcorrstats starting 658.8 seconds late 2012-05-14 00:32:58,730 INFO repeater task phins_hcorrstats ended after 5.9 seconds 2012-05-14 00:32:58,730 INFO repeater task phins_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 00:32:58,731 INFO repeater task phins2_hcorrstats starting 664.7 seconds late 2012-05-14 00:33:03,887 INFO repeater task phins2_hcorrstats ended after 5.2 seconds 2012-05-14 00:33:03,887 INFO repeater task phins2_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 00:33:08,897 INFO repeater task wh300_lastensq starting 623.9 seconds late 2012-05-14 00:33:15,668 INFO repeater task wh300_lastensq ended after 6.8 seconds 2012-05-14 00:33:15,669 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 00:32:35 2012-05-14 00:33:15,671 INFO repeater task os75bb_lastensq starting 210.7 seconds late 2012-05-14 00:33:25,347 INFO repeater task os75bb_lastensq ended after 9.7 seconds 2012-05-14 00:33:25,348 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 00:29:35 2012-05-14 00:33:25,349 INFO repeater task os75nb_lastensq starting 220.3 seconds late 2012-05-14 00:33:30,205 INFO repeater task os75nb_lastensq ended after 4.9 seconds 2012-05-14 00:33:30,206 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 00:29:35 2012-05-14 00:34:46,108 INFO repeater task wh300_lastensq starting 1.1 seconds late 2012-05-14 00:35:04,660 INFO repeater task wh300_lastensq ended after 18.6 seconds 2012-05-14 00:35:04,661 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 00:34:35 2012-05-14 00:35:04,672 INFO repeater task os75bb_lastensq starting 19.7 seconds late 2012-05-14 00:35:22,842 INFO repeater task os75bb_lastensq ended after 18.2 seconds 2012-05-14 00:35:22,843 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 00:34:35 2012-05-14 00:35:22,844 INFO repeater task os75nb_lastensq starting 37.8 seconds late 2012-05-14 00:35:33,711 INFO repeater task os75nb_lastensq ended after 10.9 seconds 2012-05-14 00:35:33,711 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 00:34:35 2012-05-14 00:36:49,185 INFO repeater task wh300_lastensq starting 4.2 seconds late 2012-05-14 00:36:54,405 INFO repeater task wh300_lastensq ended after 5.2 seconds 2012-05-14 00:36:54,406 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 00:36:35 2012-05-14 00:36:54,413 INFO repeater task wh300_quick starting 0.4 seconds late 2012-05-14 00:39:42,746 INFO repeater task wh300_quick ended after 168.3 seconds 2012-05-14 00:39:42,747 INFO repeater task wh300_3dayplots starting 168.7 seconds late 2012-05-14 00:40:58,523 INFO repeater task wh300_3dayplots ended after 75.8 seconds 2012-05-14 00:40:58,524 INFO repeater task os75bb_lastensq starting 73.5 seconds late 2012-05-14 00:41:08,618 INFO repeater task os75bb_lastensq ended after 10.1 seconds 2012-05-14 00:41:08,619 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 00:39:35 2012-05-14 00:41:08,620 INFO repeater task os75bb_quick starting 254.6 seconds late 2012-05-14 00:43:08,181 INFO repeater task os75bb_quick ended after 119.6 seconds 2012-05-14 00:43:08,182 INFO repeater task os75bb_3dayplots starting 374.2 seconds late 2012-05-14 00:44:06,000 INFO repeater task os75bb_3dayplots ended after 57.8 seconds 2012-05-14 00:44:06,002 INFO repeater task os75nb_lastensq starting 261.0 seconds late 2012-05-14 00:44:16,808 INFO repeater task os75nb_lastensq ended after 10.8 seconds 2012-05-14 00:44:16,809 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 00:39:35 2012-05-14 00:44:16,810 INFO repeater task os75nb_quick starting 442.8 seconds late 2012-05-14 00:46:16,426 INFO repeater task os75nb_quick ended after 119.6 seconds 2012-05-14 00:46:16,433 INFO repeater task os75nb_3dayplots starting 562.4 seconds late 2012-05-14 00:47:25,680 INFO repeater task os75nb_3dayplots ended after 69.2 seconds 2012-05-14 00:47:25,681 INFO repeater task wh300_beamplots starting 331.7 seconds late 2012-05-14 00:47:36,310 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 00:47:36,310 INFO repeater task wh300_beamplots ended after 10.6 seconds 2012-05-14 00:47:36,311 INFO repeater task wh300_beamplots starting 342.3 seconds late 2012-05-14 00:47:46,414 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 00:47:46,415 INFO repeater task wh300_beamplots ended after 10.1 seconds 2012-05-14 00:47:46,416 INFO repeater task os75bb_beamplots starting 352.4 seconds late 2012-05-14 00:47:51,739 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 00:47:51,739 INFO repeater task os75bb_beamplots ended after 5.3 seconds 2012-05-14 00:47:51,740 INFO repeater task os75bb_beamplots starting 357.7 seconds late 2012-05-14 00:47:56,889 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 00:47:56,889 INFO repeater task os75bb_beamplots ended after 5.1 seconds 2012-05-14 00:47:56,890 INFO repeater task os75nb_beamplots starting 362.9 seconds late 2012-05-14 00:48:01,939 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 00:48:01,939 INFO repeater task os75nb_beamplots ended after 5.0 seconds 2012-05-14 00:48:01,940 INFO repeater task os75nb_beamplots starting 367.9 seconds late 2012-05-14 00:48:07,778 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 00:48:07,778 INFO repeater task os75nb_beamplots ended after 5.8 seconds 2012-05-14 00:48:07,779 INFO repeater task phins_hcorrstats starting 673.8 seconds late 2012-05-14 00:48:14,832 INFO repeater task phins_hcorrstats ended after 7.1 seconds 2012-05-14 00:48:14,832 INFO repeater task phins_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 00:48:14,836 INFO repeater task phins2_hcorrstats starting 680.8 seconds late 2012-05-14 00:48:20,105 INFO repeater task phins2_hcorrstats ended after 5.3 seconds 2012-05-14 00:48:20,106 INFO repeater task phins2_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 00:48:25,113 INFO repeater task wh300_lastensq starting 580.1 seconds late 2012-05-14 00:48:33,751 INFO repeater task wh300_lastensq ended after 8.6 seconds 2012-05-14 00:48:33,751 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 00:46:35 2012-05-14 00:48:33,759 INFO repeater task os75bb_lastensq starting 228.8 seconds late 2012-05-14 00:48:43,345 INFO repeater task os75bb_lastensq ended after 9.6 seconds 2012-05-14 00:48:43,345 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 00:44:35 2012-05-14 00:48:43,347 INFO repeater task os75nb_lastensq starting 238.3 seconds late 2012-05-14 00:48:48,005 INFO repeater task os75nb_lastensq ended after 4.7 seconds 2012-05-14 00:48:48,006 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 00:44:35 2012-05-14 00:48:53,019 INFO repeater task wh300_lastensq starting 8.0 seconds late 2012-05-14 00:48:58,232 INFO repeater task wh300_lastensq ended after 5.2 seconds 2012-05-14 00:48:58,233 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 00:48:35 2012-05-14 00:49:48,490 INFO repeater task os75bb_lastensq starting 3.5 seconds late 2012-05-14 00:49:57,857 INFO repeater task os75bb_lastensq ended after 9.4 seconds 2012-05-14 00:49:57,858 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 00:49:35 2012-05-14 00:49:57,860 INFO repeater task os75nb_lastensq starting 12.9 seconds late 2012-05-14 00:50:02,776 INFO repeater task os75nb_lastensq ended after 4.9 seconds 2012-05-14 00:50:02,777 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 00:49:35 2012-05-14 00:50:47,909 INFO repeater task wh300_lastensq starting 2.9 seconds late 2012-05-14 00:50:53,100 INFO repeater task wh300_lastensq ended after 5.2 seconds 2012-05-14 00:50:53,101 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 00:50:35 2012-05-14 00:51:58,369 INFO repeater task wh300_quick starting 4.4 seconds late 2012-05-14 00:54:46,845 INFO repeater task wh300_quick ended after 168.5 seconds 2012-05-14 00:54:46,860 INFO repeater task wh300_3dayplots starting 172.9 seconds late 2012-05-14 00:55:43,262 INFO repeater task wh300_3dayplots ended after 56.4 seconds 2012-05-14 00:55:43,263 INFO repeater task os75bb_lastensq starting 58.3 seconds late 2012-05-14 00:55:53,469 INFO repeater task os75bb_lastensq ended after 10.2 seconds 2012-05-14 00:55:53,469 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 00:54:35 2012-05-14 00:55:53,471 INFO repeater task os75bb_quick starting 239.5 seconds late 2012-05-14 00:57:57,589 INFO repeater task os75bb_quick ended after 124.1 seconds 2012-05-14 00:57:57,593 INFO repeater task os75bb_3dayplots starting 363.6 seconds late 2012-05-14 00:59:12,506 INFO repeater task os75bb_3dayplots ended after 74.9 seconds 2012-05-14 00:59:12,507 INFO repeater task os75nb_lastensq starting 267.5 seconds late 2012-05-14 00:59:17,418 INFO repeater task os75nb_lastensq ended after 4.9 seconds 2012-05-14 00:59:17,419 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 00:54:35 2012-05-14 00:59:17,420 INFO repeater task os75nb_quick starting 443.4 seconds late 2012-05-14 01:00:58,358 INFO repeater task os75nb_quick ended after 100.9 seconds 2012-05-14 01:00:58,359 INFO repeater task os75nb_3dayplots starting 544.4 seconds late 2012-05-14 01:01:53,052 INFO repeater task os75nb_3dayplots ended after 54.7 seconds 2012-05-14 01:01:53,057 INFO repeater task wh300_beamplots starting 599.0 seconds late 2012-05-14 01:02:04,184 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 01:02:04,184 INFO repeater task wh300_beamplots ended after 11.1 seconds 2012-05-14 01:02:04,185 INFO repeater task wh300_beamplots: added 1 extra increments to time_to_run 2012-05-14 01:02:04,189 INFO repeater task wh300_beamplots starting 610.2 seconds late 2012-05-14 01:02:16,660 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 01:02:16,660 INFO repeater task wh300_beamplots ended after 12.5 seconds 2012-05-14 01:02:16,661 INFO repeater task wh300_beamplots: added 1 extra increments to time_to_run 2012-05-14 01:02:16,661 INFO repeater task os75bb_beamplots starting 622.6 seconds late 2012-05-14 01:02:25,562 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 01:02:25,563 INFO repeater task os75bb_beamplots ended after 8.9 seconds 2012-05-14 01:02:25,563 INFO repeater task os75bb_beamplots: added 1 extra increments to time_to_run 2012-05-14 01:02:25,580 INFO repeater task os75bb_beamplots starting 631.6 seconds late 2012-05-14 01:02:34,691 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 01:02:34,692 INFO repeater task os75bb_beamplots ended after 9.1 seconds 2012-05-14 01:02:34,692 INFO repeater task os75bb_beamplots: added 1 extra increments to time_to_run 2012-05-14 01:02:34,698 INFO repeater task os75nb_beamplots starting 640.7 seconds late 2012-05-14 01:02:42,456 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 01:02:42,456 INFO repeater task os75nb_beamplots ended after 7.8 seconds 2012-05-14 01:02:42,457 INFO repeater task os75nb_beamplots: added 1 extra increments to time_to_run 2012-05-14 01:02:42,458 INFO repeater task os75nb_beamplots starting 648.4 seconds late 2012-05-14 01:02:48,201 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 01:02:48,201 INFO repeater task os75nb_beamplots ended after 5.7 seconds 2012-05-14 01:02:48,202 INFO repeater task os75nb_beamplots: added 1 extra increments to time_to_run 2012-05-14 01:02:48,205 INFO repeater task phins_hcorrstats starting 654.2 seconds late 2012-05-14 01:02:54,391 INFO repeater task phins_hcorrstats ended after 6.2 seconds 2012-05-14 01:02:54,392 INFO repeater task phins_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 01:02:54,397 INFO repeater task phins2_hcorrstats starting 660.4 seconds late 2012-05-14 01:02:59,633 INFO repeater task phins2_hcorrstats ended after 5.2 seconds 2012-05-14 01:02:59,634 INFO repeater task phins2_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 01:03:04,637 INFO repeater task wh300_lastensq starting 619.6 seconds late 2012-05-14 01:03:13,024 INFO repeater task wh300_lastensq ended after 8.4 seconds 2012-05-14 01:03:13,025 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 01:02:35 2012-05-14 01:03:13,028 INFO repeater task os75bb_lastensq starting 208.0 seconds late 2012-05-14 01:03:22,887 INFO repeater task os75bb_lastensq ended after 9.9 seconds 2012-05-14 01:03:22,887 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 00:59:35 2012-05-14 01:03:22,890 INFO repeater task os75nb_lastensq starting 217.9 seconds late 2012-05-14 01:03:27,760 INFO repeater task os75nb_lastensq ended after 4.9 seconds 2012-05-14 01:03:27,761 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 00:59:35 2012-05-14 01:04:47,973 INFO repeater task wh300_lastensq starting 3.0 seconds late 2012-05-14 01:04:53,188 INFO repeater task wh300_lastensq ended after 5.2 seconds 2012-05-14 01:04:53,190 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 01:04:35 2012-05-14 01:04:53,192 INFO repeater task os75bb_lastensq starting 8.2 seconds late 2012-05-14 01:05:02,399 INFO repeater task os75bb_lastensq ended after 9.2 seconds 2012-05-14 01:05:02,399 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 01:04:35 2012-05-14 01:05:02,407 INFO repeater task os75nb_lastensq starting 17.4 seconds late 2012-05-14 01:05:06,885 INFO repeater task os75nb_lastensq ended after 4.5 seconds 2012-05-14 01:05:06,886 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 01:04:35 2012-05-14 01:06:47,557 INFO repeater task wh300_lastensq starting 2.6 seconds late 2012-05-14 01:06:52,472 INFO repeater task wh300_lastensq ended after 4.9 seconds 2012-05-14 01:06:52,473 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 01:06:35 2012-05-14 01:06:57,518 INFO repeater task wh300_quick starting 3.5 seconds late 2012-05-14 01:09:47,890 INFO repeater task wh300_quick ended after 170.4 seconds 2012-05-14 01:09:47,891 INFO repeater task wh300_3dayplots starting 173.9 seconds late 2012-05-14 01:11:07,148 INFO repeater task wh300_3dayplots ended after 79.3 seconds 2012-05-14 01:11:07,149 INFO repeater task os75bb_lastensq starting 82.1 seconds late 2012-05-14 01:11:19,628 INFO repeater task os75bb_lastensq ended after 12.5 seconds 2012-05-14 01:11:19,629 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 01:09:35 2012-05-14 01:11:19,630 INFO repeater task os75bb_quick starting 265.6 seconds late 2012-05-14 01:13:14,904 INFO repeater task os75bb_quick ended after 115.3 seconds 2012-05-14 01:13:14,907 INFO repeater task os75bb_3dayplots starting 380.9 seconds late 2012-05-14 01:14:15,216 INFO repeater task os75bb_3dayplots ended after 60.3 seconds 2012-05-14 01:14:15,218 INFO repeater task os75nb_lastensq starting 270.2 seconds late 2012-05-14 01:14:29,224 INFO repeater task os75nb_lastensq ended after 14.0 seconds 2012-05-14 01:14:29,225 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 01:09:35 2012-05-14 01:14:29,226 INFO repeater task os75nb_quick starting 455.2 seconds late 2012-05-14 01:16:29,067 INFO repeater task os75nb_quick ended after 119.8 seconds 2012-05-14 01:16:29,068 INFO repeater task os75nb_3dayplots starting 575.1 seconds late 2012-05-14 01:17:26,384 INFO repeater task os75nb_3dayplots ended after 57.3 seconds 2012-05-14 01:17:26,385 INFO repeater task wh300_beamplots starting 332.4 seconds late 2012-05-14 01:17:36,876 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 01:17:36,877 INFO repeater task wh300_beamplots ended after 10.5 seconds 2012-05-14 01:17:36,878 INFO repeater task wh300_beamplots starting 342.9 seconds late 2012-05-14 01:17:46,943 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 01:17:46,944 INFO repeater task wh300_beamplots ended after 10.1 seconds 2012-05-14 01:17:46,945 INFO repeater task os75bb_beamplots starting 352.9 seconds late 2012-05-14 01:17:52,214 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 01:17:52,214 INFO repeater task os75bb_beamplots ended after 5.3 seconds 2012-05-14 01:17:52,215 INFO repeater task os75bb_beamplots starting 358.2 seconds late 2012-05-14 01:17:57,384 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 01:17:57,384 INFO repeater task os75bb_beamplots ended after 5.2 seconds 2012-05-14 01:17:57,388 INFO repeater task os75nb_beamplots starting 363.4 seconds late 2012-05-14 01:18:02,372 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 01:18:02,373 INFO repeater task os75nb_beamplots ended after 5.0 seconds 2012-05-14 01:18:02,377 INFO repeater task os75nb_beamplots starting 368.4 seconds late 2012-05-14 01:18:08,525 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 01:18:08,526 INFO repeater task os75nb_beamplots ended after 6.1 seconds 2012-05-14 01:18:08,529 INFO repeater task phins_hcorrstats starting 674.5 seconds late 2012-05-14 01:18:18,101 INFO repeater task phins_hcorrstats ended after 9.6 seconds 2012-05-14 01:18:18,101 INFO repeater task phins_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 01:18:18,104 INFO repeater task phins2_hcorrstats starting 684.1 seconds late 2012-05-14 01:18:26,740 INFO repeater task phins2_hcorrstats ended after 8.6 seconds 2012-05-14 01:18:26,740 INFO repeater task phins2_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 01:18:31,746 INFO repeater task wh300_lastensq starting 586.7 seconds late 2012-05-14 01:18:41,204 INFO repeater task wh300_lastensq ended after 9.5 seconds 2012-05-14 01:18:41,205 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 01:16:35 2012-05-14 01:18:41,206 INFO repeater task os75bb_lastensq starting 236.2 seconds late 2012-05-14 01:18:50,621 INFO repeater task os75bb_lastensq ended after 9.4 seconds 2012-05-14 01:18:50,622 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 01:14:35 2012-05-14 01:18:50,627 INFO repeater task os75nb_lastensq starting 245.6 seconds late 2012-05-14 01:18:55,419 INFO repeater task os75nb_lastensq ended after 4.8 seconds 2012-05-14 01:18:55,420 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 01:14:35 2012-05-14 01:19:00,429 INFO repeater task wh300_lastensq starting 15.4 seconds late 2012-05-14 01:19:05,850 INFO repeater task wh300_lastensq ended after 5.4 seconds 2012-05-14 01:19:05,850 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 01:18:35 2012-05-14 01:19:46,029 INFO repeater task os75bb_lastensq starting 1.0 seconds late 2012-05-14 01:19:55,357 INFO repeater task os75bb_lastensq ended after 9.3 seconds 2012-05-14 01:19:55,358 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 01:19:35 2012-05-14 01:19:55,365 INFO repeater task os75nb_lastensq starting 10.4 seconds late 2012-05-14 01:20:00,171 INFO repeater task os75nb_lastensq ended after 4.8 seconds 2012-05-14 01:20:00,172 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 01:19:35 2012-05-14 01:20:45,513 INFO repeater task wh300_lastensq starting 0.5 seconds late 2012-05-14 01:20:50,900 INFO repeater task wh300_lastensq ended after 5.4 seconds 2012-05-14 01:20:50,901 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 01:20:35 2012-05-14 01:21:56,229 INFO repeater task wh300_quick starting 2.2 seconds late 2012-05-14 01:24:50,796 INFO repeater task wh300_quick ended after 174.6 seconds 2012-05-14 01:24:50,801 INFO repeater task wh300_3dayplots starting 176.8 seconds late 2012-05-14 01:25:46,772 INFO repeater task wh300_3dayplots ended after 56.0 seconds 2012-05-14 01:25:46,776 INFO repeater task os75bb_lastensq starting 61.8 seconds late 2012-05-14 01:25:57,569 INFO repeater task os75bb_lastensq ended after 10.8 seconds 2012-05-14 01:25:57,570 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 01:24:35 2012-05-14 01:25:57,571 INFO repeater task os75bb_quick starting 243.6 seconds late 2012-05-14 01:28:09,868 INFO repeater task os75bb_quick ended after 132.3 seconds 2012-05-14 01:28:09,879 INFO repeater task os75bb_3dayplots starting 375.9 seconds late 2012-05-14 01:29:25,872 INFO repeater task os75bb_3dayplots ended after 76.0 seconds 2012-05-14 01:29:25,874 INFO repeater task os75nb_lastensq starting 280.9 seconds late 2012-05-14 01:29:32,755 INFO repeater task os75nb_lastensq ended after 6.9 seconds 2012-05-14 01:29:32,756 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 01:24:35 2012-05-14 01:29:32,757 INFO repeater task os75nb_quick starting 458.7 seconds late 2012-05-14 01:31:26,986 INFO repeater task os75nb_quick ended after 114.2 seconds 2012-05-14 01:31:26,987 INFO repeater task os75nb_3dayplots starting 573.0 seconds late 2012-05-14 01:32:31,076 INFO repeater task os75nb_3dayplots ended after 64.1 seconds 2012-05-14 01:32:31,081 INFO repeater task wh300_beamplots starting 637.1 seconds late 2012-05-14 01:32:46,831 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 01:32:46,832 INFO repeater task wh300_beamplots ended after 15.8 seconds 2012-05-14 01:32:46,832 INFO repeater task wh300_beamplots: added 1 extra increments to time_to_run 2012-05-14 01:32:46,837 INFO repeater task wh300_beamplots starting 652.8 seconds late 2012-05-14 01:32:58,439 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 01:32:58,439 INFO repeater task wh300_beamplots ended after 11.6 seconds 2012-05-14 01:32:58,440 INFO repeater task wh300_beamplots: added 1 extra increments to time_to_run 2012-05-14 01:32:58,440 INFO repeater task os75bb_beamplots starting 664.4 seconds late 2012-05-14 01:33:03,914 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 01:33:03,915 INFO repeater task os75bb_beamplots ended after 5.5 seconds 2012-05-14 01:33:03,915 INFO repeater task os75bb_beamplots: added 1 extra increments to time_to_run 2012-05-14 01:33:03,917 INFO repeater task os75bb_beamplots starting 669.9 seconds late 2012-05-14 01:33:09,210 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 01:33:09,211 INFO repeater task os75bb_beamplots ended after 5.3 seconds 2012-05-14 01:33:09,211 INFO repeater task os75bb_beamplots: added 1 extra increments to time_to_run 2012-05-14 01:33:09,212 INFO repeater task os75nb_beamplots starting 675.2 seconds late 2012-05-14 01:33:14,099 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 01:33:14,100 INFO repeater task os75nb_beamplots ended after 4.9 seconds 2012-05-14 01:33:14,100 INFO repeater task os75nb_beamplots: added 1 extra increments to time_to_run 2012-05-14 01:33:14,104 INFO repeater task os75nb_beamplots starting 680.1 seconds late 2012-05-14 01:33:18,972 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 01:33:18,972 INFO repeater task os75nb_beamplots ended after 4.9 seconds 2012-05-14 01:33:18,973 INFO repeater task os75nb_beamplots: added 1 extra increments to time_to_run 2012-05-14 01:33:18,974 INFO repeater task phins_hcorrstats starting 685.0 seconds late 2012-05-14 01:33:25,063 INFO repeater task phins_hcorrstats ended after 6.1 seconds 2012-05-14 01:33:25,063 INFO repeater task phins_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 01:33:25,066 INFO repeater task phins2_hcorrstats starting 691.0 seconds late 2012-05-14 01:33:30,385 INFO repeater task phins2_hcorrstats ended after 5.3 seconds 2012-05-14 01:33:30,385 INFO repeater task phins2_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 01:33:35,389 INFO repeater task wh300_lastensq starting 650.4 seconds late 2012-05-14 01:33:42,911 INFO repeater task wh300_lastensq ended after 7.5 seconds 2012-05-14 01:33:42,911 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 01:32:35 2012-05-14 01:33:42,918 INFO repeater task os75bb_lastensq starting 237.9 seconds late 2012-05-14 01:33:52,722 INFO repeater task os75bb_lastensq ended after 9.8 seconds 2012-05-14 01:33:52,723 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 01:29:35 2012-05-14 01:33:52,725 INFO repeater task os75nb_lastensq starting 247.7 seconds late 2012-05-14 01:33:57,638 INFO repeater task os75nb_lastensq ended after 4.9 seconds 2012-05-14 01:33:57,639 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 01:29:35 2012-05-14 01:34:48,121 INFO repeater task wh300_lastensq starting 3.1 seconds late 2012-05-14 01:35:11,528 INFO repeater task wh300_lastensq ended after 23.4 seconds 2012-05-14 01:35:11,529 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 01:34:35 2012-05-14 01:35:11,537 INFO repeater task os75bb_lastensq starting 26.5 seconds late 2012-05-14 01:35:24,835 INFO repeater task os75bb_lastensq ended after 13.3 seconds 2012-05-14 01:35:24,836 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 01:34:35 2012-05-14 01:35:24,845 INFO repeater task os75nb_lastensq starting 39.8 seconds late 2012-05-14 01:35:38,431 INFO repeater task os75nb_lastensq ended after 13.6 seconds 2012-05-14 01:35:38,432 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 01:34:35 2012-05-14 01:36:48,761 INFO repeater task wh300_lastensq starting 3.8 seconds late 2012-05-14 01:36:54,795 INFO repeater task wh300_lastensq ended after 6.0 seconds 2012-05-14 01:36:54,795 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 01:36:35 2012-05-14 01:36:54,796 INFO repeater task wh300_quick starting 0.8 seconds late 2012-05-14 01:39:46,364 INFO repeater task wh300_quick ended after 171.6 seconds 2012-05-14 01:39:46,365 INFO repeater task wh300_3dayplots starting 172.4 seconds late 2012-05-14 01:41:00,976 INFO repeater task wh300_3dayplots ended after 74.6 seconds 2012-05-14 01:41:00,977 INFO repeater task os75bb_lastensq starting 76.0 seconds late 2012-05-14 01:41:11,075 INFO repeater task os75bb_lastensq ended after 10.1 seconds 2012-05-14 01:41:11,075 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 01:39:35 2012-05-14 01:41:11,077 INFO repeater task os75bb_quick starting 257.1 seconds late 2012-05-14 01:43:10,485 INFO repeater task os75bb_quick ended after 119.4 seconds 2012-05-14 01:43:10,486 INFO repeater task os75bb_3dayplots starting 376.5 seconds late 2012-05-14 01:44:10,878 INFO repeater task os75bb_3dayplots ended after 60.4 seconds 2012-05-14 01:44:10,879 INFO repeater task os75nb_lastensq starting 265.9 seconds late 2012-05-14 01:44:26,843 INFO repeater task os75nb_lastensq ended after 16.0 seconds 2012-05-14 01:44:26,844 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 01:39:35 2012-05-14 01:44:26,847 INFO repeater task os75nb_quick starting 452.8 seconds late 2012-05-14 01:46:21,822 INFO repeater task os75nb_quick ended after 115.0 seconds 2012-05-14 01:46:21,828 INFO repeater task os75nb_3dayplots starting 567.8 seconds late 2012-05-14 01:47:21,240 INFO repeater task os75nb_3dayplots ended after 59.4 seconds 2012-05-14 01:47:21,241 INFO repeater task wh300_beamplots starting 327.2 seconds late 2012-05-14 01:47:31,957 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 01:47:31,957 INFO repeater task wh300_beamplots ended after 10.7 seconds 2012-05-14 01:47:31,959 INFO repeater task wh300_beamplots starting 337.9 seconds late 2012-05-14 01:47:42,041 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 01:47:42,042 INFO repeater task wh300_beamplots ended after 10.1 seconds 2012-05-14 01:47:42,043 INFO repeater task os75bb_beamplots starting 348.0 seconds late 2012-05-14 01:47:47,434 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 01:47:47,434 INFO repeater task os75bb_beamplots ended after 5.4 seconds 2012-05-14 01:47:47,435 INFO repeater task os75bb_beamplots starting 353.4 seconds late 2012-05-14 01:47:52,599 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 01:47:52,599 INFO repeater task os75bb_beamplots ended after 5.2 seconds 2012-05-14 01:47:52,600 INFO repeater task os75nb_beamplots starting 358.6 seconds late 2012-05-14 01:47:57,690 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 01:47:57,691 INFO repeater task os75nb_beamplots ended after 5.1 seconds 2012-05-14 01:47:57,692 INFO repeater task os75nb_beamplots starting 363.7 seconds late 2012-05-14 01:48:02,748 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 01:48:02,749 INFO repeater task os75nb_beamplots ended after 5.1 seconds 2012-05-14 01:48:02,750 INFO repeater task phins_hcorrstats starting 668.7 seconds late 2012-05-14 01:48:10,419 INFO repeater task phins_hcorrstats ended after 7.7 seconds 2012-05-14 01:48:10,419 INFO repeater task phins_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 01:48:10,423 INFO repeater task phins2_hcorrstats starting 676.4 seconds late 2012-05-14 01:48:19,623 INFO repeater task phins2_hcorrstats ended after 9.2 seconds 2012-05-14 01:48:19,624 INFO repeater task phins2_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 01:48:24,629 INFO repeater task wh300_lastensq starting 579.6 seconds late 2012-05-14 01:48:34,018 INFO repeater task wh300_lastensq ended after 9.4 seconds 2012-05-14 01:48:34,019 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 01:46:35 2012-05-14 01:48:34,021 INFO repeater task os75bb_lastensq starting 229.0 seconds late 2012-05-14 01:48:43,738 INFO repeater task os75bb_lastensq ended after 9.7 seconds 2012-05-14 01:48:43,739 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 01:44:35 2012-05-14 01:48:43,740 INFO repeater task os75nb_lastensq starting 238.7 seconds late 2012-05-14 01:48:48,718 INFO repeater task os75nb_lastensq ended after 5.0 seconds 2012-05-14 01:48:48,718 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 01:44:35 2012-05-14 01:48:53,729 INFO repeater task wh300_lastensq starting 8.7 seconds late 2012-05-14 01:48:59,219 INFO repeater task wh300_lastensq ended after 5.5 seconds 2012-05-14 01:48:59,219 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 01:48:35 2012-05-14 01:49:49,362 INFO repeater task os75bb_lastensq starting 4.4 seconds late 2012-05-14 01:49:58,936 INFO repeater task os75bb_lastensq ended after 9.6 seconds 2012-05-14 01:49:58,937 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 01:49:35 2012-05-14 01:49:58,939 INFO repeater task os75nb_lastensq starting 13.9 seconds late 2012-05-14 01:50:04,027 INFO repeater task os75nb_lastensq ended after 5.1 seconds 2012-05-14 01:50:04,028 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 01:49:35 2012-05-14 01:50:49,173 INFO repeater task wh300_lastensq starting 4.2 seconds late 2012-05-14 01:50:54,498 INFO repeater task wh300_lastensq ended after 5.3 seconds 2012-05-14 01:50:54,498 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 01:50:35 2012-05-14 01:51:54,725 INFO repeater task wh300_quick starting 0.7 seconds late 2012-05-14 01:54:45,773 INFO repeater task wh300_quick ended after 171.0 seconds 2012-05-14 01:54:45,774 INFO repeater task wh300_3dayplots starting 171.8 seconds late 2012-05-14 01:55:44,297 INFO repeater task wh300_3dayplots ended after 58.5 seconds 2012-05-14 01:55:44,298 INFO repeater task os75bb_lastensq starting 59.3 seconds late 2012-05-14 01:55:55,961 INFO repeater task os75bb_lastensq ended after 11.7 seconds 2012-05-14 01:55:55,962 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 01:54:35 2012-05-14 01:55:55,963 INFO repeater task os75bb_quick starting 242.0 seconds late 2012-05-14 01:58:02,500 INFO repeater task os75bb_quick ended after 126.5 seconds 2012-05-14 01:58:02,508 INFO repeater task os75bb_3dayplots starting 368.5 seconds late 2012-05-14 01:59:17,560 INFO repeater task os75bb_3dayplots ended after 75.1 seconds 2012-05-14 01:59:17,565 INFO repeater task os75nb_lastensq starting 272.6 seconds late 2012-05-14 01:59:25,197 INFO repeater task os75nb_lastensq ended after 7.6 seconds 2012-05-14 01:59:25,198 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 01:54:35 2012-05-14 01:59:25,198 INFO repeater task os75nb_quick starting 451.2 seconds late 2012-05-14 02:01:11,880 INFO repeater task os75nb_quick ended after 106.7 seconds 2012-05-14 02:01:11,881 INFO repeater task os75nb_3dayplots starting 557.9 seconds late 2012-05-14 02:02:06,314 INFO repeater task os75nb_3dayplots ended after 54.4 seconds 2012-05-14 02:02:06,315 INFO repeater task wh300_beamplots starting 612.3 seconds late 2012-05-14 02:02:21,829 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 02:02:21,829 INFO repeater task wh300_beamplots ended after 15.5 seconds 2012-05-14 02:02:21,830 INFO repeater task wh300_beamplots: added 1 extra increments to time_to_run 2012-05-14 02:02:21,834 INFO repeater task wh300_beamplots starting 627.8 seconds late 2012-05-14 02:02:39,021 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 02:02:39,022 INFO repeater task wh300_beamplots ended after 17.2 seconds 2012-05-14 02:02:39,022 INFO repeater task wh300_beamplots: added 1 extra increments to time_to_run 2012-05-14 02:02:39,023 INFO repeater task os75bb_beamplots starting 645.0 seconds late 2012-05-14 02:02:45,457 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 02:02:45,457 INFO repeater task os75bb_beamplots ended after 6.4 seconds 2012-05-14 02:02:45,457 INFO repeater task os75bb_beamplots: added 1 extra increments to time_to_run 2012-05-14 02:02:45,458 INFO repeater task os75bb_beamplots starting 651.4 seconds late 2012-05-14 02:02:51,087 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 02:02:51,087 INFO repeater task os75bb_beamplots ended after 5.6 seconds 2012-05-14 02:02:51,088 INFO repeater task os75bb_beamplots: added 1 extra increments to time_to_run 2012-05-14 02:02:51,093 INFO repeater task os75nb_beamplots starting 657.1 seconds late 2012-05-14 02:02:56,000 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 02:02:56,001 INFO repeater task os75nb_beamplots ended after 4.9 seconds 2012-05-14 02:02:56,001 INFO repeater task os75nb_beamplots: added 1 extra increments to time_to_run 2012-05-14 02:02:56,005 INFO repeater task os75nb_beamplots starting 662.0 seconds late 2012-05-14 02:03:00,878 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 02:03:00,878 INFO repeater task os75nb_beamplots ended after 4.9 seconds 2012-05-14 02:03:00,878 INFO repeater task os75nb_beamplots: added 1 extra increments to time_to_run 2012-05-14 02:03:00,879 INFO repeater task phins_hcorrstats starting 666.9 seconds late 2012-05-14 02:03:07,373 INFO repeater task phins_hcorrstats ended after 6.5 seconds 2012-05-14 02:03:07,373 INFO repeater task phins_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 02:03:07,374 INFO repeater task phins2_hcorrstats starting 673.4 seconds late 2012-05-14 02:03:12,971 INFO repeater task phins2_hcorrstats ended after 5.6 seconds 2012-05-14 02:03:12,972 INFO repeater task phins2_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 02:03:17,980 INFO repeater task wh300_lastensq starting 633.0 seconds late 2012-05-14 02:03:26,062 INFO repeater task wh300_lastensq ended after 8.1 seconds 2012-05-14 02:03:26,063 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 02:02:35 2012-05-14 02:03:26,066 INFO repeater task os75bb_lastensq starting 221.1 seconds late 2012-05-14 02:03:35,900 INFO repeater task os75bb_lastensq ended after 9.8 seconds 2012-05-14 02:03:35,901 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 01:59:35 2012-05-14 02:03:35,903 INFO repeater task os75nb_lastensq starting 230.9 seconds late 2012-05-14 02:03:40,511 INFO repeater task os75nb_lastensq ended after 4.6 seconds 2012-05-14 02:03:40,512 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 01:59:35 2012-05-14 02:04:45,741 INFO repeater task wh300_lastensq starting 0.7 seconds late 2012-05-14 02:04:50,816 INFO repeater task wh300_lastensq ended after 5.1 seconds 2012-05-14 02:04:50,817 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 02:04:35 2012-05-14 02:04:50,818 INFO repeater task os75bb_lastensq starting 5.8 seconds late 2012-05-14 02:05:00,230 INFO repeater task os75bb_lastensq ended after 9.4 seconds 2012-05-14 02:05:00,231 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 02:04:35 2012-05-14 02:05:00,233 INFO repeater task os75nb_lastensq starting 15.2 seconds late 2012-05-14 02:05:04,924 INFO repeater task os75nb_lastensq ended after 4.7 seconds 2012-05-14 02:05:04,925 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 02:04:35 2012-05-14 02:06:45,273 INFO repeater task wh300_lastensq starting 0.3 seconds late 2012-05-14 02:06:50,197 INFO repeater task wh300_lastensq ended after 4.9 seconds 2012-05-14 02:06:50,198 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 02:06:35 2012-05-14 02:06:55,249 INFO repeater task wh300_quick starting 1.2 seconds late 2012-05-14 02:09:44,518 INFO repeater task wh300_quick ended after 169.3 seconds 2012-05-14 02:09:44,533 INFO repeater task wh300_3dayplots starting 170.5 seconds late 2012-05-14 02:11:02,342 INFO repeater task wh300_3dayplots ended after 77.8 seconds 2012-05-14 02:11:02,343 INFO repeater task os75bb_lastensq starting 77.3 seconds late 2012-05-14 02:11:13,320 INFO repeater task os75bb_lastensq ended after 11.0 seconds 2012-05-14 02:11:13,320 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 02:09:35 2012-05-14 02:11:13,321 INFO repeater task os75bb_quick starting 259.3 seconds late 2012-05-14 02:13:12,703 INFO repeater task os75bb_quick ended after 119.4 seconds 2012-05-14 02:13:12,705 INFO repeater task os75bb_3dayplots starting 378.7 seconds late 2012-05-14 02:14:13,087 INFO repeater task os75bb_3dayplots ended after 60.4 seconds 2012-05-14 02:14:13,091 INFO repeater task os75nb_lastensq starting 268.1 seconds late 2012-05-14 02:14:27,712 INFO repeater task os75nb_lastensq ended after 14.6 seconds 2012-05-14 02:14:27,713 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 02:09:35 2012-05-14 02:14:27,714 INFO repeater task os75nb_quick starting 453.7 seconds late 2012-05-14 02:16:24,824 INFO repeater task os75nb_quick ended after 117.1 seconds 2012-05-14 02:16:24,825 INFO repeater task os75nb_3dayplots starting 570.8 seconds late 2012-05-14 02:17:26,881 INFO repeater task os75nb_3dayplots ended after 62.1 seconds 2012-05-14 02:17:26,882 INFO repeater task wh300_beamplots starting 332.9 seconds late 2012-05-14 02:17:37,501 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 02:17:37,501 INFO repeater task wh300_beamplots ended after 10.6 seconds 2012-05-14 02:17:37,502 INFO repeater task wh300_beamplots starting 343.5 seconds late 2012-05-14 02:17:47,711 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 02:17:47,711 INFO repeater task wh300_beamplots ended after 10.2 seconds 2012-05-14 02:17:47,712 INFO repeater task os75bb_beamplots starting 353.7 seconds late 2012-05-14 02:17:53,029 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 02:17:53,029 INFO repeater task os75bb_beamplots ended after 5.3 seconds 2012-05-14 02:17:53,030 INFO repeater task os75bb_beamplots starting 359.0 seconds late 2012-05-14 02:17:58,271 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 02:17:58,272 INFO repeater task os75bb_beamplots ended after 5.2 seconds 2012-05-14 02:17:58,272 INFO repeater task os75nb_beamplots starting 364.3 seconds late 2012-05-14 02:18:03,395 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 02:18:03,396 INFO repeater task os75nb_beamplots ended after 5.1 seconds 2012-05-14 02:18:03,397 INFO repeater task os75nb_beamplots starting 369.4 seconds late 2012-05-14 02:18:10,718 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 02:18:10,718 INFO repeater task os75nb_beamplots ended after 7.3 seconds 2012-05-14 02:18:10,723 INFO repeater task phins_hcorrstats starting 676.7 seconds late 2012-05-14 02:18:21,204 INFO repeater task phins_hcorrstats ended after 10.5 seconds 2012-05-14 02:18:21,205 INFO repeater task phins_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 02:18:21,211 INFO repeater task phins2_hcorrstats starting 687.2 seconds late 2012-05-14 02:18:28,471 INFO repeater task phins2_hcorrstats ended after 7.3 seconds 2012-05-14 02:18:28,472 INFO repeater task phins2_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 02:18:33,484 INFO repeater task wh300_lastensq starting 588.5 seconds late 2012-05-14 02:18:43,077 INFO repeater task wh300_lastensq ended after 9.6 seconds 2012-05-14 02:18:43,078 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 02:16:35 2012-05-14 02:18:43,080 INFO repeater task os75bb_lastensq starting 238.1 seconds late 2012-05-14 02:18:52,360 INFO repeater task os75bb_lastensq ended after 9.3 seconds 2012-05-14 02:18:52,361 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 02:14:35 2012-05-14 02:18:52,366 INFO repeater task os75nb_lastensq starting 247.4 seconds late 2012-05-14 02:18:57,163 INFO repeater task os75nb_lastensq ended after 4.8 seconds 2012-05-14 02:18:57,164 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 02:14:35 2012-05-14 02:19:02,173 INFO repeater task wh300_lastensq starting 17.2 seconds late 2012-05-14 02:19:07,349 INFO repeater task wh300_lastensq ended after 5.2 seconds 2012-05-14 02:19:07,349 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 02:18:35 2012-05-14 02:19:47,474 INFO repeater task os75bb_lastensq starting 2.5 seconds late 2012-05-14 02:19:56,827 INFO repeater task os75bb_lastensq ended after 9.4 seconds 2012-05-14 02:19:56,828 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 02:19:35 2012-05-14 02:19:56,830 INFO repeater task os75nb_lastensq starting 11.8 seconds late 2012-05-14 02:20:01,485 INFO repeater task os75nb_lastensq ended after 4.7 seconds 2012-05-14 02:20:01,486 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 02:19:35 2012-05-14 02:20:46,817 INFO repeater task wh300_lastensq starting 1.8 seconds late 2012-05-14 02:20:52,002 INFO repeater task wh300_lastensq ended after 5.2 seconds 2012-05-14 02:20:52,003 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 02:20:35 2012-05-14 02:21:57,593 INFO repeater task wh300_quick starting 3.6 seconds late 2012-05-14 02:24:52,739 INFO repeater task wh300_quick ended after 175.1 seconds 2012-05-14 02:24:52,741 INFO repeater task wh300_3dayplots starting 178.7 seconds late 2012-05-14 02:25:49,591 INFO repeater task wh300_3dayplots ended after 56.9 seconds 2012-05-14 02:25:49,592 INFO repeater task os75bb_lastensq starting 64.6 seconds late 2012-05-14 02:26:00,509 INFO repeater task os75bb_lastensq ended after 10.9 seconds 2012-05-14 02:26:00,510 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 02:24:35 2012-05-14 02:26:00,515 INFO repeater task os75bb_quick starting 246.5 seconds late 2012-05-14 02:28:06,003 INFO repeater task os75bb_quick ended after 125.5 seconds 2012-05-14 02:28:06,016 INFO repeater task os75bb_3dayplots starting 372.0 seconds late 2012-05-14 02:29:22,377 INFO repeater task os75bb_3dayplots ended after 76.4 seconds 2012-05-14 02:29:22,378 INFO repeater task os75nb_lastensq starting 277.4 seconds late 2012-05-14 02:29:28,745 INFO repeater task os75nb_lastensq ended after 6.4 seconds 2012-05-14 02:29:28,745 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 02:24:35 2012-05-14 02:29:28,746 INFO repeater task os75nb_quick starting 454.7 seconds late 2012-05-14 02:31:14,894 INFO repeater task os75nb_quick ended after 106.1 seconds 2012-05-14 02:31:14,895 INFO repeater task os75nb_3dayplots starting 560.9 seconds late 2012-05-14 02:32:08,669 INFO repeater task os75nb_3dayplots ended after 53.8 seconds 2012-05-14 02:32:08,671 INFO repeater task wh300_beamplots starting 614.7 seconds late 2012-05-14 02:32:23,379 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 02:32:23,379 INFO repeater task wh300_beamplots ended after 14.7 seconds 2012-05-14 02:32:23,379 INFO repeater task wh300_beamplots: added 1 extra increments to time_to_run 2012-05-14 02:32:23,381 INFO repeater task wh300_beamplots starting 629.4 seconds late 2012-05-14 02:32:40,234 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 02:32:40,234 INFO repeater task wh300_beamplots ended after 16.9 seconds 2012-05-14 02:32:40,235 INFO repeater task wh300_beamplots: added 1 extra increments to time_to_run 2012-05-14 02:32:40,241 INFO repeater task os75bb_beamplots starting 646.2 seconds late 2012-05-14 02:32:48,064 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 02:32:48,064 INFO repeater task os75bb_beamplots ended after 7.8 seconds 2012-05-14 02:32:48,064 INFO repeater task os75bb_beamplots: added 1 extra increments to time_to_run 2012-05-14 02:32:48,069 INFO repeater task os75bb_beamplots starting 654.1 seconds late 2012-05-14 02:32:53,684 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 02:32:53,685 INFO repeater task os75bb_beamplots ended after 5.6 seconds 2012-05-14 02:32:53,685 INFO repeater task os75bb_beamplots: added 1 extra increments to time_to_run 2012-05-14 02:32:53,689 INFO repeater task os75nb_beamplots starting 659.7 seconds late 2012-05-14 02:32:58,618 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 02:32:58,618 INFO repeater task os75nb_beamplots ended after 4.9 seconds 2012-05-14 02:32:58,619 INFO repeater task os75nb_beamplots: added 1 extra increments to time_to_run 2012-05-14 02:32:58,633 INFO repeater task os75nb_beamplots starting 664.6 seconds late 2012-05-14 02:33:03,609 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 02:33:03,610 INFO repeater task os75nb_beamplots ended after 5.0 seconds 2012-05-14 02:33:03,610 INFO repeater task os75nb_beamplots: added 1 extra increments to time_to_run 2012-05-14 02:33:03,611 INFO repeater task phins_hcorrstats starting 669.6 seconds late 2012-05-14 02:33:09,769 INFO repeater task phins_hcorrstats ended after 6.2 seconds 2012-05-14 02:33:09,770 INFO repeater task phins_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 02:33:09,772 INFO repeater task phins2_hcorrstats starting 675.7 seconds late 2012-05-14 02:33:15,377 INFO repeater task phins2_hcorrstats ended after 5.6 seconds 2012-05-14 02:33:15,377 INFO repeater task phins2_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 02:33:20,385 INFO repeater task wh300_lastensq starting 635.4 seconds late 2012-05-14 02:33:28,197 INFO repeater task wh300_lastensq ended after 7.8 seconds 2012-05-14 02:33:28,198 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 02:32:35 2012-05-14 02:33:28,200 INFO repeater task os75bb_lastensq starting 223.2 seconds late 2012-05-14 02:33:37,752 INFO repeater task os75bb_lastensq ended after 9.6 seconds 2012-05-14 02:33:37,753 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 02:29:35 2012-05-14 02:33:37,755 INFO repeater task os75nb_lastensq starting 232.8 seconds late 2012-05-14 02:33:42,609 INFO repeater task os75nb_lastensq ended after 4.9 seconds 2012-05-14 02:33:42,610 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 02:29:35 2012-05-14 02:34:47,968 INFO repeater task wh300_lastensq starting 3.0 seconds late 2012-05-14 02:35:11,503 INFO repeater task wh300_lastensq ended after 23.5 seconds 2012-05-14 02:35:11,504 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 02:34:35 2012-05-14 02:35:11,516 INFO repeater task os75bb_lastensq starting 26.5 seconds late 2012-05-14 02:35:31,266 INFO repeater task os75bb_lastensq ended after 19.8 seconds 2012-05-14 02:35:31,267 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 02:34:35 2012-05-14 02:35:31,279 INFO repeater task os75nb_lastensq starting 46.3 seconds late 2012-05-14 02:35:44,345 INFO repeater task os75nb_lastensq ended after 13.1 seconds 2012-05-14 02:35:44,346 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 02:34:35 2012-05-14 02:36:49,777 INFO repeater task wh300_lastensq starting 4.8 seconds late 2012-05-14 02:36:54,987 INFO repeater task wh300_lastensq ended after 5.2 seconds 2012-05-14 02:36:54,988 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 02:36:35 2012-05-14 02:36:54,989 INFO repeater task wh300_quick starting 1.0 seconds late 2012-05-14 02:39:48,043 INFO repeater task wh300_quick ended after 173.1 seconds 2012-05-14 02:39:48,044 INFO repeater task wh300_3dayplots starting 174.0 seconds late 2012-05-14 02:41:04,006 INFO repeater task wh300_3dayplots ended after 76.0 seconds 2012-05-14 02:41:04,007 INFO repeater task os75bb_lastensq starting 79.0 seconds late 2012-05-14 02:41:13,623 INFO repeater task os75bb_lastensq ended after 9.6 seconds 2012-05-14 02:41:13,623 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 02:39:35 2012-05-14 02:41:13,624 INFO repeater task os75bb_quick starting 259.6 seconds late 2012-05-14 02:43:15,312 INFO repeater task os75bb_quick ended after 121.7 seconds 2012-05-14 02:43:15,313 INFO repeater task os75bb_3dayplots starting 381.3 seconds late 2012-05-14 02:44:15,583 INFO repeater task os75bb_3dayplots ended after 60.3 seconds 2012-05-14 02:44:15,589 INFO repeater task os75nb_lastensq starting 270.6 seconds late 2012-05-14 02:44:26,388 INFO repeater task os75nb_lastensq ended after 10.8 seconds 2012-05-14 02:44:26,389 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 02:39:35 2012-05-14 02:44:26,390 INFO repeater task os75nb_quick starting 452.4 seconds late 2012-05-14 02:46:12,561 INFO repeater task os75nb_quick ended after 106.2 seconds 2012-05-14 02:46:12,563 INFO repeater task os75nb_3dayplots starting 558.6 seconds late 2012-05-14 02:47:23,323 INFO repeater task os75nb_3dayplots ended after 70.8 seconds 2012-05-14 02:47:23,324 INFO repeater task wh300_beamplots starting 329.3 seconds late 2012-05-14 02:47:34,311 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 02:47:34,311 INFO repeater task wh300_beamplots ended after 11.0 seconds 2012-05-14 02:47:34,321 INFO repeater task wh300_beamplots starting 340.3 seconds late 2012-05-14 02:47:44,380 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 02:47:44,380 INFO repeater task wh300_beamplots ended after 10.1 seconds 2012-05-14 02:47:44,384 INFO repeater task os75bb_beamplots starting 350.4 seconds late 2012-05-14 02:47:49,934 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 02:47:49,935 INFO repeater task os75bb_beamplots ended after 5.6 seconds 2012-05-14 02:47:49,936 INFO repeater task os75bb_beamplots starting 355.9 seconds late 2012-05-14 02:47:55,074 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 02:47:55,074 INFO repeater task os75bb_beamplots ended after 5.1 seconds 2012-05-14 02:47:55,075 INFO repeater task os75nb_beamplots starting 361.1 seconds late 2012-05-14 02:48:00,043 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 02:48:00,043 INFO repeater task os75nb_beamplots ended after 5.0 seconds 2012-05-14 02:48:00,044 INFO repeater task os75nb_beamplots starting 366.0 seconds late 2012-05-14 02:48:05,219 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 02:48:05,219 INFO repeater task os75nb_beamplots ended after 5.2 seconds 2012-05-14 02:48:05,220 INFO repeater task phins_hcorrstats starting 671.2 seconds late 2012-05-14 02:48:11,959 INFO repeater task phins_hcorrstats ended after 6.7 seconds 2012-05-14 02:48:11,960 INFO repeater task phins_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 02:48:11,968 INFO repeater task phins2_hcorrstats starting 677.9 seconds late 2012-05-14 02:48:17,514 INFO repeater task phins2_hcorrstats ended after 5.5 seconds 2012-05-14 02:48:17,515 INFO repeater task phins2_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 02:48:22,518 INFO repeater task wh300_lastensq starting 577.5 seconds late 2012-05-14 02:48:30,577 INFO repeater task wh300_lastensq ended after 8.1 seconds 2012-05-14 02:48:30,578 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 02:46:35 2012-05-14 02:48:30,580 INFO repeater task os75bb_lastensq starting 225.6 seconds late 2012-05-14 02:48:40,512 INFO repeater task os75bb_lastensq ended after 9.9 seconds 2012-05-14 02:48:40,513 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 02:44:35 2012-05-14 02:48:40,515 INFO repeater task os75nb_lastensq starting 235.5 seconds late 2012-05-14 02:48:45,370 INFO repeater task os75nb_lastensq ended after 4.9 seconds 2012-05-14 02:48:45,371 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 02:44:35 2012-05-14 02:48:50,381 INFO repeater task wh300_lastensq starting 5.4 seconds late 2012-05-14 02:48:55,593 INFO repeater task wh300_lastensq ended after 5.2 seconds 2012-05-14 02:48:55,594 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 02:48:35 2012-05-14 02:49:45,729 INFO repeater task os75bb_lastensq starting 0.7 seconds late 2012-05-14 02:49:54,418 INFO repeater task os75bb_lastensq ended after 8.7 seconds 2012-05-14 02:49:54,419 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 02:49:35 2012-05-14 02:49:54,426 INFO repeater task os75nb_lastensq starting 9.4 seconds late 2012-05-14 02:49:59,063 INFO repeater task os75nb_lastensq ended after 4.6 seconds 2012-05-14 02:49:59,064 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 02:49:35 2012-05-14 02:50:49,213 INFO repeater task wh300_lastensq starting 4.2 seconds late 2012-05-14 02:50:54,379 INFO repeater task wh300_lastensq ended after 5.2 seconds 2012-05-14 02:50:54,380 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 02:50:35 2012-05-14 02:51:54,797 INFO repeater task wh300_quick starting 0.8 seconds late 2012-05-14 02:54:43,123 INFO repeater task wh300_quick ended after 168.3 seconds 2012-05-14 02:54:43,129 INFO repeater task wh300_3dayplots starting 169.1 seconds late 2012-05-14 02:55:39,950 INFO repeater task wh300_3dayplots ended after 56.8 seconds 2012-05-14 02:55:39,951 INFO repeater task os75bb_lastensq starting 55.0 seconds late 2012-05-14 02:55:51,632 INFO repeater task os75bb_lastensq ended after 11.7 seconds 2012-05-14 02:55:51,633 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 02:54:35 2012-05-14 02:55:51,636 INFO repeater task os75bb_quick starting 237.6 seconds late 2012-05-14 02:57:54,836 INFO repeater task os75bb_quick ended after 123.2 seconds 2012-05-14 02:57:54,837 INFO repeater task os75bb_3dayplots starting 360.8 seconds late 2012-05-14 02:59:14,338 INFO repeater task os75bb_3dayplots ended after 79.5 seconds 2012-05-14 02:59:14,339 INFO repeater task os75nb_lastensq starting 269.3 seconds late 2012-05-14 02:59:21,617 INFO repeater task os75nb_lastensq ended after 7.3 seconds 2012-05-14 02:59:21,618 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 02:54:35 2012-05-14 02:59:21,619 INFO repeater task os75nb_quick starting 447.6 seconds late 2012-05-14 03:01:10,256 INFO repeater task os75nb_quick ended after 108.6 seconds 2012-05-14 03:01:10,257 INFO repeater task os75nb_3dayplots starting 556.2 seconds late 2012-05-14 03:02:04,606 INFO repeater task os75nb_3dayplots ended after 54.3 seconds 2012-05-14 03:02:04,613 INFO repeater task wh300_beamplots starting 610.6 seconds late 2012-05-14 03:02:19,266 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 03:02:19,267 INFO repeater task wh300_beamplots ended after 14.7 seconds 2012-05-14 03:02:19,267 INFO repeater task wh300_beamplots: added 1 extra increments to time_to_run 2012-05-14 03:02:19,268 INFO repeater task wh300_beamplots starting 625.3 seconds late 2012-05-14 03:02:35,891 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 03:02:35,891 INFO repeater task wh300_beamplots ended after 16.6 seconds 2012-05-14 03:02:35,891 INFO repeater task wh300_beamplots: added 1 extra increments to time_to_run 2012-05-14 03:02:35,900 INFO repeater task os75bb_beamplots starting 641.9 seconds late 2012-05-14 03:02:43,763 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 03:02:43,764 INFO repeater task os75bb_beamplots ended after 7.9 seconds 2012-05-14 03:02:43,764 INFO repeater task os75bb_beamplots: added 1 extra increments to time_to_run 2012-05-14 03:02:43,769 INFO repeater task os75bb_beamplots starting 649.8 seconds late 2012-05-14 03:02:50,568 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 03:02:50,568 INFO repeater task os75bb_beamplots ended after 6.8 seconds 2012-05-14 03:02:50,569 INFO repeater task os75bb_beamplots: added 1 extra increments to time_to_run 2012-05-14 03:02:50,571 INFO repeater task os75nb_beamplots starting 656.6 seconds late 2012-05-14 03:02:55,521 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 03:02:55,522 INFO repeater task os75nb_beamplots ended after 5.0 seconds 2012-05-14 03:02:55,522 INFO repeater task os75nb_beamplots: added 1 extra increments to time_to_run 2012-05-14 03:02:55,526 INFO repeater task os75nb_beamplots starting 661.5 seconds late 2012-05-14 03:03:00,321 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 03:03:00,321 INFO repeater task os75nb_beamplots ended after 4.8 seconds 2012-05-14 03:03:00,322 INFO repeater task os75nb_beamplots: added 1 extra increments to time_to_run 2012-05-14 03:03:00,326 INFO repeater task phins_hcorrstats starting 666.3 seconds late 2012-05-14 03:03:06,516 INFO repeater task phins_hcorrstats ended after 6.2 seconds 2012-05-14 03:03:06,516 INFO repeater task phins_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 03:03:06,521 INFO repeater task phins2_hcorrstats starting 672.5 seconds late 2012-05-14 03:03:11,785 INFO repeater task phins2_hcorrstats ended after 5.3 seconds 2012-05-14 03:03:11,785 INFO repeater task phins2_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 03:03:16,789 INFO repeater task wh300_lastensq starting 631.8 seconds late 2012-05-14 03:03:24,813 INFO repeater task wh300_lastensq ended after 8.0 seconds 2012-05-14 03:03:24,814 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 03:02:35 2012-05-14 03:03:24,815 INFO repeater task os75bb_lastensq starting 219.8 seconds late 2012-05-14 03:03:34,507 INFO repeater task os75bb_lastensq ended after 9.7 seconds 2012-05-14 03:03:34,508 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 02:59:35 2012-05-14 03:03:34,517 INFO repeater task os75nb_lastensq starting 229.5 seconds late 2012-05-14 03:03:39,409 INFO repeater task os75nb_lastensq ended after 4.9 seconds 2012-05-14 03:03:39,410 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 02:59:35 2012-05-14 03:04:49,721 INFO repeater task wh300_lastensq starting 4.7 seconds late 2012-05-14 03:04:54,971 INFO repeater task wh300_lastensq ended after 5.2 seconds 2012-05-14 03:04:54,971 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 03:04:35 2012-05-14 03:04:54,973 INFO repeater task os75bb_lastensq starting 10.0 seconds late 2012-05-14 03:05:04,187 INFO repeater task os75bb_lastensq ended after 9.2 seconds 2012-05-14 03:05:04,187 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 03:04:35 2012-05-14 03:05:04,198 INFO repeater task os75nb_lastensq starting 19.2 seconds late 2012-05-14 03:05:09,016 INFO repeater task os75nb_lastensq ended after 4.8 seconds 2012-05-14 03:05:09,017 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 03:04:35 2012-05-14 03:06:49,681 INFO repeater task wh300_lastensq starting 4.7 seconds late 2012-05-14 03:06:55,085 INFO repeater task wh300_lastensq ended after 5.4 seconds 2012-05-14 03:06:55,086 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 03:06:35 2012-05-14 03:06:55,090 INFO repeater task wh300_quick starting 1.1 seconds late 2012-05-14 03:09:46,444 INFO repeater task wh300_quick ended after 171.4 seconds 2012-05-14 03:09:46,456 INFO repeater task wh300_3dayplots starting 172.5 seconds late 2012-05-14 03:11:07,056 INFO repeater task wh300_3dayplots ended after 80.6 seconds 2012-05-14 03:11:07,057 INFO repeater task os75bb_lastensq starting 82.1 seconds late 2012-05-14 03:11:17,781 INFO repeater task os75bb_lastensq ended after 10.7 seconds 2012-05-14 03:11:17,782 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 03:09:35 2012-05-14 03:11:17,783 INFO repeater task os75bb_quick starting 263.8 seconds late 2012-05-14 03:13:14,054 INFO repeater task os75bb_quick ended after 116.3 seconds 2012-05-14 03:13:14,055 INFO repeater task os75bb_3dayplots starting 380.0 seconds late 2012-05-14 03:14:13,197 INFO repeater task os75bb_3dayplots ended after 59.1 seconds 2012-05-14 03:14:13,198 INFO repeater task os75nb_lastensq starting 268.2 seconds late 2012-05-14 03:14:21,631 INFO repeater task os75nb_lastensq ended after 8.4 seconds 2012-05-14 03:14:21,632 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 03:09:35 2012-05-14 03:14:21,633 INFO repeater task os75nb_quick starting 447.6 seconds late 2012-05-14 03:16:06,540 INFO repeater task os75nb_quick ended after 104.9 seconds 2012-05-14 03:16:06,545 INFO repeater task os75nb_3dayplots starting 552.5 seconds late 2012-05-14 03:17:14,515 INFO repeater task os75nb_3dayplots ended after 68.0 seconds 2012-05-14 03:17:14,521 INFO repeater task wh300_beamplots starting 320.5 seconds late 2012-05-14 03:17:25,658 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 03:17:25,658 INFO repeater task wh300_beamplots ended after 11.1 seconds 2012-05-14 03:17:25,659 INFO repeater task wh300_beamplots starting 331.6 seconds late 2012-05-14 03:17:35,628 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 03:17:35,628 INFO repeater task wh300_beamplots ended after 10.0 seconds 2012-05-14 03:17:35,633 INFO repeater task os75bb_beamplots starting 341.6 seconds late 2012-05-14 03:17:41,207 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 03:17:41,208 INFO repeater task os75bb_beamplots ended after 5.6 seconds 2012-05-14 03:17:41,208 INFO repeater task os75bb_beamplots starting 347.2 seconds late 2012-05-14 03:17:46,325 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 03:17:46,325 INFO repeater task os75bb_beamplots ended after 5.1 seconds 2012-05-14 03:17:46,326 INFO repeater task os75nb_beamplots starting 352.3 seconds late 2012-05-14 03:17:51,277 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 03:17:51,277 INFO repeater task os75nb_beamplots ended after 5.0 seconds 2012-05-14 03:17:51,278 INFO repeater task os75nb_beamplots starting 357.3 seconds late 2012-05-14 03:17:56,126 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 03:17:56,127 INFO repeater task os75nb_beamplots ended after 4.8 seconds 2012-05-14 03:17:56,133 INFO repeater task phins_hcorrstats starting 662.1 seconds late 2012-05-14 03:18:02,115 INFO repeater task phins_hcorrstats ended after 6.0 seconds 2012-05-14 03:18:02,115 INFO repeater task phins_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 03:18:02,116 INFO repeater task phins2_hcorrstats starting 668.1 seconds late 2012-05-14 03:18:07,712 INFO repeater task phins2_hcorrstats ended after 5.6 seconds 2012-05-14 03:18:07,713 INFO repeater task phins2_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 03:18:12,721 INFO repeater task wh300_lastensq starting 567.7 seconds late 2012-05-14 03:18:19,342 INFO repeater task wh300_lastensq ended after 6.6 seconds 2012-05-14 03:18:19,343 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 03:16:35 2012-05-14 03:18:19,345 INFO repeater task os75bb_lastensq starting 214.3 seconds late 2012-05-14 03:18:28,753 INFO repeater task os75bb_lastensq ended after 9.4 seconds 2012-05-14 03:18:28,754 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 03:14:35 2012-05-14 03:18:28,756 INFO repeater task os75nb_lastensq starting 223.8 seconds late 2012-05-14 03:18:33,597 INFO repeater task os75nb_lastensq ended after 4.8 seconds 2012-05-14 03:18:33,598 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 03:14:35 2012-05-14 03:18:48,669 INFO repeater task wh300_lastensq starting 3.7 seconds late 2012-05-14 03:18:53,916 INFO repeater task wh300_lastensq ended after 5.2 seconds 2012-05-14 03:18:53,916 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 03:18:35 2012-05-14 03:19:49,237 INFO repeater task os75bb_lastensq starting 4.2 seconds late 2012-05-14 03:19:58,447 INFO repeater task os75bb_lastensq ended after 9.2 seconds 2012-05-14 03:19:58,448 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 03:19:35 2012-05-14 03:19:58,450 INFO repeater task os75nb_lastensq starting 13.5 seconds late 2012-05-14 03:20:03,341 INFO repeater task os75nb_lastensq ended after 4.9 seconds 2012-05-14 03:20:03,342 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 03:19:35 2012-05-14 03:20:48,501 INFO repeater task wh300_lastensq starting 3.5 seconds late 2012-05-14 03:20:53,790 INFO repeater task wh300_lastensq ended after 5.3 seconds 2012-05-14 03:20:53,791 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 03:20:35 2012-05-14 03:21:54,069 INFO repeater task wh300_quick starting 0.1 seconds late 2012-05-14 03:24:48,288 INFO repeater task wh300_quick ended after 174.2 seconds 2012-05-14 03:24:48,289 INFO repeater task wh300_3dayplots starting 174.3 seconds late 2012-05-14 03:25:45,729 INFO repeater task wh300_3dayplots ended after 57.4 seconds 2012-05-14 03:25:45,730 INFO repeater task os75bb_lastensq starting 60.7 seconds late 2012-05-14 03:25:57,060 INFO repeater task os75bb_lastensq ended after 11.3 seconds 2012-05-14 03:25:57,061 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 03:24:35 2012-05-14 03:25:57,061 INFO repeater task os75bb_quick starting 243.1 seconds late 2012-05-14 03:28:07,462 INFO repeater task os75bb_quick ended after 130.4 seconds 2012-05-14 03:28:07,463 INFO repeater task os75bb_3dayplots starting 373.5 seconds late 2012-05-14 03:29:23,874 INFO repeater task os75bb_3dayplots ended after 76.4 seconds 2012-05-14 03:29:23,875 INFO repeater task os75nb_lastensq starting 278.9 seconds late 2012-05-14 03:29:30,453 INFO repeater task os75nb_lastensq ended after 6.6 seconds 2012-05-14 03:29:30,454 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 03:24:35 2012-05-14 03:29:30,455 INFO repeater task os75nb_quick starting 456.4 seconds late 2012-05-14 03:31:11,125 INFO repeater task os75nb_quick ended after 100.7 seconds 2012-05-14 03:31:11,129 INFO repeater task os75nb_3dayplots starting 557.1 seconds late 2012-05-14 03:32:05,050 INFO repeater task os75nb_3dayplots ended after 53.9 seconds 2012-05-14 03:32:05,057 INFO repeater task wh300_beamplots starting 611.0 seconds late 2012-05-14 03:32:20,465 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 03:32:20,465 INFO repeater task wh300_beamplots ended after 15.4 seconds 2012-05-14 03:32:20,465 INFO repeater task wh300_beamplots: added 1 extra increments to time_to_run 2012-05-14 03:32:20,473 INFO repeater task wh300_beamplots starting 626.5 seconds late 2012-05-14 03:32:36,837 INFO repeater task wh300_beamplots output: Starting procsetup 2012-05-14 03:32:36,837 INFO repeater task wh300_beamplots ended after 16.4 seconds 2012-05-14 03:32:36,838 INFO repeater task wh300_beamplots: added 1 extra increments to time_to_run 2012-05-14 03:32:36,838 INFO repeater task os75bb_beamplots starting 642.8 seconds late 2012-05-14 03:32:44,913 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 03:32:44,914 INFO repeater task os75bb_beamplots ended after 8.1 seconds 2012-05-14 03:32:44,914 INFO repeater task os75bb_beamplots: added 1 extra increments to time_to_run 2012-05-14 03:32:44,915 INFO repeater task os75bb_beamplots starting 650.9 seconds late 2012-05-14 03:32:50,866 INFO repeater task os75bb_beamplots output: Starting procsetup 2012-05-14 03:32:50,866 INFO repeater task os75bb_beamplots ended after 6.0 seconds 2012-05-14 03:32:50,867 INFO repeater task os75bb_beamplots: added 1 extra increments to time_to_run 2012-05-14 03:32:50,868 INFO repeater task os75nb_beamplots starting 656.8 seconds late 2012-05-14 03:32:55,862 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 03:32:55,863 INFO repeater task os75nb_beamplots ended after 5.0 seconds 2012-05-14 03:32:55,863 INFO repeater task os75nb_beamplots: added 1 extra increments to time_to_run 2012-05-14 03:32:55,864 INFO repeater task os75nb_beamplots starting 661.8 seconds late 2012-05-14 03:33:00,888 INFO repeater task os75nb_beamplots output: Starting procsetup 2012-05-14 03:33:00,888 INFO repeater task os75nb_beamplots ended after 5.0 seconds 2012-05-14 03:33:00,889 INFO repeater task os75nb_beamplots: added 1 extra increments to time_to_run 2012-05-14 03:33:00,889 INFO repeater task phins_hcorrstats starting 666.9 seconds late 2012-05-14 03:33:07,113 INFO repeater task phins_hcorrstats ended after 6.2 seconds 2012-05-14 03:33:07,114 INFO repeater task phins_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 03:33:07,117 INFO repeater task phins2_hcorrstats starting 673.1 seconds late 2012-05-14 03:33:12,403 INFO repeater task phins2_hcorrstats ended after 5.3 seconds 2012-05-14 03:33:12,404 INFO repeater task phins2_hcorrstats: added 2 extra increments to time_to_run 2012-05-14 03:33:17,410 INFO repeater task wh300_lastensq starting 632.4 seconds late 2012-05-14 03:33:27,085 INFO repeater task wh300_lastensq ended after 9.7 seconds 2012-05-14 03:33:27,085 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 03:32:35 2012-05-14 03:33:27,087 INFO repeater task os75bb_lastensq starting 222.1 seconds late 2012-05-14 03:33:36,590 INFO repeater task os75bb_lastensq ended after 9.5 seconds 2012-05-14 03:33:36,591 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 03:29:35 2012-05-14 03:33:36,593 INFO repeater task os75nb_lastensq starting 231.6 seconds late 2012-05-14 03:33:41,549 INFO repeater task os75nb_lastensq ended after 5.0 seconds 2012-05-14 03:33:41,550 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 03:29:35 2012-05-14 03:34:48,047 INFO repeater task wh300_lastensq starting 3.0 seconds late 2012-05-14 03:35:10,160 INFO repeater task wh300_lastensq ended after 22.1 seconds 2012-05-14 03:35:10,161 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 03:34:35 2012-05-14 03:35:10,169 INFO repeater task os75bb_lastensq starting 25.2 seconds late 2012-05-14 03:35:29,353 INFO repeater task os75bb_lastensq ended after 19.2 seconds 2012-05-14 03:35:29,354 INFO repeater task os75bb_lastensq: Time from /home/data/at20/proc/os75bb/load/wrote_ens_time: 2012/05/14 03:34:35 2012-05-14 03:35:29,355 INFO repeater task os75nb_lastensq starting 44.4 seconds late 2012-05-14 03:35:37,870 INFO repeater task os75nb_lastensq ended after 8.5 seconds 2012-05-14 03:35:37,871 INFO repeater task os75nb_lastensq: Time from /home/data/at20/proc/os75nb/load/wrote_ens_time: 2012/05/14 03:34:35 2012-05-14 03:36:48,909 INFO repeater task wh300_lastensq starting 3.9 seconds late 2012-05-14 03:36:54,485 INFO repeater task wh300_lastensq ended after 5.6 seconds 2012-05-14 03:36:54,486 INFO repeater task wh300_lastensq: Time from /home/data/at20/proc/wh300/load/wrote_ens_time: 2012/05/14 03:36:35 2012-05-14 03:36:54,486 INFO repeater task wh300_quick starting 0.5 seconds late