|
| View previous topic :: View next topic |
| Author |
Message |
davephan
Joined: 19 Oct 2007
Posts: 14
|
 Posted: Wed Sep 21, 2011 11:54 am Post subject: Host Monitor e-mail storm when restarting the monitoring |
 |
|
I periodically make changes to the Host Monitor tests. When I make the changes, I run a batch file which sets the two Host Monitor services to disabled and reboots the server. I then launch the Host Monitor application to make the changes. I then reboot the server again with a batch file that enables the two Host Monitor services.
When the Host Monitor server is put back into service, all the tests rapidly run sending out many e-mail notifications. The tests are set to run at specific times or periodic time intervals.
Is it possible to start two Host Monitor services, starting the monitoring, and not running the tests until each test's scheduled time or next periodic time interval? This would prevent the Host Monitor server from generating an e-mail storm whenever the monitoring is started after making changes.
Dave |
|
| Back to top |
|
 |
KS-Soft Europe
Joined: 16 May 2006
Posts: 2832
|
| Posted: Wed Sep 21, 2011 12:04 pm Post subject: |
|
|
You may try to use "Startup delay" option, located on Service page of Options dialog.
Quote from manual:
| Quote: | Startup delay
In some cases, initialization of dependent applications and services on the HostMonitor machine may consume a substantial amount of time. If HostMonitor's tests depend on these applications, false alerts may be generated while they are initialized.
This setting will cause HostMonitor to wait at least (n) minutes before executing the first monitoring cycle thus allowing applications and services to start and initialize properly. A setting of "1" (one minute) should be adequate for most systems. |
Please check the manual or visit our web site for more information at:
http://www.ks-soft.net/hostmon.eng/mframe.htm#options.htm#Service |
|
| Back to top |
|
|
KS-Soft
Joined: 03 Apr 2002
Posts: 12795
Location: USA
|
| Posted: Wed Sep 21, 2011 12:19 pm Post subject: |
|
|
| Quote: | | Is it possible to start two Host Monitor services |
2 services on the same system? Different systems?
| Quote: | | starting the monitoring, and not running the tests until each test's scheduled time or next periodic time interval? |
Actually this is default behavior. Probably you disabled some options like "Store statistics in HML file" or "Store Recurrences in HML file"? These options located on Preferences page in HostMonitor Options dialog.
Also check "When service being stopped" option located on Service page in HostMonitor Options dialog.
| Quote: | | When the Host Monitor server is put back into service, all the tests rapidly run sending out many e-mail notifications |
Why HostMonitor starts these actions?
a) Because some test probes fail? What exactly test status do you see? Any errors in Reply field?
b) Because you setup HostMonitor do start alerts regardless of test status?
Regards
Alex |
|
| Back to top |
|
|
davephan
Joined: 19 Oct 2007
Posts: 14
|
| Posted: Wed Sep 21, 2011 1:08 pm Post subject: |
|
|
The two services that I am talking about are "KS Advanced Host Monitor" and "KS Web Service". Both services run on the Host Monitor server when it is in "service" mode. I switch off the services when I want to run the Host Monitor program in application mode to test changes.
The e-mails are triggered from "bad" test conditions.
An example of the type of alerting we do is we test for C drive free space. We run scripts independently on each server that runs a Microsoft PSINFO utility into a log file. The PSINFO lists the disk space free space for each drive. The script on each server scrapes the log file to determine if the free space is less than 10% for any drive on the system. If the free space is less than 10%, a flag file is created on the server. When the free space is over 10%, the flag file is deleted from the server. The flag files are detected by the Host Monitor server. Each drive can then be monitored for 10% free space regardless of the drive sizes.
We have a number of template files like the example below. We have a custom written Perl script that subsitutes variable for each template file with variables stored in an Excel spreadsheet. Each test is listed in an Excel spreadsheet along with the server for each test. We are then able to assign tests for each server using the Excel spreadsheet. The assignments in the Excel spreadsheet are processed with the Perl script which subsitutes the variable and creates a very long text file with thousands of tests. The text file is imported, creating all the tests. This is a very easy way to manage thousands of tests.
;------------
Method = FileExists
;--- Common properties ---
;DestFolder = FolderName\
Title = 040-~SERVER NAME~-~HOSTNAME~-(C_drive_free_space)
Comment = C drive free space status
RelatedURL =
ScheduleMode= Regular
Schedule = 7 Days, 24 Hours
Interval = 3600
Alerts = Low_C_Drive_Free_Space_EMail
ReverseAlert= No
UnknownIsBad= No
WarningIsBad= No
UseCommonLog= Yes
PrivLogMode = Default
CommLogMode = Default
;--- Test specific properties ---
File = \\~HOSTNAME~\E$\netmgt\Status\Low_C_Drive_Free_Space.txt
OkIfExists = No
UseMacros = No
;ScheduleMode= OneTestPerDay
;ScheduleTime= 02:00:00
The problem is the 'default' behavior of Host Monitor. When Host Monitor's Monitoring is first started, the Host Monitor program runs through all the tests one by one. This can create an e-mail storm if there are a couple hundred alerts (red items in the Host Monitor console). It would be better if the tests were not processed until the next scheduled test time, either the next interval or time of day, rather than starting immediately when the Host Monitor's monitoring is started.
Dave |
|
| Back to top |
|
|
KS-Soft
Joined: 03 Apr 2002
Posts: 12795
Location: USA
|
| Posted: Wed Sep 21, 2011 1:28 pm Post subject: |
|
|
| Quote: | | The two services that I am talking about are "KS Advanced Host Monitor" and "KS Web Service". Both services run on the Host Monitor server when it is in "service" mode. I switch off the services when I want to run the Host Monitor program in application mode to test changes. |
You don't need to stop Web Service in such case.
Also, why you cannot work with HostMonitor directly? Because you login to server from remote system? or because you are using Windows Vista, Windows Server 2008 or Windows 7?
Why you cannot use RCC (Remote Control Console)?
Regards
Alex |
|
| Back to top |
|
|
KS-Soft
Joined: 03 Apr 2002
Posts: 12795
Location: USA
|
| Posted: Wed Sep 21, 2011 1:32 pm Post subject: |
|
|
| Quote: | | The problem is the 'default' behavior of Host Monitor. When Host Monitor's Monitoring is first started, the Host Monitor program runs through all the tests one by one. This can create an e-mail storm if there are a couple hundred alerts (red items in the Host Monitor console). It would be better if the tests were not processed until the next scheduled test time, either the next interval or time of day, rather than starting immediately when the Host Monitor's monitoring is started. |
Could you please check options listed in my 1st answer?
When I said about "default behavior" I meant HostMonitor should work just like you want unless you disabled some options or stopped monitoring for long time (as I understand you don't stop monitoring, just switching between application or service mode)
Regards
Alex |
|
| Back to top |
|
|
davephan
Joined: 19 Oct 2007
Posts: 14
|
| Posted: Wed Sep 21, 2011 3:36 pm Post subject: |
|
|
| KS-Soft wrote: | | Quote: | | The problem is the 'default' behavior of Host Monitor. When Host Monitor's Monitoring is first started, the Host Monitor program runs through all the tests one by one. This can create an e-mail storm if there are a couple hundred alerts (red items in the Host Monitor console). It would be better if the tests were not processed until the next scheduled test time, either the next interval or time of day, rather than starting immediately when the Host Monitor's monitoring is started. |
Could you please check options listed in my 1st answer?
When I said about "default behavior" I meant HostMonitor should work just like you want unless you disabled some options or stopped monitoring for long time (as I understand you don't stop monitoring, just switching between application or service mode)
Regards
Alex |
I do stop monitoring while I am making changes to the tests. I set the services to disabled, then reboot to make sure that the services actually stop. Sometimes when I try to simply stop the services, then hang going to "stopping" and never actually "stop". When I set the services to disabled, then reboot, after the reboot, the services never are running, they are always stopped. To start the services, I set them to enabled, then reboot. After the reboot, the services are started.
I delete all tests. Then I use the test templates and my Excel spreadsheet which are processed by a Perl script which creates the new text file. The new text file containing all the tests are imported as a text file to load all the tests. After all the tests are imported, the monitoring is started again.
The problem isn't that I need to delay the startup of the Host Monitor testing away in time from when the server starts up after the reboot. The problem is I do not want all the tests to be tested when the Host Monitor monitoring process starts. Could it be possible to have a change in the program so that the tests would follow the scheduling strictly when Host Monitor monitoring is started? Then when the Host Monitor monitoring is started, tests would only be tested according to their programmed schedules, not extra tests when the monitoring process begins.
Dave |
|
| Back to top |
|
|
KS-Soft
Joined: 03 Apr 2002
Posts: 12795
Location: USA
|
| Posted: Wed Sep 21, 2011 3:50 pm Post subject: |
|
|
| Quote: | | I delete all tests. Then I use the test templates and my Excel spreadsheet which are processed by a Perl script which creates the new text file. |
Aha, I see. Then for sure HostMonitor will start all these tests because checks never were performed.
| Quote: | | The problem is I do not want all the tests to be tested when the Host Monitor monitoring process starts. Could it be possible to have a change in the program so that the tests would follow the scheduling strictly when Host Monitor monitoring is started? |
How this can help? You will receive the same alerts just a little bit later 
| Quote: | | Then when the Host Monitor monitoring is started, tests would only be tested according to their programmed schedules, not extra tests when the monitoring process begins |
Well, if you setup test to be performed every 10 min and test was never performed then its time to test, right? What other interval/time can be used?
Regards
Alex |
|
| Back to top |
|
|
davephan
Joined: 19 Oct 2007
Posts: 14
|
| Posted: Thu Sep 22, 2011 8:03 am Post subject: |
|
|
| KS-Soft wrote: | | Quote: | | I delete all tests. Then I use the test templates and my Excel spreadsheet which are processed by a Perl script which creates the new text file. |
Aha, I see. Then for sure HostMonitor will start all these tests because checks never were performed.
| Quote: | | The problem is I do not want all the tests to be tested when the Host Monitor monitoring process starts. Could it be possible to have a change in the program so that the tests would follow the scheduling strictly when Host Monitor monitoring is started? |
How this can help? You will receive the same alerts just a little bit later
| Quote: | | Then when the Host Monitor monitoring is started, tests would only be tested according to their programmed schedules, not extra tests when the monitoring process begins |
Well, if you setup test to be performed every 10 min and test was never performed then its time to test, right? What other interval/time can be used?
Regards
Alex |
You stated the problem exactly. After deleting all tests and re-importing them, Host Monitor sees that the tests were never run, and tries to run all the tests when first starting up the monitoring. I suspect the reason Host Monitor's behavior was setup that way is to insure all the tests are run.
I would like to see a way to configure Host Monitor's default behavior to strickly follow the schedules, at least when starting up. Perhaps there could be a checkbox that could change the behavor to only run tests when they are scheduled.
Or, perhaps there is a way that all the tests could be 'marked' as if they were already ran upon startup, so that the tests would only start running the next time they were scheduled to run.
The tests are either at intervals, such as once an hour or at specific set times.
The process of managing the import text file with templates, a spreadsheet, and a Perl application to create the import text file is a very easy way to allocate the tests to different servers. The same or different tests can be easily allocated to the servers being monitored using the spreadsheet that is used to create the import text file.
Please let me know if it would be possible to change the default startup behavior in a future version.
I would also like to see one change in the web portal which I asked about before. When starting the web portal, it opens up into the "Folder" view. I would like the startup to default to the "Views" view. I know that I can start the web portal and click on "Views". This is an annoyance. We manage servers first by the type of problem, such as Nightly Backup Failure, then which servers have that problem. Rather than which servers have probems, and what kind of problems do they have. Differnent people manage different types of problems. All problems types for a given server are not assigned to one person. If the default view that the web portal could configured, that would be an improvement.
Dave |
|
| Back to top |
|
|
KS-Soft
Joined: 03 Apr 2002
Posts: 12795
Location: USA
|
| Posted: Thu Sep 22, 2011 9:07 am Post subject: |
|
|
| Quote: | | I would like to see a way to configure Host Monitor's default behavior to strickly follow the schedules, at least when starting up. Perhaps there could be a checkbox that could change the behavor to only run tests when they are scheduled. |
HostMonitor does follow the schedule. If you setup test to be started every Friday at 9:00 AM, it will be started every Friday at 9:00 AM
If you setup test to be performed every 10 min, it will be performed every 10 min.
Also I do not understand how some schedule-related modification can help in your case?
If test fails because of low disk space (just for example), it will fail regardless of the schedule.
If you do not want to start "bad" actions for "new" test items, this has nothing to do with schedule or test intervals.
Perhaps you need to modify your alert profiles. E.g. you can use "advanced mode" action with expression like
('%SimpleStatus%'=='DOWN') and (%Recurrences%==1) and ('%LastSimpleStatus%'=='UP')
In such case HostMonitor will not start "bad" action if previous status was not "good".
Regards
Alex |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in
