Running very slow

grussell · Post by **grussell** » Wed Mar 10, 2010 9:24 am

Hi Guys,

We've just bought the full version and have finished importing a lot of our infrastruture.

We support about 1600 servers, and each server has between 10 and 50 tests. We have currently placed about 800 servers into AHM.

This results in around 10000 test.

The .HML file is about 40Mb

The vast majority of the tests are currently disabled. And the active tests (about 100) use the passive RMA. Only about 10 tests use HostMon to perform the action.

The CPU used by hostmon.exe is constantly between about 25% and 65%. And the PC runs very slow.

It's memory footprint (hostmon.exe) isn't too bad at 119Mb

It's running on a 2.6Ghz Pentium 4 PC with 2Gb RAM, under Windows XP SP3. The PC still has plenty of memory available and it isn't heavily using the swap file.

I am worried that it won't cope with all our servers. What can I do to increase it's performance? And how can I troubleshoot where all the resources are going?

Many thanks.

Gavin.

KS-Soft · Post by **KS-Soft** » Wed Mar 10, 2010 11:47 am

You may use Aditing Tool (menu View) to check for performance problems.
http://www.ks-soft.net/hostmon.eng/mfra ... tm#LoadDlg

What is average test frequency? What kind of tests do you use?

Regards
Alex

grussell · Post by **grussell** » Wed Mar 10, 2010 1:14 pm

A mix of tests - mostly performance counters, but a fair few scripts too.

I tried chosing the "Auditing tool..." option - but nothing happens?

KS-Soft · Post by **KS-Soft** » Wed Mar 10, 2010 1:45 pm

Nothing happens? You do not see Auditing Tools dialog window?
What version of HostMonitor do you use?

Could you check resource usage for each process? You may use standard Windows Task Manager to check Handles, GDI and USER objects. What is the total resource usage on the system? How many handles/threads/GDI objects used by hostmon.exe process?

Do you use ODBC logging or ODBC test method? If yes, what ODBC driver do you use?
Do you have installed some antivirus monitors, personal firewall, content monitoring software? Non stanard winsock components?

Regards
Alex

grussell · Post by **grussell** » Thu Mar 11, 2010 3:13 am

I've just upgraded to 8.50 - but got the same problem.

hostmon.exe
CPU ~60%
Mem Usage 89M
VM Size 85M
Handles 300
Threads 12
User Objects 156
GDI Objects 211

I was using a database connection for logging (although it was set to only log changes - and as I mentioned, there is only about 100 active tests). I set the logging to "none" and still the CPU runs high.

When I do use ODBC, I use the Native SQL driver.

Anti-virus is disabled. Firewall turned off.

If I turn off monitoring the CPU returns to normal.

What is strange, is most tests are disabled. If I remove all the disabled tests the the CPU returns to normal.

Thanks.

grussell · Post by **grussell** » Thu Mar 11, 2010 3:17 am

And the Auditing tool (which runs ok now) returns

"Conclusion: system is able to perform given tests without significant load"

KS-Soft · Post by **KS-Soft** » Thu Mar 11, 2010 12:43 pm

Could you please send your configuration files to us (support@ks-soft.net)?
We need HML file with tests, all *.LST and *.INI files

Regards
Alex

grussell · Post by **grussell** » Thu Mar 11, 2010 1:05 pm

Emailing now - you should get it in a few minutes.

KS-Soft · Post by **KS-Soft** » Thu Mar 11, 2010 2:04 pm

Thank you.
As I see Auditing Tools shows 195(!) warning related to unspecified folder variable. Theoretically this should not lead to performance problems... we will check this.

Another warning you may see relates to GUI option "Show tests in subfolders". You have a lot of tests so its better to disable this option as recommended by Auditing Tool.
If you need to see all test items on one screen, you may use Super Matrix window

Regards
Alex

grussell · Post by **grussell** » Thu Mar 11, 2010 2:11 pm

Yes, the unspecified folder variables are tests within our "templates" folder - these are always disabled, and are used by our engineers to populate customer folders.

I've disabled "show tests in subfolders" - doesn't seem to affect CPU running speed.

Also, I've deleted the "unspecified variable" tests. It doesn't seem to have any effect either.

Let me know if I can perform any tests?

Thanks.

KS-Soft · Post by **KS-Soft** » Thu Mar 11, 2010 3:33 pm

Let me know if I can perform any tests?

Sure, why not.

So far we cannot reproduce the problem using your configuration files. Resource usage looks fine (memory, handles, threads).
HostMonitor uses between 2 and 15% of CPU on Intel Core 2 system. We do not use Pentium 4 systems anymore, they are way too old (1st Pentium 4 was available 10 or 11 years ago?).

However if you enable all test items, HostMonitor will not be able to perform all your tests in time. You will need to increase time interval for some items and increase value of "Do not start more than 32 tests per second" option (Auditing Tool will provide some recommendations).
Also, if you plan to enable all items and monitor over 10,000 test items, we would recommend to use more modern system (I think your system is 6-7 years old?)

Regards
Alex

grussell · Post by **grussell** » Thu Mar 11, 2010 3:43 pm

Thanks.

grussell · Post by **grussell** » Mon Apr 19, 2010 5:26 am

Hi Guys,

We've finished building our new monitoring system.

Two 2.4Ghz cores. 6Gb of RAM. Windows Server 2008 R2 x64

This is a virtualised server.

I've just installed host monitor, and restored our tests, and still having a very slow response with monitoring is running (if I stop monitoring it seems fine); exactly the same as when I ran it on our older test machine.

Currently the vast majority of tests are disabled - there are only about 30 running!!

Seems strange. My "guess" would be something to do with the database structure. If I remove all disabled tests it is far happier with the 30 active ones?

Gavin.

KS-Soft · Post by **KS-Soft** » Mon Apr 19, 2010 1:22 pm

If I remove all disabled tests it is far happier with the 30 active ones?

No, "disabled" tests should not use CPU resources.

As I remember we tested HostMonitor with your configuration files and it worked fine on our systems.
Have you used "Auditing Tools" on your new system? Any warnings?
Do you use ODBC logging now? Could you try to disable it?
Do you have installed some antivirus monitors, personal firewall, content monitoring software? Non stanard winsock components?

Regards
Alex

grussell · Post by **grussell** » Thu Apr 22, 2010 2:41 am

- Have you used "Auditing Tools" on your new system? Any warnings?

I've got it down to 3 errors about a variable not being defined.

- Do you use ODBC logging now? Could you try to disable it?

Turned it off - didn't make any difference.

- Do you have installed some antivirus monitors, personal firewall, content monitoring software? Non stanard winsock components?

No, all firewalls have been turned off.

Also, when I delete a root folder containing alot of sub-folders a warning popup starts apearing every second saying "folder not found!".

I am currently using 10,000 tests across 3900 folders. And only about 100 tests are actually enabled.

I've spent the last few days trying to triangulate the issue. If I delete all active tests the CPU still runs at about 50% until I turn off monitoring, and it returns to 0-1%. It seems something to do with how it enumerates the tree structure?

How can I provide you more detail?

KS-Soft

Running very slow

Running very slow

Still problem

Slow after all tests disabled?