KS-Soft. Network Management Solutions
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister    ProfileProfile    Log inLog in 

Out of Memory on WMI tests

 
Post new topic   Reply to topic    KS-Soft Forum Index -> Configuration, Maintenance, Troubleshooting
View previous topic :: View next topic  
Author Message
Kris



Joined: 12 May 2010
Posts: 375

PostPosted: Tue Nov 03, 2020 8:52 am    Post subject: Out of Memory on WMI tests Reply with quote

Hi all,

Several times a day I get a 'Out of Memory' error on WMI tests.
It's not one specific test or machine, it looks totally random, but on WMI tests only.
When I instantly refresh the test it will be fine.
No big deal, but maybe we're doing something wrong...

Specs:
HM 12.50 on Windows 10 64bit
on VMware 6.7 with 2 cores @3.2GHz and 8GB memory.
Back to top
View user's profile Send private message
KS-Soft



Joined: 03 Apr 2002
Posts: 12791
Location: USA

PostPosted: Tue Nov 03, 2020 10:03 am    Post subject: Reply with quote

Does Auditing Tool show some warnings/errors (menu View)?

Could you check resource usage for each process? You may use standard Windows Task Manager to check Handles, GDI and USER objects. What is the total resource usage on the system? How many handles/threads/GDI objects used by hostmon.exe process?

Do you use ODBC logging or ODBC test method? If yes, what ODBC driver do you use?
Some ODBC drivers have many bugs and leads to various problems.

Number of handles/threads/user objects not always can help because problem may depend on what exactly kind of handle is used, e.g. how much memory this resource used in Windows internal buffers.
Sometimes 200,000 handles is not a problem; other time 80,000 is a problem. However its a starting point that may tell us what process leads to the problems.

Regards
Alex
Back to top
View user's profile Send private message Visit poster's website
Kris



Joined: 12 May 2010
Posts: 375

PostPosted: Wed Nov 04, 2020 2:15 am    Post subject: Reply with quote

Hi Alex,

Thanks for your quick response.
Please see below.

----------
Auditing Tool:
Errors: No errors other than Syslog file size and password strength
Workload: Low, avg. 2 tests/sec

System:
CPU load: 10-50%
Mem usage: 30-50%

HostMonitor process:
HM #Threads: 30-50
HM #Handles: 1500-2000
HM #GDI Handles: 0
HM #User Handles# 0
HM #Wait Chain Handles: 5

We don't use ODBC logging or tests.
----------

Hope this is useful info...
Back to top
View user's profile Send private message
KS-Soft



Joined: 03 Apr 2002
Posts: 12791
Location: USA

PostPosted: Wed Nov 04, 2020 5:42 am    Post subject: Reply with quote

Please check ALL processes running on the system, some may use too much Windows resources

Regards
Alex
Back to top
View user's profile Send private message Visit poster's website
Kris



Joined: 12 May 2010
Posts: 375

PostPosted: Wed Nov 04, 2020 8:37 am    Post subject: Reply with quote

Hi Alex,

There are plenty system resources available.
On average 5GB of 8GB free, around 20% cpu load.

I ran a logging with ProcessExplorer, and even when HM threw a 'Out of Memory' error, system resources were availabe in abundance.

Can't get my head around it
Back to top
View user's profile Send private message
KS-Soft



Joined: 03 Apr 2002
Posts: 12791
Location: USA

PostPosted: Wed Nov 04, 2020 8:44 am    Post subject: Reply with quote

Total free memory means nothing in this case.
System may have 60 GB of "free" memory and still thow "out of memory" errors because there are various internal Windows buffers (e.g. desktop heap). If there is no room in such buffer, system will not work properly. Unfortunately there is no way to check status of such buffers (there are some Microsoft tools but only for Windows XP/2003).

Please check my 1st reply.
Quote:
Could you check resource usage for each process? You may use standard Windows Task Manager to check Handles, GDI and USER objects. What is the total resource usage on the system? How many handles/threads/GDI objects used by hostmon.exe process?

Number of handles/threads/user objects not always can help because problem may depend on what exactly kind of handle is used, e.g. how much memory this resource used in Windows internal buffers.
Sometimes 200,000 handles is not a problem; other time 80,000 is a problem. However its a starting point that may tell us what process leads to the problems.

or just stop couple processes with significant number of Handles and see if this helps.

Regards
Alex
Back to top
View user's profile Send private message Visit poster's website
Kris



Joined: 12 May 2010
Posts: 375

PostPosted: Wed Nov 04, 2020 8:48 am    Post subject: Reply with quote

Wasn't this what you asked?
Quote:
HostMonitor process:
HM #Threads: 30-50
HM #Handles: 1500-2000
HM #GDI Handles: 0
HM #User Handles# 0
HM #Wait Chain Handles: 5
Back to top
View user's profile Send private message
Kris



Joined: 12 May 2010
Posts: 375

PostPosted: Wed Nov 04, 2020 9:27 am    Post subject: Reply with quote

Oh wait,

Total # handles: about 60000
Total # threads: about 2200
Back to top
View user's profile Send private message
KS-Soft



Joined: 03 Apr 2002
Posts: 12791
Location: USA

PostPosted: Wed Nov 04, 2020 10:11 am    Post subject: Reply with quote

Quote:
Could you check resource usage for each process? You may use standard Windows Task Manager to check Handles, GDI and USER objects. What is the total resource usage on the system? How many handles/threads/GDI objects used by hostmon.exe process?

Number of handles/threads/user objects not always can help because problem may depend on what exactly kind of handle is used, e.g. how much memory this resource used in Windows internal buffers.
Sometimes 200,000 handles is not a problem; other time 80,000 is a problem. However its a starting point that may tell us what process leads to the problems.

"Could you check resource usage for each process" means - please check ALL processes.
This way you may find most "hungry" one and try to stop or restart it.

Also you may setup Dominant Process tests with full logging mode (better use private log file for these tests), this may help to find some process with "leaks".

Also you may start Auditing Tool (menu View), it may detect some problems.
What the average load (tests/sec)?
WMI+DominantProcess+Memory+HyperV tests/sec rate?

Regards
Alex
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    KS-Soft Forum Index -> Configuration, Maintenance, Troubleshooting All times are GMT - 6 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group

KS-Soft Forum Index