Logging pool is full during midnight logging

NetProf · Post by **NetProf** » Tue Mar 22, 2022 12:21 am

Hello,

We have around 3000 active tests and 2000 deactivated tests. We store the changes + midnight logging in a MS SQL database.

Logging the changes works without any problems. Unfortunately I have a problem with the midnight logging.

Apparently the logging pool should be running full during midnight logging (Says in the syslog from the hostmonitor). It seems that the logging pool then does not record any more entries. I am missing about 1000 tests in the database.

We use the current version: 13.14.
The MS SQL Server is connected to the hostmonitor via ODBC. There we use the ODBC driver "ODBC Driver 18 for SQL Server".

Is it possible to increase the logging pool size? Or is there another solution for this?

Best regards,
Patrick

KS-Soft · Post by **KS-Soft** » Tue Mar 22, 2022 2:05 am

There is no reason to increase logging pool size.
This message means your database performance are very very bad and you should fix database, not HostMonitor.

You may check logging average time consumption using Auditing Tool (menu View->Auditing Tool). Normally this counter below 1ms/record and HostMonitor may easily write over 1000 records/sec without even using logging pool.
While some defected systems shows 100-300ms/record. May be there is network problem, bad network card, bad hard drive on MS SQL server side, bad antivirus software, system out of resources (memory, cpu), etc.

If you are not using this MS SQL database for some 3rd party log analyzer software, you may switch to default file logging (TXT, HTML, DBF) instead of SQL server..

Regards
Alex

NetProf · Post by **NetProf** » Tue Mar 22, 2022 5:06 pm

Hello,

it is a very fresh MSSQL Server installation in a VMWare virtualization environment on the same subnet. The SQL server has 4 CPUs, 12GB RAM. The hostmonitor server has 10 CPUs, 12GB RAM.

We have analyzed all logs and resources. The SQL server is constantly at 0-5% CPU utilization. So it does nothing and is not overloaded. We have also run a trace log. Hostmonitor sent only 3000 tests out of about 5000 to the MSSQL server. We could also see this in the trace log.

We have a load of: 35 tests/sec. This is not a problem at all, it works very well.
As already written, this problem only exists when midnight logging is active (which we do need).

I don't think that the MSSQL server is the bottleneck, since neither hard disk, CPU nor RAM are busy at this time.

Regards
Patrick

KS-Soft · Post by **KS-Soft** » Wed Mar 23, 2022 2:15 am

Please show Auditing Tool data, at least logging average time consumption

Regards
Alex

NetProf · Post by **NetProf** » Wed Mar 23, 2022 8:00 am

KS-Soft · Post by **KS-Soft** » Wed Mar 23, 2022 8:05 am

Another page, performance history.
Or send .HML file to us, we can extract more details

Regards
Alex

NetProf · Post by **NetProf** » Wed Mar 23, 2022 8:33 am

i'm sorry, but how do i create the hml file?

Regards
Patrick

KS-Soft · Post by **KS-Soft** » Wed Mar 23, 2022 8:36 am

If you have HostMonitor, you have this file.
Auditing Tool -> System/License shows location of the file.
E.g.
Config files location: C:\ProgramData\HostMonitor\
Test list file: C:\ProgramData\HostMonitor\NetProf.hml

Just don't publish the file, send it by e-mail, zip if possible.

Regards
Alex

NetProf · Post by **NetProf** » Wed Mar 23, 2022 9:00 am

I am not sure if I should do that. Are access data from servers also stored there?

Screenshot of Performance History:

KS-Soft · Post by **KS-Soft** » Wed Mar 23, 2022 9:51 am

5-20 msec per record. Its not fast..
5000 records, 25-100 seconds to store data..

Try to mark "Enable connection pooling" option
(HostMonitor Options dialog -> Misc page -> ODBC tests & logging)

Are access data from servers also stored there?

What exactly means "access data"?
hostname or IP address? Yes.
accounts and passwords? Usually not. Unless you are using "Connect as" option for some tests instead of domain account for the service and/or Connection Manager
https://www.ks-soft.net/hostmon.eng/mfr ... htm#conmgr

Regards
Alex

NetProf · Post by **NetProf** » Thu Mar 24, 2022 5:06 am

Unfortunately, that's how we did it on some tests.

The connection pooling is already enabled (in Hostmonitor and in the ODBC settings)

Do you have experience with this amount of data and the MSSQL server?

KS-Soft · Post by **KS-Soft** » Thu Mar 24, 2022 9:45 am

MS SQL 2017: 20 cores CPU, 32GB memory: 0.5 - 1.0 ms/record
MS SQL 2017: 4 cores CPU, 4GB memory: 1-2 ms/record
Everything works fine, 500 tests/second - ok, Midnight logging - Ok

May be slow ODBC driver on HostMonitor side?
What version of the driver you use?

Regards
Alex

NetProf · Post by **NetProf** » Tue Sep 06, 2022 8:41 am

Hello Alex,

sorry for the long break. However, we were able to run some more tests.

According to some stress tests we write to the database in 2ms. We tested the following drivers (with the same result):

-) ODBC Driver 17 for SQL Server
-) ODBC Driver 18 for SQL Server
-) SQL Server Native Client 11.0
-) SQL Server

We still have the problem that with Midnight logging not all tests are written to the database. It is not always the same tests which are not written to the database.

The Hostmonitor log still shows the error message: "Monitoring service warning: Logging pool is full (common logs)."

The host monitor server has the following resources:
10 cores
24GB memory

SQL Server has the following resources:
4 cores
12GB memory

Both servers have almost no load. During a stress test on the SQL (from the host monitor server), we were able to write 100,000 entries without any problems.

Best regards,
Patrick

KS-Soft · Post by **KS-Soft** » Tue Sep 06, 2022 12:22 pm

Could you please provide some internal statistics from HostMonitor?
We sent you e-mail with details

Regards
Alex