Question about logging

[argis]

Is it possible to log custom variables when logging to file? My primary log is to database (MySQL) and the secondary is to text file.
I've found that the database log misses a lot of results, especially if I do midnight logging, and therefore isn't usable, but the text log doesn't miss anything.
However I include some custom variables in the query that creates the database logs, and these are very useful, but I can't do that with the text file log.
I understand the database problem is due to the ODBC driver so there's not much that can be done about that. We have an SPL license and are doing over 8,000 tests so it's quite busy.
Is there a way to record custom variables to a text log?

[KS-Soft]

I've found that the database log misses a lot of results,

What exactly SQL server and ODBC driver do you use?
Do you see ODBC related errors in syslog (HostMonitor "system" log file)?
Any errors detected by Auditing Tool (menu View)?

Is it possible to log custom variables when logging to file?

I think you may setup 2nd ODBC log using "Microsoft text" ODBC driver.

Regards
Alex

[argis]

What exactly SQL server and ODBC driver do you use?

MariaDB 10.3 and MariaDB ODBC 3.1 driver

Do you see ODBC related errors in syslog (HostMonitor "system" log file)?

No there are no errors other than an occasional one saying the reply is too big. These are from tests returning a long error message, and aren't a problem. The reply field can be made longer to handle this.

Any errors detected by Auditing Tool (menu View)?

It's telling me that it's doing 50 tests per second and wants more threads.

I think you may setup 2nd ODBC log using "Microsoft text" ODBC driver.

I'm testing this and it works well, but is there a way to limit the size of the CSV file? The ODBC text driver can read the CSV into a database on another server, which is what we want, but it's read-only and won't delete records from the CSV after they've been read.
Is there a way to limit the size of the CSV file?

I'm having some other unrelated issues with the HostMonitor VM, so I'm going to create a new one and reinstall everything and see if that helps. It will probably help with the other issues at least. The next server will have only essential software installed, the current one has been something of a test-bed.

If the CSV file doesn't work out, is there a preferred option for a database? It won't be storing much data, that will be done on another server which will read the data from the HostMonitor server and then delete it.

[argis]

OK I see there is an option for log processing that will manage the size of the log file, so I'll try that with the CSV file.

[argis]

That failed to move or delete the CSV file, I guess the ODBC driver locks the file.
Unless someone knows of another possible solution I'll have to use a database. As I mentioned before I'd welcome a suggestion for a preferred database.

[KS-Soft]

OK I see there is an option for log processing that will manage the size of the log file, so I'll try that with the CSV file

This option is not applicable for ODBC logging.
You may setup File/Folder Size test to check file size but I am not sure what HostMonitor should do when it detects big file?

The ODBC text driver can read the CSV into a database on another server, which is what we want, but it's read-only and won't delete records from the CSV after they've been read.

If you need access to the log from different systems, then I think you just need SQL server that works properly.
May be MariaDB update will help (latest version is 10.5), may be there is some hot fix. May be you need different server. We would suggest MS SQL or MS SQL Express.

It's telling me that it's doing 50 tests per second and wants more threads

What exactly error is displayed?
...considerable system load. Also you must increase maximum thread limit (Behavior page in the Options dialog)?
Then please increase maximum thread limit.

Regards
Alex