In this tutorial, we will learn how to avoid script, SNMP, or Zabbix agent timeout errors by increasing the Zabbix timeout value on the server, agent, or proxy.
Before we start editing important files, let me explain why would someone even want to increase the Zabbix timeout value in the configuration files?
A key function of Zabbix-server is collecting data (metrics/items) from end-devices using various communication protocols and scripts so that user can view graphs and receive alerts that are based on that data.
Sometimes, monitored devices need more time to respond to the Zabbix server because they may have a large number of monitored metrics, or their CPU is overloaded, SNMP agent is badly implemented, or there is a huge lag on a network, etc.
Also, custom scripts may need more time to handle data or to wait for another system to complete query (e.g copying and parsing large files from FTP server, waiting for a SQL query to finish, etc.).
For example, very often SNMP enabled devices can timeout with error: “Timeout while connecting to […
” (usually via port 161) or some script with the error “timeout while executing a shell script
“. In that case, you can increase the timeout value.
By default, Zabbix will wait 3 sec. for a response from device and after that will declare metric as “Not supported
” and data would be lost. If there is no data in metric then there is no alerting for problems that relay on that metric and performance graphs would show “no data
” for that metric.
Okay, enough of talk! Let’s change that timeout on Zabbix!
Change the Zabbix timeout value on server, agent or proxy
Locate Zabbix configuration file “zabbix_server.conf
” (probably “/usr/local/etc/zabbix_server.conf
“) and add “Timeout=10
” anywhere in file.
Save file and restart Zabbix server afterwords.
Here we changed the timeout from default 3 to 10 sec. but you can choose any number from 1-30 (Zabbix documentation):
Parameter | Mandatory | Range | Default | Description |
Timeout | no | 1-30 | 3 | Specifies how long we wait for agent, SNMP device or external check (in seconds). |
Similar procedure applays to Zabbix agent: edit file “zabbix_agentd.conf",
add “Timeout=10
” or higher value, save file and restart Zabbix agent.
Increase timeout value beyond the max. limit of 30 sec.
This one is a bit tricky. If you would like to increase timeout beyond 30 sec. then you will have to change “Timeout
” in source file “src/zabbix_server/server.c
” and recompile Zabbix server.
Steps:
- Download Zabbix source files that matches your version of Zabbix server.
- Extract files, enter newly extracted directory and with text editor open file “
src/zabbix_server/server.c
“ - Change the maximum timeout value of 30 to higher value
{"Timeout", &CONFIG_TIMEOUT, TYPE_INT,
PARM_OPT, 1, 30},
- Recompile Zabbix server
- Change “
Timeout
” value in Zabbix configuration file (“zabbix_server.conf
“) to match the one you defined in source file - Restart Zabbix Server
Similar procedure applays to Zabbix agent: edit file “src/zabbix_agent/zabbix_agentd.c
“, replace value of 30 with higher value, recompile and restart Zabbix agent.
Negative aspects of increasing Zabbix timeout value
As a rule of thumb, try not to increase timeout value more than 10 sec. unless you have no option. Why? Becase you can lock your pollers if you have tons of script, Zabbix agents or SNMP devices that timeout regullary.
But, if you are going to use high timeout value then try to overprovision the number of pollers in Zabbix configuration file and check regularly Zabbix health statistic.
Another downside of increasing timeout value over 30 sec. is that you need to do the “hack” every time you upgrade Zabbix-server….as if you do not have enough things to remember.
Thank you for reading.
We were stuck with an issue where we would often get Zabbix not supported error as the zabbix_get command took more than 30 seconds. Were in search of a solution for this for many years now. This actually looks promising. Thanks
Thank you very much!!