Rocky Linux 8, Zabbix proxy and permission denied in logs.
Unfortunately during migrate from RHEL 7 based OS to RHEL 8 (Rocky Linux 8) i got permission denied in logs, even when ALL things was OK! So, „who” is our nightmare?
37868:20220518:165840.230 Unable to connect to the server [IP]:PORT [cannot connect to [[IP]:PORT]: [13] Permission denied]. Will retry every 120 second(s)Maybe firewall issue?
[kamil@zabbix50-prx ~]$ sudo firewall-cmd --list-all
public (active)
target: default
icmp-block-inversion: no
interfaces: ens192
sources:
services: cockpit dhcpv6-client ssh
ports: 10050/tcp
protocols:
forward: no
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules:
We got it! Port 10051/tcp is missing, so let’s add it.
[kamil@zabbix50-prx ~]$ sudo firewall-cmd --add-port=10051/tcp --permanent
success
[kamil@zabbix50-prx ~]$ sudo firewall-cmd --reload
success
Check firewall, restart zabbix-proxy service and read logs.
[kamil@zabbix50-prx ~]$ sudo firewall-cmd --list-all
public (active)
target: default
icmp-block-inversion: no
interfaces: ens192
sources:
services: cockpit dhcpv6-client ssh
ports: 10050/tcp 10051/tcp
protocols:
forward: no
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules:
[kamil@zabbix50-prx ~]$ sudo systemctl restart zabbix-proxy.service
[kamil@zabbix50-prx ~]$ sudo tail -f /var/log/zabbix/zabbix_proxy.log
38286:20220518:170923.031 cannot connect to preprocessing service: Cannot connect to service "preprocessing": [13] Permission denied.
Now we have different issue, so let’s we try to manage our loves selinux!
[kamil@zabbix50-prx ~]$ sudo semodule -l | grep -i zabbix
zabbix
[kamil@zabbix50-prx ~]$ sestatus
SELinux status: enabled
SELinuxfs mount: /sys/fs/selinux
SELinux root directory: /etc/selinux
Loaded policy name: targeted
Current mode: enforcing
Mode from config file: enforcing
Policy MLS status: enabled
Policy deny_unknown status: allowed
Memory protection checking: actual (secure)
Max kernel policy version: 33
OK, so we have zabbix modules installed and selinux is enabled in enforcing mode.
[kamil@zabbix50-prx ~]$ sudo setsebool -P zabbix_can_network=1
[kamil@zabbix50-prx ~]$ sudo systemctl restart zabbix-proxy.service
[kamil@zabbix50-prx ~]$ sudo tail -f /var/log/zabbix/zabbix_proxy.log
...
39133:20220518:173303.027 cannot connect to preprocessing service: Cannot connect to service "preprocessing": [13] Permission denied.
...
[kamil@zabbix50-prx ~]$ sudo grep zabbix_proxy /var/log/audit/audit.log | grep denied | tail -1
type=AVC msg=audit(1652887880.476:1647139): avc: denied { connectto } for pid=39092 comm="zabbix_proxy" path="/run/zabbix/zabbix_proxy_preprocessing.sock" scontext=system_u:system_r:zabbix_t:s0 tcontext=system_u:system_r:zabbix_t:s0 tclass=unix_stream_socket permissive=0
OK! So now we see another issue related to permission! Look at permissive=0, we need change this value. So time to tell selinux few things! But first we need to find correct RPM with semanage binary (if you don’t have it).
[kamil@zabbix50-prx ~]$ dnf provides "semanage"
Last metadata expiration check: 17:30:03 ago on Wed 18 May 2022 12:00:46 AM CEST.
policycoreutils-python-utils-2.9-19.el8.noarch : SELinux policy core python utilities
Repo : baseos
Matched from:
Filename : /usr/sbin/semanage
[kamil@zabbix50-prx ~]$ sudo dnf install policycoreutils-python-utils-2.9-19.el8.noarch
Last metadata expiration check: 0:22:13 ago on Wed 18 May 2022 05:12:59 PM CEST.
Dependencies resolved.
=====================================================================================================================
Package Architecture Version Repository Size
=====================================================================================================================
Installing:
policycoreutils-python-utils noarch 2.9-19.el8 baseos 252 k
Installing dependencies:
checkpolicy x86_64 2.9-1.el8 baseos 345 k
python3-audit x86_64 3.0.7-2.el8.2 baseos 86 k
python3-libsemanage x86_64 2.9-8.el8 baseos 127 k
python3-policycoreutils noarch 2.9-19.el8 baseos 2.2 M
python3-setools x86_64 4.3.0-3.el8 baseos 623 k
Transaction Summary
=====================================================================================================================
Install 6 Packages
Total download size: 3.6 M
Installed size: 10 M
Is this ok [y/N]: y
Downloading Packages:
(1/6): python3-audit-3.0.7-2.el8.2.x86_64.rpm 472 kB/s | 86 kB 00:00
(2/6): policycoreutils-python-utils-2.9-19.el8.noarch.rpm 1.2 MB/s | 252 kB 00:00
(3/6): checkpolicy-2.9-1.el8.x86_64.rpm 1.6 MB/s | 345 kB 00:00
(4/6): python3-libsemanage-2.9-8.el8.x86_64.rpm 3.4 MB/s | 127 kB 00:00
(5/6): python3-setools-4.3.0-3.el8.x86_64.rpm 7.2 MB/s | 623 kB 00:00
(6/6): python3-policycoreutils-2.9-19.el8.noarch.rpm 8.2 MB/s | 2.2 MB 00:00
---------------------------------------------------------------------------------------------------------------------
Total 5.0 MB/s | 3.6 MB 00:00
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : python3-setools-4.3.0-3.el8.x86_64 1/6
Installing : python3-libsemanage-2.9-8.el8.x86_64 2/6
Installing : python3-audit-3.0.7-2.el8.2.x86_64 3/6
Installing : checkpolicy-2.9-1.el8.x86_64 4/6
Installing : python3-policycoreutils-2.9-19.el8.noarch 5/6
Installing : policycoreutils-python-utils-2.9-19.el8.noarch 6/6
Running scriptlet: policycoreutils-python-utils-2.9-19.el8.noarch 6/6
Verifying : checkpolicy-2.9-1.el8.x86_64 1/6
Verifying : policycoreutils-python-utils-2.9-19.el8.noarch 2/6
Verifying : python3-audit-3.0.7-2.el8.2.x86_64 3/6
Verifying : python3-libsemanage-2.9-8.el8.x86_64 4/6
Verifying : python3-policycoreutils-2.9-19.el8.noarch 5/6
Verifying : python3-setools-4.3.0-3.el8.x86_64 6/6
Installed:
checkpolicy-2.9-1.el8.x86_64 policycoreutils-python-utils-2.9-19.el8.noarch
python3-audit-3.0.7-2.el8.2.x86_64 python3-libsemanage-2.9-8.el8.x86_64
python3-policycoreutils-2.9-19.el8.noarch python3-setools-4.3.0-3.el8.x86_64
Complete!
[kamil@zabbix50-prx ~]$ sudo semanage permissive -a zabbix_t
[kamil@zabbix50-prx ~]$ sudo systemctl restart zabbix-proxy.service
[kamil@zabbix50-prx ~]$ sudo tail -f /var/log/zabbix/zabbix_proxy.log
39989:20220518:174146.195 proxy #19 started [history syncer #4]
39990:20220518:174146.196 proxy #20 started [self-monitoring #1]
39991:20220518:174146.196 proxy #21 started [task manager #1]
39992:20220518:174146.198 proxy #22 started [poller #1]
39993:20220518:174146.199 proxy #23 started [poller #2]
39994:20220518:174146.200 proxy #24 started [poller #3]
39995:20220518:174146.202 proxy #25 started [poller #4]
39996:20220518:174146.203 proxy #26 started [poller #5]
39997:20220518:174146.204 proxy #27 started [unreachable poller #1]
39998:20220518:174146.218 proxy #28 started [icmp pinger #1]
And no more permission issue in our logs! You’re welcome.