SELinux Debugging

Reading AVC denials, translating them with audit2allow, fixing labels with semanage/restorecon, toggling booleans, and shipping a minimal custom policy module.

Debugging order of operations
  • Never disable SELinux to "fix" a failure — set it to permissive briefly, reproduce, collect the AVCs, then put it back to enforcing.
  • 95% of production AVCs are one of: wrong file label, missing boolean, non-standard port, or a confined daemon writing outside its writable types. Check those before writing policy.
  • Labels live in the filesystem xattr security.selinux. Copying a file with cp carries old labels; moving with mv preserves them. Both can bite. restorecon is the fix.
  • audit2allow is a translator, not a thinking tool. Always read what it produces before semodule -i.
  • If a vendor RPM ships a policy module (-selinux package), install it instead of hand-rolling one.

Mode: enforcing vs permissive

Only three modes matter: enforcing, permissive, disabled. Disabled is a last resort — the kernel stops labelling files and re-enabling later requires a full relabel.

getenforce                 # current mode
sestatus                   # verbose: policy, mode, from-config vs current

setenforce 0               # temporarily permissive (reverts on reboot)
setenforce 1               # back to enforcing

grep -E '^SELINUX=' /etc/selinux/config    # persistent setting
Permissive domains. You don't have to put the whole system in permissive to debug one daemon. Mark only that domain permissive with semanage permissive -a httpd_t; remove with -d. The rest of the system stays enforcing.

Reading audit.log

SELinux denials are logged by the kernel as audit events of type AVC (Access Vector Cache). On systems running auditd they land in /var/log/audit/audit.log. When auditd is stopped, kernel AVCs appear in dmesg / journalctl -k.

tail -f /var/log/audit/audit.log | grep -i avc
journalctl -f _TRANSPORT=audit
journalctl -k | grep -i 'type=1400'     # 1400 = AVC

A raw AVC line looks like this:

type=AVC msg=audit(1713200000.123:456): avc:  denied  { read } for
  pid=1234 comm="nginx" name="index.html" dev="dm-0" ino=98765
  scontext=system_u:system_r:httpd_t:s0
  tcontext=unconfined_u:object_r:admin_home_t:s0
  tclass=file permissive=0

Searching AVCs with ausearch

Don't grep raw — ausearch parses records and reassembles multi-line events.

ausearch -m AVC,USER_AVC -ts recent            # last 10 minutes
ausearch -m AVC,USER_AVC -ts today
ausearch -m AVC -ts today -i                   # interpret numeric fields
ausearch -m AVC -c nginx                       # by command
ausearch -m AVC --success no -ts recent        # only denials, not grants

ausearch -m AVC -ts recent --raw | audit2why   # pipe into audit2why
ausearch -m AVC -ts today   --raw | audit2allow -m mylocal
On dontaudit. Some denials are silently suppressed by dontaudit rules to keep logs quiet. If a daemon is failing and nothing shows in audit.log, disable dontaudit temporarily: semodule -DB (rebuilds base policy with dontaudits stripped). Re-enable with semodule -B.

AVC anatomy and common shapes

Every denial has the same skeleton. Learn the fields and you can triage without tools.

FieldMeaningWhat to look at
scontextSource (subject) — who tried the accessThe domain of the running process, e.g. httpd_t
tcontextTarget — the thing accessedFile type, port type, socket type
tclassObject classfile, dir, tcp_socket, unix_stream_socket, capability
{ }Permissions deniedread write open getattr connectto name_bind
comm / pidProcessIdentify who was denied
permissiveEnforcement at the moment0 = blocked, 1 = logged-but-allowed

Shape recognition saves minutes:

audit2why — what does this denial mean

Given an AVC, audit2why classifies it: missing allow rule, constraint violation, policy boolean, RBAC, or MLS. Crucially it will suggest a boolean to flip if one exists.

ausearch -m AVC -ts recent | audit2why
type=AVC msg=audit(1713200000.123:456): avc: denied { name_connect } for
    pid=1234 comm="httpd" dest=3306
    scontext=system_u:system_r:httpd_t:s0
    tcontext=system_u:object_r:mysqld_port_t:s0 tclass=tcp_socket

    Was caused by:
    The boolean httpd_can_network_connect_db was set incorrectly.
    Description: Allow httpd to connect to database servers.
    Allow access by executing:
    # setsebool -P httpd_can_network_connect_db 1
When audit2why names a boolean, use it. Booleans are the supported knobs — they survive policy updates. Hand-written allow rules do not benefit from the same maintenance.

audit2allow — generate candidate policy

When there is no boolean, audit2allow turns raw AVCs into Type Enforcement (TE) rules. Read the output before loading. A daemon that legitimately needs read on a specific directory is one thing; a blanket allow httpd_t default_t:file { read write open }; is a hole.

ausearch -m AVC -ts recent | audit2allow -m mylocal
ausearch -m AVC -ts recent | audit2allow -M mylocal   # -M = write mylocal.pp + mylocal.te

semodule -i mylocal.pp       # load
semodule -l | grep mylocal   # verify
semodule -r mylocal          # remove
module mylocal 1.0;

require {
    type httpd_t;
    type app_data_t;
    class dir { read search open };
    class file { read getattr open };
}

#============= httpd_t ==============
allow httpd_t app_data_t:dir { read search open };
allow httpd_t app_data_t:file { read getattr open };

File contexts: semanage fcontext and restorecon

A running process has a type (httpd_t). Files have types too (httpd_sys_content_t). The policy says which process-type can do what to which file-type. Get the labels right and most denials disappear.

ls -Z /var/www/html/                         # current labels
matchpathcon /var/www/html/index.html        # what SHOULD be labelled
matchpathcon -V /var/www/html/index.html     # difference from current

semanage fcontext -l | grep httpd             # installed rules
semanage fcontext -a -t httpd_sys_content_t \
  '/srv/myapp(/.*)?'                          # persistent rule
semanage fcontext -a -t httpd_sys_rw_content_t \
  '/srv/myapp/cache(/.*)?'                    # writable subtree

restorecon -Rv /srv/myapp                     # apply rules to the filesystem

semanage fcontext -d '/srv/myapp(/.*)?'       # remove a rule
restorecon -RF /srv/myapp                     # force relabel to default

Regex patterns use SELinux file-context syntax, not PCRE. The trailing (/.*)? is the idiom for "this directory and everything inside it".

Do not chcon in production. chcon sets a label directly but a relabel (restorecon -R /, fixfiles, or an autorelabel on boot) will overwrite it. Always use semanage fcontext -a + restorecon so the change is durable.

Booleans: setsebool

Booleans are policy toggles that let admins change behaviour without compiling new modules. They are the supported knobs for common cases.

getsebool -a                                 # list all
getsebool -a | grep httpd                    # only httpd
getsebool httpd_can_network_connect_db       # single value

setsebool    httpd_can_network_connect_db 1  # runtime only
setsebool -P httpd_can_network_connect_db 1  # persistent

semanage boolean -l | grep -i httpd          # descriptions

Booleans worth knowing:

BooleanUse case
httpd_can_network_connectReverse-proxy / app-server outgoing TCP
httpd_can_network_connect_dbhttpd → Postgres/MySQL over TCP
httpd_can_sendmailPHP mail(), web apps emitting mail
httpd_unifiedSimplify content type rules for mostly-static sites
nis_enabledNIS/Kerberos auxiliary network accesses from many daemons
samba_export_all_rwSamba exporting arbitrary paths read-write
ftpd_full_accessvsftpd users accessing anything readable as their UID
ssh_sysadm_loginSSH into the sysadm_r role for confined users

Ports: semanage port

Ports have types too. httpd_t may bind http_port_t (80, 443, 8080, 8443, …). To run httpd on 8081 you must label 8081 as http_port_t.

semanage port -l | grep http                   # labelled ports
semanage port -a -t http_port_t -p tcp 8081     # add
semanage port -m -t http_port_t -p tcp 9000     # modify existing
semanage port -d -t http_port_t -p tcp 8081     # delete

Same pattern for ssh_port_t, postgresql_port_t, mysqld_port_t, dns_port_t, smtp_port_t, etc.

Custom policy modules

When the problem is not a label, a boolean, or a port, you need a custom module. The modern workflow uses CIL or TE source and the semodule toolchain.

mkdir -p ~/myapp-selinux && cd ~/myapp-selinux
cat > myapp.te <<'EOF'
module myapp 1.0;

require {
    type httpd_t;
    type myapp_data_t;
    class dir { search read open };
    class file { read getattr open };
}

allow httpd_t myapp_data_t:dir  { search read open };
allow httpd_t myapp_data_t:file { read getattr open };
EOF

make -f /usr/share/selinux/devel/Makefile myapp.pp
semodule -i myapp.pp
semodule -l | grep myapp

Where myapp_data_t comes from is the usual next question: either it already exists (seinfo -t | grep myapp) because the vendor shipped a module, or you declare it yourself with type in the TE and register an fcontext entry mapping your paths to that type.

Ship policy with the app. A package that needs a policy should ship an <app>-selinux subpackage or install its .pp into /usr/share/selinux/packages/. The admin should not be hand-editing policy on hosts.

setroubleshoot and sealert

The setroubleshoot-server package adds a daemon that watches audit.log, classifies denials, and writes human-readable analyses to /var/log/messages (and via D-Bus to the GUI on desktops). Each alert includes an ID you pass to sealert for detail.

dnf install -y setroubleshoot-server
systemctl enable --now setroubleshootd

grep sealert /var/log/messages | tail -5
sealert -l e5f6a7b8-1234-4abc-9def-0123456789ab      # by alert ID
sealert -a /var/log/audit/audit.log                   # analyse whole log
SELinux is preventing /usr/sbin/nginx from name_bind access on the tcp_socket port 8081.

*****  Plugin bind_ports (99.5 confidence) suggests  ***********************
If you want to allow /usr/sbin/nginx to bind to network port 8081
Then you need to modify the port type.
Do
# semanage port -a -t http_port_t -p tcp 8081
Confidence is not correctness. The "99.5% confidence" is heuristic. Plugins sometimes suggest loose allow rules when the real answer is a label fix. Read the suggestion, then verify.

Walkthrough: a confined service failure

You deploy a new Flask app under gunicorn behind nginx. Systemd reports the unit active but curl http://localhost/ returns 502. nginx logs show connect() to unix:/run/myapp/app.sock failed (13: Permission denied).

  1. Confirm it is SELinux, not a Unix permission problem:
    ls -lZ /run/myapp/app.sock
    # srw-rw---- 1 myapp nginx system_u:object_r:var_run_t:s0 /run/myapp/app.sock
    The mode is fine; the label var_run_t is generic — nginx wants to connectto something it is allowed to connect to.
  2. Collect the AVC:
    ausearch -m AVC -ts recent -c nginx
    # avc: denied { connectto } for pid=1234 comm="nginx"
    #   path="/run/myapp/app.sock"
    #   scontext=system_u:system_r:httpd_t:s0
    #   tcontext=system_u:system_r:init_t:s0
    #   tclass=unix_stream_socket
  3. Try the boolean first:
    getsebool httpd_can_network_connect       # off
    setsebool -P httpd_can_network_connect 1
    systemctl reload nginx
    curl -I http://localhost/
    502 gone — the boolean is the right answer because the upstream socket is a network-style resource from nginx's point of view.
  4. If the boolean had not covered it, the next step would be labelling /run/myapp(/.*)? as httpd_sys_content_t or shipping a tiny module granting httpd_tinit_t:unix_stream_socket connectto.

Common AVC shapes and fixes

Denial shapeLikely causeFix
httpd_t denied read on admin_home_t / user_home_t docroot moved or created under a home dir; labels carried over semanage fcontext -a -t httpd_sys_content_t '/srv/site(/.*)?' && restorecon -Rv /srv/site
httpd_t denied name_bind on port_t Service bound a port not in http_port_t semanage port -a -t http_port_t -p tcp 8081
httpd_t denied name_connect on mysqld_port_t App server to DB over TCP setsebool -P httpd_can_network_connect_db 1
postgresql_t denied write on default_t under /data Moved data dir without relabeling semanage fcontext -a -t postgresql_db_t '/data/pgsql(/.*)?' && restorecon -Rv /data/pgsql
ssh_t denied read on user_home_t for .ssh/authorized_keys Home dir on a non-standard path (e.g. /data/home) without equivalence semanage fcontext -a -e /home /data/home && restorecon -Rv /data/home
Unit file execute denied by init_t Binary has wrong type (often admin_home_t after cp from root's home) restorecon -Rv /usr/local/bin/myapp (or add fcontext first)
ftpd_t denied search on arbitrary user dirs vsftpd trying to list /home/* without boolean setsebool -P ftpd_full_access 1 (scope carefully)
unconfined_service_t denied execmem JIT/JVM; some runtimes need writable-executable pages Prefer domain-specific boolean (httpd_execmem, etc.) over broad allows
Silent failure, no AVC dontaudit is suppressing semodule -DB; reproduce; then semodule -B
Denial only after reboot Runtime-only setsebool/semanage port change Always use -P (boolean) / default persistence, then re-test
Whole filesystem mis-labelled after /.autorelabel failure Relabel interrupted fixfiles -F onboot && reboot
Collect, don't guess. When a user reports "SELinux is broken", run ausearch -m AVC -ts today --raw > /tmp/avcs.log before touching anything. The raw AVCs are the evidence; everything else is reasoning on top of them.

Cross-reference