fail2ban Said 218 Bans. It Had Blocked Nothing

Until August 2026, the fail2ban service on our server looked busy. It detected attacks, counted them and logged bans: 218 of them for SSH alone. It had never blocked a single address. This post explains how a ban can exist only on paper, the four separate ways our firewall setup reported success while doing nothing, and the checks we now run instead of trusting the output.

218 bans, no firewall rules

The first sign was an address fail2ban had "banned" that was still trying passwords against SSH 77 minutes later, with 2,679 lines in the authentication log to show for it. A firewalled address cannot do that. One command settled it:

$ fail2ban-client get sshd actions
No actions for jail sshd

A fail2ban jail has two halves: a filter that decides an address has misbehaved, and an action that does something about it, usually adding a firewall rule. Our jails had the first half only. There were no fail2ban chains in iptables and nothing in nftables. Every "ban" was a row in fail2ban's own bookkeeping.

The cause was the configuration file. jail.conf had at some point been replaced with a stripped file of about 5 KB, where the stock one is about 23 KB. Its [DEFAULT] section still set the ban time, the retry limit and the backend, but defined no action or banaction at all. The only action lines left sat inside two sections that were not enabled, and one of them was named [ssh] rather than [sshd], so the jail that was running never saw it. Every enabled jail inherited an empty action.

We did not fix it by editing jail.conf. That file belongs to the fail2ban package, and an upgrade may replace it. Every setting that matters now lives in jail.local and jail.d/, which no package owns and which fail2ban reads after jail.conf, so they win whatever an upgrade restores.

Ban through the firewall you actually run

Our server's firewall is ufw. fail2ban's stock iptables action would technically work, because it inserts its rules ahead of ufw's chains. But any ufw reload rebuilds those chains and silently discards every fail2ban rule until fail2ban is next restarted. Bans would evaporate with no error, which is the same kind of invisible failure we had just found.

So the action we wrote issues ufw prepend deny from <ip> to any. A ban made that way survives a ufw reload, shows up in ufw status numbered next to hand-made rules, and sits at position 1. That last point matters because ufw is first-match-wins: a ban placed below the rules that allow web traffic would never be reached for a web request. prepend needs ufw 0.36 or later. We also give the action a name of our own rather than ufw, so no package can ever ship a file that replaces it.

"OK" from reload is not OK

Here is the trap that cost us the most time. With the configuration corrected, fail2ban-client -d showed the new action fully resolved, and fail2ban-client reload answered OK. The daemon still reported No actions for jail sshd. A reload re-reads the configuration but does not attach actions to jails that are already running. Only systemctl restart fail2ban applied it.

The rule that never matched

While investigating we also found out why our own scripts, which opened many SSH connections in quick succession, kept getting locked out of the server, and it was not fail2ban at all. ufw had a LIMIT rule on SSH, which drops an address that opens six or more connections in thirty seconds. Below it sat an explicit allow rule for the administrator's address. Because ufw stops at the first match, the limit rule always matched first and the allow rule was dead code. The kernel log showed the block on 6 August 2026; the rules were reordered the next day.

The lesson generalises: in a first-match-wins firewall, an allow-list entry only works if it sits above the rule it is meant to bypass. Check the order, not just the presence:

sudo ufw status numbered

Since fail2ban now prepends its bans at position 1, the numbers shift as bans come and go. What matters is that your allow rule stays above the limit rule.

A jail that stopped reading its log

On 22 August 2026 a set of addresses sent 2,946 requests to one of the sites on the server in six minutes. Running fail2ban-regex over that log afterwards showed 1,437 lines matching the jail's filter. fail2ban had logged zero matches and banned nobody, while staying alive and correctly following other logs. There had been no restart, no reload and no log rotation in that window. It simply stopped being told the file had grown, and recovered only when log rotation handed it a new file hours later.

That is a lost inotify watch. fail2ban's auto backend picks inotify where it can, and a lost watch is not something it recovers from. Our web jails now set backend = polling, which re-checks each file's size on an interval instead. For a handful of small log files the cost is nil.

fail2ban keeps its read position for each log in its SQLite database, so you can check whether a jail is keeping up by comparing that position with the file's real size:

sudo sqlite3 -readonly /var/lib/fail2ban/fail2ban.sqlite3 \
  "SELECT jail, path, lastfilepos FROM logs;"
sudo stat -c '%s %n' /var/log/apache2/your-site-access.log

Equal numbers mean the jail is caught up now. They do not prove it was caught up earlier; for that, compare its logged matches with what fail2ban-regex finds in the same file.

Verifying it actually works

Three commands in this stack report success while changing nothing:

The checks that mean something:

# 1. Is an action attached to every jail? (list yours from: sudo fail2ban-client status)
#    "No actions" means the jail bans nothing.
for j in sshd your-other-jails; do
    printf '%-20s: ' "$j"; sudo fail2ban-client get "$j" actions | tail -1
done

# 2. Are the bans real firewall rules?
sudo ufw status numbered | grep -c 'DENY IN'

# 3. After any configuration change, restart rather than reload.
sudo fail2ban-client -t && sudo systemctl restart fail2ban

Conclusion

None of these failures raised an error. fail2ban counted bans it never enforced, a reload said OK while leaving the jails unarmed, an allow rule sat where it could never match, and a jail stopped reading a log while staying perfectly healthy. Each time, the tool's own report was the thing that was wrong. A security control is only working if you can see its effect in the layer that does the blocking, so check the firewall, not the tool that claims to drive it.

Our Linux Administration course covers firewalls, services and logs on a real server, including how to check that a control does what it says.