Enterprise Linux Kernel Tuning, eBPF & Hardening Guide

Originally published on the HostingPodium Enterprise Engineering Blog.

Free SRE & Developer Diagnostics: Test your stack live with our DMARC Record Checker, Generator & SPF/DKIM Validator, Free Website Speed & Core Web Vitals Checker, or explore all 9 Free Webmaster & Cloud Engineering Tools.

This guide delivers a step-by-step reference architecture for Enterprise Linux in cloud environments, covering kernel tuning, storage layout, security hardening, and operational runbooks. It equips engineers with concrete configurations and clear trade-off analyses for production deployments.

Introduction

This guide presents a production-grade reference architecture for Enterprise Linux workloads running on public or private cloud infrastructure. It assumes a baseline of RHEL 9 (kernel 5.14) with extended support contracts, but the principles apply to any binary-compatible distribution.

Scope

The document covers kernel tuning, compute provisioning, storage layout, network stack, configuration management, observability, security hardening, high-availability design, disaster-recovery procedures, and a concrete runbook for incident response.

Baseline Platform Selection

Three common Enterprise Linux variants dominate cloud deployments:

• Red Hat Enterprise Linux (RHEL) 9 – 10-year lifecycle, certified images for AWS, Azure, GCP.

• Oracle Linux 9 – compatible kernel, optional Unbreakable Enterprise Kernel (UEK) with 4.14-based performance patches.

• CentOS Stream 9 – rolling preview of RHEL, suitable for development but not recommended for SLA-bound production.

When selecting a variant, compare the following metrics:

RHEL 9 remains the safest choice for workloads with strict SLA requirements.

Kernel Parameter Baselines

Baseline /etc/sysctl.d/99-enterprise-linux.conf should contain:

# Networking net.core.somaxconn = 65535 net.ipv4.tcp_tw_reuse = 1 net.ipv4.tcp_fin_timeout = 15 net.ipv4.ip_local_port_range = 1024 65535 # Virtual memory vm.swappiness = 10 vm.dirty_ratio = 15 vm.dirty_background_ratio = 5 # File system fs.file-max = 2097152 fs.nr_open = 1048576 

Apply with sysctl --system. These values have been validated on m5.large (2 vCPU, 8 GiB) instances handling 10 k concurrent connections.

Compute Provisioning

Cloud VM sizing must respect NUMA boundaries to avoid cross-node memory latency. Use the following pattern for instance families that expose multiple sockets:

# Example for a 4-socket, 64 vCPU instance cloud-init: cpu_affinity: - socket: 0 cores: 0-15 - socket: 1 cores: 16-31 - socket: 2 cores: 32-47 - socket: 3 cores: 48-63 

When the cloud provider does not expose socket topology, pin processes manually via taskset or cset and verify with numactl --hardware.

Storage Architecture

Enterprise workloads typically require a mix of block and file storage. The recommended layout:

• Root volume: XFS, mkfs.xfs -f -d su=64k,sw=1, 30 GiB, encrypted with LUKS.

• Data volume: LVM on top of NVMe SSDs, striped across all devices (e.g, lvcreate -i 4 -I 64K -L 500G -n data vg0).

• Backup volume: EBS gp3 with provisioned IOPS 10 k, mounted via noatime,nodiratime.

Performance target: ≥ 2 GB/s sequential read on striped LVM, ≤ 0.5 ms latency for 4 KB random reads.

Network Stack

For latency-sensitive services, enable SR-IOV and offload features where supported. Example /etc/sysconfig/network-scripts/ifcfg-eth0 for an AWS ENA interface:

DEVICE=eth0 BOOTPROTO=dhcp ONBOOT=yes HWADDR=02:7b:64:2e:1c:5a ETHTOOL_OPTS="-K tx off rx off tso off gso off gro off" 

Bond two NICs in active-backup mode for redundancy:

DEVICE=bond0 TYPE=Bond BONDING_OPTS="mode=1 miimon=100" 

Switch from iptables to nftables for reduced rule processing overhead. A minimal /etc/nftables.conf:

table inet filter { chain input { type filter hook input priority 0; policy drop; ct state established,related accept; iif "lo" accept; tcp dport { 22, 80, 443 } accept; } } 

Configuration Management

Ansible remains the most widely adopted tool for declarative provisioning on Enterprise Linux. A sample playbook to enforce the kernel parameters shown earlier:

- hosts: all become: true tasks: - name: Deploy sysctl config copy: dest: /etc/sysctl.d/99-enterprise-linux.conf content: | net.core.somaxconn = 65535 net.ipv4.tcp_tw_reuse = 1 vm.swappiness = 10 notify: Reload sysctl handlers: - name: Reload sysctl command: sysctl --system 

All roles must be version-locked in requirements.yml to guarantee repeatable builds across environments.

Observability Stack

Metrics collection relies on node_exporter (v1.6) and systemd-journal-gatewayd for log access. Prometheus scrape configuration for a fleet of 200 instances:

scrape_configs: - job_name: 'linux_nodes' static_configs: - targets: ['10.0.0.1:9100','10.0.0.2:9100',..] relabel_configs: - source_labels: [__address__] regex: '(.*):9100' target_label: instance replacement: '${1}' 

Alerting rule for CPU saturation:

- alert: CPUHighUtilization expr: avg by (instance) (rate(node_cpu_seconds_total{mode="user"}[5m])) > 0.85 for: 2m labels: severity: critical annotations: summary: "CPU usage > 85% on {{ $labels.instance }}" runbook: "{{ .Values.runbookUrl }}/cpu_saturation.md" 

Log rotation is handled by logrotate with a 7-day retention policy for /var/log/messages and a 30-day policy for application logs.

Security Hardening

Enable SELinux in enforcing mode and lock down the policy to targeted with custom modules for container runtimes. Example /etc/selinux/config:

SELINUX=enforcing SELINUXTYPE=targeted 

Auditd rule to capture all privileged command execution:

-w /usr/bin/sudo -p x -k privileged_exec 

Run OpenSCAP against the xccdf_org.ssgproject.content_profile_cis profile weekly. Non-compliant findings must be triaged within 48 hours.

High Availability Design

Two patterns dominate HA for Linux services:

• Active-passive – keepalived VRRP provides a floating IP; Pacemaker manages resource failover.

• Active-active – distributed databases (e.g, Galera) synchronize state; load balancer (HAProxy) performs health checks.

Sample keepalived configuration for a virtual IP 10.0.0.100:

vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 51 priority 150 advert_int 1 authentication { auth_type PASS auth_pass secret123 } virtual_ipaddress { 10.0.0.100/24 dev eth0 } } 

Fencing is mandatory for split-brain protection. Use stonith:external/ipmi on bare-metal and cloud-provider-fence for virtualized nodes.

Disaster Recovery

Backup strategy combines snapshot-based recovery for block devices and file-level archiving for configuration files.

• Daily EBS snapshots retained 30 days (RPO ≈ 12 h).

• Nightly borg backup of /etc, /var/lib/mysql, and /opt/app to an S3 bucket (RPO ≈ 24 h, RTO ≈ 15 min).

Recovery test cadence: quarterly full-restore drills on a separate VPC.

Operational Runbook

Incident: CPU usage spikes above 90 % for more than 5 minutes.

• Confirm alert in Prometheus UI.

• SSH to the affected host (use bastion, MFA).

• Run top -b -n1 | head -15 to identify offending process.

• If the process is a known service, restart via systemctl restart service. Record the action in the ticket.

• If the process is unknown or malicious, capture a core dump with coredumpctl dump -o /tmp/core.


• Check for runaway threads using ps -eo pid,ppid,cmd,%cpu --sort=-%cpu | head.

• Apply a temporary cpulimit rule if immediate mitigation is required.

• After stabilization, review /var/log/audit/audit.log for privilege escalation attempts.

• Update the runbook if a new pattern emerges.

Trade-off Summary

Choosing the right combination depends on SLA targets, cost constraints, and operational skill set.

For organizations looking to deploy bare-metal Linux infrastructure without virtualized CPU steal or noisy neighbor contention, explore our DigitalOcean Dedicated Alternative guide.

Free Webmaster, SRE & Cloud Diagnostic Tools (No Signup Required)

Verify and benchmark your production infrastructure directly from your browser using the HostingPodium Free Webmaster & Developer Suite:

Related Enterprise Architecture & Benchmarks

• Canonical Engineering Guide: Enterprise Linux Kernel Tuning, eBPF & Hardening Guide
• Managed Engineering Practice: Cloud Platform Engineering Services & Solutions
• Infrastructure Benchmark Comparison: AWS Cost Alternative: Zero Egress Fees & Dedicated Bare Metal