Server Monitoring Basics: What to Watch and Why

Monitoring Is Not Optional

Every server should be monitored. Not because things fail constantly, but because when things fail, you want to know before your users do. A server running out of disk space at 3am because no one was watching is an avoidable incident.

This guide covers what to monitor, what alerting thresholds make sense, and the tools that get out of your way.

The Four Horsemen

Every server has four fundamental resources that can run out. Monitor these first:

1. CPU — How much processing power is in use 2. RAM — How much memory is in use 3. Disk — How much storage is used, and how fast it's being consumed 4. Network — Is the server reachable, and is it sending/receiving data normally?

Everything else (database connections, HTTP error rates, application-specific metrics) is secondary. Get these four right first.

Essential Metrics

CPU

What to measure:

• Overall CPU usage (0-100%)
• Per-core usage (a runaway process on one core is visible here)
• Load average (shows queue depth — 1.0 = one process waiting per core)

How to check:

# Quick status
top
# Better view
htop
# Load average (1, 5, 15 minute averages)
uptime
# Per-core stats
mpstat -P ALL 1 5

Thresholds:

• Warning: CPU > 80% sustained for 5+ minutes
• Critical: CPU > 95% sustained for 2+ minutes
• Load average > number of cores: Investigation needed

What high CPU actually means:

• Legitimate: High legitimate traffic, batch jobs running
• Problem: Infinite loop in code, cryptominer, runaway process

The distinction is usually visible in top — if one process is at 100% CPU and everything else is low, that's usually bad. If CPU is spread across many processes proportionally, that's usually fine.

Memory (RAM)

What to measure:

• Used vs total RAM
• Swap usage (non-zero swap is often fine; constantly used swap is bad)
• Available memory (free + buffers + cached)

How to check:

# Memory overview
free -h
# Detailed per-process
ps aux --sort=-%mem | head -20
# Swap in detail
vmstat 1 10

Thresholds:

• Warning: RAM > 85% used
• Critical: RAM > 95% used, or swap > 0 used on a system with available RAM
• Problem: OOM killer active (dmesg | grep -i "out of memory")

Understanding Linux memory: Linux uses available memory for disk caching (makes reads faster). free -h shows this as "available" — the number you care about. "Used" memory that is mostly cache is not a problem. The OOM killer kicks in when actual memory (not cache) runs out.

              total        used        free      shared  buff/cache   available
Mem:          7.7Gi       2.1Gi       3.2Gi       45Mi       2.3Gi       5.3Gi
Swap:         2.0Gi          0B       2.0Gi

Here, 5.3Gi is available — that's what matters.

Disk

What to measure:

• Filesystem usage (df)
• Disk I/O wait (how much CPU is waiting on disk)
• Inode usage (yes, you can run out of inodes before space)
• Disk I/O throughput and IOPS

How to check:

# Filesystem usage
df -h
# Inode usage (if you're having mysterious "disk full" issues)
df -i
# I/O stats
iostat -xz 1 5
# What's consuming disk space
du -sh /var/log/*
du -sh /var/www/*

Thresholds:

• Warning: Filesystem > 80% used
• Critical: Filesystem > 90% used
• Emergency: Filesystem > 95% used (you have minutes, not hours)

Common disk issues:

• Log files growing unbounded (/var/log)
• Old kernel packages accumulating (apt autoremove)
• Database growth without maintenance
• Temporary files not cleaned up

Network

What to measure:

• Server reachability (ping/ICMP)
• Port availability (web server on 80/443, SSH on configured port)
• Bandwidth usage (not always actionable, but useful context)
• Unexpected open ports (a new open port might be compromise)

How to check:

# Active connections (what's connected to what)
ss -tunapl
# Bandwidth per interface
nload
# Listening ports
ss -tlnp
# Firewall rules
iptables -L -n -v

Thresholds:

• Critical: Server unreachable from monitoring location
• Warning: Unusual number of connections (DDoS, compromise)
• Info: Bandwidth approaching plan limits (for metered connections)

The Tools

Quick and Dirty: Shell Scripts + Cron

For a single server, this is often enough:

#!/bin/bash
# /usr/local/bin/check-server.sh
USED=$(df / | tail -1 | awk '{print $5}' | sed 's/%//')
if [ $USED -gt 90 ]; then
    echo "Disk usage critical: ${USED}%" | mail -s "Server Alert: Disk" admin@example.com
fi
MEM=$(free | grep Mem | awk '{print ($3/$2) * 100}')
if [ $(echo "$MEM > 90" | bc) -eq 1 ]; then
    echo "Memory usage critical: ${MEM}%" | mail -s "Server Alert: Memory" admin@example.com
fi

Run it from cron every 5 minutes. Crude but works.

Better: Netdata

[Netdata](https://www.netdata.cloud/) is an open-source monitoring agent that installs in seconds and gives you real-time metrics without configuration. It has a web dashboard, supports alerts, and has reasonable defaults.

# Install
wget -O /tmp/netdata-kickstart.sh https://my-netdata.io/kickstart.sh
sh /tmp/netdata-kickstart.sh

Point your browser at http://your-server:19999. It's impressive for a free tool.

Production: Prometheus + Grafana

For multiple servers and serious monitoring, Prometheus + Grafana is the standard stack:

• Prometheus collects metrics via exporters
• Grafana visualises and alerts on them
• Alertmanager handles deduplication and routing

This is overkill for 1-2 servers but scales to thousands.

Lightweight Alternative: Grafana Cloud (Free Tier)

If you don't want to self-host Prometheus, Grafana Cloud offers a free tier with 10K metrics series, 50GB logs, and 50GB traces. Install the Grafana Agent on your servers, point it at Grafana Cloud, and you get hosted Prometheus + Grafana with alerting.

Alerting That Doesn't Suck

The goal: alerts that require action, nothing else.

Rules for Good Alerts

1. Alert on symptoms, not causes. "Disk full" is a symptom. "Database growth rate + disk monitoring" is a cause. Alert on the symptom, investigate the cause.

2. Set thresholds based on history, not defaults. If your server normally runs at 60% disk, an 80% alert is fine. If it normally runs at 78%, set the alert at 90%.

3. Use multiple severity levels. Warning = look into it. Critical = fix it now. Separate these.

4. Alert on rate of change, not just absolute values. A server filling disk at 1GB/hour is more urgent than one at 80% that's been stable for months.

5. No alert storms. If 20 services fail simultaneously, you want one alert, not 20. Group related alerts.

Example Alert Rules

groups:
  - name: server_resources
    rules:
      - alert: HighMemory
        expr: (node_memory_MemTotal_bytes - node_memory_MemAvailable_bytes) / node_memory_MemTotal_bytes * 100 > 85
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "Server {{ $labels.instance }} memory usage above 85%"
      - alert: CriticalMemory
        expr: (node_memory_MemTotal_bytes - node_memory_MemAvailable_bytes) / node_memory_MemTotal_bytes * 100 > 95
        for: 2m
        labels:
          severity: critical
        annotations:
          summary: "Server {{ $labels.instance }} memory critical"
      - alert: DiskSpaceLow
        expr: (node_filesystem_avail_bytes{fstype!~"tmpfs|fuse.lxcfs"} / node_filesystem_size_bytes) * 100 < 15
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "Server {{ $labels.instance }} disk space below 15%"

What to Monitor Beyond the Basics

Once the four horsemen are covered, consider:

• HTTP error rate — 5xx errors per minute, not just availability
• SSL certificate expiry — Alert at 30, 14, 7, 1 days before expiry
• DNS resolution — If your app depends on DNS, monitor it
• Database connections — Connection pool exhaustion is a common failure mode
• Queue depths — If you use Redis, RabbitMQ, etc., monitor queue length
• Backup success/failure — Don't just run backups, verify they worked

The Monitoring Stack Priority

1. First: Basic availability (can I reach the server?) 2. Second: The four horsemen (CPU, RAM, disk, network) 3. Third: Application-level health (HTTP status, error rates) 4. Fourth: Business metrics (conversions, signups, revenue if visible to the server) 5. Fifth: Advanced profiling (traces, profiles, detailed performance)

Don't skip steps. You can't do useful application profiling if you don't know whether the server itself is healthy.

Retention

Keep metrics:

• 1-second resolution: 24 hours (for debugging)
• 1-minute resolution: 30 days (for analysis)
• 1-hour resolution: 1 year (for capacity planning)

Most hosted monitoring services handle this automatically. If self-hosting, factor in storage costs.

Summary

The minimum viable monitoring for any server:

• CPU, RAM, disk, network metrics collected every 60 seconds
• Alerts on critical thresholds (90%+ disk, 95%+ RAM, 95%+ CPU)
• Alert delivery to somewhere you'll actually see (email, Slack, PagerDuty)
• Retention of at least 30 days for investigation

Everything else is refinement. Start here.