All posts

Your firewall dashboards might be overstating traffic by 14x

FortiGate interim logs carry running totals. Sum them and every long session gets counted many times over.

Bar chart: summing cumulative byte counters showed 9.9 TB where actual traffic was about 710 GB, a 14x overstatement

Your firewall traffic dashboards might be overstating volume by 14x. Ours were.

If you ship FortiGate logs to Elasticsearch (or Splunk, or anywhere) and compute traffic volume with SUM(network.bytes), here is a trap worth checking today.

The trap

FortiOS emits an interim log every two minutes or so for long-lived sessions. That sounds helpful, until you realize each interim log carries the session's cumulative byte counter, not the bytes since the last log. A 40-minute video call doesn't log its total once. It logs a running total about 20 times.

Sum those, and you re-count the whole session over and over.

What it cost us

When we caught it, we measured one 15-minute window two ways:

  • Naive SUM(network.bytes): 9.9 TB
  • Actual traffic: about 710 GB

That is roughly 14x inflation, concentrated exactly where you'd least want bad numbers: the long-lived sessions on high-value ports.

The fix

The fix was small.

  1. FortiOS already gives you the answer. fortinet.firewall.deltabytes is the bytes since the previous log of the same session. It is only present on interim and final logs of long sessions.
  2. We added one ingest processor: network.bytes_delta is deltabytes when present, otherwise network.bytes (short, single-log sessions).
  3. Every volume metric now sums bytes_delta. network.bytes remains correct for a single session's total: take the last log per session ID.

A processor along these lines does step 2. [VERIFY: replace with the real pipeline before publishing]

{
  "script": {
    "description": "Per-log byte delta that is safe to SUM",
    "source": "def d = ctx.fortinet?.firewall?.deltabytes; if (ctx.network == null) { ctx.network = [:]; } ctx.network.bytes_delta = d != null ? d : ctx.network.bytes;"
  }
}

The uncomfortable part

Our anomaly detection models had been trained on the inflated numbers, so we reset them and backfilled clean data. If your baselines were built on cumulative counters, your "normal" is wrong too.

Takeaway

Vendors log what's cheap to log, not what's safe to aggregate. Before you trust any SUM() over event logs, ask what one row actually represents: a per-event delta, or a running total?

Check your dashboards. If your firewall volume numbers look impressively large... maybe they are.

Discussion

No comments yet