BGP FlowSpec TCP flags: using SYN, ACK and RST matches without breaking real traffic

The operational problem

The problem with TCP flags is that they look more meaningful than they really are. A SYN packet starts a connection, an ACK often belongs to an established flow, and RST may close a session. But FlowSpec sees one packet at a time. It does not know whether a SYN is part of a legitimate burst, whether an ACK belongs to a real connection seen on another path, or whether RST traffic is a symptom of an overloaded application.

This becomes even more delicate with asymmetric routing. If inbound traffic is scrubbed through one provider and outbound traffic leaves elsewhere, an upstream FlowSpec rule cannot see the whole state machine. A rule that seems safe in a lab can become too aggressive in production when the return path, NAT, load balancers or proxies are involved.

Why it matters before an attack

TCP floods can be highly disruptive even when bandwidth is not the only issue. High packet rates, SYN backlog pressure, connection table exhaustion and application timeouts can damage a service before links are full. TCP flag matches can help remove the obvious part of that pressure upstream.

But because TCP is stateful, false positives are costly. Dropping the wrong ACK traffic can make existing sessions freeze. Dropping RST blindly can make recovery slower. Dropping SYN too widely can block new users. TCP flags must therefore be used as part of a measured incident response, not as a permanent recipe copied from another network.

TCP flags are useful but incomplete

They describe packet intent, not connection legitimacy by themselves.

ACK and RST rules are especially sensitive

They can collide with real established traffic or recovery behavior.

State belongs in the right layer

FlowSpec can reduce volume; downstream filters should validate context when needed.

Possible technical approaches

For SYN floods, a narrow rule may target excessive SYN packets toward a specific protected host or port, especially when downstream SYN validation or proxying is also present. For ACK floods, the rule should be more cautious: confirm that the traffic is not legitimate return traffic, monitor established sessions and avoid broad prefix-wide matches. For RST storms, prefer short-lived containment and downstream analysis before making the rule wider.

The best solution combines layers. FlowSpec removes the obvious flag-heavy flood upstream. The mitigation stack behind it tracks baselines, rate anomalies, service ports and behavior. If the customer runs a game or API service, additional logic can validate sessions instead of assuming that one flag pattern tells the whole story.

SYN matching

Useful against obvious connection-open floods when scoped to the attacked destination.

ACK matching

Powerful but risky; validate asymmetric routing and established traffic first.

RST matching

Use briefly and carefully because RST can be part of recovery or failure behavior.

Combined validation

Use downstream logic for context that FlowSpec cannot see.

A SYN match is not evidence of an attack

Consider a SYN flood toward an API on TCP port 443. A rule that discards every SYN to that destination also discards legitimate connection attempts. Narrowing the destination does not make this rule safe for the users of that service.

Before discarding traffic, identify an additional criterion that separates the unwanted packets from valid openings, and verify it against representative legitimate traffic. If no reliable distinction exists at the router, use a suitable downstream TCP validation mechanism. A policer limits the combined matching traffic; attack packets can consume that allowance too.

Specify both the match and the action, verify how the receiving router interprets the TCP flag mask, and observe new connections as well as established sessions. Keep a withdrawal procedure and an owner for expiry. FlowSpec itself does not prove that a packet belongs to a valid TCP session.

• RFC 8955 — Flow Specification Rules ↗

Frequent mistakes to avoid

• Blocking all ACK traffic because the attack is called an ACK flood.
• Forgetting that FlowSpec does not track TCP sessions.
• Using the same flag rule across multiple unrelated customer services.
• Leaving a temporary flag rule in place after the attack changes.
• Ignoring load balancers, NAT or proxies that alter the connection path.

FAQ