BGP FlowSpec TCP flags: using SYN, ACK and RST matches without breaking real traffic
TCP flags can make FlowSpec rules precise against SYN, ACK or RST floods, but they become risky when they ignore connection state, asymmetric routing and legitimate protocol behavior.
TCP flags are useful but incomplete
They describe packet intent, not connection legitimacy by themselves.
ACK and RST rules are especially sensitive
They can collide with real established traffic or recovery behavior.
State belongs in the right layer
FlowSpec can reduce volume; downstream filters should validate context when needed.
TCP flag matching is one of the most tempting BGP FlowSpec features because many attacks appear to have an obvious flag pattern: SYN floods, ACK floods, RST storms or strange combinations that normal stacks rarely generate. Used carefully, TCP flags can remove a large amount of attack traffic upstream. Used carelessly, they can drop legitimate sessions, break asymmetric paths or hide the real problem behind a rule that looks elegant on paper. This article explains when TCP flag matching helps, how to scope it, why state still matters, and how to combine it with protected transit instead of treating it as a universal TCP firewall.
The operational problem
The problem with TCP flags is that they look more meaningful than they really are. A SYN packet starts a connection, an ACK often belongs to an established flow, and RST may close a session. But FlowSpec sees one packet at a time. It does not know whether a SYN is part of a legitimate burst, whether an ACK belongs to a real connection seen on another path, or whether RST traffic is a symptom of an overloaded application.
This becomes even more delicate with asymmetric routing. If inbound traffic is scrubbed through one provider and outbound traffic leaves elsewhere, an upstream FlowSpec rule cannot see the whole state machine. A rule that seems safe in a lab can become too aggressive in production when the return path, NAT, load balancers or proxies are involved.
TCP floods can be highly disruptive even when bandwidth is not the only issue. High packet rates, SYN backlog pressure, connection table exhaustion and application timeouts can damage a service before links are full. TCP flag matches can help remove the obvious part of that pressure upstream.
But because TCP is stateful, false positives are costly. Dropping the wrong ACK traffic can make existing sessions freeze. Dropping RST blindly can make recovery slower. Dropping SYN too widely can block new users. TCP flags must therefore be used as part of a measured incident response, not as a permanent recipe copied from another network.
TCP flags are useful but incomplete
They describe packet intent, not connection legitimacy by themselves.
ACK and RST rules are especially sensitive
They can collide with real established traffic or recovery behavior.
State belongs in the right layer
FlowSpec can reduce volume; downstream filters should validate context when needed.
Possible technical approaches
For SYN floods, a narrow rule may target excessive SYN packets toward a specific protected host or port, especially when downstream SYN validation or proxying is also present. For ACK floods, the rule should be more cautious: confirm that the traffic is not legitimate return traffic, monitor established sessions and avoid broad prefix-wide matches. For RST storms, prefer short-lived containment and downstream analysis before making the rule wider.
The best solution combines layers. FlowSpec removes the obvious flag-heavy flood upstream. The mitigation stack behind it tracks baselines, rate anomalies, service ports and behavior. If the customer runs a game or API service, additional logic can validate sessions instead of assuming that one flag pattern tells the whole story.
SYN matching
Useful against obvious connection-open floods when scoped to the attacked destination.
ACK matching
Powerful but risky; validate asymmetric routing and established traffic first.
RST matching
Use briefly and carefully because RST can be part of recovery or failure behavior.
Combined validation
Use downstream logic for context that FlowSpec cannot see.
How Peeryx designs this layer
Peeryx uses TCP flag matching only when it makes the upstream rule more precise. The preferred model is not to block “all ACK” or “all SYN” because an attack graph looks scary. Instead, the rule must be tied to the attacked destination, port, rate pattern and expected legitimate behavior.
Behind the upstream rule, Peeryx can keep more nuanced filtering in the mitigation layer. That separation is important: FlowSpec saves capacity and pps headroom, while the downstream stack protects session quality. For customers using protected transit, GRE/IPIP/VXLAN delivery or router VM, this gives room to react quickly without turning TCP into a blind upstream deny list.
Flag rules stay targeted
TCP flags are used to sharpen rules, not to create broad permanent blocks.
Session quality remains visible
Accepted traffic and connection behavior are watched after each change.
Works with protected delivery
The design supports transit, tunnel or router VM handoff depending on topology.
A web API receives a SYN flood toward TCP/443. The attack is heavy enough to overload downstream state tables. A narrow FlowSpec rule matching SYN traffic to the attacked destination can remove a major part of the flood, while downstream systems still handle real handshakes. That is a good use of TCP flag filtering.
Now consider an ACK flood where the protected path is asymmetric. A broad ACK drop may look effective on the upstream graph, but it may also remove real established traffic that FlowSpec cannot correlate. The safer option is to start with destination and rate constraints, watch legitimate sessions, and move more complex validation to the mitigation layer.
1. Identify the flag pattern
Confirm whether SYN, ACK, RST or a combination is actually dominant.
2. Scope by destination and port
Avoid prefix-wide TCP flag rules unless there is no safer alternative.
3. Check symmetry and state
Know whether return traffic and connection state are visible to the filtering layer.
4. Withdraw when the pattern changes
TCP flag rules should evolve with the attack rather than remain static.
Frequent mistakes to avoid
Blocking all ACK traffic because the attack is called an ACK flood.
Forgetting that FlowSpec does not track TCP sessions.
Using the same flag rule across multiple unrelated customer services.
Leaving a temporary flag rule in place after the attack changes.
Ignoring load balancers, NAT or proxies that alter the connection path.
FAQ
Are TCP flags supported by every FlowSpec provider?
Not always in the same way. You must verify supported components and actions with the upstream provider.
Is SYN matching safer than ACK matching?
Often yes, because SYN is tied to connection opening, but it still needs destination and rate scope.
Can FlowSpec replace SYN cookies or proxy validation?
No. It can reduce volume upstream, but validation still belongs in a state-aware layer.
Should TCP flag rules be used for gaming?
Only with care. Some game platforms also use TCP for control or HTTP/TLS flows, and false positives can affect login or resource loading.
Conclusion
The safest Anti-DDoS architecture is the one that gives each layer a clear job: routing steers traffic, upstream rules reduce obvious pressure, and downstream mitigation protects the service context.
Peeryx focuses on that operational clarity: protected IP transit, controlled delivery models and filtering decisions that are strong enough to stop attacks without turning legitimate traffic into collateral damage.
Resources
Related reading
To go deeper, here are other useful pages and articles.
Need to validate the right Anti-DDoS architecture?
Peeryx can review your prefixes, delivery model and attack exposure to propose protected IP transit, tunnel delivery or a gaming reverse proxy when it is the right fit.