Skip to content
Field Notes

E2 · C&Thoth · Aug 2020 – Feb 2023

Stopping 3 a.m. DDoS attacks with no one on the keyboard

Automated detection and blocking using logs that were already there

Detect → block automated0 human steps

  • Azure
  • DDoS
  • Automation
  • PowerShell
  • Response automation
03:00Office lights outGame serviceMitigation flow logsSources with abnormal SYN volumeNSG block rules203.0.113.x203.0.113.x203.0.113.xNo one stepped in. The service kept running.
The office stays dark and the game service stays green.

In one line

Connected Azure logs that already existed so DDoS detection through blocking runs with no human steps.

Background

I was the dedicated engineer for a global game running on 3,500 servers. The service regularly faced DDoS attacks (floods of traffic meant to knock a service offline).

Problem

Attacks often clustered at night and on weekends, when the fewest people were available.

While someone read the alert, logged in, found the attacking IPs, and added block rules, the service shook.

Judgment

I compared three options.

Add night-shift staff

Attacks are unpredictable, so round-the-clock standby is expensive and hard to sustain.

Buy a commercial security product

Procurement and contracts take time, and the client bears the cost.

Use the logs already being written

Azure's DDoS protection was already logging attack sources. No new data was needed, only a connection.

Plain explanationWhat is a SYN flood?

A TCP connection starts with a three-step greeting, like a phone call: SYN ("Can we talk?") → SYN-ACK ("Sure, ready?") → ACK ("Yes").

An attacker sends only the first step, over and over, and never answers. The server keeps seats open while it waits.

Soon there are no seats left for real users. It is like a restaurant taking thousands of reservation calls from people who never show up.

SYN → SYN-ACK → ACKSYN SYN SYN

What I did

  1. Confirm the logs: Azure DDoS protection writes Mitigation Flow Logs, and they were already landing in Azure Log Analytics (a log storage and query service).
  2. Define detection: Sources sending abnormally many TCP SYN packets, unlike normal connections, became attack candidates.
  3. Query on a schedule: A query in KQL (Azure’s log query language) runs on a fixed interval and pulls the source IPs over the threshold.
  4. Block automatically: A PowerShell script adds those IPs to the block rules of an NSG (Azure network security group, a set of firewall rules).

Result

Detection through blocking runs with no human steps, which removed the night and weekend response delay.

It took no new product and no extra staff, only connecting data that already existed.

What I learned

Security automation starts by asking what the logs you already have can do, before adding a new tool.

The higher the pressure, the more you need both the immediate block and a structure that holds when it happens again.

Connected stories

Open to work · Cloud infrastructure and security