I built a lab in AWS to watch an attack happen. Terraform provisioned a VPC, I put a PostgreSQL database on the public internet on purpose, pointed a script at it, and piped VPC Flow Logs into Splunk so my partner could watch from the SOC side. Then I wrote an incident report describing a single external threat actor conducting a systematic port scan. Months later I went back and read my own flow logs line by line. There were twenty-eight source addresses in the sample. I had written about one of them, and it was the wrong one. The setup One VPC at 10.0.0.0/16, two subnets across two availability zones, both public. An RDS PostgreSQL instance with the line that makes security people wince: publicly_accessible = true and a security group allowing 0.0.0.0/0 inbound on 5432, 22, and 80. Deliberately wrong, in the specific way production gets it wrong. Flow logs to S3, S3 into Splunk. A Node script ran a dictionary attack against db_admin on 5432, knocked on 22 and 80, and mixed in successful logins with the real password so the SOC side had to separate authorized traffic from attacks. Three different jitter ranges so none of it looked like a metronome. What was actually in the logs A 1,000-line sample from that capture: Source Lines What it was 10.0.1.149 445 My own traffic, inside the VPC 194.87.190.127 245 The simulated bot 136.35.186.87 183 A real internet actor 25 other addresses 1 to 3 each Internet background radiation 876 ACCEPT, 124 REJECT. The three noisy sources are the ones I designed. The twenty-five quiet ones are the story, and I never looked at them. The quiet ones were not mine Here are destination ports that appear in the REJECT lines: 6379 Redis 5902, 5931, 5937, 5959 VNC 10250, 10255 Kubernetes kubelet 137 NetBIOS 8008, 8022, 8081, 8088, 8090, 8103, 8389 assorted alt-HTTP Nothing in my lab ran Redis. Nothing ran VNC. There was no Kubernetes cluster and no NetBIOS. My script targeted exactly three ports. Twenty-nine distinct ports show up in the rejections. That is not one attacker sweeping a host. That is the internet's ambient scanning layer finding a fresh public IP and checking whether anything soft is listening. The source addresses back it up: several sequential hosts in one cloud range, a scattering of others across hosting providers known for this. Every one of those lines arrived because I set one boolean to true. Not one of them appears in the incident report I wrote. What I put in the ticket instead Threat Actor IP Address: 10.0.1.179 ... an external threat actor is systematically scanning for open database and secure shell ports. 10.0.1.179 sits inside 10.0.1.0/24, one of the two subnets I had defined twelve lines earlier in my own Terraform. RFC 1918 reserves 10.0.0.0/8, 172.16.0.0/12, and 192.168.0.0/16 for private use, and none of it routes on the public internet. If a private address is in your srcaddr field, whatever produced it was already inside your network. I had a genuine external actor in the same dataset, 136.35.186.87, doing a real multi-port scan. I escalated the wrong IP and labeled my own traffic as the threat. Three things flow logs will not tell you Not who authenticated. A flow log record carries source and destination address, ports, protocol, packet and byte counts, timestamps, and ACCEPT or REJECT. No payload, no usernames, no auth result. My script really was throwing bad passwords, and my analyst notes really did say credential stuffing, but the flow logs were never the evidence for that. A backup job with a broken retry loop looks identical at that layer. Authentication failures live in the RDS PostgreSQL error log, which is an entirely separate pipeline. Not what a big number means. My ticket led with 15,025 blocked events because it was the largest number on the dashboard. I never worked out which rows produced it. Looking at the sample now, rejections are spread across twenty-nine ports, most of which nothing in my lab ever touched, so that headline figure was mostly counting strangers rather than my simulation. I could have known that in five minutes by grouping by port. Instead I put it in a SEV-1 and moved on, and by the time I thought to check, the account was closed and the data was gone. Not intent. ACCEPT means the packet passed your security group. It does not mean something answered, and REJECT does not mean you blocked an attack. Both are network-layer facts about your own configuration, not judgments about the traffic. The part that held up One line: publicly_accessible = false terraform apply, then re-run the simulator: $ node attack_sim.js [+] Initiating connection to database... [-] Error: Connection Timeout. Network unreachable. Worth being precise about what that flag does, because I got this wrong too. It stops RDS from handing out a publicly resolvable endpoint. It does not move the instance to a different subnet, and my DB subnet group still contained the same two public subnets it always had. My report claimed the database was migrated into a private subnet. The code does no such thing. The narrower claim is the true one, and it is still worth something: the database is no longer reachable from outside the VPC, the change is version controlled, and anyone can clone the repo and reproduce the before and after. What I would tell someone building this Read the long tail. I spent all my attention on the three sources generating volume, because volume is what dashboards are built to show. The twenty-five sources with one line each were the only unplanned thing in the entire dataset, and they were the answer to the question the lab was supposed to ask. Check whether your attacker is actually external. Look at srcaddr, compare it to your own CIDR blocks. It takes ten seconds and it is embarrassing in a very specific way to skip. Work out what a number counts before you escalate it. If you cannot say which rows produced a figure, it does not belong in a ticket. Know what each log source can prove. Flow logs answer who talked to what, on which port, and whether the packet landed. They cannot answer whether someone tried to log in. Different question, different log, different pipeline. The lab was supposed to demonstrate that exposing a database to the internet gets you found. It did that on day one, in writing, in my own S3 bucket. I just described the wrong attacker. Terraform, the attack script, and the flow log sample are all here if you want to check my work: aws-incident-response-honeypot Detection and dashboards by Kenny Barr. If you have run something like this, what was in your long tail?