Put a listener on a public IP and it will be found within minutes. Not by anyone interested in you specifically — by the permanent background scanning of the internet. Within a day you have tens of thousands of events. Within a week, logs nobody will ever open.
The instinct is to collect more. The actual problem is that you already have too much.
Almost all of it is the same four things
Sort a week of honeypot events by behaviour and the shape is consistent:
- Port sweeps. Connect, record that the port is open, leave. No payload.
- Banner grabs. Connect, read whatever is offered, leave.
- Credential stuffing. The same short list of default logins, endlessly.
- Known exploit attempts. A fixed payload for a specific old vulnerability, fired at everything.
None of these are about you. They are inventory. Treating each event as individually meaningful is the mistake that buries the small number that are.
Group by behaviour, not by source
Grouping by source address feels natural and is nearly useless — addresses rotate constantly, so one actor becomes hundreds of rows.
Group by what was attempted instead: which ports, in what order, with what payload fingerprint. Now a thousand rotating addresses running one scanner collapse into a single line with a count of one thousand. That line is genuinely informative, and it is one line.
Hash payloads early
Storing every byte of every probe is how honeypots fill disks. Hash the payload on the way in, keep one representative sample per hash, and count the rest.
fingerprint = sha256(normalise(payload))[:16]
if fingerprint in seen:
seen[fingerprint].count += 1
else:
seen[fingerprint] = Group(sample=payload, count=1)Normalising first matters. Strip the parts that vary per connection — timestamps, session identifiers, random padding — or every probe from the same tool gets its own fingerprint and you have gained nothing.
The digest is the whole point
Everything above exists so that a person can read the result. My digest answers three questions and stops:
- What is new? Fingerprints not seen in the previous baseline window.
- What changed sharply? Existing groups whose volume moved by more than an order of magnitude.
- What stopped? Groups that were steady and then disappeared.
That last one gets overlooked and is often the most interesting. A scanner that has run daily for two months and then goes quiet usually means something changed upstream — a takedown, a rewrite, or a shift in target selection.
What I would do differently
I baselined against the previous day for too long. Day-to-day variance is high enough that everything looks anomalous, so you learn to ignore the digest — which defeats it entirely. A trailing multi-week window with a volume threshold produced a report I actually read.
I also emulated protocols too deeply at the start. Every layer of convincing emulation is more code exposed to hostile input, and it did not improve the data. Shallow listeners tell you intent, which is all the digest needs.
The honeypot in question is shadowgate, and the summarisation described here is most of what it does.