The problem with honeypot data
A honeypot on a public address gets hit constantly, and almost all of it is the same automated background scanning. Raw logs grow fast and say very little. The hard part is not collection — it is compression.
shadowgate is built around that: capture faithfully, then aggressively summarise.
How it works
- Listeners accept connections on configured ports and record the exchange without pretending to be more than they are.
- Normalisation turns each interaction into a flat event: source, port, timing, payload fingerprint.
- Grouping collapses events by behaviour, so a thousand identical probes become one line with a count.
- Summary emits the digest — what is new, what changed, what stopped.
A honeypot is a deliberate liability. It is isolated, it holds nothing real, and it is never given a route to anything that matters. Anyone running one should assume it will eventually be compromised and design for that from the start.
Design decisions
Fingerprint payloads, do not store them all
Retaining every byte of every probe is how a honeypot fills a disk. Payloads are hashed and a representative sample kept per group. Repetition is counted, not duplicated.
Emulate shallowly, on purpose
Deep protocol emulation increases both attack surface and maintenance. The listeners are intentionally shallow: enough to see intent, not enough to become the vulnerability.
The digest is the product
If the daily summary is too long to read, it will not be read, and the whole thing was pointless. The digest is tuned to fit on one screen.
Getting started
git clone https://github.com/thecrewx/shadowgate
cd shadowgate
pip install -r requirements.txt
cp config.example.yml config.yml
python -m shadowgate --config config.ymlNext
- Longer-horizon baselining, so “unusual” is measured against weeks rather than yesterday.
- Optional export into aegiscore, so honeypot findings land in the same report as everything else.