About a month ago, I gave an AI agent permission to try to break out of its security controls. A few of days ago, I discovered my Smart TV was hacked.
Here is how it happened.
How it started
After attending the AI Engineer World's Fair and returning on the hype of building my first Software Factory and being thoroughly creeped out by Erik Meijer's talk on how dangerous LLMs with tools really are I ended up setting up a dedicated local environment for running sandboxed coding agents both for my own interactive use and for the factory's agents. I felt pretty proud of my little project. Each agent had an ephemeral isolated environment with no access to the rest of the machine, my own machine, or any website I hadn't given explicit access to. With a mitmproxy between the agents and the LLMs I could also watch everything the agents thought or did.
Erik saw the post and replied:
Nice! Ask your model to find a way to escape the sandbox, even if it cannot find a way out, it will be fun to see what it tries. They are like octopi!
Obvious in hindsight - if you are building something to be secure, it's probably a good idea to test if it's actually secure, right? So I loaded a fresh session inside the sandbox, explained to the agent that it was in a locked down environment and instructed it to try to break out.
Fable 5 and Opus 5 both turned down the offer and flagged the conversation as a cyber security risk. Opus 4.8 took up the challenge, poked around for a couple of minutes, found an open SSH port, and ended the session seeming pretty satisfied with itself. This was kind of boring. Was my security posture really that effective?
I loaded up opencode (US-hosted inference and ZDR by default) and the very capable Kimi K3, and gave it the same prompt, and that's when it got interesting! The first pass spun for 20 minutes straight, mapped every repo it could reach with the GitHub credential I had in the sandboxes, worked out how my software factory invoked agents, and wrote a file explaining how it would take the whole thing apart. Snippet:
Full kill chain: from this sandbox I poison git hooks [...] your next git commit in any managed repository on the host executes my payload [...] your real-scoped CLI token, cluster credentials, production deployment credentials [...] The blast radius is the entire company.
The octopus had found its way out.
Ok, that was actually pretty fun. I pointed trusty Anthropic models at the findings, worked out some fixes, tried again. Followed a simple loop - find additional holes in one session, fix them in another, and repeat until I finally felt confident that this environment could be trusted with a fully autonomous coding agent running without supervision.
Expanding the scope
Curiosity got the best of me and I started wondering: if a single model in a fresh setup intentionally designed to be secure could be this vulnerable, what else could the octopi get into?
I started from my own laptop, made a list, fixed issues. Ran a session inside my homelab network, made a list, fixed issues. Put my laptop on my kid's isolated VLAN, made a list, fixed issues. The dedicated IoT network (because I don't trust my lightbulbs), made a list, fixed issues. I didn't have much of a master plan, would just spin up a session, work out a plan, let it run and move on to other things.
Three weeks of this led to massive changes in my security posture:
- Created a complete map of my network, well over 100 devices at any given time, now with complete identification and owner attribution.
- Isolated the guest network so anything on it can reach the internet and nothing else internally.
- Isolated 28 IoT devices from each other in case the lightbulb gets hacked.
- Found my NAS and one of my AI hosts sitting on the public internet via Syncthing!
- Discovered my 12-year-old had figured out how to leave the kid-friendly network (with parental controls and schedules) and joined the guest network in order to sneak hours of secret Roblox late at night.
- Went full Zero Trust:
- North-South: No more port forwarding homelab services through my firewall, Cloudflare tunnels with Cloudflare Access. My WAN IP is effectively dead to the world.
- East-West: Tailnet completely overhauled, all long-lived tokens removed. Only a couple of trusted devices can approve anything coming into the tailnet. All of my external nodes are accessible via Tailscale only now.
- NAS only shares explicitly with the machines that need it.
- Logging everywhere
- Segmented DNS monitoring and filtering
- IDS setup, with Signal alerts
- Moved every SSID from WPA2 to WPA3 and stood up the RF sentinel that later caught both the kid and the TV.
Over the weeks a couple hundred issues were found and hardened, some of them absolutely horrifying to consider what could have happened if a malicious actor had discovered them first.
The crowning jewel of this project is the new helper I named 'vigil'. It's an old Raspberry Pi 4B running Kali Linux with the full set of tools installed, mounted at the highest point on my property. Multiple WiFi adapters, one for constant passive listening with Kismet, a dedicated offensive one for drills and targeted recon. An SDR for monitoring RF, fingerprinting idling cars by their TPMS signatures, Bluetooth noise from nearby phones, alarm system interference attempts. When I'm not actively using it to investigate and harden, it's watching everything. It can take a phone probing for open wifi from someone's pocket, identify the model, line it up with whatever my security cameras caught during that window. The cameras use my local AI and an onboard Coral TPU for rapid object detection and image classification, license plate detection, etc. Every bit of evidence ends up on an encrypted disk for future investigation if needed.
It still needs some tuning as I don't believe this fox was actually attempting to do anything malicious over Bluetooth.
The payoff
You may not believe this, but I'm not actually paranoid. Or I wasn't. It's been more of a fun side hobby than anything serious. In all of the decades of messing with defensive computer security, I've never actually stopped anything interesting.
Until Monday.
Vigil kept alerting me on noise. Old kindles, phones, and other devices that hadn't been online when I first mapped the network. I would get an alert, figure out what it was and who it belonged to and update my map. It was tedious, but it's how you get a trusted baseline.
I kept getting alerts from the TV in the in-law apartment. My mother-in-law had been in Florida for a couple of months, and I knew everything was off in there, so I was initially just annoyed that her Smart TV was phoning home even when off. I added a few rules for some of the initial alerts that were clearly just checking for updates or sending privacy-invading tracking data back. But then the IDS fired an inbound phishing alert against it. Why was inbound traffic coming to a TV? A TV that had been turned off for months? I loaded my agent on Vigil and had it perform a complete investigation. It pulled connection history, bandwidth usage, DNS lookup history, IDS alerts, and did some packet sniffing. This thing is sending 10 GB of traffic a week! It is connecting to Russian email servers and Chinese URLs.
It turns out, the hardware maker behind it was a Chinese display manufacturer, and the box itself was a cheap Android TV running residential-proxy malware, part of the family researchers call BadBox 2.0. My TV (that was turned off!) was a proxy server that had been renting out my home network and geolocation to who knows who in order to do who knows what since May 2024. The FBI put out an alert about exactly this class of device last year.
Well, it's been sufficiently quarantined and I will do more forensics on it soon.
Open models made this possible
The model that wrote "the kill chain" was an open model. A significant number of the drills were performed by the cheap and effective Ox Alpha GLM-5.3 Flash. DeepSeek V4 Pro was used heavily. My Frigate security camera setup has been using Qwen 3.6 for months. I still love Anthropic models for complex tasks and my professional life, but open models are catching up fast and I'm genuinely rooting for their success. Comparable performance at a fraction of the cost, you can see the weights, and many of them have versions you can run on your own hardware!
My own primary AI model was recently migrated to Ornith-1.5-35B-A3B, 36B parameters total, but a mixture-of-experts design that activates only ~3B per token, so you pay for 36B in RAM and 3B in speed. It replaced Qwen3.6-35B-A3B, the same size and shape from Alibaba, which I'd run happily for months. I'm getting 70 tok/s against Qwen3.6's 67, and my Hermes agents that run on it have a noticeable improvement in quality: Ornith posts 79.0 on SWE-bench Verified. To put that in perspective, this is a model that is in the same range as Claude Opus 4.5 (80.9) and Gemini 3.1 Pro (80.6), that is running on a quiet machine in my basement that averages ~200 watts of power under load.
Stay curious
I guess the main takeaway here is to stay curious. This is the most exciting time to be in tech in my lifetime. The rate of change is going parabolic, and opportunities are exploding. And risks are certainly growing at least at the same pace. Stay on top of things by directly tinkering and exploring more than just reading from the sidelines. Point your agent at a side project or a random idea and see where it takes you. Erik is right, at the very least you will have fun!