Running AI Models Locally on Consumer Hardware
In the latest Playful Python newsletter, I cover the key concepts needed to run AI models locally on mid-tier consumer hardware, with a follow-up next week on running Qwen 3.6 35B on an RTX 5080.
In the latest Playful Python newsletter, I cover the key concepts needed to run AI models locally on mid-tier consumer hardware, with a follow-up next week on running Qwen 3.6 35B on an RTX 5080.
At the Chennaipy meetup, I demonstrated AI coding with the Pi harness using a local Qwen 3.6 35B model running on my home desktop over Tailscale — no token costs required.
At the Chennaipy meetup, I demoed Pi, an open-source agentic coding harness, and showed how it can extend itself to create a custom plan mode — including a funny moment where it wrote a plan to delete all my files instead of actually doing it.
Clean code matters more than ever in the AI age. Well-organized, documented, and modular code helps AI agents navigate codebases efficiently, reducing bugs and token costs.
Effective context management is crucial in agentic coding to avoid confusing the agent and driving up token costs.
A large part of human memory is learning to forget. If our brain remembered everything perfectly, it would get stuffed with irrelevant facts, making it harder to retrieve relevant information. This is what makes implementing memory for AI agents so hard - the ability to differentiate what is truly relevant is absolutely needed or else the memory just fills up with junk.
We are already there. I was listening to a talk yesterday and the first question was "how long before we are allowed to install and run local coding models?"
Axios, one of the most widely used JavaScript libraries, has been compromised in a supply chain attack.
The $200/month pricing for Claude Code may seem expensive, but it's actually heavily subsidized compared to the true cost of running AI models.
Claude Code may end up becoming the personal agentic framework that I've been looking for.
In the past when I needed some agentic behaviour (example: answer questions based on some documents), I used to look at hand rolling a solution. It's time consuming and not particularly easy to build and …