Running AI Models Locally on Consumer Hardware
In the latest Playful Python newsletter, I cover the key concepts needed to run AI models locally on mid-tier consumer hardware, with a follow-up next week on running Qwen 3.6 35B on an RTX 5080.
In the latest Playful Python newsletter, I cover the key concepts needed to run AI models locally on mid-tier consumer hardware, with a follow-up next week on running Qwen 3.6 35B on an RTX 5080.
At the Chennaipy meetup, I demonstrated AI coding with the Pi harness using a local Qwen 3.6 35B model running on my home desktop over Tailscale — no token costs required.
At the Chennaipy meetup, I demoed Pi, an open-source agentic coding harness, and showed how it can extend itself to create a custom plan mode — including a funny moment where it wrote a plan to delete all my files instead of actually doing it.
Clean code matters more than ever in the AI age. Well-organized, documented, and modular code helps AI agents navigate codebases efficiently, reducing bugs and token costs.
Prompt instructions alone are not enough to constrain AI agents. Using hard boundaries like OAuth scopes and CI pipelines ensures agents stay within limits even when they ignore instructions.
Effective context management is crucial in agentic coding to avoid confusing the agent and driving up token costs.
A large part of human memory is learning to forget. If our brain remembered everything perfectly, it would get stuffed with irrelevant facts, making it harder to retrieve relevant information. This is what makes implementing memory for AI agents so hard - the ability to differentiate what is truly relevant is absolutely needed or else the memory just fills up with junk.
We are already there. I was listening to a talk yesterday and the first question was "how long before we are allowed to install and run local coding models?"
When I first started freelancing more than a decade ago, many people asked me how it was.
The tradeoff I used to talk about is that freelancing gives you time and flexibility, but in return you take on uncertainity and volatility. On the other hand, a corporate job is rigid …
Axios, one of the most widely used JavaScript libraries, has been compromised in a supply chain attack.