Last week we noticed an annoying behaviour in Claude Code that could have cost us a lot if we did not catch it in time. Keep always in mind that an AI coding agent is as so many other LLMs trained to agree with the user. However, that is often not what we want, especially when we try to come up with new ways to do things and ask for a review. Let us see what happened and how we pushed back.
In last week’s post we connected two NVIDIA DGX Sparks. With this connection in place, we can now continue and run vLLM on our two-machine cluster to access the 256GB RAM of both machines.
The NVIDIA DGX Spark comes in many variations, like the Dell Pro Max with GB10 or the HP ZGX Nano AI Station. They vary in the case, but inside we have basically the same GB10 board with the same AI capabilities. The integrated 128GB RAM is great, but for mid-sized models more RAM would be helpful. Luckily for us, we can connect multiple DGX Sparks together and increase the model size we can load. Let us see how we can do that.
When we run LM Studio on a NVIDIA GB 10, we cannot use the graphical user interface. But there is the helpful lms command that allows us to do all the configuration in the command line. Let us see how we can work with this tool.
It is already two months since I covered GSD and Superpowers that give a more structured development approach to Claude Code. In the meantime, GSD released a fully autonomous application called GSD2 that works outside of Claude Code but still uses their infrastructure. With the new pricing model this kind of application is not covered by the monthly subscription and requires API pricing.
Let us see if we can run GSD2, or better GSD PI as it is now called, with our local LLMs.
In the first part we saw how easy it is to connect Claude Code to a local LLM and in the second part we measured the performance we can get with various local models. In this final part we explore the code we created while measuring the performance aspects.
In the first part last week we saw what we need to run Claude Code with a local LLM. In this second part we take a closer look at the different models and how they perform on different machines. Then the "right" model does not help us much if we cannot run it with the needed context size or when it only produces a few tokens per second. This is the hard part of running Claude Code against a local LLM and there is no solution that works everywhere.
Running Claude Code against a self-hosted LLM is much simpler than I expected. All we need are environment variables and the local LLM itself and we are good to go. However, that is just the start, and the challenges arrive when we try to do some real work. Let us see what we can do to tackle those challenges.
Creating the first CLAUDE.md file is not much work as we saw in last week's post. However, getting an initial file and something that helps us is not necessarily the same. Why not use Claude to improve our CLAUDE.md file?
As we saw in the insights report from last week, there are usually a few suggestions on how to improve our CLAUDE.md file. But if you never used a CLAUDE.md file that may not help you enough. In this post we take a close look at this helpful file and how we can use it to our advantage.