How to Run vLLM on Two NVIDIA DGX Sparks
In last week’s post we connected two NVIDIA DGX Sparks. With this connection in place, we can now continue and run vLLM on our two-machine cluster to access the 256GB RAM of both machines.
Blog
How to Connect Two NVIDIA DGX Sparks
The NVIDIA DGX Spark comes in many variations, like the Dell Pro Max with GB10 or the HP ZGX Nano AI Station. They vary in the case, but inside we have basically the same GB10 board with the same AI capabilities. The integrated 128GB RAM is great, but for mid-sized models more RAM would be helpful. Luckily for us, we can connect multiple DGX Sparks together and increase the model size we can load. Let us see how we can do that.
Helpful Commands for LM Studio's CLI Tool
When we run LM Studio on a NVIDIA GB 10, we cannot use the graphical user interface. But there is the helpful lms command that allows us to do all the configuration in the command line. Let us see how we can work with this tool.
How to Run GSD PI With a Local LLM
It is already two months since I covered GSD and Superpowers that give a more structured development approach to Claude Code. In the meantime, GSD released a fully autonomous application called GSD2 that works outside of Claude Code but still uses their infrastructure. With the new pricing model this kind of application is not covered by the monthly subscription and requires API pricing.
Let us see if we can run GSD2, or better GSD PI as it is now called, with our local LLMs.
How to Run Claude Code With a Local LLM (3/3)
In the first part we saw how easy it is to connect Claude Code to a local LLM and in the second part we measured the performance we can get with various local models. In this final part we explore the code we created while measuring the performance aspects.
How to Run Claude Code With a Local LLM (2/3)
In the first part last week we saw what we need to run Claude Code with a local LLM. In this second part we take a closer look at the different models and how they perform on different machines. Then the "right" model does not help us much if we cannot run it with the needed context size or when it only produces a few tokens per second. This is the hard part of running Claude Code against a local LLM and there is no solution that works everywhere.
How to Run Claude Code With a Local LLM (1/3)
Running Claude Code against a self-hosted LLM is much simpler than I expected. All we need are environment variables and the local LLM itself and we are good to go. However, that is just the start, and the challenges arrive when we try to do some real work. Let us see what we can do to tackle those challenges.
Use Claude to Improve the CLAUDE.md File
Creating the first CLAUDE.md file is not much work as we saw in last week's post. However, getting an initial file and something that helps us is not necessarily the same. Why not use Claude to improve our CLAUDE.md file?
Stop Repeating Yourself: Put Project Context in CLAUDE.md
As we saw in the insights report from last week, there are usually a few suggestions on how to improve our CLAUDE.md file. But if you never used a CLAUDE.md file that may not help you enough. In this post we take a close look at this helpful file and how we can use it to our advantage.
Get Insights Into Your Claude Code Usage
Claude Code gave us in April a new feature that allows us to see how we use Claude Code. Even better, it is not only a tool to show us what we did, but it helps us to get better. Let us see how it works.