Inference Optimisation: make your local model faster, one measured change at a time
Inference Optimisation is a Claude Code skill by sybil-solutions for people who run a model on their own GPU. It works out how fast the hardware could go, then tries one change at a time against interleaved controls and keeps only what beats them.
git clone https://github.com/sybil-solutions/inference-optimisation-skill && mkdir -p ~/.claude/skills && cp -r inference-optimisation-skill/skills/inference-optimisation ~/.claude/skills/Ask Claude Code to "hill-climb decode and prefill for the model on :8000 under a 40 GiB cap" and this skill runs a tuning loop on your own model server. It measures tokens per second for one user and for several, prompt processing, time to first token, startup and memory. GitHub organization sybil-solutions publishes it under the MIT license. Anthropic doesn't publish it. For how skills load, see what Claude skills are.
Ceiling first, then the loop
Before changing anything, the agent computes a theoretical ceiling from your measured memory bandwidth and compute and writes it to THEORY.md. That tells you whether your current speed is poor or already close to the limit, and flags a target the hardware can't reach.
Then it changes one setting per experiment, each with its own config file. Run-to-run noise is typically ±5–10%, by the skill's own count, so it takes at least six decode samples, uses the median, and runs control, candidate, control. A change is kept only if it beats both controls and slows nothing else by more than about 3%. Failures go into PLAN.md too, so a rejected idea isn't tried twice.
A config isn't called best until it passes a long-context retrieval test and a concurrent soak run. For a lossless claim, the output with greedy decoding has to match the original.
How to install it
For Claude Code, clone and copy the skill folder:
git clone https://github.com/sybil-solutions/inference-optimisation-skill
mkdir -p ~/.claude/skills
cp -r inference-optimisation-skill/skills/inference-optimisation ~/.claude/skills/
Other agents can be pointed at the SKILL.md, or given PROMPT.md as the task. You need a model already served behind an OpenAI-compatible endpoint; the scripts use only Python 3's standard library and bash.
Example request
Use inference-optimisation on the model at [http://localhost:8000]. Hardware: [GPU or iGPU, memory]. Engine and version: [e.g. vLLM 0.x]. Model and quantization: [name, bits]. Targets: [decode tok/s for 1 user, prefill tok/s, time to first token]. Memory cap: [GiB]. Context I need to stay stable: [tokens]. Lossless changes only: [yes/no].
You get THEORY.md with the ceilings, PLAN.md with every experiment and why it was kept or dropped, a ledger.jsonl line per run, one config per run, and best-config.json with the winner.
Limits
- It needs a quiet machine. The harness wants AC power, a fixed performance profile and an idle host, and runs one GPU job at a time.
- No promised speedup. On the laptop iGPU the skill grew from, decode reached about 55% of its practical target.
- Some tricks fit one stack. Its list of settings to try marks which ones are general and which only apply to a specific engine or GPU.
- Young project. On November 18, 2026, its history held a single commit.
When to use it
Use it when you serve a model on your own hardware and want each speedup checked against controls before you switch configs or post a benchmark. If you call models through ChatGPT or an API, there's nothing for it to tune. For long jobs tracked in files, see Planning with Files.
Sources
- sybil-solutions/inference-optimisation-skill README on GitHub, accessed November 2026
- inference-optimisation SKILL.md on GitHub, accessed November 2026