Skip to content AI News: GPT-6.2 Astra and Sol are out: Astra gets cheaper, Sol keeps its price →
SK-65 · Skill · coding · rev. 11/2026

Inference Optimisation: make your local model faster, one measured change at a time

Inference Optimisation is a Claude Code skill by sybil-solutions for people who run a model on their own GPU. It works out how fast the hardware could go, then tries one change at a time against interleaved controls and keeps only what beats them.

Publishersybil-solutions Community
LicenseMIT
Works inClaude Code
Checked18 Nov 2026
Install
git clone https://github.com/sybil-solutions/inference-optimisation-skill && mkdir -p ~/.claude/skills && cp -r inference-optimisation-skill/skills/inference-optimisation ~/.claude/skills/
Source on GitHub ↗★ 962 18 Nov 2026

Ask Claude Code to "hill-climb decode and prefill for the model on :8000 under a 40 GiB cap" and this skill runs a tuning loop on your own model server. It measures tokens per second for one user and for several, prompt processing, time to first token, startup and memory. GitHub organization sybil-solutions publishes it under the MIT license. Anthropic doesn't publish it. For how skills load, see what Claude skills are.

Ceiling first, then the loop

Before changing anything, the agent computes a theoretical ceiling from your measured memory bandwidth and compute and writes it to THEORY.md. That tells you whether your current speed is poor or already close to the limit, and flags a target the hardware can't reach.

Then it changes one setting per experiment, each with its own config file. Run-to-run noise is typically ±5–10%, by the skill's own count, so it takes at least six decode samples, uses the median, and runs control, candidate, control. A change is kept only if it beats both controls and slows nothing else by more than about 3%. Failures go into PLAN.md too, so a rejected idea isn't tried twice.

A config isn't called best until it passes a long-context retrieval test and a concurrent soak run. For a lossless claim, the output with greedy decoding has to match the original.

How to install it

For Claude Code, clone and copy the skill folder:

git clone https://github.com/sybil-solutions/inference-optimisation-skill
mkdir -p ~/.claude/skills
cp -r inference-optimisation-skill/skills/inference-optimisation ~/.claude/skills/

Other agents can be pointed at the SKILL.md, or given PROMPT.md as the task. You need a model already served behind an OpenAI-compatible endpoint; the scripts use only Python 3's standard library and bash.

Example request

Try it
Use inference-optimisation on the model at [http://localhost:8000]. Hardware: [GPU or iGPU, memory]. Engine and version: [e.g. vLLM 0.x]. Model and quantization: [name, bits]. Targets: [decode tok/s for 1 user, prefill tok/s, time to first token]. Memory cap: [GiB]. Context I need to stay stable: [tokens]. Lossless changes only: [yes/no].

You get THEORY.md with the ceilings, PLAN.md with every experiment and why it was kept or dropped, a ledger.jsonl line per run, one config per run, and best-config.json with the winner.

Limits

When to use it

Use it when you serve a model on your own hardware and want each speedup checked against controls before you switch configs or post a benchmark. If you call models through ChatGPT or an API, there's nothing for it to tune. For long jobs tracked in files, see Planning with Files.

Sources

  1. sybil-solutions/inference-optimisation-skill README on GitHub, accessed November 2026
  2. inference-optimisation SKILL.md on GitHub, accessed November 2026
Weekly

New prompts in your inbox

One email a week: the best new prompts and one short guide. Confirm by email and get the free Prompt Cheat Sheet (PDF). Unsubscribe any time. Privacy policy