The problem
I wanted to run language models on the machine I already own: one ordinary 16 GB machine with an integrated GPU, not a rack. Starting a model there is not hard. Trusting one is. A model can load and answer while a flag copied from a smoke test has quietly made it a different model, and a server can report that it is serving while it writes one token over and over. Every number I read about local models came with settings I could not see.
So 1-llm is two things, kept apart. One is a control plane: llmctl installs, starts, benchmarks and stops models behind one contract, and a stack of small services lets programs, people and other machines use what is running. The other is a research record of what I measured on this machine, how I measured it, and what turned out to be wrong.
What is in it
llmctl, a Python package and command line. It is one wrapper over the model runtimes it calls loaders: llama.cpp, koboldcpp, an attached Ollama daemon, whisper.cpp and NeMo for transcription, and several hosted services, eleven enabled in all. The same config produces the same arguments, the same environment and the same recorded conditions, so a number measured last week can be compared with one measured today.- A service stack on top: a gateway, a pool for parallel agents, a LAN proxy, a mesh service and an MCP server for coding agents. The service stack takes them one by one.
- A multi-user web chat, and a terminal chat, a menu and a desktop window over the same core.
- Instruments that measure and keep dated reports: a scorer for checkable tasks, a tool-calling harness, a Romanian speaker-label test set, and canaries that check a started model really answers. The research is what they found.
Why one 16 GB machine
The constraint is the point. The machine's integrated GPU has no memory of its own, so the CPU and the GPU share one RAM channel, and that channel is the ceiling. I measured it: a dense 7B model generating tokens used 85% of the memory bus. Faster compute would not help; more bandwidth would. About 7 GiB of RAM is left for a model's weights, its cache and its buffers.
That turns loading a model into a decision the system has to take. If a load takes more GPU memory than is free, the GPU faults, and the reset ends the desktop session. So a load is checked against free memory first. When it would not fit, it is refused, and the refusal names the models that are loaded.
The evidence discipline
A local model is easy to run and easy to misread, so the repository has rules for what counts as a result. They sit in its instructions file, which every Claude Code session reads, and each one was paid for in time before it became a rule.
- Every claim carries a mark. A tick means I measured it on this machine; a page icon means I read it somewhere and have not checked. A page icon never becomes a tick without running the thing.
- Predictions are written before the run, and kept after it, wrong or not. The tool-calling findings go further and name the commit that holds the prediction, so the order can be checked in
git loginstead of believed. - No model judges another model. Every verdict is an integer, a regular expression, an exact string or a JSON key. The checkers are checked too, in both directions: each gold answer must pass and each known-wrong answer must fail, before a score is trusted.
- A status code is not a result. A hosted endpoint once answered 200 with an empty message, because the model had spent its whole token budget thinking, and
llmctl runreported it as serving. An empty completion is now an error, never a wrong answer. - Measure at least twice, in separate server lifetimes. Repeats inside one process cannot see a cold cache or swap pressure, and that is what produced the one wrong number in the record.
- A correction stays in place. When a number turns out wrong, the document keeps the old one, the new one and the reason the first was wrong.
The rule I would put first is the one this next result taught me. The same model, on the same 26 tasks, scored two very different numbers on consecutive days, and the only thing between them was a flag my own scoring harness had copied from a smoke test.
Switching thinking off
- Tried
The same 26 tasks, scored by code rather than by another model, three repetitions each, on the same local model (gemma-4-E4B): once with thinking off, which every run before 2026-08-15 used, and once with it on.
- Measured
Thinking off: 54/75. Thinking on: 78/78. Told not to think and asked for "the number only", the model complied and gave a bare wrong number in four tokens.
- So
Thinking is part of the model, not a tuning knob. Every report now records whether it was on, a score is never compared across the two settings, and a pinned profile carries the flag.
My first reading of the low score was "weak at arithmetic". It was the harness. It took a 20B reference model and a run of the smaller sibling to catch it, and it is the most expensive lesson the repository records. Every report now carries the flag, a score is never compared across the two settings, and a pinned profile holds the flag with the score.
The research page holds the other cards. Most of the expectations on it were wrong, and each stays on the record with its stamp.
Scale, and how I built it
2,116 commits since 2026-08-10 and about 7,100 test functions, counted on 2026-10-26. The core uses only the Python standard library, with no third-party dependency at runtime or in the tests. The desktop window is the one exception, and it lives in a folder that nothing else imports. Every commit runs the test suite first. Services, front ends and instruments are fifteen optional modules that can be switched off one at a time, and every suite passes with any one of them off.
I built it as AI-orchestrated engineering, measured and reviewed. Each piece starts as a written spec with its open questions answered in it, then a plan, and Claude Code co-authored 2,047 of those 2,116 commits against them. The rules above are mine, written into the repository so that every session starts from them. So is the rule that nothing is deferred at a merge: every minor finding a final review raises is fixed in the same lane before it merges.
Before this repository
In June 2026, two months before the first commit here, a model on one laptop answered questions from my own notes, indexed in a vector database on a second laptop. The model, Qwen2.5-Coder-7B, wrote its tool calls out as JSON text instead of making them, so a script ran the search and the model only summarised the result. Told to say "not in the notes" when the context lacked the answer, it said so with the right passage in front of it, until a firmer prompt allowed that answer only when the information was really missing. The next day Qwen3-8B, with thinking off, called the search tool natively and answered with its source cited.
Read on
Measured research is the findings, each with its lab-notebook card: scoring models, small models as agents, Romanian speaker labels, and small models on an Android tablet. Service stack is how programs, people and other machines reach the models, as a diagram, plus the multi-user web chat.