Service · model optimization
Before you buy a bigger AI server, find out what is slow.
We tune self-hosted AI model servers, starting with Ollama on NVIDIA GPUs: models fitted to GPU memory, fewer reloads, and a straight answer on hardware before you spend.
On our own GPU server, one model ran at 48 tokens (word pieces) a second at a 64K context, because a setting to serve four requests at once pushed 28% of it off the graphics card onto the CPU. Cutting its context to 20K, to fit the card's memory, brought it to 173 tokens a second: 3.6 times faster, with no new hardware (30 October 2026). The other model keeps 64K. Context size is how much text a model holds at once; the bigger it is, the more GPU memory it takes.
A call. No card, no server access needed. Bring your graphics card model, the AI models you run and the app that feels slow; we tell you what we would check first and whether it looks like hardware or setup. Our measured work is Ollama on Linux; on anything else we tell you plainly if we have not done it.
Three ways to work with us: audit, tune, or keep it tuned
Each step stands alone: stop after the free hour or after the audit. After the free hour we send a fixed price in writing; it depends on how many servers, models and apps you run. The audit fee is credited if you go on to the tune.
Find where the time goes
Read-only access to one model server. We measure it under your real load: where models load from, how much runs on the GPU, context sizes per app, requests at once, power and heat, backups, scheduled jobs and unused models.
- A findings table: each finding has its evidence, what the fix could break, and the simplest fix
- A ranked fix plan
- A hardware and storage call. "Don't buy anything" is a valid answer
Apply the fixes and prove them
The audit, then the changes, each approved by you in writing first: fewer context sizes per model where your apps allow, requests at once sized to the GPU memory left over, power limits, model clean-up, backup repair.
- Before-and-after benchmarks on your own prompts
- An undo step for every change
- A runbook your IT team can follow
Keep it tuned
Models and apps change every month. We re-run your benchmark, test model updates before they go live, and watch heat, disk and backups.
- A monthly scorecard: speed, reloads, uptime, backup state
- One agreed change window a month
- Through your IT provider, if you have one
What an audit found on our own server
Everything measured on this page is from our own GPU server: a Xeon W-2133 workstation with 128 GB of memory and one RTX 3090 (24 GB), running Ollama for our agent, test and batch jobs. It is not a customer. Measured 30 October 2026 unless noted.
| Question | What we found | Label |
|---|---|---|
| Why could the 30B model be slow? | With Ollama set to serve four requests at once (a setting added 19 October), Qwen3 30B-A3B at a 64K context across those four request slots needed 32 GB in total on a 24 GB card. Ollama put 28% of it on the CPU and it ran at 48 tokens/s. At a 20K context, same four slots, it needs 22 GB in total, runs fully on the GPU, and reached 173 tokens/s. | Measured |
| And the 27B model? | It fits at a 64K context: 17 GB, fully on the GPU. Rule now: 20K for the 30B, 64K for the 27B, set per model. | Measured |
| Is the box starved? | No. With the context sizes matched, both models load fully into GPU memory (Ollama reports 100% GPU). Warm reloads took 6.8 s (30B) and 10.5 s (27B), both 4-bit. | Measured |
| Is the disk the bottleneck? | Not for warm reloads. The 27B model takes about 17 GB on the GPU. Reading that much from the SSD at its measured 479 MB/s takes over 30 s, so we infer the 10.5 s reloads were served from the operating system's file cache in memory; the 62 s cold load below fits that. A cold load after a reboot, with that cache empty, is disk-bound: 62 s (6 October, 30B model). | Measured |
| What else was slow? | Apps asked Ollama for different context sizes for the same model. On our setup each change forced a full reload of 10 to 14 s. Other engines are checked separately. | Measured |
| What changed? | Each model now has one default context size (20K for the 30B, 64K for the 27B), whatever name an app calls it by. A separate 64K setting of the 30B is kept for rare long prompts and does force a reload. In our tests, requests with the same model settings reused the loaded copy and no reload was seen (0 s, against 10 to 14 s when settings differed). | Done |
| Does a lower power limit cost speed? | 30B model, short prompts: 172 to 176 tokens/s at 280 W (2 October) against 177 at 350 W (30 September). GPU under normal load: 77 °C against 81 °C (1 October). Separate runs on different days with different prompts, so indicative. | Measured |
| Housekeeping | A local model backup was reporting errors that nobody read. We replaced it with a checked copy on the network drive, removed two models we no longer needed (restorable from the network copy) and paused 13 scheduled jobs. | Done |
| Wait time for users, before and after | Not measured yet on real traffic. | Open |
How we review: each finding needs evidence, what the fix could break, and the simplest fix. A second, separate review checks it, and we repeat until a full pass finds nothing new (two passes on this server). A person approves every change. Speeds are generation tokens per second on a loaded model from our benchmark script, not end-to-end wait time; user wait times before and after are not measured yet. These rows are examples of what an audit checks, not the whole list.
Who this is for
You already bought a GPU box
A workstation or server running Ollama for your team, and it feels slow or someone is asking for a bigger one.
IT providers
Your clients ask for private AI and you want a tuned, monitored setup you can support. Partner terms are on request at partners@staffbox.ai. How we work with IT providers.
Small teams self-hosting models
You run open models on your own GPUs to cut API bills and need steady speed when several apps share one card.
Where our measured work is: Ollama on Linux with NVIDIA GPUs. vLLM, llama.cpp, Windows, Mac and multi-GPU setups: ask in the free hour, and we say plainly if we have not done it yet. We do not train models from scratch, and we ask in writing before anything leaves your box. Security and data handling: Trust Center.
Questions
Which software do you tune?
Ollama on Linux with NVIDIA GPUs is where our measured tuning work is. We also run llama.cpp, and Ollama on Mac minis, but have not tuned those for anyone else. We have not tuned a vLLM server; if that is what you run, we say so in the free hour.
Do we have to buy a Staffbox box?
No. This service is for servers you already own. If you also want the Staffbox AI worker, see how Staffbox works.
What access do you need?
None for the free hour. For the audit, read-only remote access to the server, time-boxed and removed at the end. For the tune, each change is approved by you in writing first and has an undo step. We ask in writing before anything leaves the box.
What does it cost?
The first hour is free. After it we send a fixed price in writing for the audit and the tune, and a monthly price per server for managed. Each step stands alone, and the audit fee is credited if you go on to the tune.
Or call us: 404 493 6248