Hermes Agent with Ollama: settings that worked on a business task

The Hermes docs set two hard requirements: a 64,000-token window and a model that stays loaded

Checked 14 October 2026, the Hermes providers docs say agent use with tools needs at least 64,000 tokens of context (room for the system prompt, tool schemas and working conversation state), and list the same default as Ollama's own page (docs.ollama.com/context-length): 4,096 tokens under 24 GB, such as our 16 GB unit. Either way the window has to be raised by hand. The Hermes local Ollama guide's example builds a new model from a Modelfile with PARAMETER num_ctx 64000.

The second requirement is about time, not size. The same guide says Ollama unloads idle models after 5 minutes and suggests OLLAMA_KEEP_ALIVE=24h; for slow CPU setups it suggests HERMES_API_TIMEOUT=1800. These are statements from the Hermes docs, not from our tests. Ollama also keeps its own Hermes integration page; both are linked under Sources.

Our unit meets both with four server settings, and the server only listens to its own machine

Source: our demo-environment notes of 1 October 2026. The values are what the unit runs. We have not tested other values systematically, so treat them as one working setup and not as a tuned optimum.

To apply the same settings on a business install, work in this order:

1. Install Ollama and pull one model. Do not pull five.
2. Set keep-alive and context size in the service environment, then restart it once.
3. Point Hermes at the local endpoint.
4. Turn off the tools you do not need (see the security page).
5. Run your own test set cold, then warm, and write both timings down.
6. Only then connect a mailbox (email page).

The one failure we hit was a hang, cleared by restarting the service and keeping one window size

We saw Ollama stop answering when the disk was busy or when models unloaded between steps. The fix we use is to restart the service with launchctl kickstart -k gui/$UID/ai.staffbox.ollama and to keep one context size per run. That label is specific to our install; yours will differ.

A related cost: changing num_ctx on a 16 GB machine takes about 100 seconds because the model reloads. If a test run mixes context sizes, timings stop meaning anything. Pick one size, warm the model, then measure.

On these settings the unit got 15 of 15 unseen requests (6 Oct), but the notes and quote action, not the settings, moved the score

With Qwen3 8B, company notes and the quote action on this setup, the unit got 15 of 15 on an unseen messy set (6 Oct 2026), 40/40 on the fictional Fieldstone IT set and 39/40 on the fictional Peachtree Cabinet Works set (6 October 2026). Average time per answer was 1.1 s, 2.3 s and 2.5 s, one request at a time. Full tables are on the Mac mini page. On a new trade it had never seen, an electrical distributor (14 Oct, Dell test box), the default Qwen3 8B setup scored 30 of 40 and 2 of 15 on messy requests, with 8 wrong prices in 55, because per-foot units, price levels and trade shorthand are not handled yet; that is why we score your own list first (the electrical result).

What the numbers do not tell you: whether these settings caused the scores. We did not test keep-alive or context size against score. What we did test is the notes: on our Dell test machine, Qwen3 8B scored 1/40 on Fieldstone IT alone, 33/40 with the notes and 40/40 with the notes plus the quote action. Those are the parts we know move the score; the settings are what keep the server up long enough for it to matter.

Sources

• Hermes docs: local Ollama setup, checked 13 Oct 2026
• Ollama: Hermes integration, seen 13 Oct 2026
• github.com/Staffbox-ai/staffbox (Staffbox code and evals/results, MIT)
• huggingface.co/spaces/staffbox/scorecard (every graded answer)