Hermes Agent · Model choice

A non-Chinese model for Hermes Agent: gpt-oss and the Georgia rule

Short answer: If Alibaba models are ruled out, gpt-oss-20b scored 39/40, 40/40 and 15/15 on our fictional tests; it needs a 32 GB unit.

On a new trade it had never seen, an electrical distributor (14 Oct, Dell test box), the default Qwen3 8B setup scored 30 of 40 and 2 of 15 on messy requests, with 8 wrong prices in 55, because per-foot units, price levels and trade shorthand are not handled yet; that is why we score your own list first (the electrical result).

Georgia state agencies work under a published standard that lists Alibaba as prohibited and DeepSeek as a concerned good, and Staffbox's default model, Qwen3, comes from Alibaba. Whether a locally run copy of an open-weight Qwen model falls under that list is for the agency to decide, and we will not guess. The faster route is a model outside the question: gpt-oss-20b, scored on the same fictional tests. Buying for an agency? Tell us in the form below; a person on our team replies.

Updated 14 October 2026 · built on Hermes Agent, not affiliated with Nous Research

Staffbox is built on Hermes Agent (Nous Research, MIT license). Staffbox is not affiliated with Nous Research.

Georgia's GTA standard SS-22-002 prohibits Alibaba and lists DeepSeek as a concerned good for state agencies

Some organisations cannot use certain vendors' products. The clearest case in Georgia is state government. The Georgia Technology Authority (GTA) publishes standard SS-22-002, "Concerned Goods and Prohibited Software and Services", which lists technology state agencies must remove from state-owned, controlled or leased systems and devices. Its prohibited vendors and services list includes "Alibaba products and services". Its concerned goods list, under services, includes "DeepSeek".

The standard applies to executive-branch state agencies, boards, authorities and commissions. It is dated: effective 1 December 2022; the web version shows a revision date of 29 January 2026 (read 14 October 2026), while the PDF copy we read on 13 October 2026 showed 30 June 2025. Always open the current version, on the web page or as a PDF: GTA SS-22-002 (PDF). The GTA also has a web page for the standard.

Qwen is an Alibaba model, and whether a local copy counts is the agency's call

Qwen is the model family from Alibaba, and Staffbox's default model is qwen3. So any buyer who must avoid Alibaba products needs to ask the question. Whether an open-weight Qwen file running on your own Mac counts as an "Alibaba product" under the standard is for the agency and the GTA to decide. Staffbox does not give legal advice and we will not guess at a reading.

What we can do is give you a model that is not in the question. That removes the argument. For the wider Georgia purchasing picture see Staffbox for government offices and Georgia AI procurement rules.

gpt-oss-20b removes the question and scores within one test of Qwen3 8B

Qwen3 8B vs gpt-oss-20b on three quoting test sets Fieldstone IT, out of 40 Qwen3 8B · 16 GB Mac mini M4 40 gpt-oss-20b · Dell RTX 3090 39 Peachtree Cabinet Works, out of 40 Qwen3 8B · 16 GB Mac mini M4 39 gpt-oss-20b · Dell RTX 3090 40 Unseen messy set, out of 15 Qwen3 8B · 16 GB Mac mini M4 15 gpt-oss-20b · Dell RTX 3090 15 gpt-oss-20b needs 32 GB. Its Apple speed: not measured.
Score per test set, zero-based; each track is the set size. Different hardware and dates, so read it as "both pass", not as a ranking. gpt-oss-20b on a Dell RTX 3090, 2 Oct 2026; Qwen3 8B on a 16 GB Mac mini M4, 6 Oct 2026; fictional companies, machine-graded. Source: results files on GitHub.

A non-Qwen option exists: gpt-oss-20b, an open-weight model released by OpenAI. It needs a 32 GB machine.

Test set (all fictional companies)Result with gpt-oss-20bHardwareSource
Fieldstone IT, 40 requests39 of 40Dell with an RTX 3090Results file, 2 Oct 2026
Peachtree Cabinet Works, 40 requests40 of 40Dell with an RTX 3090Results file, 2 Oct 2026
Messy held-out set, 15 requests15 of 15Dell with an RTX 3090Results file, 2 Oct 2026

These runs were on a Dell graphics card, not on Apple hardware. Speed of gpt-oss on a Mac mini or Mac Studio is not measured yet. Fifteen requests is a small sample, and none of the companies is real. A pilot on a 32 GB unit is measured on the same tests before day 0, so the speed and the score on your hardware will be known before you commit.

Qwen3 8B (default)gpt-oss-20b (non-Qwen option)
Memory neededScored on a 16 GB Mac mini M432 GB
Fieldstone IT 40 requests40 of 40 (6 Oct, Mac mini M4)39 of 40 (Dell RTX 3090)
Peachtree 40 requests39 of 40 (6 Oct, Mac mini M4)40 of 40 (Dell RTX 3090)
SpeedAbout 1.1 to 3.2 s average per answer on the Mac mini, one request at a time, model warmNot measured on Apple hardware
Hermes minimum context (reported 64K)Runs with OLLAMA_CONTEXT_LENGTH=65536 and 65536 in the Hermes profile, where every score was measured; native window 40,960, longer prompts not tested128K reported by others; passes the check

The two columns are not a fair race: different hardware, different dates. Use them to see that both options score well on our fictional tests, not to rank the models. The context numbers are explained on the 64K context page. All local model results are collected on which local model for Hermes Agent.

  • We have not scored any other non-Chinese model beyond gpt-oss-20b. A different model may suit you; we would test it the same way.

Get the agency's answer in writing, then score a 32 GB unit before day 0

Our suggestion, in order. It is guidance, not legal advice.

  1. Read the current SS-22-002 PDF yourself and note its revision date.
  2. Ask your agency's security or procurement office whether a locally run open-weight model from a listed vendor is in scope. Get the answer in writing.
  3. If there is any doubt, ask for the gpt-oss-20b configuration and a 32 GB unit.
  4. Start with the free 30-quote test on past requests with the price list. Nothing is installed for that step.
  5. Agree the pass bar before day 0, and have your own staff grade 10 held-back answers blind.

What this page is not:

  • We are not a certified or approved vendor under any Georgia programme, and we do not claim to be. No SOC 2 or other certification yet.
  • We have no state or local government customer. There is no customer install of any kind yet.
  • We have not asked the GTA for an opinion on locally run open-weight models. Do not read this page as one.

Related pages

Sources

Or call us: 404 493 6248 · How the free test works

Ask usTextCall