Maryland, September 22, 2026 — EcoWestern AI, a division of EcoWestern, today announced the release of ESAC (EcoWestern Short and Cheap Benchmark), an evaluation program built on the premise that measuring a model's intelligence should not require an expensive test, a five-hour runtime, or blind trust in whoever ran it.

The first suite, ESAC-GI (General Intelligence), is available today at github.com/EcoWestern/ESAC-GI, released under the MIT license with a runnable harness anyone can execute against any model in minutes, for a fraction of the cost of existing large-scale benchmarks. Both suites are governed by a shared specification at github.com/EcoWestern/ESAC, the program's parent repository and information hub. A companion suite, ESAC-AG (Agentic Work and Capability), is fully specified there and will follow as its own implementation repository.

"Most benchmarks end up measuring your API budget as much as the model's capability," said Xav, founder of EcoWestern. "We wanted something short enough that anyone can rerun it themselves and confirm a score, not just read it off a vendor's page and hope it holds up."

A benchmark designed to resist being gamed

Both ESAC suites, each scored out of 75 points, are governed by a single public specification and share a set of design principles aimed directly at the two problems that undermine most benchmarks: cost and trust.

  • Items generate rather than repeat. Every question is produced from a deterministic seed rather than stored as a fixed string, so memorizing an answer does not work: the same item becomes a different problem under a different seed.
  • No single vendor grades anyone. The portion of each suite that requires judgment is scored by a fixed, open-weight model, published in advance and never substituted for a vendor's own.
  • A public set for self-checking, a held-out set for real claims. Anyone can download the public item pool and check a model's performance directly. Official comparative results are drawn from a separate, private pool, so training toward the public set does not move the number that counts.
  • Passing is a gate, not an average. A qualifying score requires clearing the threshold in every category and overall, so strength in one area cannot mask a weakness in another.
  • Infrastructure failure is never scored as model failure. A dropped API call or a harness timeout is retried, never counted against the model being tested.

ESAC-GI evaluates eight categories, including logic, math reasoning, factual knowledge, reading comprehension, abstraction, instruction-following, writing quality, and response-depth calibration, the ability to judge how much explanation a given question actually warrants. Sixty of its 75 points are graded with no judge model involved at all, using exact match, regex, and programmatic checks alone.

ESAC-AG, once implemented, will evaluate tool-call correctness, planning under a constrained step budget, code generation, error recovery, state tracking across multi-turn tasks, ambiguity handling, and resistance to instructions injected through tool output, run entirely against a deterministic mock environment rather than live services.

Open specification, closed contribution path

The full ESAC constitution and its amendments are published in the parent EcoWestern/ESAC repository, which serves as the hub for the program and every suite under it. EcoWestern AI is not accepting external pull requests to the specification, a deliberate choice intended to keep every published score traceable to a single, trusted revision of the benchmark that produced it. Issues and forks remain open, and the team says it welcomes challenges to individual items or the specification itself.

One design question remains explicitly unresolved: whether official held-out evaluations should be run by a single maintainer or a small trusted rotation. EcoWestern AI says it is leaving the question open in the published specification rather than settling it unilaterally.

ESAC-GI v1.0.0 is available now at github.com/EcoWestern/ESAC-GI. The full ESAC specification, constitution, and amendment history live at github.com/EcoWestern/ESAC.

About EcoWestern AI

EcoWestern AI is the artificial intelligence division of EcoWestern, which builds an interconnected technology ecosystem spanning operating systems, developer infrastructure, and productivity software. ESAC is EcoWestern AI's contribution to independent, reproducible model evaluation.

Media Contact
Michael Tavarus
press@ecowestern.net