StartLux open model ranks first on Decision Index benchmark

On a leaderboard run by the community, the 27B Instant model of StartLux-Decision v1.1 is now first, with a Full score of 69.12 as of October 16. The ranking is measured on Decision Index 0.3.2 on Hugging Face, which is maintained by volunteers and has no link to TypeSafe. The top model took first place in all three parts of the test: 72.88 on public benchmarks, 64.82 on private tests of the same skills, and 61.41 on private tasks from new domains. The runner-up, vLLM-SR d3, reached 66.26.

The task these models are built for is narrow and practical. Give a model a state, which can be a block of text, a JSON file or an image, along with a set of typed questions. It returns a probability for every answer: a choice from a list, a yes or no, or a level on a scale. The Instant models produce that answer in a single forward pass through the network. The AutoThink variants work the same way when they are confident, but when the top probability drops below a gate, set by default at 0.6, they write a short reasoning trace and then read the answer again. On the public subset of Decision Index 0.3, StartLux reports 73.37 for the 27B AutoThink model with the gate raised to 0.9, a figure the company ran itself.

The lineup spans eight open models, in Instant and AutoThink versions, available in GGUF and MLX formats and running on vLLM and MLX. The authors describe a three-stage training pipeline and a reinforcement learning method called RLCD-squared, which rewards models for putting honest probabilities on their answers and lightly penalizes long reasoning. They say it cut average reasoning length by up to about 65% while keeping accuracy largely intact. StartLux also introduced DecisionPivot, its own benchmark built from pairs of cases that differ in one detail that should change the answer, across policies, ledgers, schedules, permissions and tool calls. On that test, the 27B AutoThink model reaches 86.9% pair accuracy. Those figures come from the company, not from independent labs.

For a developer building an agent that must choose a next step, the gain is in cost and control: a model that answers in one pass most of the time, and reasons only when it hesitates, can be cheaper to run than one that always writes out its thinking. The demos on the project site show this kind of step-by-step decision-making in chess, Super Mario Bros., StarCraft II and a shopping task performed on a computer, with one typed request per step. The licence is the immediate limit: the weights are open for research and personal use, but commercial teams must wait for a different agreement. What remains open is whether the leaderboard lead holds up on tests the authors have not designed, and whether the gap in reasoning length survives outside their own benchmarks.