JetBrains Mellum2.1 turns a fast coding model into an agent worker

JetBrains released Mellum2.1 on October 8 with the same 12-billion-parameter mixture-of-experts architecture as Mellum2, but with post-training aimed at agents that navigate repositories, edit files and check their work. In JetBrains’ own matched evaluation, SWE-bench Verified rose from 2.0% for Mellum2 Thinking to 47.0% for Mellum2.1 Thinking; Qwen3.5-9B still scored 50.0%, and its lead was wider on SWE-bench Pro. The open weights and an official repository of GGUF files are listed now. The multi-token-prediction head behind a separate speed claim is still marked coming soon (JetBrains release announcement; Mellum2.1 model card; official GGUF repository).

That distinction matters if you are choosing a model for an actual agent rather than admiring a coding score. A model can finish a function in an editor and still struggle when the next task is to find the right file, run a test, understand the failure and repair its first attempt. Mellum2.1 is an unusually clear release about that change of job. It is also a good example of why a large within-family improvement and a market-wide victory are two very different claims.

Mellum2.1 changed the training job, not the model’s skeleton

Imagine a coding agent confronted with a failing test in a repository it has never seen. Its work is not one autocomplete suggestion. It has to look through the project, form a hypothesis, change a file, run a command and decide whether the output confirms the fix. A plausible patch that never reaches a passing test is not the same outcome as a completed task. This is an illustrative workflow, not a record of our own run of Mellum2.1.

JetBrains describes Mellum2 as fast but not strong enough at working inside a repository. For version 2.1, it says the architecture remained the same while almost all the effort moved to post-training, chiefly reinforcement learning. The new task mix spans math, competitive programming, science, tool use and software engineering. In the software-engineering environments, the model works inside repositories with shell and file-editing tools and receives a reward when tests pass. JetBrains says it launched millions of sandboxed runs across thousands of environments during training (release announcement; model-card highlights). Those are the developer’s descriptions of its training, not counts we independently audited.

This changes the target from “write a likely next piece of code” to “take a sequence of actions that survives a test.” It does not establish that any specific user repository, test suite or agent framework will work out of the box. The release provides evaluation evidence for the broader shift; it does not give a controlled ablation separating more training runs from task filtering, reward design and the other post-training changes.

The model remains 12B parameters in total, with 2.5B active per token. Its card lists 64 experts, eight activated for a token, and a maximum context of 131,072 tokens. Active parameters describe computation during a token, not the size of all the weights that must be stored or the memory an agent will occupy at a long context. The practical question in this release is whether that existing architecture can now do useful multi-step work (model overview).

The same-pipeline scores show a leap and a limit

JetBrains evaluated Mellum2.1, Mellum2, Qwen3.5-9B and Gemma 4 E4B in thinking mode using what its model card calls the same evaluation pipeline. The table below selects the figures most relevant to a coding-agent decision; all figures are JetBrains’ own percentages, not tests by this publication (model-card evaluation table).

The most consequential comparison is within the Mellum family. Verified rises by 45 percentage points under this pipeline, from a predecessor that barely solved these agent tasks to one that resolves nearly half. Pro rises from a reported 0.0 to 28.0; Terminal-Bench from 0.6 to 17.4. That pattern fits the emphasis on repository work. But the figures alone do not prove which training ingredient caused how much of the improvement, and the comparison concerns the evaluated Thinking variants, not every version ever named Mellum.

The cross-model comparison is less comfortable and more useful. Qwen3.5-9B is ahead on all three agent-heavy rows shown here: 50.0 versus 47.0 on Verified; 38.0 versus 28.0 on Pro; 21.7 versus 17.4 on Terminal-Bench. Mellum2.1 leads on LiveCodeBench v6 and BFCL v4 in this same table. A model can do well at producing a solution to a coding prompt while completing fewer tasks in a tool-using agent harness. Neither finding cancels the other. “Best coding model” collapses those distinct jobs into a claim the data cannot support.

For the agentic benchmarks, JetBrains specifies the open-source Pi v0.73.1 harness with shell and file tools, a 114K-token context and up to 16K generated tokens per turn. Every model used the same agent harness but its own default sampling; the card states a temperature of 1.0 for Mellum2.1. That is a stronger basis for comparing these particular rows than combining different companies’ published leaderboards. It is still vendor-reported evaluation, not an independent replication, and a harness score does not tell you your agent’s wall-clock time, hardware bill or failure rate in a different repository (evaluation notes).

A coding-agent score is a starting point for an agent design

What should a team do with 47% on SWE-bench Verified? First, understand what the number represents: task resolution in a specified setup, not a promise that 47% of its customers’ tickets will be fixed. A team’s own backlog may have different languages, dependency installation, hidden services, flaky tests or repo permissions. The model’s 131,072-token supported context length does not establish that every task can use that much context cheaply or effectively. The benchmark’s agent setup used 114K, not a proof that serving at the model maximum is the best default for every deployment.

The interesting use case is a bounded worker in a larger coding workflow. An agent could assign a candidate model a concrete patch-and-test task, capture its tool trace, and accept the result only after the repository’s checks run. JetBrains explicitly presents Mellum2.1 as a worker for parts of an agent’s plan, such as identifying the cause of a failing test or drafting and checking a fix (release use cases). This is a design possibility, not evidence that we have deployed or integrated it. Where mistakes are expensive, compare it on representative tasks and retain a human or another verification step for changes that tests do not cover.

An evaluation for that decision needs more than one pass rate. Track completed tasks against attempted tasks, tool-call errors, retries, elapsed time and the environment in which each task ran. A low-cost candidate that needs repeated attempts can consume the time it was meant to save. Conversely, a model that resolves fewer of the hardest benchmark tasks might still be useful for smaller, tightly verified subtasks. That is an editorial inference from the difference between benchmark tasks and production workflows, not a measured cost advantage for Mellum2.1.

The downloadable model and the promised acceleration are different releases

There are two artifact sets available to inspect today. The main model repository lists the model under Apache 2.0, and JetBrains supplies a vLLM serving example with --reasoning-parser qwen3; its optional tool-calling form adds --enable-auto-tool-choice --tool-call-parser hermes. These are instructions in a manufacturer model card, not a verified installation on our machines. A full agent deployment also needs its own tool execution boundary, task harness and validation; a model-server endpoint alone is not an agent.

A separate official GGUF repository has instructions for llama.cpp and Ollama. When its file tree was checked on October 9, it listed five actual GGUF variants: BF16, Q8_0, Q6_K, Q4_K_M and MXFP4_MOE. The Q4_K_M file is 8,071,295,264 bytes, rounded to 8.1 GB in the GGUF card. The list is evidence that those files were present at that check, not proof that we downloaded and successfully served each variant. Quantization also calls for task-quality checks; the card’s quantization comparison measures logits against BF16 on Wikitext-2 at n_ctx=512, not coding-agent resolution (GGUF model card).

There is a documentation timing mismatch worth spelling out. Both the October 8 announcement and the main model card still say GGUF builds are “coming soon,” while the official GGUF repository and its file listing already existed at our October 9 check. The narrow, defensible statement is that official GGUF files were listed at that point; it would be misleading either to say no GGUF files exist or to claim that a published file list proves every local runtime configuration works. Repository state and release text should be checked again before anybody treats this as an evergreen availability guide.

An 8.1 GB Q4 file is not an 8.1 GB machine requirement. The complete weights are distinct from the 2.5B active per token, and serving adds runtime overhead and KV-cache memory that grows with context and concurrency. The 131K-token context in the specifications should not be converted into a promise that a typical laptop will serve an agent at that length. For local planning, choose a quantization and context budget, then measure the real deployment rather than buying hardware based on the download size alone.

The speed claims require another separation. JetBrains says that under heavy load on one H200, Mellum2.1 serves almost twice as many tokens as Qwen3.5-9B; it also says multi-token prediction makes a single request about 1.6 times faster (announcement speed section). Those are vendor results tied to its benchmark setup, not general performance promises. More importantly, the MTP head for speculative decoding in vLLM is still labeled coming soon in the announcement and primary card. Downloading the available model weights or GGUF files is not the same as obtaining that head, so the 1.6× claim must not be presented as a speedup readers can reproduce today from the currently documented release.

The sensible next test is small and specific: run your own representative repository tasks in a pinned harness, record completed work and tool failures, and measure latency and memory with the context you actually use. Mellum2.1 earns attention because post-training appears to have changed what this architecture can do in JetBrains’ agent evaluation. The available weights make that question testable. The harder-agent lead held by Qwen in the same report, and the still-unreleased MTP head, are reasons to test rather than shortcuts around testing.