Nemotron 3 Ultra Harness Profile
Open-source Deep Agents harness profile that brought NVIDIA's open Nemotron 3 Ultra within a point of Opus 4.8 at ~10x lower cost per run
$ apt list --installed
The open-source harness profile behind the NemoClaw Deep Agents Blueprint, launched jointly by LangChain and NVIDIA. It tunes everything around NVIDIA's open Nemotron 3 Ultra model — the system prompt, the tool descriptions, and the middleware around model and tool calls — so the model spends its capability on the task instead of fighting the scaffolding. The weights and generation settings never changed.
What It Does
- •Ships as a built-in profile in the Deep Agents SDK (deepagents #4192), so Deep Agents configures its harness for Nemotron 3 Ultra out of the box
- •Targeted prompt blocks — short, single-purpose instructions aimed at behaviors that failed in traces, instead of broad system prompt rewrites
- •Tool-call shims plus compatibility and execution-repair middleware between the model and the Deep Agents tool loop
- •In-band context engineering — guidance delivered where the model will read it: a "keep reading" notice placed in full-page file results (the same words did nothing in the tool description), and plan-then-review messages injected mid-run
- •Loop control — caps on model and tool calls end a looping run before it times out, and a one-shot retry absorbs transient tool failures
Eval-Driven Tuning
Every change ran through a trace-driven loop in LangSmith: evaluate, observe, diagnose, engineer, re-evaluate. Candidates were screened on a small, representative slice of the Deep Agents eval suite first and promoted to the full suite only once they proved out. A change stayed only if the win repeated across trials and regressed nothing else, and it had to fix a behavior class rather than a benchmark — one change that leaned on a phrase from a single eval was thrown out.
Results
- •No-profile baseline of ~0.80 to ~0.84 on a typical run, with a best run of 0.86 against Opus 4.8's best of 0.87
- •~10x cheaper per run than Opus at matched best-run quality ($4.48 vs $43.48 on the full suite, 3–10x depending on precision and prompt caching), with median latency at parity
- •Summarization had the most headroom going in and tuning cleared it; tool use improved significantly, and retrieval and file operations, already strong, went higher
Where It Shipped
- •The NemoClaw Deep Agents Blueprint with NVIDIA, which runs Deep Agents Code on Nemotron 3 Ultra inside an OpenShell sandbox — deny-by-default networking, human approval on sensitive operations, and a full audit trail, set up with one command
- •Runnable OpenShell examples in langchain-nvidia-samples: a contained incident agent workshop and a Deep Agent coding example
- •Featured on the NVIDIA Developer Blog and the NVIDIA Blog