Nemotron 3.5 Super VL in SageMaker: a 120B model for documents and GUI agents, deployed in 10 minutes

If you are building an agent that needs to do more than chat — inspect a screenshot, read an invoice or interpret a chart — you usually end up assembling a patchwork: a separate OCR system, a separate language model and glue code between them. NVIDIA and AWS offer another approach. Nemotron 3.5 Super VL is now available in SageMaker JumpStart, and you can deploy it with a few clicks in Studio or a few lines of Python.

What's inside

This is a multimodal model: it takes images, video and text as input and produces text. It has 120 billion parameters in total, but its Mixture-of-Experts architecture activates only 12 billion per token. The language backbone is a hybrid: Mamba-2 layers alternate with MoE layers, alongside eight attention layers. A C-RADIO encoder handles vision. The advertised context window is up to 1 million tokens, but we will return to that figure.

The model supports tool calling and JSON output that follows a specified schema, while reasoning can be switched on or off for each request using the enable_thinking parameter. For extracting invoice fields, it is better to disable reasoning: it is faster and cheaper. For complex questions about charts, it is worth leaving it on.

JumpStart offers two versions. NVFP4 takes up around 75 GB and is designed for fast inference; BF16 takes up around 249 GB and is intended for those planning to fine-tune, distill or create their own quantized version of the model.

Hardware, speed and quality

The NVFP4 version runs by default on ml.g7.24xlarge: four RTX PRO 4500 Blackwell GPUs with 128 GB of memory. BF16 requires ml.p5en.48xlarge with eight H200s. AWS measurements, using 250 input tokens, 1500 output tokens and five concurrent streams, show the following:

• g7.24xlarge, NVFP4: around 63 tokens/s per request, 0.45 s to the first token;
• g7.48xlarge, NVFP4: around 64 tokens/s, 0.51 s;
• p5en.48xlarge, BF16: around 141 tokens/s, 0.53 s.

These figures do not provide a fair comparison between NVFP4 and BF16 because the hardware differs. In terms of quality, however, the quantized version gives up almost nothing. Both score 62.82 on MMLongBench-Doc; NVFP4 is slightly weaker on CharXiv (79.88 versus 80.82), and even slightly better on OSWorld-Verified, a computer-use benchmark (52.30 versus 51.14). These metrics come from NVIDIA's model cards, so treat them as reported results rather than independently verified findings.

Pitfalls to know about before deployment

First: you will not get a million-token context window in JumpStart. The serving configurations support 520–15,600 tokens. A 1536×1024 page consumes around 1536 tokens, so a long, multipage document will not fit into a single request.

Second: a single request accepts one image and one video. The model cannot process raw PDFs; you must first convert them into page images. Media must be passed inline as base64 because the endpoint runs in network isolation and cannot fetch content from links.

Third: long prompts slow things down. With 10,000 input tokens, time to first token on g7.24xlarge is 12.6 seconds. That is a lot for an interactive chat, but tolerable for batch document processing.

Fourth: the checkpoint includes an MTP head for speculative decoding, but it is not enabled in the JumpStart configuration. The endpoint does not load LoRA adapters; only the base weights are served. Quotas for g7 instances start at zero in new accounts, so submit a quota increase request in advance. You will need SageMaker Python SDK v3, version 3.25 or later, and must explicitly specify the instance type in Compute; otherwise, the build will fail on the default ml.m5.large.

What to do with it

If you have a stream of invoices, contracts or reports and want to keep the data within AWS, this is a quick way to test an idea: deploy NVFP4 on g7.24xlarge, render the pages as PNGs and ask for JSON. For GUI agents, an OSWorld result of around 52% suggests the model is suitable for experiments, but not unattended automation. The license is OpenMDW 1.1; be sure to read it on the model card before deployment. Product managers should also budget for endpoint idle time: a 120B model on four GPUs costs money even when nobody is sending requests.