Overview

NVIDIA’s DeepSeek-V4.1-Flash-1017-NVFP4 is a quantized, multimodal MoE language model intended for deployment in agent systems, chatbots, RAG, and other AI applications. It is based on DeepSeek-V4.1-Flash-1017 and uses a Causal Encoder-Decoder architecture with Compressed Sparse Attention 2. Maintainer: nvidia. The stated license is MIT; the source describes commercial and non-commercial use.

Best use cases

Text and image input with text generation; chatbot and RAG applications; AI-agent deployments, including tool calling through the documented vLLM configuration. Evaluation covers scientific reasoning, long-context recall, scientific coding, instruction following, visual understanding, and terminal-based agent tasks.

Limitations

The base model may produce biased or toxic responses, inaccuracies, omissions, or irrelevant and redundant text. This checkpoint requires a vLLM build with MXFP4 Engram support; stock vLLM images cannot load its Engram tables, and SGLang has not been validated. Speculative decoding was not exercised. Validate the intended runtime and application with task-specific testing. The source reports benchmark results but does not establish that this checkpoint is superior overall to its source-precision baseline.

How it compares

The supplied related model DeepSeek-V4.1-Flash-NVFP4 is a different version: its name omits the 1017 suffix, so do not assume its checkpoint or results are interchangeable. Against the stated MXFP4 source baseline, this checkpoint's reported scores were 92.93 vs. 92.77 on GPQA Diamond, 80.19 vs. 80.41 on AA-LCR, 57.32 vs. 57.58 on SciCode, 76.47 vs. 76.33 on IFBench, 78.73 vs. 78.38 on MMMU-Pro, and 81.11 vs. 81.81 on Terminal-Bench 2.1. These are benchmark-specific results, not evidence of a general winner.

Technical specifications

Transformer model, DeepseekV41ForCausalLM; Causal Encoder-Decoder MoE with Compressed Sparse Attention 2. The backbone has 552B total parameters, with 8B activated during prefill and 16B during decode, plus 196B Engram conditional memory. Quantization uses Model Optimizer v0.48.0rc0: routed expert weights and activations are NVFP4 W4A4, group size 16; Engram lookup tables are MXFP4. Other components retain source precision, including MXFP8 where applicable. Expert weight conversion preserved dequantized values, while Engram conversion is lossy; this does not guarantee identical inference outputs. The checkpoint is about 400 GiB in 48 safetensors shards. Supported hardware is NVIDIA Blackwell; preferred OS is Linux. Reported inference used vLLM on four NVIDIA GB300 GPUs. Evaluation scores are percentages: MXFP4 source / this NVFP4 checkpoint—GPQA Diamond 92.77 / 92.93; AA-LCR 80.41 / 80.19; SciCode 57.58 / 57.32; IFBench 76.33 / 76.47; MMMU-Pro 78.38 / 78.73; Terminal-Bench 2.1 81.81 / 81.11. Training data details are undisclosed. Calibration used 1,024 samples total from CNN/DailyMail and selected Nemotron-Post-Training-Dataset-v2 subsets.

Model inputs and outputs

Inputs

Text and images; image input is specified as RGB. The model card states a context length of up to 1 million tokens. Text and image input were validated with the documented vLLM command. Tool calling can be enabled with the listed vLLM flags.

Outputs

Text strings, generated as one-dimensional sequences.

Getting started

VLLM_ENGINE_READY_TIMEOUT_S=3600 vllm serve nvidia/DeepSeek-V4.1-Flash-1017-NVFP4 \ --tensor-parallel-size 4 \ --tokenizer-mode deepseek_v41 \ --attention-config '{"backend":"FLASHINFER_MLA_SPARSE_DSV41","indexer_kv_dtype":"mxfp4","indexer_sparse_logits":true}' \ --kv-cache-dtype fp8 \ --max-num-seqs 128 \ --engram-config '{"cpu_offload":true}' \ --reasoning-parser deepseek_v41

Use a vLLM build with MXFP4 Engram support; stock vLLM images cannot load this checkpoint. The documented command was used with four GB300 GPUs. For tool calling, add --enable-auto-tool-choice --tool-call-parser deepseek_v41.

Frequently asked questions

Q: What runtime does this checkpoint support?

A: The supplied documentation supports vLLM with MXFP4 Engram support. Stock vLLM images cannot load the checkpoint's Engram lookup tables; SGLang has not been validated.

Q: What hardware and operating system are specified?

A: NVIDIA Blackwell GPUs and Linux are specified. Reported inference used four NVIDIA GB300 GPUs.

Q: Can it accept images?

A: Yes. The documented inputs include text and RGB images, and both text and image input were validated with the stated vLLM command.

Q: How large is the context window?

A: The model card states a context length of up to 1 million tokens.

Q: Does the checkpoint support tool calling?

A: The vLLM usage notes specify enabling it with --enable-auto-tool-choice --tool-call-parser deepseek_v41.

Q: What is the checkpoint's license?

A: The supplied model information states MIT and describes commercial and non-commercial use.

Q: Are speculative decoding and SGLang validated?

A: No. Speculative decoding was not exercised, and SGLang has not been validated with this checkpoint.

This is a simplified guide to an AI model called DeepSeek-V4.1-Flash-1017-NVFP4 maintained by nvidia. If you like these kinds of analysis, join AIModels.fyi or follow us on Twitter.