Deploy supported Hugging Face models for end-to-end TensorRT inference in just two commands.
trtmc build Qwen/Qwen3-0.6B -o qwen3-0.6b.bundle
trtmc run ./qwen3-0.6b.bundle --prompt "What is the capital of France? Answer in one word." --chat-template --no-thinking
# Generated text: ParisThe same bundle works from C++:
auto pipeline = trtmc::load("./qwen3-0.6b.bundle");
std::cout << pipeline->generate("What is the capital of France? Answer in one word.").text << '\n';TensorRT Model Connect is an extensive collection of AI Model reference implementations in C++, on top of NVIDIA TensorRT. Model Connect is powered by an agentic workflow that continuously adds support for upcoming models, drastically reducing integration effort on user side and time until new models become compatible.
- Use TensorRT-Model-Connect to explore models quickly and evaluate broad model coverage.
- For production LLM/VLM deployment on NVIDIA edge platforms where performance is the priority, start directly with TensorRT Edge-LLM.
- Start from a supported Hugging Face or local checkpoint and build TensorRT engines without an intermediate ONNX export step.
- Hand a versioned
.bundleartifact from the Python-first build environment to native C++ task APIs such as text generation, transcription, image and video generation, segmentation, embedding, and forecasting. - Use model-family-owned builders, runtime pipelines, helper kernels, and validation contracts as concrete blueprints for modification and customization.
- Keep native TensorRT execution and exactly qualified optimized-runtime dispatch behind the same task-oriented application boundary.
Read the Project Overview for the architecture boundary, intended users, and comparison with other TensorRT integration paths.
TensorRT-Model-Connect is a reference implementation. Users are responsible for trusting the checkpoints, bundles, native libraries, and local environment they provide when building or running models.
Recommended Quick Start: AI-Native
Give an AI coding agent with terminal, Docker, and NVIDIA GPU access this prompt:
/goal Use the current TensorRT-Model-Connect checkout, or clone
https://github.com/NVIDIA/TensorRT-Model-Connect.git if none is provided. Read
AGENTS.md, then follow website/docs/getting-started/source-build.md and
website/docs/getting-started/quick-start.md exactly. Do not modify source,
tests, Dockerfiles, git history, or remote state. Report the selected GPU,
exact commands, bundle path, inference output, and any deviation from the
documentation.
Want to know more? See the Quick Start documentation.
| Goal | Start here |
|---|---|
| Complete the first Qwen inference | Quick Start |
| Select and install an environment | Get Started |
| Compile the CLI, backends, and model DSOs | Build from Source |
| Find an exact checkpoint or model recipe | Models & Recipes |
| Look up task and feature workflows | User Guides |
| Learn through progressive labs and self-checks | Tutorials |
| Look up CLI, Python, C++, bundle, and config contracts | Reference |
| Understand architecture or extend the repository | Developer Guide |
| Review compatibility, limitations, and lifecycle policy | Release & Support |
| Give a coding agent repository-specific guidance | AI & Agent Guide |
The Supported Models page is the single source of truth for exact checkpoints, Hugging Face architectures, TRTMC profiles, precision, quantization, optimized-runtime dispatch, configuration, and qualification evidence.
Start with Get Help and File an Issue to choose the right support route and collect the model, environment, command, and log details maintainers need. Use the issue chooser for usage questions, reproducible bugs, feature or model requests, and documentation corrections.
Do not disclose suspected security vulnerabilities in a public issue. Follow SECURITY.md to report them privately to NVIDIA PSIRT.
- Read CONTRIBUTING.md before proposing source or model integration changes.
- TensorRT-Model-Connect is licensed under the terms in LICENSE.