NVIDIA’s GTC 2025 Key Takeaways: A look inside Dynamo AI OS, Open-Source Innovation & the AI Stack

NVIDIA’s GTC 2025 Keynote, led by Jensen Huang, unveiled a bold vision for the future of AI. More than just hardware or software, NVIDIA is revolutionizing the entire AI stack—from chips to data centers, inference platforms, and communication tools. Here’s a breakdown of the biggest announcements and the open-source technologies powering one of its stars, NVIDIA Dynamo, the open-source AI inference platform designed to optimize large-scale model serving.

Here’s a breakdown of the biggest announcements and the open-source technologies that are shaping NVIDIA’s AI future.

NVIDIA’s AI Stack: Innovations Across Every Layer

NVIDIA is building an end-to-end AI ecosystem, innovating at multiple layers of the stack:

NVIDIA AI Stack

Let’s explore each of these layers in detail.

Compute Layer: The Biggest Leap Yet with the Blackwell Architecture

The Blackwell Architecture is NVIDIA’s most powerful AI GPU, designed to dramatically speed up AI processing—making model training and inference up to 30 times faster than previous generations (H100 GPUs) on resource-intensive inference tasks—like a 1.8-trillion-parameter GPT-MoE model. Huang emphasized that AI inference at scale is extreme computing, with an unprecedented demand for FLOPS, memory, and processing power. NVIDIA introduced Dynamo, an AI-optimized operating system that enables Blackwell NVL systems to achieve 40x better performance. More about Dynamo below.

Infrastructure Layer: AI Factories are The New Data Center Paradigm

Huang introduced the concept of “AI Factories”: hubs designed to transform raw compute into intelligence, replacing traditional data centers. Built to handle trillion-parameter models, these AI powerhouses generate revenue through token production.

Scalability & Efficiency Layer: Networking & Power Efficiency

AI scaling requires more than just compute power. NVIDIA tackled key constraints in networking and energy efficiency:

• Quantum-X800 InfiniBand (800Gb/s): High-speed networking that enables faster model training and inference by ensuring rapid data transfer between GPUs and AI clusters.

• Liquid-cooled Blackwell systems: Advanced cooling technology that reduces energy consumption and improves thermal efficiency, making large-scale AI computation more sustainable.

These innovations in networking and cooling technology enable Blackwell GPUs and AI Factories to function at their full potential.

Software Layer: NVIDIA Dynamo, the Operating System for AI Inference

What’s missing when you have a faster AI inference chip and AI Factories? An operating system to power them, right?

Enter NVIDIA Dynamo: an open-source framework optimized for large-scale inference. It doubles throughput on models like Llama while reducing costs through efficient GPU cluster orchestration.

The Open-Source Backbone of Dynamo

NVIDIA Dynamo is built on a powerful ecosystem of open-source technologies, each playing a crucial role in optimizing large-scale AI inference:

Communication & Coordination Layer

• NATS.io : A high-performance messaging system that enables real-time communication within Dynamo. It efficiently routes requests, distributes tasks to workers, and handles key-value (KV) events for cache-aware operations.

• etcd: A distributed key-value store responsible for managing worker states and configurations across nodes.

Optimization Layer

• TensorRT-LLM: NVIDIA’s open-source library for optimizing LLM inference, integrated into Dynamo for maximum performance.

• vLLM: A framework designed to enhance LLM serving efficiency, featuring dynamic batching and memory optimization.

• PyTorch: A supported framework ensuring model compatibility and flexibility, making it easier to integrate diverse AI workloads.

Orchestration & Development Layer

• Kubernetes : Automates deployment and scaling of AI workloads across multiple GPUs.

• Rust: Used for performance-critical components in AI infrastructure, ensuring safety and speed.

• Python Software Foundation : The go-to language for AI and ML development, widely used for scripting and model training.

Monitoring and Observability Layer

• Prometheus: A powerful open-source monitoring and alerting toolkit that collects and analyzes performance metrics within Dynamo.

• Grafana: A visualization and analytics platform that works alongside Prometheus to provide real-time dashboards, helping teams monitor AI inference performance and system health.

Together, these technologies power Dynamo’s disaggregated serving architecture and dynamic GPU scaling, making it one of the most efficient AI inference platforms to date.

How Does NVIDIA Dynamo Work?

OK, you’ve read this far, so are you curious how Dynamo actually works?

I spent some time analyzing different docker-compose and YAML files, and here’s my understanding:

Dynamo System Architecture

1. Clients send HTTP requests to the Frontend (port 8000). Dynamo is designed to handle HTTP requests from clients, serving as the entry point for inference tasks.

2. The Frontend forwards requests to the Processor via NATS. Dynamo utilizes NATS, a high-performance messaging system, to facilitate communication between components like the Frontend and Processor.

3. The Processor routes tasks to vLLMWorker (decode) and PrefillWorker (prefill) using NATS (port 4222). Dynamo's architecture supports disaggregated prefill and decode inference, optimizing GPU utilization by separating these tasks.

4. vLLMWorker and PrefillWorker communicate with each other for KV cache transfers via NIXL. The NVIDIA Inference Xfer Library (NIXL) accelerates point-to-point communications in AI inference frameworks like Dynamo, providing efficient data transfers between components

5. etcd-server (port 2379) provides configuration and state data to all Dynamo components (Frontend, Processor, and Workers).

6. Prometheus (port 9090) scrapes metrics from Dynamo components (e.g., via HTTP endpoints) and NATS (port 8222 for monitoring). Dynamo integrates with Prometheus for metrics collection, enabling monitoring of various components.

7. Grafana (port 3001) pulls data from Prometheus to visualize metrics. Grafana is used in conjunction with Prometheus to visualize collected metrics, providing insights into system performance.

Final Thoughts: NVIDIA’s Open-Source Vision

NVIDIA’s full-stack AI platforms are accelerating enterprise adoption of AI, offering end-to-end solutions from hardware to software. AI-powered virtual agents are also on the rise, reshaping productivity across industries.

Finally, NVIDIA is applying its AI leadership to robotics. Huang outlined a future where general-purpose robots will be trained in virtual environments using synthetic data, reinforcement learning, and digital twins before being deployed in the real world. This marks the beginning of AI-driven automation at an industrial scale.

GTC 2025 wasn’t just about hardware or software—it was about a fully integrated AI ecosystem. NVIDIA is innovating at every layer of the stack, from chips (Blackwell) to infrastructure (AI Factories) to inference OS (Dynamo) to microservices inter communication (NATS & ETCd).

And Dynamo’s reliance on NATS, etcd, TensorRT-LLM, vLLM, PyTorch, Kubernetes, Prometheus, and Grafana highlights NVIDIA’s commitment to open-source innovation.