⚙️ Building Arion: an infrastructure for language at scale
I work at DeepL, and our mission is simple: enable people and businesses to communicate clearly across languages. That goal sounds straightforward until you start to unpack what "clearly" really means. Accurate literal translation is necessary, but not sufficient. What users want is translation that reads like a native wrote it — fluent, nuanced, and context aware. Delivering that level of quality at global scale is an engineering and research challenge that demands purpose-built infrastructure.
To meet that challenge we built Arion, our internal AI infrastructure designed for both training and inference at industrial scale. Arion is centered around an NVIDIA DGX SuperPOD populated with DGX GB200 systems. Those systems bring the compute, memory, and interconnect bandwidth required for large-scale model development, while the software stack — including NVIDIA TensorRT-LLM and NVFP4 inference optimizations — helps us take those models to production with near-zero latency.
We designed Arion with three priorities in mind:
- Accuracy and fluency — ensuring translations capture nuance, idiom, and tone.
- Latency — delivering results in milliseconds to feel instantaneous for the end user.
- Efficiency and sustainability — minimizing compute cost and environmental impact while scaling to millions of daily users.
🧠 Training models on billions of sentences
Good translation models learn from huge corpora. To produce the level of nuance we aim for, our models are trained on billions of sentences, characters, and context windows. The data variety must be broad: formal documents, conversational text, web content, and domain-specific materials. The models must learn both linguistic structure and the real-world context in which words are used.
Training at that scale requires specialized hardware. The Blackwell-based GB200 systems in our SuperPOD provide a step change in performance compared with previous generations. They let us iterate faster and explore larger model architectures — including Mixture of Experts models — without prohibitive training time. Faster training cycles mean our researchers can try more hypotheses, push for better contextual understanding, and move improvements into production more quickly.
"For that, we had to build those large data centers where our models train, where they are going through billions of sentences of words, of characters, which allow them to draw those insights out of those texts."
That training data is not just raw text. We carefully curate, filter, and align datasets to preserve quality and reduce bias. Context-dependent translation is especially sensitive to alignment and metadata: source of text, register, domain, and even locale. All of these factors feed into the training pipeline so that the models can learn to choose the right tone and vocabulary for a given input.
⏱️ Inference at near-zero latency
Training a great model is only half the battle. The real-world user experience depends on inference: how fast and reliably those models can produce a translation when millions of users request one simultaneously. Our customers expect high quality delivered instantly — no waiting, no awkward phrasing that reveals a machine translation.
To get that responsiveness, we invested heavily in inference optimization. A few of the levers we use:
- TensorRT-LLM — this inference runtime accelerates large language model execution with kernel fusion, optimized attention kernels, and efficient memory management. It reduces latency without degrading model output quality.
- Quantization with NVFP4 — lowering numerical precision from FP16/FP32 to NVFP4 on GPU inference reduces memory footprint and increases throughput while preserving translation fidelity for our tasks.
- Efficient model architectures — techniques such as Mixture of Experts let us scale model capacity without linear increases in inference cost.
- Careful batching and request handling — dynamic batching, predictive request routing, and priority handling ensure interactive requests complete in milliseconds.
"TensorRT-LLM ... allows us to create more efficiency in the inference process, in how we run our models. It allows us to reduce the latency for our customers without sacrificing any of the quality and accuracy that DeepL is known for."
With these optimizations, we can reliably serve translations with practically no latency for users, even at peak scale. The system is engineered to prioritize low-latency interactive requests while still maximizing GPU utilization for bulk translation workloads.
🌱 Efficiency and sustainability: cooling and green power
High-performance computing and sustainability are often seen in tension. But there are clear design choices that let you have both high throughput and a small environmental footprint. We partnered with EcoDataCenter to host our infrastructure and take advantage of modern data center design that emphasizes energy efficiency and renewable power.
One of the technology advantages EcoDataCenter and NVIDIA bring to the table is liquid cooling. Liquid cooling dramatically improves heat transfer efficiency over air cooling, reducing overall power usage and enabling higher-density racks. Combining liquid cooling with renewable electricity means we can run intense workloads while keeping our carbon footprint low.
Operationally, this matters for two reasons. First, energy is one of the largest ongoing costs for large GPU clusters. Efficiency directly reduces operating expense. Second, sustainability matters to our customers. Many businesses want responsible AI operations, and demonstrating a commitment to green power and efficient infrastructure is part of that responsibility.
🤝 Software and hardware co-design: collaborating with NVIDIA
High-performance AI systems are the result of close collaboration between hardware vendors, software engineers, and research teams. Our engineers and researchers worked directly with NVIDIA to test and tune models on the GB200 systems and to adopt the evolving software stack such as TensorRT-LLM.
That collaboration accelerated our rollout. When hardware, libraries, and models are developed in concert, the result is an optimized end-to-end experience. Latency optimizations from TensorRT-LLM and the new capabilities in the Blackwell architecture translate into concrete user benefits: faster translations and increased throughput with the same or better quality.
That partnership also lets us adopt new capabilities quickly. For example, when NVFP4 inference became available on GB200 NVL72 systems, we moved rapidly to benchmark and deploy it in production workloads where the quality tradeoffs were acceptable. Fast feedback cycles between our researchers and NVIDIA's engineers helped us tune the stack and set new performance baselines.
🌍 Real-world impact: what this delivers for users and businesses
The practical result of this work is simple and profound: people and businesses can communicate across borders more naturally. Millions of users visit our service daily and expect translations that are accurate, fluent, and delivered on the instant. Businesses use our APIs to power customer interactions, internal knowledge sharing, and multilingual content workflows.
Here are tangible outcomes that matter to our users:
- Near-native fluency — translations that preserve tone and context so the message feels written by a human.
- Fast responses — millisecond-scale inference so no human waits for a reply.
- Scalability — the ability to serve millions of requests per day without degradation.
- Privacy and security — deploying our own infrastructure gives us control to meet enterprise security requirements and compliance obligations.
These capabilities change how companies operate globally. Support centers can respond in multiple languages in real time. Product documentation and knowledge bases can be translated on-demand while maintaining technical accuracy. Internal teams distributed across geographies can collaborate with fewer friction points.
🔧 Technical deep dive: MoE, GB200 NVL72, and NVFP4
If you enjoy the technical nuts and bolts, here are some of the key technologies and why they matter to us.
Mixture of Experts (MoE)
MoE is a model architecture approach that increases model capacity without a proportional increase in compute for every input. The main idea is simple: the model contains many specialized "experts," and a gating network activates only a subset of those experts for any given input. That lets the model have a very large effective parameter count while keeping per-request compute low because only a few experts are used.
For translation, MoE helps because language tasks are highly diverse. A document about legal contracts uses different vocabulary and structure than a casual chat message. Experts can specialize on register, domain, or language pairings, yielding better translations for a wider variety of inputs. The tradeoff is engineering complexity: MoE requires efficient sparse routing and careful memory handling to avoid performance bottlenecks. On the GB200 architecture and with modern software runtimes, those tradeoffs become attractive at scale.
GB200 NVL72 systems and Blackwell architecture
The GB200 generation — often referred to in shorthand as Blackwell — brings improvements in raw compute, memory capacity, and interconnectivity. These advances are especially important for very large models that need high memory bandwidth and low-latency communication between GPUs.
Key benefits we leveraged:
- Increased memory capacity — larger models and longer context windows fit in GPU memory without excessive sharding.
- Higher throughput — faster matrix math and optimized kernels speed both training and inference.
- Better power efficiency — enabling more work per watt, which ties back to sustainability goals.
NVFP4 quantization
Quantization reduces numerical precision to speed up inference and reduce memory usage. NVFP4 is an NVIDIA-specific low-precision format that balances throughput gains with acceptable numerical fidelity for many language tasks. For translation, we've found that NVFP4 preserves the qualities we care about — fluency and context preservation — while significantly increasing inference capacity.
Adopting NVFP4 required careful validation. We evaluated output quality across languages and domains to ensure that reduced precision did not introduce subtle errors or degrade readability. Where the tradeoff was acceptable, the throughput gains allowed us to serve more users per GPU and reduce cost per translation.
📈 Deploying efficiently: cost, caching, and scaling strategies
Having a powerful GPU cluster is only useful if you can use it efficiently. Here are some of the operational patterns and optimizations that made our deployment successful.
Dynamic batching and priority routing
Batching multiple requests together increases GPU utilization and amortizes kernel launch overheads. However, naive batching increases latency for small, interactive requests. We implemented dynamic batching that groups compatible requests while keeping latency-sensitive traffic separate and prioritized.
Model sharding and pipeline parallelism
For the largest models we use a combination of model parallelism strategies. Data parallelism is straightforward for training; for inference on very large models we also use pipeline and tensor parallelism where appropriate. The hardware topology in GB200 clusters makes these strategies efficient by providing high-bandwidth interconnects between GPUs.
Cache and reuse
Many translation requests contain overlapping contexts: repeated phrases, shared documents, or similar sentences. We cache translation outputs and intermediate computations where it makes sense. Caching reduces unnecessary computation and speeds up response time for repeated or similar requests.
Auto-scaling and cost control
We designed capacity planning so we don’t overprovision GPUs. Auto-scaling groups expand during predictable peaks and contract during lulls. Efficient inference runtimes and NVFP4 quantization reduced the per-request cost, meaning we can handle more load without linear increases in hardware spend.
🔍 Quality at scale: how we preserve nuance
Delivering low latency and high throughput is important, but never at the cost of translation quality. That is the core of why people choose our product. Here are some of the ways we preserve nuance while scaling:
- Context windows — long-context processing helps preserve discourse-level meaning, idioms, and coreference resolution.
- Domain adapters — lightweight fine-tuning layers or adapters let the model specialize on a domain without retraining the entire model.
- Human-in-the-loop evaluation — continuous human evaluation for quality, particularly for sensitive content and low-resource languages.
- Continuous monitoring — telemetry and quality checks in production detect degradation and trigger model retraining or rollback paths.
These systems combine to ensure users get translations that not only convey facts correctly but also feel natural and suited to the context.
📣 Organizational lessons: aligning research, engineering, and ops
What enabled us to move fast and responsibly was organizational alignment. AI infrastructure projects can easily get bogged down if research, engineering, and operations are siloed. We broke down those silos in a few ways:
- Cross-functional teams — researchers, engineers, and ops staff work together on the same objectives and share ownership of deployment.
- Common tooling and metrics — shared CI/CD, monitoring, and cost/quality metrics make tradeoffs explicit.
- Rapid experimentation — small, reproducible experiments let us validate ideas before large-scale rollout.
Those practices let us adopt new hardware and software advances quickly while maintaining high quality and system reliability.
🚀 Looking ahead: translating the whole internet and beyond
Scale begets new possibilities. With the performance improvements we've unlocked, the ambition grows. When the hardware and software line up, the rate at which we can improve models and roll them out increases dramatically. I've said internally that with the new clusters, "we're going to be able to practically translate the whole internet in like roughly two weeks." That’s a way of expressing the acceleration we now have at our fingertips.
That acceleration creates responsibility. Faster iteration cycles mean we must be vigilant about model behavior, safety, and fairness. We are committed to ongoing evaluation, transparency about limits, and improving performance for low-resource languages and underrepresented dialects.
At the business level, this platform enables new products: real-time multilingual collaboration tools, integrated customer support automation with native-quality responses, and enterprise translation services with rigorous privacy controls. For individuals it means fewer language barriers, better access to information, and smoother global communication.
📝 Final thoughts
Building infrastructure for real-time language AI is a multi-dimensional engineering challenge. It demands advances in model architecture, optimized inference runtimes, powerful and efficient hardware, and sustainable operations. It also requires close collaboration between research and infrastructure teams and strong partnerships with technology providers and data center operators.
By combining the GB200-based DGX SuperPOD, NVIDIA software like TensorRT-LLM, NVFP4 inference, and efficient data center design with EcoDataCenter, we created Arion: a platform that delivers high-quality, low-latency translations to millions of users every day. The result is not just a technical milestone, it is a practical engine for better global communication.
I am excited about where this work takes us next: expanding language coverage, improving contextual understanding, and making professional-quality translation an everyday utility for businesses and individuals around the world.
