Today we’re coming out of stealth to introduce MachGen: an AI infrastructure startup building a high-performance inference and fine-tuning stack for diffusion models.
We didn’t start MachGen to shave a few percent off existing bills. The point of pushing diffusion latency and cost down by an order of magnitude is that it changes what’s possible. When a video that takes several minutes can be generated in seconds, and at a fraction of the cost, you don’t just get a cheaper version of today’s products — you unlock interactive generation, real-time personalization, customized advertising at scale, and a whole category of applications that simply can’t exist at current price-performance. We think of this as the democratization of image and video generation: putting capabilities that are today the preserve of a few well-funded labs into the hands of any developer who wants to build with them.
We’ve spent most of our careers making AI models run fast. From co-building the TRT high-performance inference teams at Nvidia, leading Datacenter AI software at Intel, serving on the PyTorch Foundation board, driving the vLLM inference effort at Google, and building the large-model training infrastructure and frameworks behind PyTorch at Meta. We’ve watched the LLM inference stack mature from research prototypes into the highly optimized, production-grade systems that now serve trillions of tokens a year. That experience is exactly why we started MachGen — and why we’re betting that diffusion is where the next wave of inference infrastructure gets built.
The lesson from LLM inference
The first generation of LLM serving was slow, expensive, and fragile. What turned it into infrastructure was years of focused systems work: paged KV caches, continuous batching, prefill/decode disaggregation, speculative decoding, and kernel-level optimization tuned to the autoregressive transformer. Each of those advances assumed a very specific computational shape — and collectively they compounded into orders-of-magnitude improvements in throughput, latency, and cost per token.
Having lived through that arc, one thing was obvious to us: diffusion models are about to walk the same path, and almost none of the LLM playbook transfers cleanly. The optimizations that made LLM serving cheap are built around the autoregressive pattern — a compute-bound prefill followed by a memory-bound, token-by-token decode, with a KV cache that grows linearly. Diffusion is a different animal. It’s non-autoregressive and compute-bound throughout the denoising loop. Prefill/decode disaggregation in its current form doesn’t apply, and KV-cache management as designed for LLMs largely doesn’t either. The scheduler, the memory model, the kernels — the critical pieces all need to be rethought for diffusion as a first-class citizen, not retrofitted.
Diffusion performance is nowhere near its ceiling
The gap between where diffusion serving is today and where it could be is enormous, and we have the numbers to prove it. The results are already compelling. Across leading image models, MachGen delivers HiDream in 1 second vs. 6 seconds and Flux in 1.5 seconds vs. 6.2 seconds — roughly 4–6× lower latency, while reducing inference cost by 2–4×.
The same pattern holds for video. Wan 2.2 runs in 16.5 seconds vs. 98 seconds, and LTX 2.3 in 10.7 seconds vs. 67 seconds— about 6× faster — while costing 2–3× less than comparable providers.
These gains come from running the original, undistilled models — not by sacrificing quality. This isn’t incremental optimization; it’s a fundamentally more efficient inference stack.
Numbers like these are the signature of a stack that hasn’t yet had its “LLM moment.” The reason the gap exists is mostly historical. The overwhelming majority of inference optimization effort over the last few years went into LLMs, because that’s where the workload and the money were. Existing serving frameworks are over-indexed on LLMs by design. Diffusion got bolted on. Model coverage is thin, performance is left on the table, and there’s no out-of-the-box story for running across heterogeneous accelerators. That’s the gap MachGen exists to close.
What we’re building
MachGen is a high-performance inference and fine-tuning stack that treats diffusion models as first-class citizens. In practice, that means several pieces working together:
Attention— Attention dominates the compute budget in most video generation. We focus on lower-precision recipes, sparse attention, and related techniques to bring that cost down.
Caching— KV caching is central to LLM inference but doesn’t transfer to diffusion models. Instead, there’s spatial and temporal redundancy to exploit. Standard approaches like TeaCache and MagCache tend to fall short, so we’ve built caching recipes that hold up across models.
Kernels— Optimized GPU kernels for LLMs (including attention) are widely available in open source. Far less exists for diffusion. We’ve developed highly optimized kernels built specifically for diffusion architectures.
Parallelism— Parallelism is essential to utilize multiple GPU’s computing power for speedup. However, open source offering of parallelism is far from being optimal in terms of communication overhead. We implement highly tailored kernels, so communications overlap with data-independent computations and get pipelined to data-dependent ones.
The optimized stack is available through the following product offerings.
- Model APIs: Generate lightning-fast image/videos now by accessing optimized model APIs
- MachGen Cloud: Our optimized inference and fine-tuning platform as a fully managed service on dedicated MachGen infrastructure.
- Self-Hosted:Deploy MachGen’s optimized inference runtime directly into your existing platform infrastructure, whether your cloud or on-premise.
- ServerLess: Bring your own model and inference runtime. MachGen handles GPU provisioning, autoscaling, routing, and global capacity.
Image and video generation raise a broader set of challenges. The gap between open- and closed-source models is wider than in the LLM world, and each model has its own characteristics. We’re tackling those. More to come.
Our North Star: world models
There’s a longer arc here. World models — systems that can understand, simulate, and reason about the physical world rather than just language — are widely seen as the next major frontier in AI, with serious efforts underway at labs led by Fei-Fei Li, Yann LeCun, xAI, and others, and over $20B already flowing into the space. Crucially, today’s world models are, at least computationally, diffusion-like. The inference stack that serves diffusion well is the natural foundation for serving world models well.
There is essentially no production-grade inference solution for world models today. We intend to be the first to build it. Diffusion is where we earn our keep now; world models are our North Star. We believe their impact will rival, and possibly exceed, that of LLMs — and when they arrive, performance won’t be a nice-to-have. It’ll be the thing that determines whether they’re usable at all.
If that’s the kind of problem you want to work on, we’d love to talk.