AI-Native Podcast E4: Fireworks AI’s Story — The $17B AI Layer Built Without Sales or Marketing

In our fourth episode of “מג’נרטים”, Eran Bielski and Ido Grady take a deep look at Fireworks AI — the inference middleware startup founded by 7 ex-Meta PyTorch engineers that sits between NVIDIA hardware and every open-source LLM, quietly routing the AI products millions of people use every day.

Here’s some of what we covered:

• The invisible layer worth $17.5B: how Fireworks identified the inference bottleneck nobody else was solving and built the middleware every AI application routes through — often without even knowing it.
• Fire Attention: the custom CUDA kernel Fireworks engineered that outperforms NVIDIA’s own optimization — and what it signals about where real value in AI infrastructure is actually being created.
• The chef and the sous-chef: how Speculative Decoding uses a small model to pre-generate drafts for the larger model, cutting inference latency in half — a structural cost advantage, not a product feature.
• 10,000 customers before the first salesperson: the open-source PLG motion that brought Cursor in through developer adoption — and how Cursor became roughly half of Fireworks’ total revenue without a single sales conversation.
• The routing thesis: why Fireworks believes every organization will eventually have its own specialized AI model built on institutional knowledge — and why the company’s job is routing that model to wherever it runs fastest and cheapest.
• Microsoft and the enterprise ceiling: the partnership that gives Fireworks access to enterprise buyers who can’t tell Sonnet from Opus from Haiku — and why that confusion is a feature, not a problem.

Most importantly for founders, we analyze Fireworks AI’s strategy through an investor’s lens — what a real infrastructure moat looks like when open source is simultaneously your distribution channel and your biggest competitive threat, where the PLG-to-enterprise motion breaks at scale, and what Lin Chao building a $17.5B company at 48 says about the durable advantage of deep technical experience in AI infrastructure.

Watch the Full Episode:

Apple Podcasts:

Available on all platforms      

About this Podcast

How do you turn a breakthrough technology into a massive company, and how can the Israeli ecosystem lead the revolution?

Welcome to the “מג’נרטים” (Meganretim) podcast, your strategic playbook for the world of AI.

In each episode, we take a leading global AI company, go behind the scenes, and break down the product moves, GTM plays, economics, and talent wars that made them a massive success—or a total failure.

No fluff, no empty buzzwords—just straight talk. We bring you the lessons, insights, and practical tools you need to build, scale, and navigate your startup.

The podcast for founders, tech professionals, and anyone who wants to understand where the real world of AI is heading.