Across industries, rising compute expenses are often cited as a barrier to AI adoption, yet top engineering leaders are shifting their focus. The new priority is not how much the cloud bills but how quickly models can be deployed, how low latency can be achieved, and how flexible the system remains under surging demand. This mindset shift is evident in companies that have moved beyond the “cost‑first” narrative to a “speed‑first” one, where economics become a secondary consideration.
Wonder, the cloud‑native food‑delivery platform, illustrates this trend. CTO James Chen notes that AI adds only a few cents per order – a fraction of the 14‑cent technology cost – and that the real challenge is capacity. The company assumed unlimited cloud resources, but rapid growth forced a move to a second region months ahead of schedule. Wonder’s ambition to train ultra‑efficient, personalized micro‑models remains hampered by the high cost of one model per user, so the team balances experimentation with tight budget controls, treating cost forecasting as an art rather than a science. The cost of repeatedly sending large context payloads to large language models can consume up to 80 % of spend, amplifying the pressure to keep models lightweight.
Recursion, a biotech firm, tackles a different side of the same problem by building a hybrid infrastructure that blends on‑premise GPU clusters with public‑cloud inference. Early on, cloud vendors offered limited options, so Recursion’s 2017 cluster used gaming GPUs that are still in use today, debunking the myth that GPUs have a three‑year lifespan. For massive training jobs that need tight data locality, on‑prem is ten times cheaper and cuts five‑year total cost of ownership in half. Shorter, low‑priority workloads, meanwhile, run on the cloud with pre‑empted GPUs and TPUs, accepting slower turnaround in exchange for cost efficiency. Mabey warns that without a long‑term commitment to compute, organizations will spend on demand and stifle innovation. The lesson is clear: investing in the right mix of capacity, location, and model size unlocks rapid deployment and sustained growth, turning AI from an expensive experiment into a strategic asset.
Want the full story?
Read on VentureBeat →