Yadong Xie / 谢亚东
ProjectsLabWritingSaved|GithubLinkedInX

Lepton AI

Founding member · Lepton AI · 2023 – 2025

Lepton AI screenshot

I was employee number one and a founding member of Lepton AI, and I built the platform end to end.

Lepton was an AI cloud. Inference deployments came with autoscaling, observability, and multi-region availability. GPU Pods gave you a full development environment you could SSH into. Training let a team share a pool of GPUs and queue distributed jobs by priority. All of this ran on dedicated compute that you could rent from us or bring from your own cloud account.

Deploying a model was meant to be quick. You picked one from the catalog (Llama 3.1 from 8B up to 405B, Mixtral, Gemma, Stable Diffusion) or pointed the same form at your own model or a container image. You named it, added a Hugging Face token if the weights were gated, picked a shape, saw its hourly price, and created it. The form also showed the idle behavior: after an hour without use, it scales to zero.

The Lepton AI console: a catalog of prebuilt models, then a short form naming an endpoint, choosing a resource shape and creating it
From the catalog to an endpoint on one screen, with the hourly price shown before you commit.

Every endpoint opened on a playground before the API, so you could try it first. You talked to the model in the browser, by voice if you turned voice mode on, adjusted temperature and token limits to see the answer change, then switched to the API tab and copied the same call as code.

A deployed Llama endpoint answering in its browser playground with voice mode on, temperature and token sliders alongside, then the API tab showing the equivalent call as code
The same endpoint in the playground, then as code.

GPU Pods were the other half of the product: compute to work inside instead of a model to call. You chose a card, with its VRAM and hourly cost shown, gave it an image and a run command, and waited. You got back an SSH command, TCP port forwarding, and a JupyterLab you could open in a browser tab.

Creating a GPU pod in the Lepton console by picking a card and an image, then the running pod's SSH and port-forwarding details, and finally JupyterLab open in the browser
A GPU rented by the hour, ending in a notebook. The AMD card between the two NVIDIA ones shows the multi-vendor support.

That covers only a small part of the product. The map below shows all of it: compute (deployments, jobs, pods, and fine-tuning), storage, networking, machines, observability, settings, and billing, down to individual controls such as the ten fields that define a health check, the autoscaling rule that can watch queries per replica or GPU utilization, and the four things a pod needs before it starts. I owned the user experience across all of it, and with a product this wide, most of the work was keeping it coherent.

A mind map of every Lepton AI feature, branching from the center into compute, storage, network, machines, billing, settings and observability, each opening into its own screens and controls
Every screen and control in the product on one sheet. Open the PDF

To show what the platform could run, I also built a set of apps on it: Elmo Chat, a Chrome extension that summarizes any page and that Vercel's CEO shared; Lepton Search, one of the first and most popular open-source LLM search engines; and ImgPilot and tryEmoji, which paint a sketch as you draw it and an emoji in an artist's style.

In SemiAnalysis's ClusterMAX review of the GPU cloud market, Lepton was rated Gold, one tier below Platinum, alongside Crusoe, Nebius, Oracle, Azure, and Together AI, and it was singled out for running across multi-cloud infrastructure.

NVIDIA acquired Lepton in 2025, and I now lead an engineering team building the platform of NVIDIA DGX Cloud.

Mind map — every Lepton AI feature · NVIDIA DGX Cloud Lepton · SemiAnalysis — the GPU cloud ClusterMAX rating system

←NVIDIA DGX CloudElmo Chat→