Realtime Compute for Apache Flink
Staff Engineer, product experience · Alibaba Cloud · 2015 – 2023

Realtime Compute is Alibaba Cloud's fully managed, serverless Apache Flink. It uses the same engine that handles the traffic behind Tmall's Double Eleven, sold as a service where a team writes SQL and never has to manage a cluster. I led the web engineering for its product experience.
Streaming jobs are hard to build a product around. A job runs for a long time and can be healthy, slowly falling behind, or producing wrong results, and users need to see which one it is before someone gets paged at 3am. So most of the design work went into the job lifecycle: what you see when a job is deployed, scaled, restarted from a checkpoint, or falling behind its source.
Two things made the console hard to build. The first was how fast the data changes: a running job reports metrics, checkpoints and logs continuously, and the pages have to keep up with them without stalling. The second was that streaming jobs never go offline. A job deployed years ago on an old engine version is still running today, so the console has to keep working with every version of the API that has ever shipped, alongside the newest one.
Branching on versions throughout the pages would not have lasted. Instead, the pages depend on an abstract service, and the implementation for each version is injected where it's needed, so supporting a new version means adding an implementation rather than editing every page. I wrote about the approach in Frontend Dependency Injection for TC39 Standards.