Blog

Rebuilding Netflix Video Processing Pipeline with Microservices

Netflix Technology Blog

Read original

TL;DR

Netflix rebuilt the system that turns source video into the streams members play. The new pipeline runs on Cosmos, a microservice platform, instead of the older monolithic path they called Reloaded. Separate services own pieces such as complexity analysis and the bitrate ladder, so encoding recipes can change without a single release of the whole pipeline.

Why it matters

Video encoding is both a research loop (better ladders, new codecs) and a production factory (every title, every partner). A monolith couples those. Splitting them let the encoding team ship experiments faster and finish moving traffic off Reloaded in September 2023. The post is the overview; later posts in the series go service by service.

Key takeaways

  • Cosmos is the next platform. Reloaded is the system it replaced, and the two ran side by side during the migration.
  • Complexity analysis and ladder generation are separate services. The ladder service reads complexity data and writes encoding recipes.
  • That split is how per-title and per-shot encoding ideas stay editable without rewriting capture and bookkeeping.
  • The switchover of necessary functionality completed in September 2023, after years of parallel operation.
  • This post is the map. The Video Encoding Service writeup is the next, deeper chapter.

The Video and Image Encoding team in Encoding Technologies (ET) has spent the last few years rebuilding the video processing pipeline on our next-generation microservice-based computing platform Cosmos.

From the Netflix Technology Blog, January 10, 2024.