Netflix has described a redesigned automated system for checking whether the short synopses shown for its titles are factually accurate, according to a post on the Netflix TechBlog published November 9 and authored by five of its engineers and researchers. [1]
The company says its existing checker sends a reasoning model all available title information, including metadata, text assets and a factuality rubric, and repeats the call five times over contexts exceeding 100,000 tokens, failing a synopsis if any run flags it. Netflix says this is costly and that the model's explanations often diverge from those of its human annotators. [1]
The new "cascade" design runs every synopsis through two specialized judges: a literal judge that checks whether statements are true, and an implicature judge that checks whether a synopsis gives a misleading picture to a new viewer. Confident verdicts are kept, while low-confidence cases are escalated to a router model that turns the judges' doubts and assumptions into discrete claims and sends them to a web-search judge or a targeted recheck of source material. [1]
Netflix reports that the approach cut total cost by 52.4% on its show-level dataset and raised recall on non-factual synopses from 0.77 to 0.86. It says the system's explanations now overlap with human rationales about 82% of the time, up from about 69%, a measure scored by another LLM acting as a judge. On episode-level synopses, the company says the cascade was 15% cheaper than its current pipeline. [1]
The company acknowledges trade-offs: median latency increased compared with the old pipeline, though Netflix calls it reasonable. It also says simpler alternatives fell short, with more reasoning effort making the checker too lenient and a smaller model making it too strict. All results come from Netflix's own internal benchmark of human-labeled synopses and have not been independently reproduced. [1]
Netflix says future work may include adding multimodal inputs and using the checker's rationales to automatically correct faulty synopses. [1]