Content & Experience Evaluator, 3335-1 Member Experience The Opportunity If you're an avid fan and serious expert in visual entertainment, the Member Experience team is seeking a unique candidate who can capture the essence of films, series, and other content, all while keeping the experience of our users top of mind. As a Content & Experience Evaluator, your work will directly contribute to the training, evaluation, and improvement of GenAI applications that power member-facing experiences. In this role, you'll apply expert judgment to rate, annotate, and review large-scale content datasets used to train and refine AI systems. By bringing nuance, consistency, and clarity to content evaluation, you'll help ensure that AI-driven solutions better understand films and series, respond accurately to member needs, and deliver high-quality, trusted experiences at scale. The Ideal Candidate Has a deep and broad knowledge of visual media and is passionate about global entertainment, with a strong understanding of films and series across genres and markets. They can succinctly and objectively communicate what is important about a piece of content, and why, distilling complex creative works into clear, structured insights. The ideal candidate is experienced as an expert annotator, rater, or dataset reviewer, particularly in structured or rubric-based environments that support AI or machine-learning use cases. They are comfortable applying expert judgment within defined frameworks, navigating nuance and ambiguity while maintaining consistency and quality at scale. They thrive in a high-volume, high-quality, deadline- and data-driven environment, and are highly detail-oriented, reliable, and disciplined in their work. They are flexible, adaptive, and curious, with the ability to evolve alongside changing tools, processes, and organizational needs, and are motivated by work that directly improves member-facing experiences through high-quality data.

Responsibilities

Rate, annotate, and review content recommendations and generated text to support training, evaluation, and quality improvement of large language models (LLMs) Apply established guidelines and expert judgment to capture nuance, intent, and context within creative content Review and validate datasets used in AI and machine-learning workflows for accuracy, consistency, and completeness Identify gaps, edge cases, or patterns in annotations that may impact model performance or member-facing outcomes Document evaluations and rationale in a clear, objective, and concise manner Collaborate with internal partners across product, data, and AI teams to continuously improve dataset quality and relevance

Qualifications

Prior experience in media as a writer, editor, content designer, or metadata analyst OR experience as a domain-expert content labeller and AI trainer Passionate about films and series with a deep knowledge of the U.S. entertainment industry and the global streaming landscape Ability to distinguish nuances within different content genres and distill the essence of a film/series, including sharing findings in an objective, concise manner Ability to master internal tools and systems successfully Fluency in English and strong conversational writing skills required Fluency in Spanish strongly preferred Experience working with large data catalogs is a plus Pay Rate Range $60–70/hr.