Senior Product Manager, AI Evals

As part of our Product Management team, we are seeking a Senior This role will define how LSEG measures the accuracy, reliability and consistency of AI-powered interactions across our data, analytics and market solutions. You will establish the frameworks, processes and standards that evaluate LLM behaviour, MCP tool usage, and natural-language output quality across multiple AI stacks-ensuring our customers receive precise, trustworthy, and explainable results. Key Responsibilities & Accountabilities Define requirements for internal tooling: scenario catalogues, result dashboards, model-versus-model comparisons, performance tracking and release gates.Contribute to group-wide AI governance by defining evaluation requirements that support responsible AI, explainability and bias mitigation. Skills & Competencies Background in product management , QA/test automation , or ML evaluation frameworks with experience designing structured test systems at scale. Strong knowledge of LLMs , prompt engineering, and tool-augmented LLM workflows. Experience with test automation, scenario design, evaluation metrics and quality frameworks in complex systems.Familiarity with financial data, analytics workflows, or MCP-style interoperability . Ability to translate ambiguous real-world user tasks into formal evaluation structures and rubrics. Experience with LLM-based evaluators, rubric scoring systems, structured data validation, and regression frameworks. Familiarity with financial data pipelines, structured/unstructured data handling, analytics APIs, and Python/JS-based tooling ecosystems. Strong analytical skills: ability to identify quality gaps, failure patterns and reliability issues in AI systems.Ability to support product teams in shipping AI features with confidence. Our purpose is the foundation on which our culture is built. Our values of Integrity, Partnership , Excellence and Change underpin our purpose and set the standard for everything we do, every day. They go to the heart of who we are and guide our decision making and everyday actions.LSEG (London Stock Exchange Group) is a leading global financial markets infrastructure and data provider. Our purpose is driving financial stability, empowering economies and enabling customers to create sustainable growth. Our culture of connecting, creating opportunity and delivering excellence shapes how we think, how we do things and how we help our people fulfil their potential.