Data Scientist

Mercor
London
1 day ago
Create job alert

Job Description: AI Task Evaluation & Statistical Analysis Specialist

Role Overview

We're seeking a data-driven analyst to conduct comprehensive failure analysis on AI agent performance across finance-sector tasks. You'll identify patterns, root causes, and systemic issues in our evaluation framework by analyzing task performance across multiple dimensions (task types, file types, criteria, etc.).

Key Responsibilities

  • Statistical Failure Analysis: Identify patterns in AI agent failures across task components (prompts, rubrics, templates, file types, tags)

  • Root Cause Analysis: Determine whether failures stem from task design, rubric clarity, file complexity, or agent limitations

  • Dimension Analysis: Analyze performance variations across finance sub-domains, file types, and task categories

  • Reporting & Visualization: Create dashboards and reports highlighting failure clusters, edge cases, and improvement opportunities

  • Quality Framework: Recommend improvements to task design, rubric structure, and evaluation criteria based on statistical findings

  • Stakeholder Communication: Present insights to data labeling experts and technical teams

Required Qualifications

  • Statistical Expertise: Strong foundation in statistical analysis, hypothesis testing, and pattern recognition

  • Programming: Proficiency in Python (pandas, scipy, matplotlib/seaborn) or R for data analysis

  • Data Analysis: Experience with exploratory data analysis and creating actionable insights from complex datasets

  • AI/ML Familiarity: Understanding of LLM evaluation methods and quality metrics

  • Tools: Comfortable working with Excel, data visualization tools (Tableau/Looker), and SQL

Preferred Qualifications

  • Experience with AI/ML model evaluation or quality assurance

  • Background in finance or willingness to learn finance domain concepts

  • Experience with multi-dimensional failure analysis

  • Familiarity with benchmark datasets and evaluation frameworks

  • 2-4 years of relevant experience

Related Jobs

View all jobs

Data Scientist

Data Scientist

Data Scientist

Data Scientist - Contract - 12 months

Data Scientist (Globally Renowned Retail Group)

Data Scientist Project Lead

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Jobs

Neurodiversity in AI Careers: Turning Different Thinking into a Superpower

The AI industry moves quickly, breaks rules & rewards people who see the world differently. That makes it a natural home for many neurodivergent people – including those with ADHD, autism & dyslexia. If you’re neurodivergent & considering a career in artificial intelligence, you might have been told your brain is “too much”, “too scattered” or “too different” for a technical field. In reality, many of the strengths that come with ADHD, autism & dyslexia map beautifully onto AI work – from spotting patterns in data to creative problem-solving & deep focus. This guide is written for AI job seekers in the UK. We’ll explore: What neurodiversity means in an AI context How ADHD, autism & dyslexia strengths match specific AI roles Practical workplace adjustments you can ask for under UK law How to talk about your neurodivergence during applications & interviews By the end, you’ll have a clearer picture of where you might thrive in AI – & how to set yourself up for success.

Jobs

AI Hiring Trends 2026: What to Watch Out For (For Job Seekers & Recruiters)

As we head into 2026, the AI hiring market in the UK is going through one of its biggest shake-ups yet. Economic conditions are still tight, some employers are cutting headcount, & AI itself is automating whole chunks of work. At the same time, demand for strong AI talent is still rising, salaries for in-demand skills remain high, & new roles are emerging around AI safety, governance & automation. Whether you are an AI job seeker planning your next move or a recruiter trying to build teams in a volatile market, understanding the key AI hiring trends for 2026 will help you stay ahead. This guide breaks down the most important trends to watch, what they mean in practice, & how to adapt – with practical actions for both candidates & hiring teams.

Jobs

How to Write an AI CV that Beats ATS (UK examples)

Writing an AI CV for the UK market is about clarity, credibility, and alignment. Recruiters spend seconds scanning the top third of your CV, while Applicant Tracking Systems (ATS) check for relevant skills & recent impact. Your goal is to make both happy without gimmicks: plain structure, sharp evidence, and links that prove you can ship to production. This guide shows you exactly how to do that. You’ll get a clean CV anatomy, a phrase bank for measurable bullets, GitHub & portfolio tips, and three copy-ready UK examples (junior, mid, research). Paste the structure, replace the details, and tailor to each job ad.

🍪 We use cookies to enhance your browsing experience on our website. By continuing to use this site, you consent to the use of cookies as described in our Cookie Policy. You can review our Cookie Policy for more information.