Be at the heart of actionFly remote-controlled drones into enemy territory to gather vital information.

Apply Now

Data Scientist

Mercor
City of London
1 day ago
Create job alert

Get AI-powered advice on this job and more exclusive features.


Job Description: AI Task Evaluation & Statistical Analysis Specialist
Role Overview

We're seeking a data-driven analyst to conduct comprehensive failure analysis on AI agent performance across finance-sector tasks. You'll identify patterns, root causes, and systemic issues in our evaluation framework by analyzing task performance across multiple dimensions (task types, file types, criteria, etc.).


Key Responsibilities

  • Statistical Failure Analysis: Identify patterns in AI agent failures across task components (prompts, rubrics, templates, file types, tags)
  • Root Cause Analysis: Determine whether failures stem from task design, rubric clarity, file complexity, or agent limitations
  • Dimension Analysis: Analyze performance variations across finance sub-domains, file types, and task categories
  • Reporting & Visualization: Create dashboards and reports highlighting failure clusters, edge cases, and improvement opportunities
  • Quality Framework: Recommend improvements to task design, rubric structure, and evaluation criteria based on statistical findings
  • Stakeholder Communication: Present insights to data labeling experts and technical teams

Required Qualifications

  • Statistical Expertise: Strong foundation in statistical analysis, hypothesis testing, and pattern recognition
  • Programming: Proficiency in Python (pandas, scipy, matplotlib/seaborn) or R for data analysis
  • Data Analysis: Experience with exploratory data analysis and creating actionable insights from complex datasets
  • AI/ML Familiarity: Understanding of LLM evaluation methods and quality metrics
  • Tools: Comfortable working with Excel, data visualization tools (Tableau/Looker), and SQL

Preferred Qualifications

  • Experience with AI/ML model evaluation or quality assurance
  • Background in finance or willingness to learn finance domain concepts
  • Experience with multi-dimensional failure analysis
  • Familiarity with benchmark datasets and evaluation frameworks
  • 2-4 years of relevant experience

Seniority level

Not Applicable


Employment type

Full-time


Job function

Engineering and Information Technology


Industries

Software Development


#J-18808-Ljbffr

Related Jobs

View all jobs

Data Scientist

Data Scientist

Data Scientist

Data Scientist

Data Scientist

Data Scientist

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

Jobs

AI Hiring Trends 2026: What to Watch Out For (For Job Seekers & Recruiters)

As we head into 2026, the AI hiring market in the UK is going through one of its biggest shake-ups yet. Economic conditions are still tight, some employers are cutting headcount, & AI itself is automating whole chunks of work. At the same time, demand for strong AI talent is still rising, salaries for in-demand skills remain high, & new roles are emerging around AI safety, governance & automation. Whether you are an AI job seeker planning your next move or a recruiter trying to build teams in a volatile market, understanding the key AI hiring trends for 2026 will help you stay ahead. This guide breaks down the most important trends to watch, what they mean in practice, & how to adapt – with practical actions for both candidates & hiring teams.

Jobs

How to Write an AI CV that Beats ATS (UK examples)

Writing an AI CV for the UK market is about clarity, credibility, and alignment. Recruiters spend seconds scanning the top third of your CV, while Applicant Tracking Systems (ATS) check for relevant skills & recent impact. Your goal is to make both happy without gimmicks: plain structure, sharp evidence, and links that prove you can ship to production. This guide shows you exactly how to do that. You’ll get a clean CV anatomy, a phrase bank for measurable bullets, GitHub & portfolio tips, and three copy-ready UK examples (junior, mid, research). Paste the structure, replace the details, and tailor to each job ad.

Jobs

AI Recruitment Trends 2025 (UK): What Job Seekers Must Know About Today’s Hiring Process

Summary: UK AI hiring has shifted from titles & puzzle rounds to skills, portfolios, evals, safety, governance & measurable business impact. This guide explains what’s changed, what to expect in interviews, and how to prepare—especially for LLM application, MLOps/platform, data science, AI product & safety roles. Who this is for: AI/ML engineers, LLM engineers, data scientists, MLOps/platform engineers, AI product managers, applied researchers & safety/governance specialists targeting roles in the UK.

🍪 We use cookies to enhance your browsing experience on our website. By continuing to use this site, you consent to the use of cookies as described in our Cookie Policy. You can review our Cookie Policy for more information.