National AI Awards 2025Discover AI's trailblazers! Join us to celebrate innovation and nominate industry leaders.

Nominate & Attend

Software Engineer - Model Evaluation and Productisation (Must be UK based)

PolyAI
London
8 months ago
Applications closed

Related Jobs

View all jobs

Software Engineer

Software Engineer

Software Engineer

Software Engineer, Hardware Control

Software Engineering Manager – Machine Learning

Lead Software Engineer

PolyAI is a leader in automating customer service through innovative voice technology. Our voice assistants empower businesses to deliver exceptional customer service at every interaction.

We are seeking a talented and hands-on Software Engineer with a strong data science background to join our team.

In this role, you will work on building software to enhance the visibility and configurability of large language models (LLMs). You will be responsible for rapidly developing tools and platforms to evaluate, iterate, and productionalize models, ensuring their reliability and accuracy.

We are looking for the right candidate, and therefore are flexible on the levelling for this position ranging from mid- level to senior!

Responsibilities:

  • Must have at least 2 years of Python experience
  • Must have at least 2+ years working experience
  • Develop software that provides visibility into LLM models and offers configurability for tuning and evaluation.
  • Build and maintain evaluation datasets and tools, enabling the measurement of model performance across key metrics.
  • Take a hands-on approach to quickly prototype, test, and iterate on solutions, moving from proof-of-concept to production.
  • Employ a data-driven methodology to drive model accuracy, leveraging evaluation results to inform decisions.
  • Collaborate with cross-functional teams to integrate developed tools and ensure they meet production standards.
  • Formulate hypotheses, design experiments, and collect data to validate model assumptions, consistently striving for improved reliability.
  • Communicate findings and ranking metrics clearly to both technical and non-technical stakeholders.

Why Join Us:

Join a dynamic and innovative team at the forefront of LLM development. You will have the opportunity to work on challenging projects, rapidly build impactful solutions, and drive data-informed improvements that push the boundaries of what LLMs can achieve.

Requirements

  • Degree in Computer Science, Data Science, or related field, or equivalent experience.
  • Strong proficiency in Python
  • Experience developing evaluation suites, datasets, and data-driven tools for model reliability testing.
  • Ability to rapidly prototype and iterate on solutions while maintaining a focus on production-level quality.
  • Strong problem-solving skills and a creative mindset, with the ability to hypothesize and validate results through experimentation.
  • Familiarity with cloud platforms such as AWS, GCP, or Azure is a plus.

Benefits

Participation in the company’s employee share options plan

25 days holiday, plus bank holidays

Flexible working from home policy plus a one-off WFH allowance when you join

Work from outside of the UK for up to 6 months each year

Enhanced parental leave

Yearly learning budget

Bike2Work scheme

Annual learning and development allowance

One-off WFH allowance when you join

‍ ‍Company-funded fertility and family-forming programmes

Menopause care programme with Maven

Private healthcare and dental cover, discounts on gym members and relaxation apps, and access to a range of mental health programs

Equal Opportunity Statement:

PolyAI is proud to be an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.

All employment decisions at PolyAI will be based on the business needs without attention to ethnicity, religion, sexual orientation, gender identity, family or parental status, national origin, neurodiversity status or disability status.

National AI Awards 2025

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

AI Jobs Skills Radar 2026: Emerging Frameworks, Languages & Tools to Learn Now

As the UK’s AI sector accelerates towards a £1 trillion tech economy, the job landscape is rapidly evolving. Whether you’re an aspiring AI engineer, a machine learning specialist, or a data-driven software developer, staying ahead of the curve means more than just brushing up on Python. You’ll need to master a new generation of frameworks, languages, and tools shaping the future of artificial intelligence. Welcome to the AI Jobs Skills Radar 2026—your definitive guide to the emerging AI tech stack that employers will be looking for in the next 12–24 months. Updated annually for accuracy and relevance, this guide breaks down the top tools, frameworks, platforms, and programming languages powering the UK’s most in-demand AI careers.

How to Find Hidden AI Jobs in the UK Using Professional Bodies like BCS, IET & the Turing Society

When it comes to job hunting in artificial intelligence (AI), most candidates head straight to traditional job boards, LinkedIn, or recruitment agencies. But what if there was a better way to find roles that aren’t advertised publicly? What if you could access hidden job leads, gain inside knowledge, or get referred by people already in the field? That’s where professional bodies and specialist AI communities come in. In this article, we’ll explore how UK-based organisations like BCS (The Chartered Institute for IT), IET (The Institution of Engineering and Technology), and the Turing Society can help you uncover AI job opportunities you won’t find elsewhere. We'll show you how to strategically use their directories, special-interest groups (SIGs), and CPD (Continuing Professional Development) events to elevate your career and expand your AI job search in ways most job seekers overlook.

How to Get a Better AI Job After a Lay-Off or Redundancy

Being made redundant or laid off can feel like the rug has been pulled from under you. Whether part of a wider company restructuring, budget cuts, or market shifts in tech, many skilled professionals in the AI industry have recently found themselves unexpectedly jobless. But while redundancy brings immediate financial and emotional stress, it can also be a powerful catalyst for career growth. In the fast-evolving field of artificial intelligence, where new roles and specialisms emerge constantly, bouncing back stronger is not only possible—it’s likely. In this guide, we’ll walk you through a step-by-step action plan for turning redundancy into your next big opportunity. From managing the shock to targeting better AI jobs, updating your CV, and approaching recruiters the smart way, we’ll help you move from setback to comeback.