About the job
Testlio’s fully managed crowdsourced testing platform, powered by our proprietary intelligence technology – LeoAI EngineTM, integrates expert, on‑demand testers directly into your release process. Ship faster and more confidently with global coverage across 600,000+ devices, 800+ payment methods, 150+ countries, and 100+ languages. To learn more, visit www.testlio.com.
We are hiring a Staff Machine Learning Scientist (Applied AI) to help design, build, and scale Testlio’s next generation of AI‑powered data products. You’ll join our product team and will apply advanced machine learning, data science, and statistical methods to transform the rich data from our software testing platform into actionable insights for our customers. Your work will directly influence how global engineering and product teams understand product quality, user experience, and business impact. This is a hands‑on, high‑impact opportunity to shape the future of AI at Testlio while working in a remote, collaborative, and fast‑paced environment.
Why will you love this job?
- Build from the ground up. You’ll be part of the team shaping Testlio’s new AI product line, creating real‑world solutions that customers will use to make smarter product decisions.
- High‑impact data. Our platform generates unique and complex datasets from software testing at scale, giving you access to rich, real‑world data to design and deploy meaningful models.
- Experiment + innovate. You’ll have freedom to test, prototype, and deploy new approaches in NLP, predictive modeling, and data‑driven insights.
- Cross‑functional collaboration. You’ll work closely with engineers and customer‑facing teams to ensure your models deliver measurable business value.
Why will you love being a part of Testlio?
- Great Culture : Testlio is a female‑founded company, and half our team identifies as women. We’re proud of our inclusive, purpose‑driven culture where people genuinely enjoy collaborating. As part of our team (we call ourselves TestLions), you’ll help create exceptional digital experiences for our customers, while also contributing to our freelance network.
- Remote Work : Our culture is built around remote work. We’ve created systems to allow us to successfully work together asynchronously as a fully remote and globally distributed team. Testlio provides the tools and guidance for everyone to succeed in their careers in a fully remote setting. Our working style encourages everyone to make decisions, communicate effectively, and work at a sustainable pace.
- Investment in You : Your growth and well‑being matter to us. You’ll have flexible paid time off—including national holidays, personal days, and sick days—plus stock options so you can grow with Testlio. We also provide a $300 annual learning stipend to support your personal and professional development.
- Winning Business : Testlio is growing, profitable, and cash‑strong. We are leading our industry with exceptional clients who provide us with a high NPS score and a 4.7 rating on G2. Our business model is global, enterprise, and subscription‑based. Several of our largest clients have been with us for 7+ years, and many spend $500K+ / year with Testlio.
What would your day look like?
- Partner with engineering leaders to define, design, and deliver AI Data products.
- Explore and model complex datasets from Testlio’s platform using statistical, ML, and deep learning techniques.
- Prototype and validate models, then work with engineering to deploy them into production at scale.
- Research and implement techniques in areas like NLP, anomaly detection, recommendation systems, and predictive analytics.
- Translate raw outputs into clear, actionable insights that are easy for customers to understand and use.
- Measure and improve model performance continuously to ensure accuracy, fairness, and scalability.
- Contribute to building Testlio’s data science practice — influencing standards, tools, and best practices.
What do you need to succeed?
Technical Skills
- Advanced degree (Master’s or PhD) in Computer Science, Data Science, Statistics, or a related field.
- 10+ years applying end‑to‑end machine learning and statistical modeling solutions to real‑world problems, ideally in SaaS or data products.
- Strong Python skills and experience with deep learning frameworks such as PyTorch and TensorFlow.
- Hands‑on experience with NLP, predictive modeling, recommendation systems, anomaly detection, and training ML and deep learning models from scratch.
- Expertise in data wrangling, feature engineering, and managing large, complex datasets.
- Knowledge of Large Language Models (LLMs) and Small Language Models (SLMs), including architecture, training, fine‑tuning (LoRA, QLoRA, SFT), and deployment strategies.
- Proven experience designing, building, and maintaining end‑to‑end ML pipelines and MLOps frameworks, including model training, deployment, monitoring, and lifecycle management.
- Hands‑on experience designing, deploying, and maintaining AI agents, including multi‑agent systems, in production with robust APIs and error handling.
- Proficiency with SQL, data visualization tools, and cloud platforms such as AWS or Azure.
Human Skills
- Curiosity + creativity. You’re eager to experiment, learn, and push the boundaries of what’s possible with data.
- Business orientation. You can translate technical outputs into meaningful customer value.
- Collaborative spirit. You enjoy working across teams to deliver impact.
- Adaptability. You thrive in fast‑changing environments where priorities evolve as we scale.
- Mentorship mindset. You enjoy sharing knowledge and uplifting teammates.
- Growth mindset. You continuously refine your craft, stay current with advances in AI/ML, and seek feedback to improve.
What is the application process?
- Application
- Recruiter interview
- TestGorilla assessment
- ~3 Team and Stakeholder interviews, inclusive of skills demonstration
- Reference checks
- Offer
Diversity and Inclusion
Testlio is an equal‑opportunity employer deeply committed to creating an inclusive environment for people of all backgrounds and identities. We are female‑founded, and 46% of our team members identify as women. For more information, see the DEI section of our website.
#LI-Remote
