Karve
Browse all jobs

Human Data Quality Engineer (Founding Team) 

Prolific · London

3 months ago

Remotehuman data qualityllm evaluationmachine learning pipelines

Workplace

Remote

Location

London

Posted

3 months ago

Summary

As a founding member of prolific's ai data services team, you will build quality systems for human data used to train advanced ai models. You will work with clients and frontier ai labs to define what high-quality human data means, engineer quality throughout the lifecycle, and create scalable evaluation and monitoring frameworks using python and sql.

Responsibilities

  • Design quality frameworks for human data programs
  • Translate client model needs into data strategies
  • Engineer quality upstream across recruitment and training
  • Build scalable quality systems and automated checks
  • Investigate data quality issues and identify root causes
  • Architect dashboards for monitoring and reporting
  • Upskill operations and mentor junior analysts
pythonsqldashboardsmonitoringstatistical thinkingmachine learning pipelinesllm evaluationrlhfred teamingdrift detection
human data qualityllm evaluationmachine learning pipelinesdata annotation qualityai safety and red teaming