Human Data Quality Engineer (Founding Team)
Prolific · London
3 months ago
Remotehuman data qualityllm evaluationmachine learning pipelines
3 months ago
Workplace
Remote
Location
London
Posted
3 months ago
Summary
As a founding member of prolific's ai data services team, you will build quality systems for human data used to train advanced ai models. You will work with clients and frontier ai labs to define what high-quality human data means, engineer quality throughout the lifecycle, and create scalable evaluation and monitoring frameworks using python and sql.
Responsibilities
- Design quality frameworks for human data programs
- Translate client model needs into data strategies
- Engineer quality upstream across recruitment and training
- Build scalable quality systems and automated checks
- Investigate data quality issues and identify root causes
- Architect dashboards for monitoring and reporting
- Upskill operations and mentor junior analysts
pythonsqldashboardsmonitoringstatistical thinkingmachine learning pipelinesllm evaluationrlhfred teamingdrift detection
human data qualityllm evaluationmachine learning pipelinesdata annotation qualityai safety and red teaming
