Summary

Senior data engineer responsible for designing, operating, and improving production ingestion pipelines across textual, time-series, and event-stream data. The role is remote-first and involves working with Python, AWS or GCP, Terraform, and modern data stack technologies to support energy transactions and AI initiatives.

Responsibilities

• Design resilient ingestion pipelines with recovery strategies
• Build and maintain etl and elt pipelines
• Choose batch or streaming approaches for pipelines
• Build monitoring and sla-based alerting systems
• Define data quality checks for sla commitments
• Optimise pipeline performance and operating costs
• Handle late arrivals, reprocessing, and data backfills