Summary

This senior backend software engineer will join the crawler team to build systems that ingest, process, and manage web-scale data for advanced search technologies. The role focuses on distributed crawler services, full-stack data acquisition features, and high-load infrastructure using python, c++, rust, go, kubernetes, and aws.

Responsibilities

• Lead large-scale web crawler development projects
• Build and optimize crawler storage and retrieval components
• Develop ui dashboards for data management
• Architect scalable distributed full-stack features
• Operate ingestion pipelines processing web-scale data
• Deploy and optimize high-load backend services