Summary

The senior machine learning engineer will optimize LLM and VLM inference systems across model internals, serving engines, and production endpoints. The role focuses on latency, throughput, memory efficiency, GPU utilization, model quality, and cost per token in a hands-on AI cloud environment.

Responsibilities

• Optimize endpoints for latency and throughput
• Deploy and benchmark inference engines
• Debug production model regressions
• Build model compression workflows
• Implement speculative decoding and KV-cache optimization
• Diagnose bottlenecks across serving layers