ML Inference Engineer – Terminal-Bench Task Designer

Toptal is seeking a Machine Learning Inference Engineer to design expert-level tasks for a benchmarking project that evaluates AI agents in terminal environments. The role involves creating challenges based on real-world ML inference work, including defining the necessary systems and environments, writing reference solutions, and implementing automated checks to ensure accuracy. Candidates should have over three years of hands-on experience in ML inference or serving, or possess a PhD with active research in the field. Familiarity with tools such as PyTorch or JAX, as well as serving stacks like vLLM or TensorRT-LLM, is essential. The position requires the ability to identify potential issues in ML outputs, such as numerical drift. This is a fully remote position, making it suitable for individuals who thrive in flexible work environments. The role is ideal for those who have recently engaged in practical ML work and can articulate the systems and tools they have used.