ML Inference Engineer – Terminal-Bench Task Designer (vLLM / SGLang / TensorRT-LLM)

Toptal is seeking a Machine Learning Inference Engineer to design expert-level tasks for a benchmarking project that evaluates AI agents' performance in terminal environments. The role involves creating challenges based on real-world ML inference tasks, defining the necessary systems and environments, and developing working solutions that can be verified for accuracy. Candidates should have at least three years of hands-on experience in ML inference or serving, or possess a PhD with active research in the field. Familiarity with tools and systems such as PyTorch, JAX, vLLM, SGLang, or TensorRT-LLM is essential. The position requires the ability to explain potential errors in ML outputs and to package tasks using Docker and Git for reproducibility. This remote role is suited for individuals who can translate their professional experience into realistic challenges for AI agents, ensuring that tasks remain expert-level and objectively gradable. The compensation model is task-based, and applicants are encouraged to review the application questions for further details.