PhysicsX
Platform Engineer - Self Hosting LLMs
- London
Builds the LLM access layer for agents on the PhysicsX platform, serving self-hosted open models and routing each task to the model or combination that best fits its constraints. Owns inference engines such as vLLM and SGLang and serving platforms such as KServe, including packaging, deployment, health checks, upgrades and rollback, plus GPU autoscaling, batching, caching and quantisation tuning. Sets technical direction for air-gapped serving and cost, quality and latency-aware routing.
vLLM · SGLang · KServe +5