Note: The job is a remote job and is open to candidates in USA. WEKA is building NeuralMesh, an AI-native data infrastructure platform for enterprise reasoning workloads. The Principal Product Manager will own the Augmented Memory Grid roadmap and go-to-market, partnering with engineering, enterprise customers, GPU cloud providers, and inference-stack partners to improve large-scale LLM inference performance, efficiency, and cost.

Responsibilities

• Own the AMG product roadmap: KV-cache/prefix-cache offload, memory tiering, and integration with inference engines and orchestration layers (vLLM, NVIDIA Triton/TensorRT-LLM/NIM, Kubernetes-based serving)
• Partner with engineering to define architecture trade-offs across GPU memory, networking (RDMA, GPUDirect, NVMe-oF), and distributed storage — translating inference performance bottlenecks (time-to-first-token, throughput, context length) into product requirements
• Work directly with enterprise customers and GPU cloud partners: Nebius, CoreWeave, TogetherAI, etc., running production inference workloads to gather requirements, validate benchmarks, and prioritize features that reduce cost-per-token and improve SLAs at scale
• Partner with NVIDIA and other silicon/inference-stack partners on joint roadmap and certification work
• Define and track benchmarks (TTFT, throughput, cache hit rate) that demonstrate AMG's value versus standard GPU-memory-only inference
• Support sales and field teams with technical positioning, competitive differentiation, and enterprise deal support

Skills

• Strong leadership skills with a history of successfully leading cross-functional teams
• You are a strategic thinker who can develop and execute product strategies that align with market trends and customer needs, as well as think critically about existing strategies
• You have a proven ability to translate strategic goals into actionable plans and deliver results
• You have excellent communication and interpersonal skills, with the ability to articulate complex technical concepts to both technical and non-technical stakeholders
• You are comfortable presenting product strategies and roadmaps to internal teams and external customers
• Hands-on product or engineering experience with LLM inference serving — vLLM, NVIDIA Triton/TensorRT-LLM/NIM, Ray Serve, or comparable — and fluency in concepts like KV-cache, prefix/context caching, quantization, and batching strategies
• Working knowledge of how modern LLMs are served in production (context windows, multi-tenant serving, GPU scheduling) well enough to translate model-level constraints into infrastructure requirements
• Comfort with the fundamentals of high-performance networking and distributed systems — RDMA, GPUDirect Storage, NVMe-oF, or equivalent — and how they affect inference performance
• Track record working directly with large enterprise accounts — requirements gathering, production deployments, SLAs — not solely self-serve/PLG products
• 10+ years of product management experience, ideally with some portion in infrastructure, ML platforms, or developer-facing technical products
• Prior experience at a GPU cloud, inference platform, or AI infrastructure startup
• Familiarity with storage systems (parallel/distributed file systems, object storage) in AI/ML pipelines
• Experience partnering directly with NVIDIA or other accelerator/silicon vendors

Company Overview

Company H1B Sponsorship