Open role
[Remote] Senior Software Engineer, AI Developer Experience
Note: The job is a remote job and is open to candidates in USA. Vanta is hiring a Senior Software Engineer to build the infrastructure and developer workflows that make AI-powered engineering reliable and useful. The role focuses on background agents, agent evaluation systems, and tools that help engineers adopt and improve AI-enabled development practices.
Responsibilities
- Plan, build, and improve background-agent infrastructure that enables reliable, trackable, and repeatable automated engineering work
- Develop systems for evaluating agent performance using task outcomes, quality measures, human intervention, cost, and other signals
- Create evaluation sets and feedback loops that help AI systems improve over time through regular review, tuning, and correction
- Improve how engineers discover, understand, and use shared AI tools and workflows
- Evaluate AI development tools, including code generation, AI-assisted review, and testing, against real engineering needs, and help integrate the ones that prove useful
- Develop visibility into which tools and workflows are useful, where they create friction, and how they should evolve
- Create clear guidance and documentation that helps engineers adopt new AI-enabled development practices
- Improve code review workflows by using feedback from real engineering work to identify and address recurring friction
- Influence the foundations of an autonomous software factory, including systems that help agents complete bounded engineering tasks safely and effectively
- Work directly with engineers and internal users to understand their needs, validate solutions, and make thoughtful product tradeoffs
- Help the team move ambiguous, dependency-heavy projects from early exploration through reliable delivery
- Raise the quality bar through thoughtful code reviews, technical design discussions, testing, observability, and attention to reliability and security
- Partner closely with engineers and cross-functional stakeholders while contributing as a dependable, collaborative teammate
Skills
- Substantial industry software engineering experience
- Experience taking technical leadership roles on teams or complex projects
- Experience improving AI systems, creating evaluation sets, and driving quality hill-climbing through measurement and iteration
- Strong software engineering fundamentals and experience designing, building, testing, and operating reliable systems
- Experience with developer tooling, AI-enabled workflows, agentic systems, infrastructure, or related technical domains
- A track record of working through ambiguity, decomposing complex problems, managing dependencies, and helping projects reach completion
- Experience working in rapidly scaling startups or larger companies, particularly in environments with a strong bias for action
- An action-oriented approach and a willingness to engage directly with users and contribute hands-on
- Product judgment and curiosity: you bring strong technical opinions while staying focused on building platforms that meet users where they are
- Strong written and verbal communication, with the ability to explain technical concepts clearly and create guidance that helps others adopt new tools
- A collaborative approach and a demonstrated ability to make meaningful contributions as part of a team
- Uses AI in day-to-day work to amplify their skills and strengthen their outcomes, applying it across routine workflows, actively building fluency through experimentation, and exercising sound judgment to ensure AI-assisted work meets a high quality bar
Benefits
- Equity
- Comprehensive medical, dental, and vision coverage, with 100% of employee-only benefit premiums covered for most medical plans
- 16 weeks paid Parental Leave for all new parents
- Health & wellness stipend
- Remote workspace, internet, and cellphone stipend
- Commuter benefits for team members who report to the SF and NYC office
- Family planning benefits
- Matching 401(k) contribution with immediate vesting
- Flexible PTO policy, plus 80 hours of Sick Time
- 11 company-paid holidays
- Virtual team building activities, lunch and learns, and other company-wide events
- Remote work
Company Overview
Company H1B Sponsorship