All roles

Open role

[Remote] Domain Expert, AI Evaluation Task Writer: Unreal Game Development (Contract, Remote)

Remote · Italy Full-time

Note: The job is a remote job and is open to candidates in USA. Patronus AI is hiring a contract Unreal Engine domain expert to create evaluation and training tasks for frontier AI models. The role involves designing realistic UE5 tasks, building reference solutions and objective scoring criteria, testing model performance, and reviewing other contributors’ tasks.

Responsibilities

  • Author tasks from a short brief for a project area: write the task the way you would hand it to a mid-level colleague, list the assets, Blueprints and source files involved, and write acceptance checks a tester could verify without opinion
  • Build the project materials a task needs: Blueprints, C++ classes, data assets, materials, Niagara systems, levels and test maps, along with the build, cook and log output that shows the starting state
  • Solve each task end to end: implement the reference solution in UE 5.8, deliver the changed files, and record each step and how you verified every check
  • Write explicit scoring criteria that let two independent graders reach the same verdict, including acceptable alternative implementations
  • Run tasks against current models, find where and why they fail, and adjust difficulty
  • Review tasks written by others: open the project, read the task, score each part on whether it is realistic and whether the checks test the right thing, and cite what you looked at for every score
  • Incorporate reviewer feedback within agreed turnaround times and flag guideline gaps or edge cases you run into

Skills

  • 3+ years of hands-on Unreal Engine development in one or more of the four areas above, as a gameplay programmer, technical designer, technical artist or level designer. You need to have built the work yourself, not only directed it
  • Working command of the engine in your area (for example the gameplay framework and Gameplay Ability System, the material and Niagara systems, or greybox, nav mesh and encounter scripting), and the ability to open a project in UE 5.8, rebuild it, and work in both C++ and Blueprint
  • The habit of verifying behavior in the editor and in a packaged build, and of citing the log line, callstack, asset or setting behind every conclusion
  • Clear, precise writing in professional English
  • Ability to work independently and asynchronously against deadlines
  • At least one shipped UE5 title, or several years on a single UE5 project in your role, ideally a shooter, action or multiplayer game
  • Experience with UE5 systems beyond your core area, such as Nanite, Lumen, World Partition, PCG, MetaSounds, Mass or Control Rig, and with shipping on more than one platform
  • Experience with the surrounding toolchain, such as Perforce or Plastic SCM, Unreal Build Tool and Unreal Automation Tool, Unreal Insights, RenderDoc and a CI system such as Horde or Jenkins
  • Prior experience writing or reviewing AI evaluation tasks, technical interview questions, tutorials, courses or engine documentation
  • Familiarity with how AI models fail on engine work (APIs that don't exist or changed between versions, Blueprint logic that can't be built as described, fixes that compile but break replication or packaged builds, confident answers about settings that don't apply)

Benefits

  • Potential bonus for each accepted task.
  • The trial task is paid.
  • Fully remote, asynchronous.

Company Overview

  • Patronus AI develops training data and simulated environments for frontier AI. It was founded in 2023, and is headquartered in San Francisco, California, USA, with a workforce of 51-200 employees. Its website is https://www.patronus.ai.
  • Company H1B Sponsorship

  • Patronus AI has a track record of offering H1B sponsorships, with 4 in 2026, 1 in 2025. Please note that this does not guarantee sponsorship for this specific role.
  • More open positions

    [Remote] Senior Systems Software Engineer, CUDA and AI HW Profiler

    Work from home Full-time role

    [Remote] Senior Manager, B2B Outbound Marketing (Remote)

    Work from home Full-time role

    [Remote] Process Engineer II

    Work from home Full-time role

    [Remote] Social Media Fellow (Iowa)

    Work from home Full-time role

    [Remote] National Account Director, Medicaid, Global Rare Diseases - Midwest Job Details | Chiesi Group

    Work from home Full-time role

    Digital Campaign Success Manager

    Work from home Full-time role

    Native Campaign Manager

    Work from home Full-time role

    Underwriting Manager - Middle Market

    Work from home Full-time role

    Switchgear Equipment Category Manager - Remote

    Work from home Full-time role

    Aircraft Engineering Manager

    Work from home Full-time role

    Intermediate I&C Engineer

    Work from home Full-time role

    Freelance LMS Trainer – Content Development Opportunity

    Work from home Full-time role

    Content Expansion Associate

    Work from home Full-time role

    Manager, Lease Administration

    Work from home Full-time role

    Senior DevOps Engineer

    Work from home Full-time role

    Senior Remote Data Entry Specialist – Expert Spreadsheet Management & Data Integrity Champion – Work From Home Opportunity at brighthush

    Work from home Full-time role

    Senior Remote Data Entry Specialist – Expert in Precision Data Management, Database Administration, and Digital Documentation – Drive Data Quality Excellence from Anywhere – Join brighthush's Remote Data Innovation Team

    Work from home Full-time role

    Vice President of Products

    Work from home Full-time role

    [Remote] Product Manager (Leadership)

    Work from home Full-time role

    MT- RemoteSpanish To English Interpreters

    Work from home Full-time role

    Lead Cloud Architect

    Work from home Full-time role