Open role
AI Analytics Workflow Evaluator
The Work You will test how AI assistants handle realistic analytics tasks connected to cloud data warehouses. You will compare AI-generated results with source data and document whether the figures, SQL, joins, filters, and time windows are correct. The work also includes managing evaluation environments, investigating product behavior, and helping keep scoring consistent across evaluators.
- Run evaluation scenarios covering anomaly investigation, KPI reporting, and data refreshes.
- Write and run SQL queries to check AI-generated results against source data.
- Maintain, reset, and manage seeded Snowflake datasets so scenarios stay accurate and repeatable.
- Manage Snowflake roles, permissions, grants, and access controls.
- Set up and document connections and authentication across analytics and SaaS platforms.
- Investigate undocumented product behavior and record findings clearly.
- Take part in calibration sessions to keep grading and scoring consistent.
What It Pays and Takes This is a part-time contractor role listed as entry level. Advanced SQL, Snowflake administration, and careful analysis are central to the work.
- Pay: $50 to $60 per hour.
- Language: Fluent English.
- Location: Open to candidates in Algeria, Argentina, Austria, Bahrain, Bangladesh, Belgium, Brazil, Canada, Chile, Colombia, Denmark, Egypt, France, Germany, Ghana, Greece, India, Indonesia, Italy, Japan, Jordan, Kazakhstan, Kenya, Kuwait, Kyrgyzstan, Lebanon, Malaysia, Mexico, Morocco, the Netherlands, Nigeria, Oman, Pakistan, Palestinian Territories, Peru, the Philippines, Portugal, Qatar, Saudi Arabia, Singapore, South Africa, Spain, Sri Lanka, Switzerland, Taiwan, Thailand, Tunisia, the United Arab Emirates, the United Kingdom, the United States, Uzbekistan, and Vietnam.
- Skills: Advanced SQL and hands-on expertise with Snowflake warehouses, access control, and query history.
- Analysis: Ability to audit reported metrics against raw data and find subtle aggregation or logic errors.
- Security: Experience with database access, roles, grants, authentication, or security integrations is valuable.
- Analytics: Familiarity with business and finance analytics, KPI definitions, and reporting for leaders or external stakeholders is helpful.
- Communication: Strong written and verbal communication, careful documentation, and sound judgment when evaluating analytical results.
- Preferred background: At least three years of hands-on data analyst or analytics engineer experience.
- Useful experience: Connectors with Claude or ChatGPT, reviewing AI-generated SQL and analytical outputs, rubric-based evaluation, quality assurance, or data labeling.
- Additional familiarity with OAuth, security integrations, Slack, Google Workspace, or Microsoft 365 is useful.
How It Works Apply on OpenTrain with your resume, then complete the application on the hiring site. About AI Training Work AI training is the human work behind systems that analyze data and generate answers. People check model outputs against reliable examples, and experienced contributors are needed when the work requires careful technical judgment.