Note: The job is a remote job and is open to candidates in USA. 24-MAG LLC offers an independent consulting opportunity for experienced Educator/Assessor Writers. The role evaluates AI-generated written responses, applies rubrics and scoring criteria, identifies content and reasoning issues, and documents assessment decisions with clear written rationales.

Responsibilities

• Evaluate and compare multiple AI-generated written responses
• Assess clarity, tone, helpfulness, relevance, and adherence to instructions
• Identify strengths and weaknesses across competing responses
• Evaluate whether content is appropriate for its intended audience
• Apply consistent professional judgement across repeated review tasks
• Apply provided rubrics and evaluation guidelines to written content
• Assign analytical quality scores using defined assessment criteria
• Distinguish between fluent, well-written responses and genuinely high-quality outputs
• Identify responses that are comprehensive but impractical or insufficiently supported
• Maintain consistency and objectivity across scoring decisions
• Detect factual errors, unsupported claims, and internal inconsistencies
• Identify weaknesses affecting reliability, usability, or instructional value
• Assess reasoning quality and whether conclusions are adequately supported
• Review written material with strong attention to detail and analytical precision
• Flag ambiguous or borderline cases requiring additional consideration
• Produce detailed written rationales explaining assessment decisions
• Clearly justify scores, rankings, and evaluation outcomes
• Document findings, observations, and recurring quality issues
• Maintain thorough records of review decisions
• Communicate complex assessment judgements in concise, professional language
• Apply experience from grading, marking, instructional design, or academic review
• Contribute insight on rubric interpretation and effective assessment practices
• Collaborate asynchronously with project managers and reviewers
• Use digital content-review platforms to complete and document assignments
• Work independently while meeting project deadlines and quality expectations

Skills

• First-language proficiency or fully equivalent fluency in written English
• Completed degree from an accredited institution in the United States, Canada, United Kingdom, Ireland, Australia, or New Zealand
• Experience marking, grading, examining, or reviewing written work against defined criteria
• Background in assessment design, instructional design, peer review, thesis supervision, or academic writing support
• Strong critical-analysis and content-evaluation skills
• Excellent written communication and editing ability
• Exceptional attention to detail and consistency
• Ability to make objective, well-articulated assessment judgements
• Comfortable interpreting and applying structured rubrics
• Able to identify factual, logical, and usability issues in written content
• Comfortable working independently in a remote environment
• Familiarity with digital content-review platforms is advantageous
• Experience with model evaluation, RLHF, annotation, or similar workflows is beneficial but not required

Benefits

• Fully remote
• Flexibility alongside other professional obligations

Company Overview