Lead AI Trainer / Prompt Evaluator Job at Social Discovery Group, Remote

d2ZFZ1Q2a1FVRmtVNzVRbDdKaDhZSW8zQUE9PQ==
  • Social Discovery Group
  • Remote

Job Description

( Social Discovery Group ( (SDG) is the 3rd largest social discovery company in the world, uniting 60+ brands with 500 million users. We solve the problems of loneliness, isolation, and disconnection by transforming virtual intimacy into the new normal. Our portfolio includes online communication platforms focusing on AI, game mechanics, and video streaming - Dating.com, DateMyAge, Cupid Media, Dil Mil, Kiseki, and others. 

SDG invests in IT startups around the world.  Our investments include Open AI, Patreon, Flo, Clubhouse, Woebot, Flure, Astry, Coursera, Academia.edu, and many others.

We bring together a team of like-minded people and IT professionals specializing in the creation and development of globally impactful social discovery products. Our international team of 1200 professionals and digital nomads works all over the world. 

Our teams of digital nomads  work remotely from  Cyprus, Malta, the USA, Armenia, Georgia, Kazakhstan, Montenegro, Poland, Latvia, Serbia, Spain, Portugal, UAE, Israel, Turkey, Thailand, Indonesia, Japan, Hong Kong, Australia and many other locations. 

In August 2024, we achieved  Great Place to Work US Certification™!  This achievement reflects our core belief that a truly exceptional workplace is built on trust, pride, and camaraderie—not just great perks.

We are looking for a meticulous Lead AI Trainer / Prompt Evaluator to oversee and enhance  the quality of AI model dialogues  with users. 

Your main tasks will be:

  • Review and assess the quality of datasets produced by the content team, ensuring accuracy, consistency, and adherence to labeling guidelines.
  • Provide constructive feedback and help maintain a high quality bar across annotators.
  • Personally perform high-quality annotation and dataset creation as a subject-matter expert.
  • Validate conversational AI model outputs (e.g., SbS evaluations, error tagging, scenario stress-testing) to detect issues before production release.
  • Work closely with the ML team to refine instructions, dataset specifications, and evaluation criteria for model training cycles.
  • Identify recurring issues in annotations or model behavior and suggest improvements to workflows, QA processes, or validation pipelines.
  • Contribute insights on prompts and data strategies to accelerate model quality improvements.

We expect from you:

  • Experience in dataset annotation and content validation for conversational AI / chat models / NLP products.
  • Hands-on experience with SbS (side-by-side) or other model evaluation frameworks (e.g., quality scoring, tagging, error categorization).
  • Ability to analyze conversational responses and identify accuracy, tone, safety, and contextual relevance issues.
  • Experience preparing or following annotation guidelines and QA checklists.
  • Strong attention to detail and ability to consistently apply evaluation criteria.
  • Ability to provide clear and constructive written feedback to other annotators.
  • Familiarity with prompt engineering practices is a plus.
  • Familiarity with LLM-based automated evaluators / AI-as-a-judge is a plus.
  • English proficiency (B2+) in reading and writing.

What do we offer:

  • REMOTE OPPORTUNITY  to work full time;
  • Vacation  28 calendar days per year;
  • 7 wellness days per year  (time off) that can be used to deal with household issues, to lie down and recover without taking sick leave;
  • Bonuses up to $5000  for recommending successful applicants for positions in the company;
  • 50%  payment for professional training, international conferences and meetings ;
  • Corporate discount for  English lessons;
  • Health benefits.  According to the paychecks, if you are not eligible for corporate medical insurance, the company will compensate you with up to $ 1,000 gross per year per employee. This can be spent on self-purchase of health insurance or on doctor's fees for yourself and close relatives (spouse, children);
  • Workplace organization.  The company provides all employees with an equipped workplace and all the necessary equipment (table, armchair, wifi, etc.) in our offices or co-working locations. In the other locations, the company provides reimbursement of workplace costs up to $ 1000 gross once every 3 years, according to the paychecks. This money can be spent on the rent of the co-working room, on equipping the working place at home (desk, chair, Internet, etc.) during those 3 years;
  • Internal gamified gratitude system:  receive bonuses from colleagues and exchange them for our merchandise, team building activities, massage certificates, etc. 

Sounds good? Join us now!  

Job Tags

Remote job, Full time, Work at office,

Similar Jobs

The Rock Group

Class A Driver - Dedicated Linehaul Lane Job at The Rock Group

 ...owned and oriented , with headquarters in Syracuse, NY . We offer our drivers one of the best packages in the area with DAILY home time, top pay, late model tractors, personalized dispatch with anopen line ofcommunication!Our team has a current need for a Local... 

PEOPLE FORCE CONSULTING INC

Principal Cloud Infrastructure Architect Job at PEOPLE FORCE CONSULTING INC

 ...meetings Specific Job Functions: Design VPCs, subnets, firewalls and IAM policies Plan for high availability, disaster recovery and performance Architect and design scalable, secure, and resilient cloud infrastructure solutions (Azure) to support... 

Dynamics ATS

QC Inspector Job at Dynamics ATS

 ...require the frequent use and knowledge of MS Windows 7 and/or MS Windows 10, MS Word, MS Excel, MS Outlook, Blue Beam PDF Software, Kronos, two-way radio, and barcode scanner While performing the duties of this job, the employee is regularly required: Regularly... 

Piper Companies

Mission Systems Integration Engineer Job at Piper Companies

Mission Systems Integration Engineer Location St. Louis, MO (Downtown area) : Zachary Piper Solutions is seeking a Mission Systems Integration Engineer to support a United States Air Force (USAF) program to deliver faster jets in support of the T7 Red Hawk program.... 

Virginia Department of Transportation

Transport Opertns Manager I Job at Virginia Department of Transportation

 ...Job Identification 11531 Job Category Transportation Operations Posting Date 01/15/2026, 07:05 PM Locations New Kent AHQ Apply Before 01/30/2026, 04:59 AM Job Schedule Full time State Role Title Transport Opertns Manager I Minimum Salary 51428....