You'll help validate the rubrics we use to assess the agent's answers, grade responses against accuracy and coverage standards, and pressure-test the scenarios that keep benefits guidance correct and trustworthy. This is a great fit for benefits and insurance professionals who want flexible, remote work — and who want their expertise to shape a product while it's still being built.
The kind of work you might do:
Engagements vary, but could include:
- Scenario design and pressure-testing — building and stress-testing the member scenarios the agent is evaluated against, so the rubric reflects the questions members actually ask and the failure modes that matter most.
- AI agent evaluation — supporting the development and validation of rubrics, grading AI-generated benefits guidance against structured accuracy and coverage standards, flagging where the agent misstates coverage or oversteps into advice it shouldn't give, and providing written feedback on how responses can be improved.
- Subject-matter input — bringing a benefits, claims, or regulatory perspective to how we define correctness, coverage, and the hard-fail boundaries the agent must never cross.
Specializations we're looking for:
We're building a multidisciplinary panel. You don't need to cover all of these — we're looking for depth in one or more of the following areas:
- Benefits plan design and administration — You validate that the agent correctly interprets a member's plan documents rather than plausibly paraphrasing them wrong. This is the foundation of every coverage answer the agent gives.
- Payer / claims operations and live-data interpretation — You ensure the agent doesn't just echo API numbers but understands what they actually mean, catching failure modes like pending claims, embedded deductibles, and accumulator confusion that only hands-on operations experience surfaces.
- Regulatory and compliance — You confirm the agent doesn't misstate legal entitlements or drift into legal or tax advice, helping define the hard-fail boundaries where an error carries real regulatory or member harm.
- Member navigation and advocacy — You keep scenarios grounded in the questions members actually ask and the emotional context around them, so the rubric rewards genuinely helpful navigation over answers that are technically correct but useless.
- Supplemental health solutions / point-solution knowledge — You validate that the agent matches supplemental solutions to real member need rather than over-steering or upselling, protecting member trust in the agent's recommendations.
Required:
- Meaningful hands-on experience in one or more of the specializations above (e.g., benefits administration, plan design, payer or claims operations, benefits regulatory/compliance, member advocacy, or supplemental/point-solution expertise)
- Based in Canada or the United States, with working knowledge of the relevant benefits and insurance landscape
- Strong command of where accurate benefits guidance ends and regulated advice (legal, tax, clinical) begins
- Reliable, detail-oriented, and dependable on deadline-driven async work
- Ability to access and operate within Google Workspace
Nice to have:
- Prior experience with evaluation or validation of AI products, or structured content/quality review
- Familiarity with self-funded vs. fully-insured plan mechanics, accumulators, coordination of benefits, or claims adjudication
- Genuine curiosity and interest about the role of AI in benefits navigation
Engagement Details:
- Fully remote and async-friendly
- Project-based, with everything from short one-off engagements to recurring cycles — steady, ongoing work available for experts who are a strong fit
- Flexible hours that work around your day job or other commitments
- Structured onboarding and calibration so expectations are clear from the start
Please note: this is a Contractor opportunity as we are building out our roster for talent for these engagement-centric projects.
AI Fluency & Ways of WorkingAt League, we are an
AI-native organization. We expect all employees regardless of role or level to thoughtfully leverage AI to improve the quality, speed, and impact of their work.
What this means in practice:
- Use AI tools as part of your daily workflow to enhance productivity, problem-solving, and decision-making (e.g., drafting, analysis, coding, research, or process automation)
- Apply judgment and accountability when using AI by reviewing outputs for accuracy, bias, and quality before use
- Continuously learn and adapt as new AI tools and capabilities emerge, incorporating them into your ways of working
- Identify opportunities to improve how work gets done from personal productivity to team-level workflows by leveraging AI effectively
- Operate with strong data responsibility and security awareness, especially when working with sensitive or regulated information
How this scales by level:
- Individual Contributors: Use AI to improve personal productivity and quality of output
- Senior ICs / Managers: Integrate AI into team workflows and improve processes
- Leaders: Drive AI adoption at the organizational level and shape how work is done across teams
What we look for:
- Demonstrated experience using AI tools in a practical, responsible way
- Curiosity and openness to experimenting with new technologies
- Ability to balance efficiency with quality and sound judgment
Our employees come from different backgrounds, and we celebrate those differences. We are looking for the best candidates for our open roles, but do not expect applicants to meet every qualification in order to be considered. If you are excited about what you could accomplish at League and believe you can add value to our team, we would love to hear from you.
We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity or Veteran status. If you are an individual in need of assistance at any time during our recruitment process, please contact us at
recruitinginfo@league.com.
Our Application Process:
Applying to a role you love can be exhausting, and understanding the next steps can feel vague and uncertain. You have done the hard part of submitting your application; let's do ours by sharing potential next steps
- You should receive a confirmation email after submitting your application.
- A recruiter (not a computer) reviews all applications at League.
- If we see alignment with League's needs, a recruiter will reach out to learn more about your goals. The recruiter will also share the team-specific interview process depending on the roles you are exploring.
- The final step is an offer, which we hope you will accept!
- Prior to joining us, we conduct reference and background checks. Additional checks could be required for US Candidates, depending on the role you are exploring.
Here are some additional resources to learn more about League:
Work Location:
We have a mix of office-centric roles based in our vibrant Toronto office, and remote-eligible roles based anywhere in Canada or US. Each job posting will indicate where the role will be based. Regardless of the role’s posted location, all Toronto-area Leaguers (living within 65 km of our downtown HQ) collaborate in-office Monday through Thursday. Depending on your distance to the office, you’ll enjoy 10 or 20 Flexible Remote Days each quarter for focus and deep-work time. We are committed to fostering a meaningful work environment and connections for all Leaguers regardless of location.
Recognize and Avoid Employment scams. Practice safe job searching.
Scammers are getting craftier and leveraging fake job postings to get personal information. Know the warning signs and protect yourself from scammers. Learn more
here.
Use of AI NoticeWe are committed to ensuring fairness and transparency throughout our hiring process. League may use Artificial Intelligence (AI) tools to assist in the screening of applicants for this position. Please check out our stance on using AI in recruitment
here.
Privacy Policy
Review our
Privacy Policy for information on how League is protecting personal data.