Automation Test Engineer

Our client is a healthcare technology company where software quality directly affects whether patients get the care and access they need. You'll own the testing strategy and automation for an AI-driven platform, making sure every behavior and integration works as expected before it ever reaches a live patient. It's a rare chance to solve testing problems where the system never gives the same answer twice.

C2C is not an option with this job opening, and all applicants must be able to work for any U.S. employer without sponsorship. Sponsorship is not provided and will not be required in the future.

Benefits & Extras

• Direct impact on patient safety and access to care.
• Ownership of test strategy and infrastructure for a cutting-edge AI platform.
• Top-tier health coverage: medical, dental, and vision.
• 401(k).
• Paid time off, holidays, and paid parental leave.

Compensation

• $150,000

What You'll Be Doing

• Design, build, and maintain automated test infrastructure covering unit, integration, end-to-end, and load testing.
• Build evaluation and regression harnesses that score AI behavior by outcome, not exact output.
• Generate synthetic conversation data and de-identified patient scenarios to cover rare edge cases.
• Build testing frameworks and tooling that help the wider engineering team write reliable tests.
• Define and enforce quality standards and best practices across engineering teams.
• Own test environments and CI/CD integration for automated test runs.
• Triage production quality issues and drive root cause analysis, including HIPAA and compliance-aware testing.

What You'll Need to be Considered

• Experience testing AI-powered applications, including LLM prompt and model regression testing.
• Experience testing non-deterministic conversational flows, with assertions that tolerate variation without hiding real regressions.
• Experience generating synthetic test data at scale.
• Built test automation frameworks from the ground up.
• Python or TypeScript skills, with CI/CD experience such as GitHub Actions or Jenkins.
• Track record of raising quality standards across a team.