Remote Lead QA Engineer

Role Overview This position focuses on developing and implementing QA processes for AI and ML systems in a banking context. You will design and execute evaluation frameworks, create automated test suites, and establish quality gates in CI/CD pipelines to ensure operational and regulatory compliance. Key Responsibilities Design and execute AI evaluation frameworks, defining behavioral contracts and regression baselines. Develop and maintain automated tests for APIs, pipelines, and AI inference workflows, including performance and load testing. Build and enforce quality gates in CI/CD pipelines, triage production bugs, and write regression tests. Produce audit-ready documentation, test artifacts, and logs meeting SOC 2 standards. Collaborate with engineering on client validation and production issue reproduction. Qualifications & Skills 7+ years in software QA, with at least 2 years testing AI/ML systems. Proficiency in Python, pytest, Playwright, or Selenium. Hands-on experience with evaluation tools like RAGAS, DeepEval, LangSmith. Ability to trace failures across systems and integrate QA in CI/CD pipelines. Experience in fintech, banking, or regulated environments is advantageous. Success Indicators Initial success includes diagnosing current test coverage, building evaluation frameworks, and establishing quality gates. Long-term success involves scaling coverage, maintaining audit-ready documentation, and embedding quality in every deployment, with the foundation laid by your direct contributions.