Postúlate en Kit Empleo: kitempleo.com.co/empleo/1at36e

Why This Role Matters
Caseware is a leading Fintech company with a general audit and accounting software platform. We are accelerating an AI‑first future by embedding generative AI and autonomous agents to deliver smarter, faster user experiences.
What You’ll Be Doing
- 1. AI‑Driven Quality Strategy & Architecture
- Architect a comprehensive “Quality Intelligence” platform to predict defect hotspots, optimize regression suites, auto‑generate tests, and enable self‑healing automation.
- Define an enterprise‑wide AI‑first testing strategy, including continuous monitoring for drift, hallucination, and bias.
- Establish ethical governance for AI testing, aligning with emerging standards.
- 2. LLM & Agent Evaluation Frameworks
- Design advanced benchmarks, red‑team protocols, and adversarial testing for AI agents.
- Build evaluation pipelines using tools such as LangFuse, LangSmith, DeepEval, RAGAS, or Arize Phoenix for faithfulness, context precision, and safety compliance.
- Architect harnesses for agentic workflows, multi‑agent simulations, and post‑deployment observability.
- 3.



Infrastructure & Automation Architecture
- Embed AI‑based testing into GitHub‑based CI/CD pipelines.
- Lead design of self‑healing test frameworks that adapt to UI and model changes.
- Architect synthetic data generation, maintain gold‑data sets, and provide AI‑powered data masking for privacy compliance.
- 4. Cross‑Functional Leadership & Evangelism
- Collaborate with product, data science, ML engineering, and security teams to influence AI feature design.
- Mentor QA engineers into AI‑augmented testers through workshops and playbooks.
- Drive adoption of AI quality best practices organization‑wide with DORA + AI indicators.
- 5. Observability, Metrics & Continuous Evolution
- Implement AI‑specific quality telemetry integrated with tools like Langfuse.
- Establish feedback loops for model iteration and proactive risk mitigation.

Success in the First 6‑12 Months
- Launch the “Quality Intellig

Postúlate en Kit Empleo: kitempleo.com.co/empleo/1at36e

📌 AI Test Architect (Antioquia)
🏢 CaseWare
📍 Antioquia