What You’ll Be Doing:
• You build it, you run it. End-to-end accountability for the AI platform in production: architecture, delivery, operations, cost, and quality, with on-call for what you ship. No hand-off to a separate ops or QA function.
• Design, build, and operate agentic AI systems: LLM services, retrieval pipelines, multi-agent orchestration, agent execution runtimes, and human-in-the-loop capabilities.
• Build and operate memory infrastructure for agent systems (storage, retrieval, promotion/demotion mechanics). Applied Science owns promotion criteria and scientific soundness.
• Own LLMOps: the harnesses, pipelines, and infrastructure that keep non-deterministic systems reliable and correct in production, including dynamic model selection by task, cost, and latency.
• Build automated eval infrastructure that continuously compares offline and online metrics and alerts on drift or regression. Applied Science owns the methodology and acceptance thresholds it checks against.
• Evaluate, adopt,



and integrate third-party LLMOps and eval platforms where they accelerate delivery over building in-house.
• Build the technical controls, telemetry, and guardrails that enforce compliance frameworks (e.g., ISO 42001, AIUC-1) in the architecture itself.
• Build and maintain the agent definition management APIs and the tool registry definitions reference, keeping definitions stable as tools and models version independently.
• Lead the engineering build of proof-of-concepts. Applied Science defines the architecture bet and validation criteria being tested.

Core Responsibilities
• Create architecture, prototypes, and design proposals that deliver near-term business value while aligning with the future-state product and platform vision.
• Influence technical direction through architecture reviews, RFCs, design discussions, and hands-on collaboration.
• Partner with senior developers, architects, and tech leads to define and execute the product/platform architecture roadmap

📌 Senior Software Engineer (Colombia)
🏢 CaseWare
📍 Colombia