Purpose This position is responsible for acting as the organization’s deep technical authority for enterprise Artificial Intelligence initiatives—particularly agentic and LLM-enabled solutions—ensuring they exceed intended business outcomes and comply with Responsible AI and Model Risk requirements. This role is designed for an expert individual contributor who leads through technical influence: defining patterns, guiding architecture decisions, strengthening evaluation discipline (including prompting), and elevating delivery quality across hybrid environments (cloud and on-prem). The Specialist is accountable for taking over complex AI initiatives already in motion, identifying critical gaps, and improving them through rigorous prompt/evaluation practices, governance-by-design, and production-grade engineering.

The role serves as a key escalation point for complex technical decisions, enabling multiple teams (including Level 6 engineers) to deliver safe, reliable, measurable, and supportable AI capabilities.

Accountabilities Technical

Authority & Architecture Leadership (IC Leadership) - Lead architecture decisions for agentic/LLM solutions, including integration patterns (APIs/event-driven), retrieval/grounding approaches, guardrails, evaluation design, and operational monitoring.

- Establish reusable reference architectures and "golden paths" that reduce rework and accelerate safe delivery across teams.
- Act as escalation point for high-impact technical risks and guide root-cause analysis to drive sustainable fixes—not short-term patches.

Prompting

Excellence (MUST – Prompt Engineering) - Own prompt engineering standards for production systems: prompt templates, system instructions, structured outputs, and context strategies that improve reliability and reduce failures.

- Design and execute controlled experiments to compare prompt variants and measure impact on quality, safety, and cost (token usage/latency).
- Define evaluation metrics and datasets for prompt/agent behavior (relevance, groundedness, safety, robustness)



and implement regression testing for prompt and model changes.
- Implement prompt lifecycle management (documentation, version control, A/B testing/rollbacks) and integrate it into delivery workflows.
- Identify and mitigate failure modes such as hallucinations, prompt injection risk, unsafe outputs, and brittle behavior under edge cases. Responsible AI / Model Risk Governance (MUST) - Implement Responsible AI controls as non-negotiable standards: privacy safeguards, auditability, traceability, and risk-based validation for prompts, responses, logs, and tool usage.
- Partner with Security, Risk, and Compliance stakeholders to ensure approvals, documentation, and controls are met before production releases. Full-Stack Delivery Enablement (MUST – Node.js / TypeScript / React) - Provide technical leadership to ensure end-to-end AI-enabled products (UI + API + AI integration) meet enterprise standards for maintainability, security, and performance.
- Review and guide Node.js/TypeScript backend services and React/TypeScript frontend implementation approaches to ensure consistency and scalability across solutions. LLMOps / Operational Excellence (Production Readiness) - Establish evaluation and monitoring practices for LLM/agentic systems (tracing, evaluation harnesses, prompt management, governed model access, production monitoring) to detect regressions early and improve reliability at scale.
- Define and monitor operational KPIs (quality, safety signals, latency, cost per task, adoption) and recommend optimization strategies.



Cross-Functional Influence & Advisory - Translate complex technical risks and trade-offs into executive-ready recommendations for Product, Architecture, and Risk stakeholders.
- Provide enablement to engineers (including Level 6) via technical reviews, standards, playbooks, and knowledge transfer sessions—without direct people management accountability.

Continuous

Improvement & Innovation - Drive continuous improvement of AI delivery practices: evaluation maturity, observability, testing automation, cost optimization, and reusable components.

- Evaluate emerging tools and practices for agentic systems (tool use, orchestration, memory, evaluation) and recommend adoption aligned with governance expectations.

Education / Experience / Other Information Bachelor’s degree in Computer Science, Engineering, or related field. 3-5 years of software engineering experience, including 2+ years in an engineering leadership/people management role (direct reports). Demonstrated experience delivering AI-enabled solutions in production (LLM applications, agentic workflows, or applied ML), including scaling and improving existing initiatives to meet defined outcomes. Responsible AI / Model Risk (MUST): proven experience implementing privacy, auditability, traceability, and validation controls aligned with enterprise governance.

Prompt

Engineering (MUST): hands-on experience designing, testing, and maintaining prompts/system instructions, including controlled experimentation, evaluation metrics, and prompt versioning/documentation in production settings. Full-Stack Engineering (MUST): strong working knowledge of Node.js/TypeScript backend services and React/TypeScript frontend delivery, sufficient to guide architecture decisions and mentor senior-equivalent ICs. Strong knowledge of SDLC and Agile methodologies;

experience leading delivery across multiple streams and dependencies. Strong stakeholder communication skills, including executive-ready reporting of risks, progress, and trade-offs.

English level: B2 (recommended for general stakeholder management).

📌 Artificial Intelligence & Agentic Solutions Specialist (Bogotá)
🏢 ScotiaTech
📍 Bogotá