Aditi Consulting

Reimagine your reality

Summary

A leading client is growing its Data & AI practice and is hiring a Senior Data Engineer who operates at Solutions Architect level. You will design and own data platforms and AI-enabled pipelines that turn raw, real-world data into trustworthy products. The scope covers data collection and validation at the point of capture, declarative Databricks pipelines, auditable data-quality gates, and RAG and agentic AI workloads grounded in governed data.

You will set reference architecture, stay hands-on, and help staff, mentor, and teach other engineers. Location, work model, contract type, and start date are not specified in the brief.

Responsibilities

- Design and own data platforms and AI-enabled pipelines across the full data lifecycle, from raw data to trusted data products
- Design data collection instruments (forms/surveys, event instrumentation, APIs, CDC, and EDC/eCRF in clinical contexts) with validation at the point of capture
- Build declarative, production-grade pipelines on Databricks using Lakeflow Declarative Pipelines (DLT), Spark/PySpark, and Delta Lake
- Stand up auditable, code-first data-quality gates following a profile → reconcile → validate → govern discipline
- Deliver RAG and agentic-AI workloads grounded in governed data, using vector databases and orchestration frameworks
- Set reference architecture and best practices, and make high-level technical decisions while remaining hands-on
- Enforce data and AI governance, including lineage, access control, and responsible-AI guardrails
- Staff, interview, mentor, and teach engineers, and communicate trade-offs to technical and executive stakeholders.

Required Qualifications

- 8-12+ years in data engineering, with a progression from hands-on engineering to architecture
- Strong Python, advanced SQL, and solid NoSQL experience
- Databricks experience with Lakeflow Declarative Pipelines (formerly DLT), Spark/PySpark, and Delta Lake,



plus lakehouse and warehouse modeling (medallion, star/snowflake)
- Experience with one major cloud (AWS or Azure), Airflow orchestration, and CI/CD for data
- Demonstrable code-first, rule-as-code data-quality engineering (dbt tests + Elementary, Great Expectations, Soda, or Deequ), including a real incident you detected, diagnosed via lineage, and fixed
- Experience designing and integrating data collection instruments with validation at source
- RAG experience with at least one vector database, and distributed-systems fundamentals (CAP/PACELC, ACID vs. BASE, batch vs. stream)
- Fluency in data and AI governance (lineage and access control with Unity Catalog or equivalent), proven staffing, interviewing, mentoring, and teaching experience, and bilingual English/Spanish

Preferred Qualifications

- Snowflake + Cortex; LangChain/LangGraph, MCP, and agentic or self-healing pipeline patterns (e.g., Databricks Genie, Snowflake Cortex Agents)
- Hands-on AI governance frameworks (EU AI Act, NIST AI RMF, ISO/IEC 42001) and LLM guardrails (OWASP LLM Top-10, PII redaction, output grounding, audit logs)
- ML-driven data observability (Monte Carlo, Anomalo, Bigeye) and performance/load testing (k6, Locust, JMeter)
- Clinical/regulated standards (CDISC/CDASH) and EDC systems, plus BI experience (Power BI, DAX, RLS, semantic models)
- Certifications such as AWS Solutions Architect, Databricks, or Google Professional Data Engineer

Soft Skills

- Comfort with uncertainty and ambiguity
- Strong critical thinking, structured problem framing, and trade-off reasoning
- Clear communication with technical, non-technical, and executive stakeholders
- Ownership and pragmatism: commits at the last responsible moment and implements as well as architects
- People leadership: mentoring, teaching, and growing engineers

Must Have Skill

- Databricks with Lakeflow Declarative Pipelines (DLT), Spark/PySpark, and Delta Lake
- Code-first data-quality engineering with lineage and governance (e.g., Unity Catalog)
- RAG with at least one vector database, with Python and advanced SQL

📌 SR Data Engineer (Bogotá)
🏢 Aditi Consulting
📍 Bogotá