We are seeking a Lead Data Platform Engineer to design, build, and optimize scalable Databricks data platforms that power analytics, reporting, and AI initiatives. You will own modern pipelines, governance, and high-quality data products in the cloud.ResponsibilitiesDesign scalable data platform architectures on Databricks to support analytics and AI use casesBuild end-to-end data pipelines using Lakeflow DLT, Auto Loader, and Medallion ArchitectureImplement data governance with Unity Catalog, including access controls, lineage, and metadata standardsOptimize Apache Spark workloads for performance, reliability, and cost efficiencyDevelop dimensional data models and curated datasets for reporting and downstream consumptionDeliver AI/ML-ready datasets and enable feature engineering or GenAI/RAG data pipelinesAutomate deployments with CI/CD using GitHub Actions and Databricks Asset BundlesIntegrate AWS services such as Amazon S3, IAM, KMS, DMS, and Aurora PostgreSQL into data solutionsEstablish data quality checks, monitoring,



and operational runbooks for production pipelinesCollaborate with stakeholders to translate requirements into well-defined data products and SLAsReview code and set engineering standards to improve maintainability and delivery speedRequirements5+ years of data engineering or analytics engineering experience1+ years of leadership experience driving platform standards and technical decisionsStrong project delivery skills owning end-to-end data platform initiativesAdvanced Databricks skills with Unity Catalog, Delta Lake, and Databricks for Data AnalyticsStrong Apache Spark skills using PySpark and SQL for large-scale processingStrong data pipeline skills with Lakeflow DLT, Auto Loader, and Medallion ArchitectureAdvanced data modeling skills with FACT/DIM, Star Schema, and SCD Type 2Strong data governance skills including metadata management, lineage, and access controlsStrong AWS skills with Amazon S3, IAM, KMS, DMS, and Aurora PostgreSQLStrong CI/

📌 Lead Data Platform Engineer (Colombia)
🏢 EPAM Systems
📍 Colombia