10 ago
|
Agileengine
|
Colombia
10 ago
Agileengine
Colombia
About the Role
We are looking for a Tech Lead Data Engineer to own the data pipeline and analytical architecture layer for a large-volume marketing analytics platform. You will make architectural decisions around partitioning strategy, file formats, schema design, and near-real-time processing for OLAP-oriented workloads built on an S3-backed data lake. You will design and govern ETL pipelines, define DAG-based orchestration strategies using Airflow, drive the AWS data stack including Athena and EKS, and lead a team of senior developers while enforcing code quality standards.
What you will do
Design and own ETL pipelines that extract, transform, and validate data from internal databases and external APIs at scale.
Make architectural decisions on partitioning strategy, file formats, schema and data-type strategy, and near-real-time processing for large-volume, OLAP-oriented data systems built on an object-storage data lake.
Own the design of scheduled batch workflows (DAGs) on the client's Airflow setup, defining pipeline structure, dependencies, and triggering strategy,
while driving architectural discussions. Not responsible for administering Airflow itself.
Drive use of the client's AWS data stack (S3-backed data lake, Athena, EKS/Kubernetes), and partner directly with the client's DevOps team to clarify functional and non-functional requirements.
Review pull requests and enforce code quality standards.
Guide senior developers and ensure alignment with the client's engineering practices.
Must haves
7+ years of engineering experience, with a proven track record designing and implementing ETL pipelines and making architectural decisions for large-volume data systems.
Hands-on experience with OLAP-style analytical data architecture. Experience with Athena, Trino/Presto, BigQuery, Snowflake, Spark SQL, ClickHouse, or similar technologies is acceptable; a specific stack isn't mandatory as long as the OLAP depth is real.
Hands-on experience designing against a data la
📌 Tech Lead Data Engineer (Colombia)
🏢 Agileengine
📍 Colombia