AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.

WHY JOIN US

If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you

ABOUT THE ROLE

We are looking for a Data Engineer to build batch and streaming pipelines on Databricks using PySpark and Delta Lake. This person migrates legacy data warehouse and ETL workloads onto a governed Lakehouse, modeling a medallion architecture that powers analytics and AI use cases. Strong SQL, Python, and experience with Unity Catalog governance round out the role.

WHAT YOU WILL DO
- - Design, build, and operate batch and streaming data pipelines on Databricks using PySpark, Delta Lake, and Databricks Workflows.
- - Model and maintain a medallion (bronze/silver/gold) architecture serving analytics, reporting, and machine learning consumers.




- - Migrate legacy ETL and data warehouse workloads onto the Lakehouse with validated data parity and minimal business disruption.
- - Use Claude or GitHub Copilot as a development accelerator, generating code scaffolding, writing and reviewing tests, creating documentation, and prototyping solutions.
- - Write clean, well-tested Python and SQL; maintain high standards through code review and documentation.
- - Optimize Spark jobs and Delta tables for performance and cost, including partitioning, clustering, caching, and cluster sizing.
- - Implement data quality, lineage, and governance controls using Unity Catalog and automated validation checks.
- - Debug, troubleshoot, and resolve pipeline failures, data defects, and production incidents.
- - Participate in Agile or product-centric delivery practices, including sprint planning and retrospectives.
- - Collaborate with DevOps, platform, and analytics

📌 Databricks Engineer Id86297 (Barranquilla)
🏢 Ingepsy
📍 Barranquilla