•

• Design, build, and maintain end-to-end data pipelines for ingestion, transformation, and delivery of large-scale data.

• Develop and optimize data processing logic using PySpark on Databricks (Apache Spark).

• Implement ETL/ELT pipelines integrating data from multiple structured and semi-structured sources.

• Contribute to the design and implementation of lakehouse architectures (Delta Lake, Medallion architecture).

• Ensure data quality, reliability, performance, and observability across pipelines.

• Optimize Spark jobs through partitioning, caching, and performance tuning techniques.

• Collaborate with data architects, analysts, and business stakeholders to translate requirements into scalable data solutions.

• Implement best practices in CI/CD, version control, and pipeline automation.

• Support the evolution of modern data platforms and analytics capabilities.

• Work with orchestration tools (ADF, Airflow, or similar) to manage workflows.

• Communicate technical solutions and trade-offs effectively to both technical and non-technical stakeholders.





Qualifications
• • • BS in Computer Science, Engineering, or related technical field, or equivalent practical experience.
• 4+ years of experience in Data Engineering or similar roles.
• Strong experience with Databricks and PySpark for large-scale data processing.
• Solid knowledge of Spark architecture and performance optimization.
• Strong proficiency in Python and SQL.
• Experience designing and building data pipelines and ETL/ELT processes.
• Understanding of modern data architectures (lakehouse, distributed systems, big data processing).
• Experience with version control (Git) and CI/CD practices.
• Familiarity with cloud platforms (Azure preferred, AWS also acceptable).
• Experience with orchestration tools (Azure Data Factory, Airflow, Fabric).
• Strong communication skills and ability to work with cross-functional teams.
• Advanced English (written and spoken).
• Proactive, analytical mindset a

📌 Senior Data Engineer (Colombia)
🏢 Perficient
📍 Colombia