• • Design, build, and maintain end-to-end data pipelines for ingestion, transformation, and delivery of large-scale data.
• Develop and optimize data processing logic using PySpark on Databricks (Apache Spark).
• Implement ETL/ELT pipelines integrating data from multiple structured and semi-structured sources.
• Contribute to the design and implementation of lakehouse architectures (Delta Lake, Medallion architecture).
• Ensure data quality, reliability, performance, and observability across pipelines.
• Optimize Spark jobs through partitioning, caching, and performance tuning techniques.
• Collaborate with data architects, analysts, and business stakeholders to translate requirements into scalable data solutions.
• Implement best practices in CI/CD, version control, and pipeline automation.
• Support the evolution of modern data platforms and analytics capabilities.
• Work with orchestration tools (ADF, Airflow, or similar) to manage workflows.
• Communicate technical solutions and trade-offs effectively to both technical and non-technical stakeholders.

Qualifications




• • • BS in Computer Science, Engineering, or related technical field, or equivalent practical experience.
• 4+ years of experience in Data Engineering or similar roles.
• Strong experience with Databricks and PySpark for large-scale data processing.
• Solid knowledge of Spark architecture and performance optimization.
• Strong proficiency in Java, Python and SQL.
• Experience designing and building data pipelines and ETL/ELT processes.
• Understanding of modern data architectures (lakehouse, distributed systems, big data processing).
• Experience with version control (Git) and CI/CD practices.
• Familiarity with cloud platforms (Azure preferred, AWS also acceptable).
• Experience with orchestration tools (Azure Data Factory, Airflow, Fabric).
• Strong communication skills and ability to work with cross-functional teams.
• Advanced English (written and spoken).
• Proactive, analytical mindset and co

📌 Senior Data Engineer (Bogotá)
🏢 Perficient
📍 Bogotá