We're seeking a hands-on Lead Data Engineer to construct and maintain scalable data pipelines along with cloud-based data solutions.

The right person for this role brings substantial background in Python, PySpark, AWS, Airflow, and Dev

Ops, with particular attention to production-grade engineering and automation practices

Responsibilities

Create, construct, and sustain scalable data pipelines running on AWS infrastructure

Build dependable data processing solutions leveraging Python, Pandas, and PySpark

Establish and oversee workflows via AWS Managed Workflows for Apache Airflow (MWAA)Craft and fine-tune data queries through AWS Athena to enable efficient analytical work

Streamline infrastructure setup and deployment workflows through automation using Terraform and Git

Hub Actions

Put in place testing, monitoring, troubleshooting practices, and data-quality safeguards to keep pipelines dependable

Take part in reviewing code and help promote sound engineering practices throughout the team

Work alongside engineering, product, and business groups within an Agile setting

Requirements

At least 5 years of applicable professional backgroundA minimum of one year spent leading and managing development teams

Solid foundation in core Python programming applied to data engineering work

Practical experience applying Python Pandas for manipulating and analyzing data

Strong capability with PySpark for handling large-scale distributed data processing

Background working with AWS MWAA or Apache Airflow to coordinate and manage data workflows





Familiarity with AWS Athena for running queries and analyzing data housed within AWSHands-on background using Terraform to automate infrastructure setup and management

Working knowledge of Git

Hub Actions for constructing automated CI/CD pipelines

Good grasp of Dev

Ops principles as applied within data engineering contexts

Strong capabilities in data engineering, troubleshooting, and resolving complex problems

Upper-intermediate English language skills or higher (B1)Nice to have

Background applying Git

Hub Copilot to assist with development, code reviews, testing, and release processes

Knowledge of CI/CD methodologies and associated tools for smoother software delivery

Background working within Agile/Scrum team structures

Experience gained within financial services or investment management sectors

Background in Gen AI application development and incorporating Gen AI practices into the software development lifecycle (SDLC)We offer

International projects with top brands

Work with general teams of highly skilled, diverse peers

Healthcare benefits

Employee financial programs

Paid time off and sick leave

Upskilling, reskilling and certification courses

Unlimited access to the Linked

In Learning library and 22,000+ courses

Global career opportunities

Volunteer and community involvement opportunitiesEPAM Employee Groups

Award-winning culture recognized by Glassdoor, Newsweek and Linked

InEPAM is an Equal Opportunity Employer.

All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, age, sexual orientation, gender identity or expression, disability, protected veteran status, or any other characteristic protected by applicable law. #J-18808-Ljbffr

📌 Lead Data Engineer (Colombia)
🏢 EPAM Systems
📍 Colombia