We are looking for a Middle Data Engineer to help build and maintain a data platform that supports data engineering, BI, Machine Learning, and AI initiatives. In this role, you will design batch and real-time data pipelines, work with distributed data processing tools, and collaborate with business and technical stakeholders to deliver reliable data solutions.

Required for this role
- 3–5 years of experience
building
batch and real-time data pipelines
using big data technologies.
- Experience with distributed data processing tools such as
Spark
,
Hadoop
,
Airflow
,
NiFi
, and
Kafka
.
- Hands-on experience designing, developing, and maintaining
ETL processes
and data pipelines.
- Proficiency in
Python
and/or
Scala
, including the ability to optimize data processing scripts.
- Strong knowledge of
SQL
and experience designing scalable
data models
.
- Familiarity with cloud-based data platforms such as
Snowflake
,
Databricks
,
AWS
,
Azure
, or
GCP
.
- Experience using
CI/CD
tools and working with
Docker
and
Kubernetes
for data pipeline automation.




- Ability to troubleshoot, monitor, and optimize data workflows for reliability and performance.
- Basic understanding of
data governance
,
data quality frameworks
, and compliance requirements.

Nice to have
- Experience with
OpenShift
,
Trino
,
Ranger
, and
Hive
.
- Knowledge of
Machine Learning
and data science concepts and tools.
- Familiarity with BI and analytics tools such as
Tableau
and
Superset
.
- Good communication skills, attention to detail, Agile mindset, and flexibility.

Your responsibilities
- Work with business stakeholders and cross-functional teams to understand data requirements and deliver scalable data solutions.
- Design, develop, and maintain ETL processes that move data from different sources into the data platform.
- Build batch and event-driven data pipelines using cloud and on-premises hybrid data platforms.
- Collaborate with data architects to review solutions and data models, ensuring alignment

📌 Middle Data Engineer (Colombia)
🏢 Sombra
📍 Colombia