We are looking for a Data Platform Engineer to build, operate, and scale a real-time data platform that transforms raw, high-throughput data into reliable datasets used for machine performance monitoring and optimization.

This position combines Data Engineering and Platform Engineering , with a strong focus on production data pipelines, real-time streaming, automation, observability, and platform reliability.

You will work closely with Physics, Analytics, and Engineering teams to transform scientific and exploratory datasets into scalable, reliable, production-ready solutions.

What You'll Do

- Build, deploy, maintain, and optimize data pipelines and real-time streaming applications.
- Integrate new data sources and machine signals into the data platform.
- Transform scientific, laboratory, and exploratory data into reliable, documented, production-ready datasets.
- Partner with Physics, Analytics, and Engineering teams to translate technical and business requirements into scalable solutions.
- Manage platform operations, including upgrades, patching, infrastructure maintenance, capacity planning, and deprecations.
- Implement observability through logging, monitoring, dashboards, alerts, and usage tracking, primarily with Grafana .




- Automate source onboarding and operational workflows using Azure DevOps, Python, and Infrastructure as Code .
- Manage RBAC, permissions, and platform security.
- Monitor pipeline health and data quality.
- Troubleshoot production failures and latency issues.
- Participate in incident management and perform root cause analysis.

Required Qualifications

- Advanced hands-on experience with Python and SQL for data engineering, automation, querying, scripting, and data transformations.
- Strong understanding of data modeling, data quality, ETL/ELT, pipeline architecture, and optimization.
- Hands-on experience with Apache Spark, PySpark, and Databricks .
- Experience working with Microsoft Azure .
- Production experience deploying and managing workloads on Kubernetes .
- Experience with Azure DevOps , including pipelines, repositories, CI/CD, and release management.
- Strong knowledge of Git, branching strategies, and code reviews .
- Experience building or supporting high-throughput, low-latency streaming data systems.
- Hands-on experience with Grafana for dashboards, monitoring, and alerts.
- Strong debugging, troubleshooting, incident management, and root cause analysis capabilities in production data environments.

📌 Sr Data Engineer (Colombia)
🏢 Unosquare
📍 Colombia