01 oct
|
Devsu
|
Colombia
We are looking for a Data Engineer to design, build, modernize, and operate scalable data pipelines supporting Tax technology platforms. This role focuses on developing and enhancing data processing workloads using Python, Apache Spark, PySpark, and Databricks, while ensuring reliable, secure, and maintainable data solutions. The adecuado candidate has strong hands-on engineering skills, experience with production ETL/ELT pipelines, data ingestion, APIs, data quality, and cloud-based data platforms, along with strong problem-solving and collaboration skills.
Responsibilities
• Design, build, and support scalable data pipelines using Python, Apache Spark, PySpark, and Databricks.
• Modernize and migrate legacy data processing workloads to secure, cloud-native platforms.
• Build and maintain batch data ingestion pipelines from structured and unstructured sources.
• Integrate data from REST APIs, SharePoint, document repositories, enterprise applications, and cloud platforms.
• Implement data quality, monitoring, observability, and operational controls.
• Optimize data workloads for performance, scalability, reliability, and cost efficiency.
• Develop document extraction, classification, metadata enrichment, and automation pipelines.
• Apply software engineering practices including Git, CI/CD, automated testing, and code reviews.
• Build, deploy, troubleshoot, and maintain production ETL/ELT pipelines.
• Collaborate with architects, developers, tax subject matter experts, and platform teams.
• Analyze existing codebases and identify opportunities to improve maintainability, security, and reliability.
• Use AI-assisted development tools to build, review, test, and maintain data pipeline code.
Must Have• 3+ years of experience in data engineering.
• Strong Python development experience.
• Strong Apache Spark and PySpark experience.
• Hands-on experience with Databricks.
• Strong SQL and data modelling skills.
• Experience building, testing, deploying, and troubleshooting
📌 Data Engineer (Colombia)
🏢 Devsu
📍 Colombia