30 sep
|
RedCloud Consulting
|
Bogotá, D.C.
30 sep
RedCloud Consulting
Bogotá, D.C.
RedCloud seeks a Data Engineer to support immediate client operations.
Location: Latin America (LATAM), Remote (must have working hours overlap with US PST and CST time zones).
Overview
The Data Engineer will play a key role on our client’s customer data team as someone who can work across large-scale batch and streaming systems, optimize Spark workloads, support production data pipelines, and review AI/LLM-generated code with good engineering judgment.
Responsibilities Include But Are Not Limited To
- Build and maintain scalable batch and streaming data pipelines.
- Optimize Spark and Scala workloads for performance, reliability, and cost efficiency.
- Monitor production pipelines, troubleshoot issues, and resolve data quality defects.
- Partner with stakeholders to define requirements, mappings, and dependable data solutions.
- Orchestrate workflows using Airflow or similar tools.
- Process real-time data using Kafka, Flink, Kafka Streams, or similar technologies.
- Support data modeling, schema design, validation, and quality controls.
- Apply strong engineering practices across testing, versioning, deployment, and environments.
- Review and validate AI/LLM-generated code for production readiness.
- Support observability, SLAs, SLOs, and ongoing platform reliability.
Required knowledge, skills, and abilities
- Strong Scala / Spark experience.
- Batch data processing and Spark job optimization.
- Streaming experience with Kafka Streams, Flink, Kafka, or similar.
- Airflow workflow orchestration.
- Data modeling, schema design, and source-to-target mapping.
- Data quality, validation, and production troubleshooting.
- Strong software engineering fundamentals: testing, deployment discipline, versioning, and environment management.
- Ability to review and validate AI/LLM-generated code.
- Awareness of frameworks and tools like spec-led development, spec-kit, agent-skills etc. preferred.
- Flink experience preferred.
- Large-scale NoSQL experience, especially Scylla,
📌 Data Engineer (Bogotá, D.C.)
🏢 RedCloud Consulting
📍 Bogotá, D.C.