30 sep
|
RedCloud Consulting
|
Bogotá
30 sep
RedCloud Consulting
Bogotá
RedCloud seeks a Data Engineer to support immediate client operations.
Location: Latin America (LATAM), Remote (must have working hours overlap with US PST and CST time zones)
Overview
The Data Engineer will play a key role on our client’s customer data team as someone who can work across large-scale batch and streaming systems, optimize Spark workloads, support production data pipelines, and review AI/LLM-generated code with good engineering judgment.
Responsibilities Include But Are Not Limited To
Build and maintain scalable batch and streaming data pipelines.
Optimize Spark and Scala workloads for performance, reliability, and cost efficiency.
Monitor production pipelines, troubleshoot issues, and resolve data quality defects.
Partner with stakeholders to define requirements, mappings, and dependable data solutions.
Orchestrate workflows using Airflow or similar tools.
Process real-time data using Kafka, Flink, Kafka Streams, or similar technologies.
Support data modeling, schema design, validation, and quality controls.
Apply strong engineering practices across testing,
versioning, deployment, and environments.
Review and validate AI/LLM-generated code for production readiness.
Support observability, SLAs, SLOs, and ongoing platform reliability.
Required knowledge, skills, and abilities:
Strong Scala / Spark experience.
Batch data processing and Spark job optimization.
Streaming experience with Kafka Streams, Flink, Kafka, or similar.
Airflow workflow orchestration.
Data modeling, schema design, and source-to-target mapping.
Data quality, validation, and production troubleshooting.
Strong software engineering fundamentals: testing, deployment discipline, versioning, and environment management.
Ability to review and validate AI/LLM-generated code.
Awareness of frameworks and tools like spec-led development, spec-kit, agent-skills etc. preferred.
Flink experience preferred.
Large-scale NoSQL experience, especially Scylla, Cassandra, DynamoDB, or similar pre
📌 Data Engineer (Bogotá)
🏢 RedCloud Consulting
📍 Bogotá