Job DescriptionInsight General is looking for a senior data engineer to design, build, and maintain production-grade batch and streaming pipelines using Databricks, Lakeflow Declarative Pipelines, and dbt for our client. This person will participate in sprint planning, standups, retrospectives, and backlog grooming. You will monitor and troubleshoot pipeline reliability, performance, and cost, implement centralized data governance and access controls with Unity Catalog and tools such as Immuta, and build DataOps standards, CI/CD automation, and automated data-quality controls. You will be responsible for partnering with cross-functional stakeholders to deliver curated, analytics-ready datasets and data models, and review pipeline designs and code while guiding mid-level and junior engineers. Lastly, you will evaluate emerging Databricks capabilities, including Lakeflow, AI/BI, and SQL AI Functions and recommend improvements to platform architecture and engineering practices. This is a 3 month contract with yearly anticipated extensions remote in Brazil or Colombia.Skills and Requirements7+ years of data engineering and architecture experienceDeep, hands-on production experience with Databricks and modern lakehouse platformsStrong Apache Spark experience, including PySpark and Spark SQLExperience building and operating scalable batch and streaming pipelinesExperience with declarative pipeline frameworks,



including Lakeflow Declarative PipelinesStrong experience with dbt, Databricks Workflows, and Apache Airflow for data transformation and orchestrationAdvanced SQL and strong Python programming skillsSolid understanding of lakehouse architecture, medallion/layered data modeling, and data warehousingProduction experience with Databricks Unity CatalogCloud Experience (AWS, Azure, GCP)Scala or Java experienceDatabricks Certified Data Engineer Professional certificationConfluent Kafka and Fivetran experienceImmuta experience alongside Unity CatalogAlation or similar data catalog and lineage toolingDeclarative Automation BundlesPhoton, Liquid Clustering, and Databricks Serverless computeWe are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances.#J-18808-Ljbffr

📌 Senior Data Engineer, Databricks (Santander)
🏢 Insight Global
📍 Santander