AgileEngine is seeking a Senior Data Engineer to build batch and streaming data pipelines on Databricks using PySpark and Delta Lake. This role migrates legacy data warehouse workloads onto a governed Lakehouse, using a medallion architecture to power analytics and AI use cases.
You will collaborate with DevOps and analytics engineers, optimize Spark jobs, implement data quality and governance with Unity Catalog, and write clean Python and SQL.