AgileEngine is seeking a Middle Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You will build batch and streaming data pipelines with PySpark and Delta Lake, following a medallion architecture across bronze, silver, and gold layers.
This role also uses AI tools like Claude and Github Copilot to speed up development. Responsibilities include migrating legacy workloads, optimizing Spark jobs, and implementing data quality and governance with Unity