AgileEngine is seeking a Middle Data Engineer to help modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You will design batch and streaming pipelines with PySpark and Delta Lake, following a medallion architecture across bronze, silver, and gold layers.
The role involves leveraging AI tools like Claude and Github Copilot to accelerate development. Responsibilities include building scalable data pipelines on Databricks, migrating legacy workloads, and maintaining data