AgileEngine is seeking a Senior Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You will build batch and streaming pipelines with PySpark and Delta Lake, using a medallion architecture while leveraging AI tools like Claude and GitHub Copilot.
You will migrate legacy workloads to the Lakehouse with data parity and work in an Agile environment, collaborating across DevOps, platform, and analytics teams to ensure security and observability.