AgileEngine is seeking a Senior Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You will build batch and streaming pipelines with PySpark and Delta Lake, following a medallion architecture across bronze, silver, and gold layers.
This role also uses AI tools like Claude and GitHub Copilot to speed up development. Responsibilities include writing clean Python/SQL, optimizing Spark jobs, and implementing data governance with Unity Catalog.