AgileEngine, LLC. is seeking a Senior Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You will build batch and streaming pipelines with PySpark and Delta Lake, following a medallion architecture across bronze, silver, and gold layers.
This role also uses AI tools like Claude and GitHub Copilot to speed up development. Responsibilities include migrating legacy ETL workloads, optimizing Spark jobs, and implementing data governance with Unity Catalog.