AgileEngine is seeking a Senior Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You will build batch and streaming pipelines using PySpark, Delta Lake, and Databricks Workflows.
You will migrate legacy ETL workloads with validated data parity and support analytics, reporting, and ML consumers. Expect to leverage AI tools like Claude and GitHub Copilot to accelerate development, write clean Python and SQL, and optimize Spark jobs for performance and