AgileEngine is seeking a Senior Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You will design batch and streaming pipelines with PySpark and Delta Lake, and drive medallion architecture across bronze, silver, and gold layers.
You will leverage AI tools like Claude and GitHub Copilot to accelerate development, write clean Python/SQL, and ensure data quality, governance, and scalable production pipelines.