AgileEngine is seeking a Senior Data Engineer to design, build, and operate batch and streaming data pipelines on Databricks using PySpark and Delta Lake. You will migrate legacy ETL workloads to a governed Lakehouse and implement data quality, governance, and testing practices.
The role emphasizes strong SQL/Python skills, Unity Catalog governance, and collaboration with DevOps and analytics teams to ensure scalable data platforms for analytics and AI use cases.