Senior Lead Data Engineer - Databricks
- Level
- Lead
Not stated in the posting: salary, work type, visa.
Open. First seen 3 October 2026.
About the role
Description
We're on the lookout for exceptional individuals to join our team as Senior Lead Data Engineers – Databricks. As part of our team, you'll play a key role in designing and delivering modern, scalable data platforms for clients across multiple industries, leveraging Azure and Databricks technologies.
You will lead the design, development, and optimization of end-to-end data engineering solutions built on modern Lakehouse architectures. Working closely with clients and cross-functional teams, you'll translate business requirements into scalable, high-performance data platforms while mentoring engineers and driving technical excellence.
Requirements
- Responsibilities
- Lead the design, development, and maintenance of scalable data engineering solutions using Azure and Databricks.
- Design and implement modern Lakehouse architectures following industry best practices.
- Build and optimize large-scale batch and streaming data pipelines.
- Develop ETL/ELT pipelines using Databricks, PySpark, SQL, and Azure Data Factory.
- Design, implement, and optimize Delta Lake solutions for reliable and performant data processing.
- Collaborate with clients to gather requirements and translate business needs into technical solutions.
- Optimize Spark workloads for performance, scalability, and cost efficiency.
- Design and maintain enterprise-grade Data Warehouses, Data Lakes, and Lakehouses.
- Implement data governance, security, and metadata management using Azure services such as Purview, Key Vault, and Microsoft Entra ID (Azure Active Directory).
- Establish CI/CD pipelines and DevOps practices for data engineering workloads.
- Mentor and coach junior data engineers while promoting engineering best practices.
- Participate in architecture discussions, solution design, and technical leadership initiatives.
- Contribute to internal knowledge sharing and continuous improvement.
- Requirements
- Required
- 10+ years of professional experience in Data Engineering.
- 4+ years of hands-on experience with Databricks.
- Strong experience developing data pipelines using PySpark and Apache Spark.
- Strong experience designing and implementing Lakehouse architectures.
- Experience working with Delta Lake and Medallion Architecture.
- Strong knowledge of SQL and Python.
- Experience with Azure Data Factory.
- Experience working with Azure Data Lake Storage Gen2 (ADLS Gen2).
- Experience with Azure Synapse Analytics.
- Experience with Data Warehousing concepts and dimensional modeling.
- Strong understanding of ETL and ELT design patterns.
- Experience optimizing Spark jobs, partitioning strategies, joins, caching, and workload performance.
- Experience with Git and Azure DevOps CI/CD pipelines.
- Experience implementing secure data platforms using Azure Key Vault, Microsoft Entra ID, and Microsoft Purview.
- Experience building scalable enterprise data platforms in Azure.
- Experience working directly with clients and gathering technical requirements.
- Experience mentoring engineers and leading technical initiatives.
- Strongly Preferred
- Experience with Snowflake.
- Experience building real-time streaming solutions using Structured Streaming, Kafka, or Event Hubs.
- Experience with Databricks Workflows.
- Experience with Delta Live Tables.
- Experience with Unity Catalog.
- Experience with Databricks SQL.
- Experience with MLflow.
- Experience with dbt.
- Experience with Airflow or other orchestration platforms.
- Experience working in Agile environments.
- Nice to Have
- MSc in Computer Science or a related field.
- Experience with DataOps practices.
- Experience with MLOps.
- Experience with Azure Machine Learning.
- Experience with Azure Cognitive Services.
- Experience with Docker and Kubernetes.
- Experience working with Microsoft Fabric.
- Experience with Power BI.
- Technical Skills
- Core Technologies
- Azure Databricks
- Apache Spark
- PySpark
- SQL
- Python
- Delta Lake
- Azure Data Factory
- Azure Synapse Analytics
- ADLS Gen2
- Snowflake
- Data Engineering
- Data Warehousing
- Data Lakes
- Lakehouse Architecture
- Medallion Architecture
- Dimensional Modeling
- ETL / ELT
- Batch Processing
- Streaming Data Pipelines
- Data Governance
- Performance Optimization
- Azure Ecosystem
- Azure DevOps
- Microsoft Purview
- Azure Key Vault
- Microsoft Entra ID
- Azure Functions
- Event Hubs
- Nice-to-Have Technologies
- Kafka
- Airflow
- dbt
- MLflow
- Docker
- Kubernetes
- Power BI
Benefits
- Salary paid in USD
- Six-month career advancing opportunities
- Supportive and friendly work environment
- Premium medical insurance [employee +family]
- English language development courses
- Interest-free loans paid over 2.5 years
- Technical development courses
- Planned overtime program (POP)
- Employment referral program
- Premium location in Maadi
- Social insurance
What this role asks for
Particular to this role
Active Directory · Airflow · Azure DevOps · Delta Lake · Git · Kafka · Power BI · Unity
Rare in Integrant's other 24 open postings.
Standard for data engineering roles
Azure · Data Warehousing · ETL · Python · SQL · Snowflake · Spark · dbt
Common in data engineering postings at any company.
In most of Integrant's ads
CI/CD
Named in most of its open postings, whatever the role.
Compared with Integrant's other open jobs.
The team: Software Development HIRES STEADILY
This role is one of 18 open in Software Development. 1 was posted in the last 30 days and the oldest in October: a team that hires all year rather than one being built.
WHAT THE TEAM WORKS WITH, ACROSS ITS POSTINGS
CI/CD (13 of 18), Azure (11 of 18), AWS (10 of 18), Agile (9 of 18), Docker (8 of 18), Machine Learning (8 of 18), Python (8 of 18)
OTHER ROLES ON THIS TEAM
- Senior Lead Agentic Developer posted Oct 29
- Lead Software Engineer - Angular/UI posted Oct 20
- Lead SysOps Engineer posted Aug 13
- Principal Data Scientist - AI & Machine Learning (Databricks) posted Aug 13
- Senior Lead Software Engineer - Java Full Stack posted Jul 29
- Lead Backend Engineer (C/C++) posted Jun 22
- Lead Software Engineer - Java Full Stack posted Jun 15
- Lead Data Scientist - AI & Machine Learning posted Jun 11
- Principal Data Scientist - AI & Machine Learning posted Jun 10
- Senior Lead Software Developer in Test (SDET) posted May 14
- Devops & SysOps Architect posted Apr 26
- Lead Software Developer in Test (SDET) posted Apr 21
- Principal Software Developer in Test (SDET) posted Apr 19
- Senior Technical Project Lead / .NET posted Apr 8
- Data Architect / Part-Time posted Mar 29
- Senior Technical Lead - .Net Fullstack posted Mar 26
- Principal SysOps Engineer posted Oct 16
From the team and dates on Integrant's own job board.
Similar jobs
- Senior Software Engineer - Python, Data Engineering, AI at Zenoti
Shares 18 skills: Delta Lake, Unity, Azure DevOps… - Resident Solution Architect (RSA) - Data & Databricks at Weekday AI
Shares 15 skills: Delta Lake, MLflow, Unity… · Lead level - Data Manager - Databricks at Unison Group
Shares 14 skills: Delta Lake, Unity, Azure DevOps… · Lead level - Senior AI Architect at Multibank Group
Shares 17 skills: Delta Lake, MLflow, Unity… · Lead level - Data Engineer (Cloud) at Incelligent
Shares 16 skills: Unity, Azure DevOps, Airflow… - Senior Lead Agentic Developer at Integrant
Also at this company
More jobs like this
Is this your posting? Email [email protected] to have it removed.