Sr Data Engineer (Databricks)
- Level
- Senior
Not stated in the posting: salary, work type, visa.
Open. First seen 3 October 2026.
About the role
Position Overview:
- Lead the design, architecture, and optimization of enterprise-scale data pipelines using Azure Databricks for ACDP projects, driving innovation in DATA & AI initiatives.
- Leverage 7-8 years of expertise to mentor teams, ensure scalability, and deliver mission-critical data solutions.
Key Responsibilities:
- Architect and implement end-to-end data pipelines with Azure Databricks (notebooks, jobs, clusters) for ingestion, ETL/ELT, and complex transformations, incorporating Medallion Architecture (Bronze, Silver, Gold) using Delta Lake for ACID compliance, versioning, schema evolution, and data quality.
- Provide hands-on leadership in Databricks Spark (PySpark, Scala, SQL), Delta Live Tables, Structured Streaming, and Unity Catalog for advanced governance, security, and multi-workspace management.
- Integrate with Azure ecosystem including ADLS Gen2, Azure SQL Database, Serverless/Dedicated SQL Pools; orchestrate via Databricks workflows and hybrid ADF triggers for high-volume ACDP workloads.
- Design and deploy CI/CD pipelines using Azure DevOps, Bicep for infrastructure-as code, and advanced automation for Databricks workspaces, emphasizing zero downtime deployments and cost optimization.
- Champion data quality, validation, cleansing with Databricks expectations/constraints, and compliance via RBAC, encryption, and Purview; mentor on best practices for Photon engine, auto-scaling, spot instances, and performance tuning
- Collaborate with senior analysts, data scientists, and executives to translate complex business requirements into reusable Databricks components, fostering AI/ML integration.
- Drive monitoring, troubleshooting, and optimization for petabyte-scale processing, including unstructured data handling, data modeling, and streaming from Kafka/Event Hubs
Requirements
Required Qualifications:
- 7-8+ years in data engineering, with deep Azure Databricks expertise for big data processing in high-demand environments
- Advanced proficiency in Python/PySpark, SQL, Scala; strong data architecture and distributed systems experience
- Proven track record in cloud-native solutions (AZ-900+ preferred), CI/CD, and mentoring junior engineers
- Excellent communication for stakeholder alignment and documentation
Preferred: Power BI, Power Automate, Azure ML, enterprise governance tools.
What this role asks for
Particular to this role
Azure DevOps · Delta Lake · Power BI · Power Platform · Scala · Serverless · Unity
Rare in Unison Group's other 170 open postings.
Standard for data engineering roles
Azure · CI/CD · ETL · Python · Spark
Common in data engineering postings at any company.
In most of Unison Group's ads
SQL
Named in most of its open postings, whatever the role.
Compared with Unison Group's other open jobs.
The team: Bharat HIRES STEADILY
This role is one of 30 open in Bharat. 9 were posted in the last 30 days and the oldest in March: a team that hires all year rather than one being built.
WHAT THE TEAM WORKS WITH, ACROSS ITS POSTINGS
SQL (23 of 30), Azure (20 of 30), Python (20 of 30), CI/CD (19 of 30), Databricks (14 of 30), ETL (13 of 30), GCP (13 of 30)
OTHER ROLES ON THIS TEAM
- Databricks Data Architect posted Nov 18
- Databricks Architect posted Nov 13
- C++ Developer posted Nov 11
- Machine Learning Manager (AI) posted Nov 10
- AI & Machine Learning Manager posted Nov 10
- AI & Machine Learning Manager (Techno-Functional) posted Nov 10
- Cloud Engineer ( Google Cloud Engineer) posted Oct 30
- GCP Cloud Engineer - GCP & GKE posted Oct 28
- Teradata Data Architect posted Oct 26
- Ontology Specialist - Data & AI posted Oct 16
- Data Manager - Databricks posted Oct 16
- Data Engineering Manager - Databricks & Lakehouse posted Oct 16
- Ontology Specialist - Data & AI posted Oct 16
- SAP HANA CLOUD Consultant posted Oct 16
- Low Code Engineer (OutSystems Platform) posted Oct 14
- Fullstack Developer posted Oct 14
- Data Engineer (Quantexa) posted Oct 13
- Java Backend Developer (Microservices) posted Oct 12
- Java & Microservices Developer posted Oct 10
- Java & Microservices Developer posted Oct 10
- Java & Microservices Developer posted Oct 10
- Quantexa Data Engineer posted Oct 8
- Resident Solution Architect Databricks posted Oct 6
- Senior Databricks Data Engineer posted Oct 2
- Data Analyst/Business Analyst (SAP BW) posted Sep 25
- Lead Data Engineer (Banking) posted Sep 25
- Techanical Business Analyst - (SAP BW, Python, SQL, Power BI) posted Sep 22
- Senior Data Analyst - SAP BW/ECC, Databricks & BI posted Sep 8
- Business Development Manager - IT Consulting posted Mar 19
From the team and dates on Unison Group's own job board.
Similar jobs
- Senior Lead Data Engineer - Databricks at Integrant
Shares 13 skills: Delta Lake, Unity, Azure DevOps… - Resident Solution Architect (RSA) - Data & Databricks at Weekday AI
Shares 10 skills: Delta Lake, Unity, ETL… · same country - AWS Databricks Architect -Sr Staff at Qualcomm
Shares 11 skills: Delta Lake, Unity, Azure DevOps… · same country - Senior Data Engineer (Databricks) at Livefront
Shares 11 skills: Delta Lake, Unity, Scala… · Senior level - Senior Azure Data Engineer (Databricks) at Capco
Shares 11 skills: Delta Lake, Unity, Scala… · Senior level - Senior Java Microservices & Spark Developer at Unison Group
Also at this company
More jobs like this
Is this your posting? Email [email protected] to have it removed.