Senior Data Engineer (Databricks on Azure)
- Level
- Senior
- Work type
- Remote · Colombia only
Not stated in the posting: salary, visa.
Open. First seen 7 October 2026.
About the role
We are looking for a Senior Data Engineer with advanced, hands-on Databricks on Azure experience to design, build, modernise and operate scalable data pipelines supporting Tax technology platforms. You will migrate and enhance existing Spark and Python workloads on Azure Databricks and build scalable ingestion and document-processing capabilities. We need someone who has owned production Databricks on Azure pipelines end to end, understands the wider system, surfaces assumptions and trade-offs, and proposes better approaches.
Responsibilities
- Design, build, and support scalable data pipelines using Python, Apache Spark, PySpark, and Databricks on Azure.
- Modernise and migrate legacy data processing workloads to secure, cloud-native platforms.
- Build and maintain batch data ingestion pipelines from structured and unstructured sources.
- Integrate data from REST APIs, SharePoint, document repositories, enterprise applications, and cloud platforms.
- Implement data quality, monitoring, observability, and operational controls.
- Optimise data workloads for performance, scalability, reliability, and cost efficiency.
- Develop document extraction, classification, metadata enrichment, and automation pipelines.
- Apply software engineering practices, including Git, CI/CD, automated testing, and code reviews.
- Build, deploy, troubleshoot, and maintain production ETL/ELT pipelines.
- Collaborate with architects, developers, tax subject matter experts, and platform teams.
- Analyse existing codebases and identify opportunities to improve maintainability, security, and reliability.
- Use AI-assisted development tools to build, review, test, and maintain data pipeline code.
Requirements
Must Have
- 5+ years of experience in data engineering.
- Strong Python development experience.
- Strong Apache Spark and PySpark experience.
- Advanced, hands-on Databricks on Azure experience in production (Delta Lake, Unity Catalogue, Workflows/Jobs, ADLS Gen2, Azure Data Factory).
- Strong SQL and data modelling skills.
- Experience building, testing, deploying, and troubleshooting production ETL/ELT pipelines.
- Experience with Git-based source control and CI/CD pipelines (e.g., Azure DevOps).
- Experience consuming REST APIs for data ingestion, including authentication, pagination, and failure handling.
- Experience writing automated tests for data pipelines.
- Strong understanding of data quality, reliability, and data engineering best practices.
- Strong problem-solving, communication, and collaboration skills, with sound judgement to analyse unfamiliar codebases, identify gaps and trade-offs, challenge requirements and propose practical improvements.
Nice to have
- Experience modernising legacy data pipelines.
- Experience with document processing, intelligent document extraction, or unstructured data.
- Familiarity with AI-assisted development tools such as Codex, GitHub Copilot, or similar.
- Experience with streaming data pipelines or large-scale data repositories.
Benefits
A stable, long-term contract. Continuous Training. Private Health insurance. Flexible schedule. Work with some of the most talented software engineers in Latin America and the US, doing challenging work and world-class software for clients in the US and around the world.
Devsu's standard text
At Devsu, we believe in creating an environment where you can thrive both personally and professionally. By joining our team, you’ll enjoy:
- A stable, long-term contract with opportunities for career growth
- Private health insurance
- A remote-friendly culture that promotes work-life balance
- Continuous training, mentorship, and learning programs to keep you at the forefront of the industry
- Free access to AI training resources and state-of-the-art AI tools to elevate your daily work
- A flexible Paid Time Off (PTO) policy as well as paid holiday days
- Challenging, world-class software projects for clients in the US and LatAm
- Collaboration with some of the most talented software engineers in Latin America and the US, in a diverse work environment
Devsu's standard text
Join Devsu and discover a workplace that values your growth, supports your well-being, and empowers you to make a global impact.
What this role asks for
Particular to this role
Azure DevOps · Delta Lake · SharePoint · Unity
Rare in Devsu's other 12 open postings.
Standard for data engineering roles
Azure · CI/CD · ETL · Python · SQL · Spark
Common in data engineering postings at any company.
92% of this description is written for the role. The rest is Devsu's standard text, folded away below.
Compared with Devsu's other open jobs.
The team: Devsu HIRES STEADILY
This role is one of 9 open in Devsu. 4 were posted in the last 30 days and the oldest in July: a team that hires all year rather than one being built.
WHAT THE TEAM WORKS WITH, ACROSS ITS POSTINGS
Python (5 of 9), CI/CD (4 of 9), Kubernetes (4 of 9), SQL (4 of 9), TypeScript (4 of 9), AWS (3 of 9), Azure (3 of 9)
OTHER ROLES ON THIS TEAM
- Full Stack Developer (Java / Angular / GCP) | Mid-Senior Level posted Nov 17
- Senior Power Platform Developer posted Nov 9
- Semi Senior / Senior Data Engineer (AWS, Python & SQL) posted Nov 6
- Head of Engineering (B2B Technology Services) posted Oct 15
- Sr Fullstack (Go/Java + React) posted Oct 13
- Forward Deployed AI Engineer posted Oct 5
- Revenue Operations Engineer (RevOps) | HubSpot, Automation & AI posted Aug 13
- Application Support Engineer (GCP & Media Platforms) posted Jul 30
From the team and dates on Devsu's own job board.
Similar jobs
- Senior Databricks Data Engineer at 49 Solutions
Shares 12 skills: Delta Lake, Unity, Azure DevOps… · Senior level - Senior Data Engineer (Databricks and Pyspark)-Mandarin Speaker at Unison Group
Shares 12 skills: Delta Lake, SharePoint, Azure DevOps… · Senior level - Azure Data Engineer at ProArch
Shares 12 skills: Delta Lake, Azure DevOps, ETL… · Senior level - Senior Software Engineer - Python, Data Engineering, AI at Zenoti
Shares 13 skills: Delta Lake, Unity, Azure DevOps… · Senior level - Senior Azure Data Engineer (Databricks) at Capco
Shares 10 skills: Delta Lake, Unity, Azure DevOps… · Senior level - Full Stack Developer (Java / Angular / GCP) | Mid-Senior Level at Devsu
Also at this company
More jobs like this
Is this your posting? Email [email protected] to have it removed.