AWS Data Engineer
- Level
- Mid
Not stated in the posting: salary, work type, visa.
Open. First seen 20 November 2026.
About the role
Job Title: Data Engineer
About Us
“Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount. With our presence across 32 cities across globe, we support 100+ clients across banking, financial and Energy sectors. We are recognized for our deep transformation execution and delivery.
WHY JOIN CAPCO?
You will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry. The projects that will transform the financial services industry.
MAKE AN IMPACT
Innovative thinking, delivery excellence and thought leadership to help our clients transform their business. Together with our clients and industry partners, we deliver disruptive work that is changing energy and financial services.
#BEYOURSELFATWORK
Capco has a tolerant, open culture that values diversity, inclusivity, and creativity.
CAREER ADVANCEMENT
With no forced hierarchy at Capco, everyone has the opportunity to grow as we grow, taking their career into their own hands.
DIVERSITY & INCLUSION
We believe that diversity of people and perspective gives us a competitive advantage.
Job Description:
Role: AWS Data Engineer
Location : Bangalore, Pune
Experience-6-12 yrs
Databricks Development and Data Engineering
- Design, develop, test, and maintain scalable data pipelines and transformation workflows using Databricks.
- Build and optimise data processing solutions using Apache Spark, PySpark, and SQL.
- Develop batch and, where required, streaming data pipelines to support enterprise data processing requirements.
- Implement robust data ingestion, cleansing, transformation, and enrichment processes across multiple data sources.
- Develop and maintain Delta Lake tables, applying appropriate data modelling, partitioning, and optimisation strategies.
- Implement reusable, modular, and maintainable data engineering frameworks and components.
- Ensure data pipelines meet agreed performance, reliability, scalability, and quality standards.
Data Platform Design and Optimisation
- Contribute to the design and implementation of cloud-based data platforms and lakehouse architectures.
- Apply Medallion Architecture principles, including Bronze, Silver, and Gold data layers, where appropriate.
- Optimise Spark workloads through efficient transformations, query tuning, cluster configuration, and resource management.
- Implement appropriate data storage, partitioning, and data lifecycle management strategies.
- Monitor and troubleshoot data processing workloads, identifying performance bottlenecks and implementing improvements.
- Support the implementation of secure and scalable data platform solutions aligned with enterprise architecture standards.
Data Integration and Orchestration
- Design and develop data ingestion and integration pipelines connecting structured, semi-structured, and unstructured data sources.
- Implement data orchestration workflows using Databricks Workflows, AWS, or equivalent technologies.
- Support integration with enterprise databases, APIs, cloud storage, and downstream analytical platforms.
- Develop data validation, reconciliation, and quality control processes to ensure data accuracy and consistency.
- Implement appropriate error handling, logging, monitoring, and recovery mechanisms.
- Support data migration and modernisation initiatives, including migration from legacy data platforms to cloud-based solutions.
Software Engineering and DevOps
- Apply software engineering best practices to data pipeline development, including modular design, code reviews, automated testing, and documentation.
- Develop and maintain CI/CD pipelines for deploying Databricks code and configurations across environments.
- Use Git, Azure DevOps or equivalent tools for source control, collaboration, and release management.
- Implement unit testing, integration testing, and data quality testing for data engineering solutions.
- Support automated deployment processes using Databricks Asset Bundles, Terraform, or equivalent tools where appropriate.
- Troubleshoot and resolve defects across development, testing, and production environments.
- Contribute to continuous improvement of development standards, reusable frameworks, and engineering practices.
- Data Governance, Security and Reliability
- Implement appropriate data access controls, security configurations, and governance standards within Databricks.
- Work with Unity Catalog or equivalent governance capabilities to support data discovery, access management, and lineage.
- Apply data quality, metadata management, and data lifecycle principles.
- Support compliance with organisational security policies and relevant financial services regulatory requirements.
- Implement monitoring and alerting to improve pipeline reliability and operational resilience.
- Collaborate with platform, security, and architecture teams to ensure solutions meet enterprise standards.
Consulting, Technical Leadership and Stakeholder Engagement
- Engage with client stakeholders to understand business requirements and translate them into scalable technical solutions.
- Take ownership of complex technical deliverables or defined data engineering workstreams.
- Contribute to solution architecture discussions, technical design decisions, and implementation planning.
- Provide technical guidance, code reviews, and mentoring to junior engineers and consultants.
- Communicate technical concepts, delivery progress, risks, and dependencies to technical and non-technical stakeholders.
- Collaborate with cross-functional teams to ensure alignment between data engineering solutions and business objectives.
- Identify opportunities to improve data engineering practices, platform performance, and delivery efficiency.
- Support technical estimation, sprint planning, and delivery coordination within Agile teams.
Required Skills and Experience
Essential Technical Skills
- Typically 6+ years of experience in data engineering, software engineering, or related technology roles, including substantial hands-on experience with Databricks.
- Strong practical experience developing data pipelines and transformation workloads using Databricks.
- Advanced proficiency in Python, PySpark, and SQL.
- Strong understanding of Apache Spark architecture, distributed data processing, and performance optimisation.
- Hands-on experience with Delta Lake, including data ingestion, transformation, optimisation, and data management.
- Experience designing and implementing ETL/ELT pipelines for large-scale data processing.
- Good understanding of lakehouse architecture, data modelling, and data warehousing concepts.
- Strong knowledge and hands-on experience of AWS development in some of the following : Lambda, Glue, Step Functions, IAM roles, Lakeformation, EventBridge, SNS, SQS, EC2,Security Groups, CloudFormation, RDS, DynamoDB, Redshift
- Experience with data orchestration tools such as Databricks Workflows or Azure Data Factory.
- Practical experience implementing data quality checks, error handling, logging, and monitoring.
- Experience using Git and CI/CD pipelines within a data engineering environment.
- Understanding of data security, access controls, and governance principles.
- Strong troubleshooting and performance-tuning capabilities for Spark and Databricks workloads.
- Experience working within Agile software delivery environments.
Consulting and Professional Skills
- Strong analytical thinking and structured problem-solving capabilities.
- Ability to independently own and deliver complex technical workstreams.
- Experience contributing to technical design and solution architecture decisions.
- Strong written and verbal communication skills, including the ability to explain technical concepts to non-technical stakeholders.
- Ability to engage effectively with client stakeholders and multidisciplinary delivery teams.
- Experience providing technical guidance and mentoring to junior team members.
- Ability to identify delivery risks, manage dependencies, and recommend appropriate solutions.
- Strong organisational skills and the ability to manage competing priorities.
- Proactive, collaborative, and client-focused approach to delivery.
- Desirable Skills and Experience
- Experience delivering data engineering solutions within banking, capital markets, insurance, or other financial services environments.
- Databricks Certified Data Engineer Associate or Professional certification.
- Experience with Unity Catalog, Delta Live Tables / Lakeflow Spark Declarative Pipelines, and Databricks SQL.
- Exposure to structured streaming and real-time data processing using Spark Structured Streaming or Kafka.
- Experience with infrastructure as code using Terraform or equivalent technologies.
- Familiarity with Microsoft Fabric, Snowflake, Synapse Analytics, or other modern data platforms.
- Experience with cloud data migration and legacy data platform modernisation programmes.
- Knowledge of dimensional modelling, Data Vault, and enterprise data architecture principles.
- Familiarity with data observability, metadata management, and data lineage solutions.
- Exposure to cost optimisation and workload management within cloud data platforms.
- Understanding of data governance and regulatory considerations relevant to financial services.
- Experience supporting technical proof-of-concepts and evaluating emerging data technologies.
- Excellent verbal communication skills.
- Good interpersonal skills.
- Ability to provide technical leadership for the team.
What this role asks for
Particular to this role
Azure DevOps · CloudFormation · Databricks · Delta Lake · DynamoDB · Git · IAM · Kafka
Rare in Capco's other 299 open postings.
Standard for data engineering roles
AWS · Azure · CI/CD · Data Warehousing · ETL · Python · SQL · Snowflake
Common in data engineering postings at any company.
Compared with Capco's other open jobs.
The team: Data & Analytics HIRES STEADILY
This role is one of 86 open in Data & Analytics. 21 were posted in the last 30 days and the oldest in October: a team that hires all year rather than one being built.
WHAT THE TEAM WORKS WITH, ACROSS ITS POSTINGS
Python (38 of 86), Azure (35 of 86), SQL (33 of 86), Agile (28 of 86), CI/CD (27 of 86), ETL (27 of 86), GCP (25 of 86)
OTHER ROLES ON THIS TEAM
- Data Lead - Financial Services posted Nov 20
- Snowflake Data Engineer posted Nov 19
- Data Scientist posted Nov 19
- Full Stack developer posted Nov 18
- Data Referral- Dummy ID posted Nov 17
- Data Architect - PSM posted Nov 17
- Senior Consultant - Data Analyst (Business & Data Transformation) posted Nov 13
- Data Manager & Governance posted Nov 12
- Data Engineer posted Nov 12
- Sr Data Engineer posted Nov 12
- Data Analytics Specialist posted Nov 9
- AI & ML Engineer | Banking posted Nov 6
- Automation Engineer posted Nov 6
- Data Modeler - HSBC posted Nov 6
- Data Privacy & Security – Senior Manager posted Nov 5
- Data Analyst – Python, SQL & PySpark (Financial Services) posted Nov 3
- AI Architect posted Nov 3
- ML AI Platform Engineer posted Nov 3
- Data Engineer posted Nov 3
- Big Data Engineer- Scala posted Nov 3
- Data Science (Banking) posted Oct 20
- Principal Data Architect posted Oct 15
- AI Governance / Ethical AI Specialist posted Oct 15
- Senior Consultant - AI Product Owner posted Oct 14
- Data Engineer | Banking posted Oct 12
- Data Architect posted Oct 5
- AI Engineer - All Levels posted Oct 2
- Principal Consultant - Technical Data Lead posted Sep 30
- GCP Data Engineer posted Sep 29
- Mid/Senior Data Engineer (Kraków/GCP) posted Sep 28
- Data Governance Specialist posted Sep 21
- GCP Data Engineer-LBG posted Sep 18
- GCP Data Engineer-LBG posted Sep 18
- AI Architect posted Sep 16
- AI Architect posted Sep 16
- AI Architect posted Sep 16
- Data Architect posted Sep 16
- Data Architect posted Sep 16
- Data Architect posted Sep 16
- Python Developer | Banking posted Sep 15
- Data Business Analyst – GCP / Hadoop posted Sep 10
- Data Engineer Snowflake - Qatar posted Sep 9
- Data Modelling Consultant (Insurance) posted Sep 7
- Data Modelling Consultant (Insurance) posted Sep 7
- Data Modelling Consultant (Insurance) posted Sep 7
- Data Modelling Consultant (Insurance) posted Sep 7
- Senior Data Architect (DataLake) posted Sep 7
- Data Engineer (DataLake) - All levels posted Sep 7
- Qliksense Developer posted Sep 3
- Data Architect Lead posted Aug 31
- Expert Data Analyst posted Aug 17
- Data Analyst (Banking experience) posted Aug 13
- Senior Data Engineering - GCP Lead posted Aug 6
- Senior Data Engineering - PSM-Libra posted Aug 6
- Senior AI Fullstack Engineer posted Jul 28
- AI Strategy & Transformation – Managing Principal posted Jul 20
- Principal Consultant - Data Governance Lead posted Jul 16
- Senior Data Consultant – Architecture, Modelling & Products posted Jul 10
- Alteryx Developer posted Jun 17
- Sr Data Modeler posted Jun 11
- Senior Data Engineer (Banking) posted May 22
- Data Architect (Banking) posted May 22
- Data Analyst - Data Remediation - Mumbai posted May 15
- AI Architect posted Mar 5
- Senior Data Architect - Wealth & Asset Management. posted Feb 19
- Management Consultant: Data Analytics Lead – Managing Principal (Director) posted Feb 6
- Applied AI & Analytics - Principal Consultant posted Jan 31
- Senior Data Analytics Consultant posted Jan 29
- Lead Data Analytics Consultant posted Jan 29
- Data & AI Strategy Consultant - Manager/Senior Manager posted Jan 28
- Senior Data Architect posted Jan 28
- Senior Data Modelling Consultant posted Jan 28
- Senior Data Scientist posted Jan 20
- Data Modeler posted Dec 15
- Data Science Manager posted Oct 29
- Senior Data Analyst posted Aug 14
- GenAI Engineer posted May 19
- GenAI Engineer posted May 19
- GenAI Engineer posted May 19
- GenAI Engineer posted May 19
- Data Analyst with German language posted Oct 3
- Generative AI Engineer posted Apr 5
- Data Analyst with German language posted Jul 24
- Data Analyst with German language posted Oct 20
- Data Analyst with German language posted Oct 20
From the team and dates on Capco's own job board.
Similar jobs
- Senior Software Engineer - Python, Data Engineering, AI at Zenoti
Shares 18 skills: Delta Lake, Redshift, CloudFormation… · same country - Data Engineer (Cloud) at Incelligent
Shares 17 skills: Redshift, CloudFormation, Unity… · same seniority - Senior Lead Data Engineer at Integrant
Shares 17 skills: Delta Lake, DynamoDB, Redshift… - AI Data Engineer at Weekday AI
Shares 13 skills: Delta Lake, Redshift, Data Warehousing… · same city - Data Platform Engineer at Verifone
Shares 12 skills: Delta Lake, Redshift, Data Warehousing… · same seniority · same city - Senior Solution Architect at Capco
Also at this company
More jobs like this
Is this your posting? Email [email protected] to have it removed.