Weekday AI

Lead Data Engineer

Delhi
Level
Lead

Not stated in the posting: salary, work type, visa.

Open. First seen 16 November 2026.

About the role

๐—ง๐—ต๐—ถ๐˜€ ๐—ฟ๐—ผ๐—น๐—ฒ ๐—ถ๐˜€ ๐—ณ๐—ผ๐—ฟ ๐—ผ๐—ป๐—ฒ ๐—ผ๐—ณ ๐˜๐—ต๐—ฒ ๐—ช๐—ฒ๐—ฒ๐—ธ๐—ฑ๐—ฎ๐˜†'๐˜€ ๐—ฐ๐—น๐—ถ๐—ฒ๐—ป๐˜๐˜€

๐—ฆ๐—ฎ๐—น๐—ฎ๐—ฟ๐˜† ๐—ฟ๐—ฎ๐—ป๐—ด๐—ฒ: ๐—ฅ๐˜€ ๐Ÿฐ๐Ÿฌ๐Ÿฌ๐Ÿฌ๐Ÿฌ๐Ÿฌ๐Ÿฌ - ๐—ฅ๐˜€ ๐Ÿฒ๐Ÿฌ๐Ÿฌ๐Ÿฌ๐Ÿฌ๐Ÿฌ๐Ÿฌ (๐—ถ๐—ฒ ๐—œ๐—ก๐—ฅ ๐Ÿฐ๐Ÿฌ-๐Ÿฒ๐Ÿฌ ๐—Ÿ๐—ฃ๐—”)

Experience: 6+ yrs

Location: NCR, Delhi, India

As a Lead Data Engineer, you will spearhead the design, development, and optimization of large-scale data pipelines that support both batch and realโ€‘time data processing initiatives across the organization. You will collaborate closely with data scientists, product managers, and infrastructure teams to ensure data integrity, scalability, and high performance at every stage of the pipeline, while driving continuous improvement through automation, monitoring, and best practices. Your leadership will shape the data architecture, protect data quality, and enable timely, actionable insights that inform business decisions.

In a fastโ€‘moving environment, you will balance technical excellence with strategic vision, fostering a culture of collaboration and innovation within the data engineering team.

Requirements

๐—ž๐—ฒ๐˜† ๐—ฅ๐—ฒ๐˜€๐—ฝ๐—ผ๐—ป๐˜€๐—ถ๐—ฏ๐—ถ๐—น๐—ถ๐˜๐—ถ๐—ฒ๐˜€

  • Design and implement robust, endโ€‘toโ€‘end batch data pipelines using tools such as Apache Spark, Hadoop, and Airflow.
  • Develop realโ€‘time streaming pipelines with technologies like Apache Kafka, Flink, or Spark Structured Streaming to deliver lowโ€‘latency analytics.
  • Optimize pipeline performance through partitioning, indexing, and efficient data format selection (e.g., Parquet, ORC).
  • Establish data governance, quality checks, and automated monitoring to ensure data reliability and compliance.
  • Lead capacity planning and scaling strategies for data infrastructure, including cluster management and cost optimization.
  • Mentor and coach junior data engineers, fostering a collaborative and growthโ€‘oriented team culture.
  • Collaborate with Data Scientists to translate analytical requirements into scalable pipeline solutions.
  • Drive integration of data pipelines with downstream applications, dashboards, and reporting tools.
  • Stay current with emerging data engineering technologies and evaluate their applicability to business needs.
  • Participate in incident response and root cause analysis to quickly resolve data pipeline failures.

๐—ช๐—ต๐—ฎ๐˜ ๐— ๐—ฎ๐—ธ๐—ฒ๐˜€ ๐—ฌ๐—ผ๐˜‚ ๐—ฎ ๐—š๐—ฟ๐—ฒ๐—ฎ๐˜ ๐—™๐—ถ๐˜

  • 6+ years of experience in data engineering with a proven track record in both batch and realโ€‘time processing.
  • Deep knowledge of distributed computing frameworks such as Hadoop, Spark, and Flink.
  • Proficiency in message queuing and stream processing systems like Kafka, Pulsar, or RabbitMQ.
  • Strong scripting skills in Python, Scala, or Java, and experience with SQL on large datasets.
  • Handsโ€‘on experience with cloud data services (AWS Glue, Redshift, BigQuery, Snowflake, or Azure Data Factory).
  • Familiarity with containerization (Docker, Kubernetes) and CI/CD pipelines for data workloads.
  • Demonstrated ability to architect and scale data pipelines to handle petabyteโ€‘scale volumes.
  • Excellent communication skills, capable of translating technical concepts for nonโ€‘technical stakeholders.
  • A proactive problemโ€‘solver with a passion for data quality, governance, and automation.
  • Leadership aptitude, with experience building, mentoring, and growing data engineering teams.

What this role asks for

Particular to this role

Airflow ยท BigQuery ยท Docker ยท Flink ยท Hadoop ยท Java ยท Kafka ยท RabbitMQ

Rare in Weekday AI's other 299 open postings.

Standard for data engineering roles

AWS ยท Azure ยท CI/CD ยท Python ยท SQL ยท Snowflake ยท Spark

Common in data engineering postings at any company.

Compared with Weekday AI's other open jobs.

The team: Weekday's Client via platform HIRES STEADILY

This role is one of 564 open in Weekday's Client via platform. 343 were posted in the last 30 days and the oldest in September: a team that hires all year rather than one being built.

WHAT THE TEAM WORKS WITH, ACROSS ITS POSTINGS

Python (186 of 564), CI/CD (171 of 564), AWS (152 of 564), Azure (131 of 564), SQL (131 of 564), REST (115 of 564), GCP (112 of 564)

OTHER ROLES ON THIS TEAM

From the team and dates on Weekday AI's own job board.

Salary context

6 other open postings titled Lead Data Engineer state a salary: median USD 132,500 to USD 189,450 a year.

Similar jobs

More jobs like this

Is this your posting? Email [email protected] to have it removed.