Data Engineer

Position Overview

We are seeking a Data Engineer to build and operate the data platform spanning both network/operational (OSS) and business/commercial data. This role is responsible for owning data ingestion, storage, transformation, and serving processes, leveraging both streaming and batch data architectures. The successful candidate will ensure operational and commercial data are governed effectively while enabling reporting, analytics, and AI/ML initiatives across the organization.

Key Responsibilities:

· Build, develop, and deploy data ingestion and streaming pipelines using Apache Flink, CDC-based ingestion, AWS Glue Schema Registry, and in-stream validation processes.
· Design and maintain raw, curated, and served data layers while optimizing query performance, storage costs, and data freshness.
· Manage table maintenance, compaction, and partitioning strategies as data volumes increase.
· Develop and maintain SQL transformation pipelines and dimensional data models.
· Design schemas and implement ingestion and transformation logic for new data domains.
· Implement and maintain data quality testing to ensure freshness, completeness, and schema conformance.
· Build and maintain BI and reporting solutions including dashboards, ad hoc query capabilities, and scheduled reports.
· Establish governed, high-quality datasets to support AI/ML initiatives such as predictive maintenance and capacity forecasting.
· Document data models, pipeline architectures, and governance controls.
· Collaborate with internal teams to gather requirements, validate assumptions, and deliver data solutions that meet business needs.

Qualifications

Education:

Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Technology, or a related field, or equivalent professional experience.

Experience:

A minimum of 4+ years of professional experience in data engineering, including both streaming and batch data pipeline development.

Preferred Qualifications:

· Production experience with Apache Flink or a comparable stream-processing platform.
· Experience with CDC-based ingestion architectures.
· Hands-on experience with Apache Iceberg, Delta Lake, and AWS Glue Data Catalog.
· Experience with Protobuf, schema registries, and schema evolution management.
· Experience operating Apache Airflow for workflow orchestration.
· Knowledge of AWS services including S3, Glue, MSK, IAM, Athena, and Redshift.
· Experience implementing data governance, access controls, and data protection measures.
· Experience supporting AI/ML data preparation and data platform initiatives.
· Experience working with operational telemetry data and commercial/business datasets.
· English language proficiency at B2 level or above.

Soft Skills:

· Strong ownership mindset with a focus on data accuracy and quality.
· Excellent interpersonal and cross-functional collaboration skills.
· Strong written and verbal communication skills with both technical and non-technical stakeholders.
· Ability to bridge operational and business data domains effectively.
· Proactive problem-solving and risk identification capabilities.
· Self-directed and highly organized approach to work.
· Meticulous attention to detail in documentation, governance, and data management activities.
· Ability to adapt and thrive in a fast-paced, globally distributed team environment.

Technology Stack:

· Apache Flink
· Apache Kafka / AWS MSK
· Protobuf
· AWS Glue Data Catalog
· AWS Glue Schema Registry
· Amazon S3
· Apache Iceberg
· Amazon Athena
· Amazon Redshift
· Apache Airflow
· dbt
· OpenTelemetry
· Prometheus
· Grafana
· SQL
· Python
· Git
· Azure DevOps CI/CD
· AWS IAM

Physical Requirements

· Ability to work in a standard office or remote home-office environment and use a computer for extended periods.
· Ability to sit, type, and view computer screens for prolonged periods.
· Ability to participate in occasional after-hours incident response activities.
· Ability to communicate effectively via virtual and in-person meetings.
· Ability to travel occasionally if required for business purposes.