Senior Data Engineer

Senior Data Platform Engineer - Spark / Flink / Kafka

About the role

Design and deliver large-scale batch and streaming data pipelines. Build reliable processing and storage flows, improve data quality and recovery, and support the operation of distributed data workloads in Kubernetes environments.

Key responsibilities

• Design and develop batch and streaming pipelines using Apache Spark, Apache Flink and Apache Kafka.
• Implement processing logic using Java or Scala and SQL.
• Deploy, scale and troubleshoot containerised data workloads on Kubernetes.
• Build data-quality checks and manage schema, lineage, checkpointing and recovery.
• Integrate S3-compatible object storage and support pipeline delivery through GitLab and CI/CD practices.

Requirements

• 5-8 years of data engineering experience, including delivery of large-scale data systems.
• Strong hands-on experience designing pipelines using Apache Spark, Apache Flink and Apache Kafka.
• Proficiency in Java or Scala for distributed processing, together with SQL.
• Experience deploying and scaling data workloads on Kubernetes.
• Experience with S3-compatible storage, data-quality controls, schema and lineage management, checkpointing, recovery, GitLab and CI/CD.
• Comparable large-scale processing frameworks may be considered where transferable distributed-data-system capability is demonstrated.

Good to have

• SAFe and safety-critical or mission-critical data platforms.
• MinIO, layered bronze/silver/gold processing, metadata preservation and storage-integrity controls such as checksums, erasure coding and healing.
• Retention and lifecycle controls, hot/cold reboot recovery, high availability, backup, disaster recovery and inter-site synchronisation.
• Apache Airflow or equivalent workflow-orchestration tools.

Interested candidates are encouraged to submit their resumes outlining their relevant experience and achievements to apply88(@)talentvis.com or click apply!

We regret to inform that only shortlisted candidates would be notified

EA License No: 04C3537

EA Personnel No: R22106683

EA Personnel Name: Yang Hui Shan, Sherri