Description du poste
Job Overview
Design, develop and maintain data engineering solutions, including large-scale batch and streaming pipelines, distributed processing and containerized data workloads.
Key Responsibilities
• Design, develop and maintain ingestion, transformation and validation pipelines using Apache Spark, Apache Flink and Apache Kafka.
• Implement distributed processing using Java or Scala and SQL, integrating data sources and S3-compatible object storage.
• Maintain schemas, metadata, data lineage and data-quality controls across processing stages, supporting layered data processing where applicable.
• Deploy and scale workloads on Kubernetes, and implement automated tests, GitLab CI/CD, monitoring and operational troubleshooting.
• Implement checkpointing, restart recovery, retention and lifecycle controls.
• Support high availability, backup, disaster recovery and inter-site synchronization in line with agreed requirements.
Requirements & Skills
• 5–8 years of data engineering experience, including building and deploying large-scale data systems.
• Strong hands-on experience designing batch and streaming pipelines using Apache Spark, Apache Flink and Apache Kafka.
• Proficiency in Java or Scala for distributed data processing, together with SQL.
• Experience deploying and scaling data workloads in containerised Kubernetes environments.
• Experience with S3-compatible object storage, data-quality controls, schema and lineage management, checkpointing and recovery, GitLab and CI/CD practices.
How to Apply
(EA Reg No: 20C0312)
Please email a copy of your detailed resume to pauline@talentsis.com.sg for immediate processing.
Only shortlisted candidates will be notified.