Information Technology Consultant (Data Engineer (Cloudera to AWS Migration)

We are seeking a highly skilled Data Engineer with 5+ years of experience in building scalable data platforms and modern data pipelines. The ideal candidate will have strong expertise in Python, SQL, PySpark, Hadoop/Cloudera technologies, and AWS data services, with hands-on experience migrating enterprise data workloads from Cloudera/Hadoop environments to AWS. This role will play a key part in designing, developing, and optimizing cloud-native data solutions that support analytics, reporting, and business-critical applications.

• Design, develop, and maintain scalable ETL processes and data pipelines using Python, SQL, and PySpark.
• Lead and support migration of data workloads, datasets, and processing frameworks from Cloudera/Hadoop environments to AWS.
• Build and optimize cloud-native data solutions leveraging AWS services such as S3, Glue, EMR, Athena, and Redshift.
• Develop robust data ingestion, transformation, and integration frameworks to support enterprise analytics and reporting needs.
• Monitor and improve data pipeline performance, reliability, and scalability across on-premises and cloud environments.
• Ensure data quality, governance, security, and compliance standards are implemented throughout the data lifecycle.
• Collaborate with data architects, business stakeholders, analysts, and engineering teams to translate business requirements into technical solutions.
• Implement data orchestration, automation, and operational best practices to improve platform efficiency and supportability.
• Support production deployments, troubleshooting, monitoring, and continuous improvement initiatives.

Required Skills

• 5+ years of experience in Data Engineering, Data Warehousing, or Big Data technologies.
• Strong programming expertise in Python, SQL, and PySpark.
• Hands-on experience with the Hadoop/Cloudera ecosystem, including:
• HDFS
• Hive
• Impala
• Spark
• Cloudera Data Platform (CDP) preferred
• Amazon S3
• AWS Glue
• EMR
• Athena
• Redshift
• Proven experience migrating data platforms and workloads from Cloudera/Hadoop to AWS.
• Extensive experience in ETL development, data integration, and large-scale data pipeline implementation.
• Strong understanding of data processing, performance tuning, and optimization techniques.
• Experience working in Agile delivery environments.

Preferred Skills

• Data Modeling and Data Architecture concepts.
• Apache Airflow for workflow orchestration.
• DevOps and CI/CD pipeline implementation.
• Banking and Financial Services domain experience.
• Experience with cloud migration, modernization, and data transformation initiatives.

Interested candidates, who wish to apply for the above position, please send in your resume to: Sign in to view email

PERSOL Singapore Pte Ltd EA License No. 01C4394 EA Reg No: R1331594 (Sasi Indurthi Venkata)

We regret to inform you that only shortlisted candidates will be contacted.

This is in partnership with Employment and Employability Institute Pte Ltd (“e2i”). e2i is the empowering network for workers and employers seeking employment and employability solutions. e2i serves as a bridge between workers and employers, connecting with workers to offer job security through job-matching, career guidance and skills upgrading services, and partnering employers to address their manpower needs through recruitment, training, and job redesign solutions. e2i is a tripartite initiative of the National Trades Union Congress set up to support nation-wide manpower and skills upgrading initiatives. By applying for this role, you consent to e2i’s PDPA.

By sending us your personal data and curriculum vitae (CV), you are deemed to consent to PERSOL Singapore Pte Ltd and its affiliates to collect, use and disclose your personal data for the purposes set out in the Privacy Policy available at https://www.persolapac.com.sg/policies. You acknowledge that you have read, understood, and agree with the Privacy Policy.