Senior Data Engineer

Key Responsibilities

• Build and test data pipelines on modern data platforms

• Support data engineering, ingestion, transformation and consumption activities

• Develop new pipelines and data products, and automation

• Enhance existing pipelines and data products, and provide support

• Maintain all the data pipelines and data products of these Solutions

• Design conceptual, logical, and physical data models

• Design data lake to support analytical and reporting needs

• Provide technical services like development, data preparation, data transformation, data

encryption, data masking, data optimization and data testing preparation

• Adhere to the standards and guidelines defined by the Authority for devops processes, provide

deployment plan, checklist and assist in production implementation/ roll out

• Provide support and integration with capability on latest AI tool stack to expedite the

development, testing and deployment process

Required Skills

• Python

• PySpark

• Snowflake

• Databricks

• Cloudera

• Migration from Cloudera to Snowflake or Databricks with meta data-driven design

Required Experience

• Minimum 5+ years of experience in the software used by the Solution

• Good understanding of data warehouse implementations

• Hands-on experience in big data engineering jobs using Python, PySpark to build and test data

pipelines on modern data platforms such as Snowflake, Databricks and Cloudera