Senior Cloud Data Engineer

As a senior data engineer, you will play a key role in developing and maintaining our cloud-based data infrastructure. This is a challenging opportunity to work on complex data integration projects, design scalable data pipelines, and ensure optimal performance and cost-effectiveness.

Key Responsibilities:

• Design, develop, and deploy cloud-based data processing systems using big data technologies such as Hadoop, Spark, and NoSQL databases.
• Develop and maintain ETL workflows for extracting, transforming, and loading data into data warehouses and other downstream systems.
• Collaborate with cross-functional teams to identify business requirements and develop technical solutions that meet those needs.
• Ensure data quality, security, and compliance with regulatory requirements.
• Stay up-to-date with emerging trends and technologies in the field of big data and cloud computing.

To be successful in this role, you will need to have the following skills and qualifications:

• 7-9 years of experience in IT, with at least 5 years in AWS-related projects.
• Associate level or professional level AWS certification.
• In-depth knowledge of AWS services, including S3, EC2, EMR, Serverless, Athena, AWS Glue, Lambda, Step Functions.
• Cloud databases (AWS Aurora, Singlestore, RedShift, Snowflake).
• Big data (Hadoop, Hive, Spark, YARN).
• Programming languages (Scala, Python, Shell Scripts, PySpark).
• Operating systems (Linux, Windows).
• Strong SQL skills.
• Orchestration tools (Apache Airflow).
• Expertise in developing ETL workflows comprising complex transformations like SCD, deduplications, aggregations etc.

The ideal candidate will have the following qualifications:

• Bachelor's degree in computer science, engineering, or related field.
• Demonstrated continued learning through one or more technical certifications or related methods.
• Minimum 5 years of experience on cloud-related projects.

This is an excellent opportunity to join a growing team and contribute to the development of innovative data solutions. We offer a competitive salary, comprehensive benefits package, and opportunities for professional growth and development.

Work and Technical Experience:

• Must-Have:
  • Overall, 7-9 years of IT Exp
  • 5+ years of AWS related project
  • Good to have Associate Level and Professional Level AWS Certification
  • In depth knowledge of following AWS Services required: S3, EC2, EMR, Severless, Athena, AWS Glue, Lambda, Step Functions
  • Cloud Databases (Must have) – AWS Aurora, Singlestore, RedShift, Snowflake
  • Big Data (Must have) - Hadoop, Hive, Spark, YARN
  • Programming Language (Must have) – Scala, Python, Shell Scripts, PySpark
  • Operating System (Must have) - Any flavor of Linux, Windows
  • Must have very strong SQL Skills
  • Orchestration Tools (Must have): Apache Airflow
  • Expertise in developing ETL workflows comprising complex transformations like SCD, deduplications, aggregations etc.
  • Should have thorough conceptual understanding of AWS VPC, Subnets, Security Groups, & Route Tables
  • Should be a quick and self-learner and be ready to adapt to new AWS Services or new Big Data Technologies as and when required

Qualities:

• Hold strong technical knowledge and experience
• Should have the capability to deep dive and research in various technical related fields
• Self-motivated and focused on delivering outcomes for a fast-growing team and firm
• Able to communicate persuasively through speaking, writing, and client presentations
• Able to consult, write, and present persuasively
• Able to work in a self-organized and cross-functional team
• Able to iterate based on new information, peer reviews, and feedback
• Prior experience of working in a large media company would be added advantage