Senior Data Engineer

The purpose of this role is to maintain, improve, clean and manipulate data in the business's operational and analytics databases. The Data Engineer works with the business's software engineers, data analytics teams, data scientists and data warehouse engineers in order to understand and aid in the implementation of database requirements, analyse performance, and troubleshoot any existent issues.

Job Description:

Role Overview:
We are seeking a highly skilled and motivated Senior GCP Data Engineer to join our team. The role is critical to the development of a cutting-edge reporting and insights platform.

The GCP Data Engineer will design, implement, and maintain scalable, reliable, and efficient data solutions on Google Cloud Platform (GCP). The role focuses on enabling data-driven decision-making by developing ETL/ELT pipelines, managing large-scale datasets, and optimizing data workflows. The ideal candidate is a proactive problem-solver with strong technical expertise in GCP, a passion for data engineering, and a commitment to delivering high-quality solutions aligned with business needs.

Key Responsibilities:

1. Data Engineering & Development:

• Design, build, and maintain scalable ETL/ELT pipelines for ingesting, processing, and transforming structured and unstructured data.
• Implement enterprise-level data solutions using GCP services such as BigQuery, Cloud Storage, Dataflow, Cloud Functions, Cloud Pub/Sub, and Cloud Composer.
• Leverage BigQuery agentic development approaches for dynamic, metadata-driven ELT pipeline creation (big plus).
• Develop RESTful APIs using Python and Flask to expose data services and integrate with upstream/downstream systems.
• Enable secure API-based interactions between data pipelines, applications, and external interfaces.

2. Cloud Infrastructure Management:

• Manage and deploy GCP infrastructure components to enable seamless data workflows.
• Ensure data solutions are robust, scalable, and cost-effective, leveraging GCP best practices.

3. Collaboration and Stakeholder Engagement:

• Work closely with cross-functional teams, including data analysts, data scientists, DevOps, and business stakeholders, to deliver data projects aligned with business goals.
• Translate business requirements into scalable, technical solutions while collaborating with team members to ensure successful implementation.

4. Quality Assurance & Optimization:

• Implement best practices for data governance, security, and privacy, ensuring compliance with organizational policies and regulations.
• Conduct thorough quality assurance, including testing and validation, to ensure the accuracy and reliability of data pipelines.
• Monitor and optimize pipeline performance to meet SLAs and minimize operational costs.

Qualifications and Certifications:

Education:

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.

Experience:

• Minimum of 3–6 years of experience in data engineering, with at least 2 years working on GCP cloud platforms.
• Proven experience designing and implementing data workflows using GCP services like BigQuery, Cloud Dataflow, Cloud Pub/Sub, and Cloud Composer.
• Experience developing and deploying RESTful APIs using Flask for data integration and system interoperability is mandatory.
• Exposure to real-time/streaming frameworks (Kafka, Spark Streaming, or Beam) in addition to batch pipelines.

Mandatory Skills:

• Advanced proficiency in Python for data pipelines and automation.
• Strong SQL skills for querying, transforming, and analyzing large datasets.
• Expertise in GCP services such as BigQuery, Cloud Functions, Cloud Storage, Dataflow, DBT.
• Experience developing and deploying RESTful APIs using Flask for data integration and system interoperability is mandatory.
• Hands-on experience with CI/CD tools such as Jenkins, Git, or Bitbucket.
• Familiarity with workflow orchestration tools like Apache Airflow or Cloud Composer.
• Google Cloud Professional Data Engineer certification preferred.

Nice-to-Have Skills:

• BigQuery agentic development of ELT pipelines (metadata-driven, automation-assisted).
• Experience with other cloud platforms like AWS or Azure is an added advantage.
• Knowledge of data visualization tools (e.g., PowerBI, Looker, Tableau) is a big plus.
• Understanding of machine learning workflows and their integration with data pipelines.
• Knowledge of monitoring/alerting tools (Stackdriver, Prometheus) to ensure pipeline health, SLAs, and governance.

Location:

DGS India - Bengaluru - Manyata N1 Block

Brand:

Merkle

Time Type:

Full time

Contract Type:

Permanent

---