Cloud Data Engineer with Expertise in Scalable ETL Pipelines

1 week ago

Bikaner, Rajasthan, India beBeeCloudDataEngineer Full time ₹ 18,00,000 - ₹ 20,00,000
About the Role

We are seeking an experienced Cloud Data Engineer to join our team. The ideal candidate will have extensive experience with cloud-native ETL tools such as AWS DMS, AWS Glue, Kafka, Azure Data Factory, GCP Dataflow, and other ETL tools like Informatica, SAP Data Intelligence.

The successful candidate will design and develop scalable ETL pipelines using cloud-native tools, architect and implement data lakes and data warehouses on cloud platforms, and develop and optimize data ingestion, transformation, and loading processes using Databricks, Snowflake, Redshift, BigQuery and Azure Synapse.

Key Responsibilities:

  • Data Integration and Management
    • Integrate various data sources, including relational databases, APIs, unstructured data, and ERP systems into the data lake.
    • Ensure data quality and integrity through rigorous testing and validation.
    • Perform data extraction from SAP or ERP systems when necessary.
  • Performance Optimization
    • Monitor and optimize the performance of data pipelines and ETL processes.
    • Implement best practices for data management, including data governance, security, and compliance.
  • Collaboration and Communication
    • Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.
    • Collaborate with cross-functional teams to design and implement data solutions that meet business needs.
  • Documentation and Maintenance
    • Document technical solutions, processes, and workflows.
    • Maintain and troubleshoot existing ETL pipelines and data integrations.
Requirements

To be considered for this role, you should have:

  • Bachelor's degree in Computer Science, Information Technology, or a related field.
  • 7+ years of experience as a Data Engineer or in a similar role.
  • Proven experience with cloud platforms: AWS, Azure, and GCP.
  • Hands-on experience with cloud-native ETL tools such as AWS DMS, AWS Glue, Kafka, Azure Data Factory, GCP Dataflow, etc.
  • Experience with other ETL tools like Informatica, SAP Data Intelligence, etc.
  • Experience in building and managing data lakes and data warehouses.
  • Proficiency with data platforms like Redshift, Snowflake, BigQuery, Databricks, and Azure Synapse.
  • Experience with data extraction from SAP or ERP systems is a plus.
  • Strong experience with Spark and Scala for data processing.
  • Strong programming skills in Python, Java, or Scala.
  • Proficient in SQL and query optimization techniques.
  • Familiarity with data modeling, ETL/ELT processes, and data warehousing concepts.
  • Knowledge of data governance, security, and compliance best practices.
Preferred Qualifications

A strong candidate will also have:

  • Experience with other data tools and technologies such as Apache Spark, or Hadoop.
  • Certifications in cloud platforms (AWS Certified Data Analytics – Specialty, Google Professional Data Engineer, Microsoft Certified: Azure Data Engineer Associate).
  • Experience with CI/CD pipelines and DevOps practices for data engineering.
What We Offer

We offer:

  • Remote working.
  • Flexibility to work where and when you like most of the time.
  • A subscription to A Cloud Guru and generous budget for taking certifications and other resources you'll find helpful.
  • An opportunity to learn and run the latest industry standard tools.
  • Colleagues who will challenge you giving the chance to learn from them and them from you.

  • ETL Engineer

    7 days ago

    Bikaner, Rajasthan, India Eficaz HR Services Private Limited Full time

    Role DescriptionThis is a full-time role for an ETL Engineer with one of our renowned client for Pune location.The ETL Engineer will be responsible for designing, deploying, and managing ETL processes to integrate data from various sources. Daily tasks include using ETL tools, performing data modeling, ensuring data quality, and providing analytical...

  • Data Strategist with Cloud Expertise

    5 days ago

    Bikaner, Rajasthan, India beBeeCloudDataStrategist Full time ₹ 75,00,000 - ₹ 1,25,00,000

    Job Title: Data Strategy Manager – Cloud Data PlatformsThe role entails strategic leadership in cloud data architecture and pipeline management. As a collaborative employee based in Hyderabad, India, you will contribute to our global data initiatives and partner with stakeholders across teams.Develop and execute product strategy, backlog, and roadmap for...

  • Data Engineering Expertise Wanted

    1 week ago

    Bikaner, Rajasthan, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Azure Data Engineer Job DescriptionWe are seeking an experienced Data Engineering Professional to join our team.The ideal candidate will have a strong background in designing and optimizing modern data platforms, with expertise in Databricks and Apache Spark. Key responsibilities include:Leading the architecture, development, and operations of large-scale...

  • Advanced Data Engineer for Scalable Systems

    6 days ago

    Bikaner, Rajasthan, India beBeeCloudBased Full time ₹ 15,00,000 - ₹ 25,00,000

    Job SummaryAzure data engineers design, build, and maintain scalable data systems using Azure technologies such as Synapse Analytics, Data Factory, and Event Grid.Key Responsibilities:Design and implement data pipelines using Azure Data Factory and Synapse AnalyticsDevelop and deploy cloud-based data processing solutions using Azure Functions and...

  • Data Scientist/ml Engineer

    2 weeks ago

    Bikaner, Rajasthan, India Quant-data Full time

    ???? We're Hiring: Machine Learning Engineer / Data Engineer (Remote | Full-Time) Build AI-powered credit decisioning systems on Microsoft AzureWe're looking for a Machine Learning Engineer / Data Engineer with 5+ years of experience to join our AI-driven credit lending platform team.In this role, you'll design and deploy scalable ML solutions that power...

  • Cloud Data Engineering Specialist

    7 days ago

    Bikaner, Rajasthan, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 30,00,000

    Job TitleGoogle Cloud (GCP) Data EngineerKey Responsibilities:Develop and implement scalable data pipelines on GCP using BigQuery, GCS, and Airflow.Design robust ETL/ELT processes in Python, integrating with data orchestration tools such as Apache Airflow.Collaborate with cross-functional teams to understand data requirements and deliver high-quality data...

  • Cloud Data Solutions Engineer

    7 days ago

    Bikaner, Rajasthan, India beBeeDataEngineer Full time ₹ 90,00,000 - ₹ 1,50,00,000

    Job OverviewWe are seeking skilled professionals to design, develop and manage robust data pipelines that power our AI solutions.Key Responsibilities:Design, develop and maintain efficient ETL/ELT workflows to process structured, unstructured and streaming data.Leverage cloud services for scalable data ingestion, transformation, storage and querying.Ensure...

  • Chief Cloud Data Architect

    3 days ago

    Bikaner, Rajasthan, India beBeeData Full time ₹ 1,00,00,000 - ₹ 1,50,00,000

    Cloud Data Engineer Job DescriptionAbout this Role:This position is responsible for designing, developing, and maintaining scalable ETL pipelines using cloud-native tools. The ideal candidate will have experience with data lakes and data warehouses on cloud platforms, as well as expertise in data ingestion, transformation, and loading processes.Key...

  • Cloud Solutions Data Engineer

    6 days ago

    Bikaner, Rajasthan, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title:Data Engineer - Cloud SolutionsWe are seeking a highly skilled Data Engineer with 3–4 years of experience in building, optimizing, and maintaining large-scale data processing solutions using PySpark, Python, SQL, and real-time data streaming.Design and develop scalable data pipelines using PySpark and Python to meet business...

  • Senior Cloud Data Integration Specialist

    6 days ago

    Bikaner, Rajasthan, India beBeeDataIntegration Full time ₹ 10,00,000 - ₹ 15,00,000

    We are seeking a highly skilled Data Integration Expert to lead the design and development of cloud-based ETL pipelines using Azure services. The ideal candidate will have extensive experience integrating data into modern data platforms like AWS Datalake.Key Responsibilities:Design, develop, and deploy scalable and efficient ETL pipelines on Azure...