Senior Cloud Data Engineer

21 hours ago

Ellore, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 1,58,40,000 - ₹ 2,58,40,000
Job Description:

We are seeking a seasoned Data Engineer to join our team. As a key member of our data infrastructure group, you will be responsible for designing, implementing, and maintaining scalable data pipelines and building robust data lakes.

  • Design and Development:
    • Design and develop scalable ETL pipelines using cloud-native tools such as AWS DMS, AWS Glue, Kafka, Azure Data Factory, GCP Dataflow, etc.
    • Architect and implement data lakes and data warehouses on cloud platforms (AWS, Azure, GCP).
    • Develop and optimize data ingestion, transformation, and loading processes using Databricks, Snowflake, Redshift, BigQuery, and Azure Synapse.
    • Implement ETL processes using tools like Informatica, SAP Data Intelligence, and others.
    • Develop and optimize data processing jobs using Spark Scala.
  • Data Integration and Management:
    • Integrate various data sources, including relational databases, APIs, unstructured data, and ERP systems into the data lake.
    • Ensure data quality and integrity through rigorous testing and validation.
    • Perform data extraction from SAP or ERP systems when necessary.
  • Performance Optimization:
    • Monitor and optimize the performance of data pipelines and ETL processes.
    • Implement best practices for data management, including data governance, security, and compliance.
  • Collaboration and Communication:
    • Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.
    • Collaborate with cross-functional teams to design and implement data solutions that meet business needs.
  • Documentation and Maintenance:
    • Document technical solutions, processes, and workflows.
    • Maintain and troubleshoot existing ETL pipelines and data integrations.

Required Skills and Qualifications:

  • 7+ years of experience as a Data Engineer or in a similar role.
  • Proven experience with cloud platforms: AWS, Azure, and GCP.
  • Hands-on experience with cloud-native ETL tools such as AWS DMS, AWS Glue, Kafka, Azure Data Factory, GCP Dataflow, etc.
  • Experience with other ETL tools like Informatica, SAP Data Intelligence, etc.
  • Experience in building and managing data lakes and data warehouses.
  • Proficiency with data platforms like Redshift, Snowflake, BigQuery, Databricks, and Azure Synapse.
  • Experience with data extraction from SAP or ERP systems is a plus.
  • Strong experience with Spark and Scala for data processing.

Benefits:

  • Remote working opportunities
  • Flexible work arrangements
  • Opportunity to learn and run the latest industry-standard tools
  • A subscription to A Cloud Guru

What We Offer:

We are an equal opportunity employer.

  • Senior Cloud Data Engineer

    4 days ago

    Ellore, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job DescriptionWe are seeking a skilled professional to fill the role of Senior Cloud Data Engineer. This is an exciting opportunity to work on complex projects and contribute to the growth of our organization.Experience: 10+ yearsWork Location: Remote, IndiaShift: Flexible hoursNotice Period: Immediate to 15 daysThe ideal candidate will have expertise in...

  • Cloud Data Engineer

    4 days ago

    Ellore, Andhra Pradesh, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Senior Data Engineer OpportunityGreeting!We are seeking a Senior Data Engineer with experience in GCP and ETL to join our team.The successful candidate will have a strong background in data engineering, with expertise in designing, building, and maintaining large-scale data pipelines using GCP services such as BigQuery, Cloud Storage, Pub/Sub, and...

  • Cloud Data Engineer Leader

    5 days ago

    Ellore, Andhra Pradesh, India beBeeDataSystems Full time ₹ 15,00,000 - ₹ 20,00,000

    Senior Data Systems ArchitectDesign and build scalable data processing systems that ensure data is accessible, reliable, and optimized for analytics and machine learning initiatives. Collaborate with cross-functional teams to drive business growth through effective data management.Key Responsibilities:Develop ETL/ELT pipelines using Cloud Dataflow, Apache...

  • Senior Cloud Data Engineer

    6 days ago

    Ellore, Andhra Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Cloud Data EngineerAs a key member of our data engineering team, you will play a pivotal role in designing and developing scalable data pipelines using Google Cloud Platform (GCP) services. Your primary responsibility will be to collaborate with cross-functional teams to deliver high-quality data solutions that meet the organization's business objectives.Key...

  • Cloud Data Engineer

    6 days ago

    Ellore, Andhra Pradesh, India beBeeCloudDataEngineer Full time ₹ 25,00,000 - ₹ 30,00,000

    Role: Cloud Data EngineerTotal years of experience: 12+ Years in cloud data engineering.Location: Major cities across India.Relevant experience for engagement: 10 Years in designing & implementing solutions.Relevant skillset:Maintain architecture principles, guidelines and standards for big data engineering.Data Warehousing on Google Cloud...

  • Senior Data Engineer

    4 days ago

    Ellore, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Description:We are seeking a highly skilled Senior Data Engineer to lead our cloud architecture team. As a key member of our organization, you will design and develop scalable data pipelines using AWS services.You will have a strong understanding of AWS services like EC2, S3, EMR, Glue, and Redshift, with hands-on experience in developing applications...

  • Senior Cloud Architect for Data Engineering

    6 days ago

    Ellore, Andhra Pradesh, India beBeeData Full time ₹ 20,00,000 - ₹ 30,00,000

    Cloud Data Engineer PositionThe role of a cloud data engineer involves designing and deploying scalable, cloud-based data solutions on AWS.Key Responsibilities:Designing, developing, and deploying scalable cloud-based data solutions on AWS.Implementing end-to-end cloud data engineering solutions like enterprise data lakes and data hubs in AWS.Developing...

  • Cloud Data Engineer

    4 days ago

    Ellore, Andhra Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000

    Cloud Data Engineering SpecialistWe are seeking a highly skilled data engineer to design, develop and optimize data pipelines and integration solutions in the cloud.Key Responsibilities:Design and build advanced ETL pipelines for data ingestion and egress.Develop scalable data solutions using Azure services and other cloud-based tools.Troubleshoot data jobs...

  • Senior Cloud Data Engineer

    1 week ago

    Ellore, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Summary:A seasoned data engineering professional with 6+ years of experience in designing, developing, and optimizing ETL/ELT data pipelines using Apache Spark (PySpark or Scala), AWS Glue, and Azure Data Factory is required for this role.The ideal candidate will collaborate with cross-functional teams to build scalable ingestion and transformation...

  • Senior Data Engineer

    6 days ago

    Ellore, Andhra Pradesh, India beBeeMachineLearningEngineer Full time ₹ 1,92,00,000 - ₹ 2,56,00,000

    As a Senior Data Engineer, you will play a crucial role in enhancing our machine learning capabilities and credit risk management processes.Key Responsibilities:Design and implement end-to-end data engineering architecture, including model deployment and production maintenance.Develop and maintain scalable data models using Python, PySpark, NumPy, Pandas,...