AWS Data Pipeline Specialist

1 day ago

Jamnagar, Gujarat, India beBeeData Full time ₹ 1,80,00,000 - ₹ 2,50,00,000
Job Description

We are seeking a skilled Data Architect to design, develop, and maintain scalable data pipelines and ETL/ELT solutions using AWS cloud services.

  • Ingest, process, and store structured/unstructured data from multiple sources (IoT devices, sensors, ERP systems, drilling logs, production data, etc.).

The ideal candidate will implement data lake and data warehouse solutions that optimize data pipelines for performance, cost, and scalability.

  • Collaborate with domain experts, data scientists, and business analysts to translate Oil & Gas business requirements into data solutions.

Sustain data governance, data quality, and security compliance while supporting real-time data streaming and analytics using AWS services (Kinesis, Kafka, Lambda).

  • Document technical solutions and provide knowledge transfer to the team.

Proficiency in AWS services: S3, Redshift, Glue, Athena, EMR, Lake Formation - Lambda, Kinesis, Step Functions, CloudWatch, RDS, DynamoDB (optional).

  • Expertise in ETL/ELT development using PySpark, Python, or SQL.

Experience with data modeling, warehousing, and performance tuning as well as familiarity with Oil & Gas industry datasets and use cases.

The selected candidate will possess hands-on experience with CI/CD and DevOps practices for data solutions and strong SQL skills along with ability to work with large datasets. Additionally, they should have knowledge of data bricks & Azure Data Factory, SQL knowledge, incremental & append data load, understanding data ingestion patterns and comprehensive expertise in AWS ecosystem.

  • AWS Cloud Architect Specialist

    7 days ago

    Jamnagar, Gujarat, India beBeeCloud Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job Title: AWS Cloud Architect SpecialistWe are seeking a skilled AWS Data Engineer to design, develop, and deploy end-to-end data pipelines on the Amazon Web Services (AWS) cloud infrastructure.The ideal candidate will have extensive experience in designing and building large-scale data processing pipelines in a production environment.Key...

  • Data Pipeline Developer

    17 hours ago

    Jamnagar, Gujarat, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Data Pipeline DeveloperWe are seeking a skilled Data Pipeline Developer to join our team. The ideal candidate will have experience designing, developing, testing, and supporting data pipelines and applications using AWS services.The successful candidate will design and develop robust ETL pipelines using PySpark, focusing on performance,...

  • Senior Data Pipeline Specialist

    5 days ago

    Jamnagar, Gujarat, India beBeeData Full time ₹ 12,00,000 - ₹ 16,00,000

    About the RoleWe're looking for a highly skilled Data Engineering professional with expertise in building scalable data pipelines using AWS services.Key Responsibilities:Design and develop production-level data pipelines handling large volumes of structured and unstructured data.Own end-to-end ETL/ELT processes ensuring reliability, scalability, and high...

  • AWS Data Architect

    2 hours ago

    Jamnagar, Gujarat, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 25,50,000

    Job TitleWe are seeking a highly skilled Data Engineer to join our data engineering team. The ideal candidate will have deep expertise in building scalable data pipelines using Apache Spark, PySpark, SQL, and Python, along with hands-on experience in the AWS ecosystem.Design, develop, and maintain robust ETL/ELT pipelines on AWS using Spark and...

  • Cloud Data Engineer

    3 days ago

    Jamnagar, Gujarat, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    We are seeking a talented professional to lead the design and implementation of scalable data pipelines and ETL solutions in a cloud environment. The ideal candidate will have hands-on experience with AWS services, including EMR, Glue, PySpark, Airflow, and Python.Requirements:A minimum of 5 years of experience in data engineering or related roles.Strong...

  • Cloud Data Pipeline Architect

    5 days ago

    Jamnagar, Gujarat, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    We are seeking a highly motivated and experienced Data Engineer to design, build, and maintain robust data pipelines. This role requires expertise in cloud-based data processing technologies using GCP, AWS, or Azure.Key ResponsibilitiesDesign, develop, and maintain scalable data pipelines using cloud services.Implement data ingestion, transformation, and...

  • Validating Complex Data Pipelines

    1 day ago

    Jamnagar, Gujarat, India beBeeDataValidator Full time ₹ 15,00,000 - ₹ 24,00,000

    ETL Data Validation SpecialistAbout the RoleEvaluate and validate data pipelines and transformations using Databricks, Python scripting, and strong SQL skills.Develop expertise in ETL testing methodologies, debugging data issues, and working with large datasets in cloud environments.The ideal candidate will have hands-on experience with Databricks, Python,...

  • AWS Data Engineer

    9 hours ago

    Jamnagar, Gujarat, India Coforge Full time

    Responsibilities-* Build and maintain data pipelines using Glue, Python, EMR/Spark and Kafka/Kinesis streaming.* Design and develop relational databases, with a preference for Aurora Postgres.* Write complex SQL queries and stored procedures.* Troubleshoot and resolve data quality issues.* Work with stakeholders to understand data requirements and develop...

  • Data Pipeline Specialist

    1 day ago

    Jamnagar, Gujarat, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Senior Data EngineerDeveloping and Maintaining Scalable Data Pipelines:The ideal candidate will be responsible for developing and maintaining scalable data pipelines to support increasing data volume and complexity.Key Responsibilities:Design efficient ETL pipelines following business requirements and development standards.Perform integration...

  • Data Pipeline Specialist

    3 days ago

    Jamnagar, Gujarat, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 30,00,000

    Job Description">We seek a seasoned Data Engineer to optimize our data infrastructure and deliver robust data pipelines.Key Responsibilities:">Design, implement, and maintain Continuous Integration/Continuous Deployment (CI/CD) pipelines using GitHub Actions or equivalent tools.Configure and maintain GitHub branch protection rules and workflows to ensure...