Data Pipeline Specialist

15 hours ago

Varanasi, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 35,00,000
Oil and Gas Data Engineer Opportunity

Job Description:

We are seeking a seasoned Oil and Gas Data Engineer with expertise in designing, building, and optimizing scalable data pipelines and cloud solutions. The ideal candidate will have hands-on experience with AWS cloud services, including S3, Redshift, Glue, Athena, EMR, Lake Formation, and Lambda.

Key Responsibilities:

  1. Design and develop scalable data pipelines and ETL/ELT solutions using AWS cloud services.
  2. Ingest, process, and store structured/unstructured data from multiple sources (IoT devices, sensors, ERP systems, drilling logs, production data, etc.).
  3. Implement data lake and data warehouse solutions.
  4. Optimize data pipelines for performance, cost, and scalability.
  5. Collaborate with domain experts, data scientists, and business analysts to translate Oil and Gas business requirements into data solutions.
  6. Ensure data governance, data quality, and security compliance.
  7. Support real-time data streaming and analytics using AWS services (Kinesis, Kafka, Lambda).
  8. Document technical solutions and provide knowledge transfer to the team.

Required Skills and Qualifications:

  • AWS services: S3, Redshift, Glue, Athena, EMR, Lake Formation.
  • Lambda, Kinesis, Step Functions, CloudWatch, RDS, DynamoDB (optional).
  • ETL/ELT development using PySpark, Python, or SQL.
  • Data modeling, warehousing, and performance tuning.
  • Data lake and modern data architecture.
  • Strong SQL skills and ability to work with large datasets.
  • Hands-on with CI/CD and DevOps practices for data solutions.
  • Familiarity with Oil and Gas industry datasets and use cases (production, drilling, exploration, sensor/IoT data).

Benefits:

The successful candidate will have the opportunity to work on high-profile projects, collaborate with experienced professionals, and contribute to the growth and development of the company's data engineering capabilities.

  • Data Pipeline Specialist

    2 days ago

    Varanasi, Uttar Pradesh, India beBeeDataEngineer Full time US$ 1,20,000 - US$ 1,50,000

    Job Description:As a skilled data engineer, you will be responsible for designing and implementing robust data pipelines using Python and SQL. You will work with modern DevOps/configuration management tools to manage multi-environment deployments.Design, implement, and manage CI/CD pipelines using Github Actions or equivalent tools.Configure and maintain...

  • Principal Data Pipeline Specialist

    3 days ago

    Varanasi, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Overview:We are seeking an experienced Data Engineer to design, develop, and maintain large-scale data pipelines using Azure Databricks and PySpark. This role involves collaboration with cross-functional teams to integrate data engineering solutions with business requirements, ensuring data quality, security, and compliance throughout the data...

  • Senior Data Pipeline Specialist

    2 days ago

    Varanasi, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 27,00,000

    Meet the requirements for a senior data engineer position to create and maintain ETL/ELT pipelines in our organization.Key ResponsibilitiesPipeline Development: Design, build, and manage efficient ETL/ELT pipelines using AWS services like Lambda, DMS, and Glue.Data Transformation: Implement complex business logic and data transformations utilizing Snowflake...

  • Cloud Data Pipelines Specialist

    8 hours ago

    Varanasi, Uttar Pradesh, India beBeeDataEngineer Full time US$ 1,80,000 - US$ 2,00,000

    Job Title:">Data Engineer - Cloud Integration">Description:">We are seeking a skilled professional to design, build, and maintain scalable data pipelines using Azure Data Factory, Azure Synapse Analytics, and Azure Data Lake.">This role requires expertise in data governance frameworks, ensuring data quality across all pipelines.">The ideal candidate will...

  • Data Pipelines Expert

    20 hours ago

    Varanasi, Uttar Pradesh, India beBeedataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job TitleA Data Engineer is needed to design, build and maintain high-quality, secure and scalable data pipelines using various technologies.

  • Expert Data Pipeline Developer

    3 hours ago

    Varanasi, Uttar Pradesh, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Remote Data Engineering OpportunityThreatXIntel is a cyber security startup that specializes in protecting businesses and organizations from cyber threats. Our team provides various services, such as cloud security, web and mobile security testing, and DevSecOps. We aim to deliver customized and affordable solutions tailored to meet the specific needs of our...

  • AI Pipeline Specialist

    6 days ago

    Varanasi, Uttar Pradesh, India beBeeMachine Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    AI Pipeline SpecialistAs a pivotal member of our team, you will play a critical role in designing, implementing, and deploying cutting-edge Artificial Intelligence (AI) and Machine Learning (ML) pipelines across various cloud environments.Key ResponsibilitiesDesign and develop AI/ML pipelines using AWS services like SageMaker, Fargate, and Bedrock.Apply...

  • Chief Data Pipeline Architect

    3 hours ago

    Varanasi, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    Key Role in Data EngineeringWe are seeking a highly skilled Data Engineer to play a key role in designing, building, and maintaining scalable data pipelines and infrastructure.Our ideal candidate will have expertise in big data technologies and be able to design robust data pipelines and ETL workflows.The successful candidate will work closely with analytics...

  • Senior Data Engineer

    9 hours ago

    Varanasi, Uttar Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 21,00,000

    Job OpportunityWe are seeking a skilled data expert to take on the challenge of designing, building, and maintaining robust data pipelines for financial data.

  • Data Architect Specialist

    24 hours ago

    Varanasi, Uttar Pradesh, India beBeeDataArchitect Full time US$ 80,000 - US$ 1,40,000

    Data Architect SpecialistAre you a data expert looking for a challenging role in the field of data engineering? Do you have experience with designing and developing scalable data pipelines using Python and AWS services such as S3, Redshift, Lambda, and Glue? If so, we want to hear from you!Key Responsibilities:Data Pipeline Development: Develop and maintain...