Large Scale Data Processing Specialist

23 hours ago

Vadodara, Gujarat, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000
Data Engineer Position Overview

We are seeking a skilled Data Engineer to design, develop, and maintain large-scale data processing systems.

  • This role involves designing and implementing scalable data pipelines using cloud-based services.
  • Developing robust data models using big data tools and technologies is a key responsibility.
  • Collaborating with cross-functional teams to integrate data products into agile software development methodologies is essential.
  • Maintaining high standards of data quality, security, and compliance is crucial.

The ideal candidate will have 5+ years of SQL development experience and 3+ years of experience working with Google Cloud Platform (GCP) services.

A bachelor's degree in computer science or a related technical field is required. Additionally, the candidate should have experience working with airflow for scheduling and orchestration of data pipelines and terraform for infrastructure as code.

Beyond the technical requirements, excellent problem-solving skills, strong communication abilities, and a willingness to learn and adapt to new technologies are highly valued.

The selected candidate will work closely with our team to drive business growth through data-driven insights and strategic decision-making.

This is an exciting opportunity for an experienced data professional to join our organization and contribute to the success of our data initiatives.

Responsibilities:

  1. Design and implement scalable data pipelines using GCP services
  2. Develop and maintain robust data models using big data tools and technologies
  3. Collaborate with cross-functional teams to integrate data products into agile software development methodologies
  4. Maintain high standards of data quality, security, and compliance

Requirements:

  • Bachelor's degree in Computer Science or a related technical field
  • 5+ years of SQL development experience
  • 3+ years of Google Cloud Platform (GCP) services experience
  • Experience working with airflow for scheduling and orchestration of data pipelines
  • Experience working with terraform for infrastructure as code
  • Excellent problem-solving skills, strong communication abilities, and a willingness to learn and adapt to new technologies

  • Mastering Large-Scale Data Engineering

    7 days ago

    Vadodara, Gujarat, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data EngineerThe ideal candidate will have 3-5 years of experience in data engineering and a strong background in distributed data processing and storage.Key Responsibilities:Design, develop, and maintain large-scale data pipelines using Python, SQL, and cloud-based technologies.Collaborate with cross-functional teams to integrate data into...

  • Large Scale Cloud Migration Expert

    2 weeks ago

    Vadodara, Gujarat, India beBeeCloudMigration Full time ₹ 25,00,000 - ₹ 30,00,000

    Senior Cloud Migration SpecialistAbout the Role:We are seeking an experienced Senior Cloud Migration Specialist to lead large-scale cloud migration projects. The ideal candidate will have a strong background in migrating on-premise data systems to the cloud.The Key Responsibilities include:Design and develop robust and scalable data pipelines for migrating...

  • Senior Large-Scale Data System Specialist

    2 weeks ago

    Vadodara, Gujarat, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    As a seasoned data engineer, you will play a pivotal role in designing and building large-scale data systems.Key Responsibilities:Design and Develop Data PipelinesOptimize SQL Queries for PerformanceCollaborate with Cross-Functional TeamsDevelop Analytical Skills to Tackle Complex Data ChallengesRequirements:Strong Experience with Trino...

  • Chief Architect

    2 weeks ago

    Vadodara, Gujarat, India beBeePlatform Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Job OverviewAs a Lead Python Platform Engineer, you will be responsible for designing and building a shared component library/SDK for pipelines, as well as defining patterns/templates for Apache Beam pipelines and Databricks jobs.Key Responsibilities:Design and build a shared component library/SDK for pipelines: ingestion, parsing/OCR, extraction,...

  • ServiceNow Developer

    3 days ago

    Vadodara, Gujarat, India beBeeService Full time ₹ 20,00,000 - ₹ 25,00,000

    Order Management Opportunity">We are seeking an experienced ServiceNow developer to support a large-scale transformation project.">Key Responsibilities">">Design and implement Order Management processes in ServiceNow.">Implement ServiceNow OM and CSM modules, including order capture and orchestration.">Collaborate with technical leads to ensure best...

  • Data Architect

    1 week ago

    Vadodara, Gujarat, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 25,00,000

    Big Data Engineer RoleJob Overview:As a key member of our team, you will be responsible for designing, developing, and deploying scalable data processing systems using Apache Spark and Hadoop technologies.Key Responsibilities include design and implementation of big data pipelines, development and maintenance of large-scale data processing systems,...

  • Data Specialist

    23 hours ago

    Vadodara, Gujarat, India beBeeDataSpecialist Full time ₹ 80,00,000 - ₹ 1,20,00,000

    We are seeking a skilled Data Specialist to spearhead our data collection initiatives from the ground up.Key ResponsibilitiesData Extraction & Processing: Develop and maintain automated systems to extract structured and unstructured data from websites, APIs, and public datasets.Scalable Systems: Create multi-threaded, distributed systems capable of handling...

  • Data Engineer

    3 days ago

    Vadodara, Gujarat, India beBeeBackend Full time ₹ 12,00,000 - ₹ 25,00,000

    Senior Data Integration SpecialistDesign and implement scalable data pipelines, including crawling, parsing, deduplication, and embeddings.Connect external systems and interfaces to enhance data quality and efficiency.The ideal candidate will have a strong background in backend/data engineering, including experience with design and implementation of data...

  • Freelance Healthcare Data Insights Specialist

    1 week ago

    Vadodara, Gujarat, India beBeeData Full time ₹ 1,20,00,000 - ₹ 2,00,00,000

    Healthcare Data Insights SpecialistJob Description:ThreatXIntel, a leading cyber security firm, seeks an experienced Freelance Data Scientist to support business growth through data-driven decision-making. The ideal consultant will leverage expertise in SQL, Python, and healthcare claims/data analysis to uncover actionable insights.Extract, clean, and...

  • Big Data Processing Expert

    5 days ago

    Vadodara, Gujarat, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Data EngineerWe are seeking a skilled Data Engineer to join our organization.Key Responsibilities:Create and maintain high-performance data processing pipelines using big data technologies like HIVE, HBase, Impala, and Parquet.Analyze complex data sets and design efficient data storage and retrieval systems.Collaborate with cross-functional teams to...