Senior Big Data Specialist

1 week ago

Jamnagar, Gujarat, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

Job Title: Data Engineer

This is a contract opportunity to develop and maintain large-scale data systems.

The ideal candidate will have at least 4 years of experience in data engineering with expertise in Python, Spark, and PySpark.

Key Responsibilities:

  • Design, build, and optimize complex data pipelines and workflows.
  • Manage and process massive datasets using Spark, PySpark, and SQL.
  • Build and maintain real-time streaming applications with Spark Streaming/Kafka.
  • Collaborate with data scientists and product teams to integrate AI/ML models into production.
  • Ensure high-quality data processing, scalability, and performance in all pipelines.
  • Deploy and manage Spark workloads on cloud platforms (AWS, Azure, GCP, Databricks).
  • Automate testing and deployment of Spark jobs via CI/CD pipelines.

Requirements:

  • Bachelor's/Master's degree in Computer Science, Computer Engineering, or related field.
  • Minimum 4 years of professional experience as a Data Engineer.
  • Strong expertise in Python, Spark, PySpark.
  • Hands-on experience with Spark SQL, DataFrames, UDFs, DAG execution.
  • Knowledge of data ingestion tools (Kafka, Flume, Kinesis) and data formats (Parquet, Avro, ORC).
  • Proficiency in Airflow/Prefect for scheduling and orchestration.
  • Familiarity with performance tuning in Spark (partitioning, caching, broadcast joins).
  • Experience deploying on Databricks, AWS EMR, Azure HDInsight, or GCP Dataproc.
  • Exposure to testing and CI/CD for data pipelines (pytest, Jenkins, GitHub Actions).

What We Offer:

A challenging role with opportunities for growth and development, working with cutting-edge technologies and collaborating with experienced professionals.

Why You'll Thrive:

This position requires a strong passion for data engineering, excellent problem-solving skills, and the ability to work effectively in a team environment.

],

  • Big Data Engineering Expert

    2 days ago

    Jamnagar, Gujarat, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Opportunity OverviewWe are seeking a skilled Big Data Specialist to join our team. The ideal candidate will have expertise in designing and developing scalable data pipelines using Big Data technologies.Key Skills We're Looking For:6-12 years of experience in Big Data and related technologies.Strong understanding of data structures and algorithms.Experience...

  • Big Data Solutions Architect

    1 week ago

    Jamnagar, Gujarat, India beBeeDataEngineer Full time ₹ 1,80,00,000 - ₹ 2,01,00,000

    Job Title: Big Data Solutions ArchitectWe are seeking an experienced Senior Data Engineer to help us implement a cutting-edge data platform for a large retailer.Key Responsibilities:Design and develop scalable big data solutions using Python, Pyspark, and SQL.Work closely with cross-functional teams to integrate data engineering solutions into existing...

  • Big Data Transformation Specialist

    11 hours ago

    Jamnagar, Gujarat, India beBeeDataExpert Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Lead ETL/Big Data TesterWe are seeking a skilled and experienced professional to join our team as a Lead ETL/Big Data Tester. The ideal candidate will have expertise in writing and executing User Acceptance tests in QA/Production labs with multiple machines/VMs.

  • Big Data Engineer

    6 days ago

    Jamnagar, Gujarat, India beBeeBigData Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Big Data EngineerWe are seeking a seasoned professional with expertise in big data platforms, tools, and technologies to drive business growth through data-driven insights.The ideal candidate will possess a deep understanding of large-scale data processing, data warehousing, and storage solutions. Key responsibilities include:Must-Have...

  • Senior Big Data Solutions Specialist

    6 days ago

    Jamnagar, Gujarat, India beBeeDataEngineer Full time ₹ 30,00,000 - ₹ 40,00,000

    Lead Data EngineerWe are seeking a Lead Data Engineer to spearhead the development and optimization of ETL/ELT pipelines using Azure Data Factory (ADF), Databricks, and Synapse Analytics.About the RoleDesign and implement scalable data architectures following Medallion Architecture and best practices.Perform data ingestion, transformation, and orchestration...

  • Big Data Solutions Architect

    1 week ago

    Jamnagar, Gujarat, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Title: Data EngineerAbout the Role:We are seeking a skilled and experienced Data Engineer to join our team. As a Data Engineer, you will play a key role in designing, building, and maintaining large-scale data systems.Key Responsibilities:Design and develop scalable data architectures using AWS services such as Amazon EMR, Amazon Athena, and AWS...

  • Big Data Expert

    6 days ago

    Jamnagar, Gujarat, India beBeeData Full time US$ 1,80,000 - US$ 2,00,000

    Key Responsibilities:Develop and maintain data pipelines using Python and AWS (S3, Redshift, Lambda, Glue).Implement and optimize ETL processes for large datasets.Ensure data integrity, security, and accessibility.Collaborate with teams to meet business data needs.With 5+ years of experience in data engineering, the ideal candidate will have a strong...

  • Big Data Developer

    2 days ago

    Jamnagar, Gujarat, India beBeeData Full time ₹ 8,00,000 - ₹ 12,50,000

    Job Title: Data ScientistWe are seeking a highly skilled and motivated professional to join our team in a Data Scientist role.Key Responsibilities:Develop and maintain scalable data pipelines using Scala, Spark, and Hadoop Big Data Eco System Tools.Design and implement efficient database queries using strong SQL experience.Work on performance enhancement and...

  • Senior Data Infrastructure Specialist

    12 hours ago

    Jamnagar, Gujarat, India beBeeDataEngineer Full time ₹ 40,00,000 - ₹ 50,00,000

    Job TitleAs a senior data engineer, you will play a crucial role in designing and implementing scalable, reliable, and secure data infrastructure for large organizations.Design and implement data platforms that meet the needs of business stakeholders.Develop and maintain data pipelines to ensure timely and accurate data delivery.Collaborate with...

  • Highly Scalable Big Data Platform Lead

    6 days ago

    Jamnagar, Gujarat, India beBeeTechnical Full time US$ 1,50,000 - US$ 2,00,000

    Lead Big Data EngineerThis is a pivotal role where you will be responsible for building a highly-scalable and extensible big data platform that provides the foundation for collecting, storing, modeling, and analyzing massive data sets from multiple channels.The successful candidate will have a proven track record of relevant work experience and a computer...