Big Data Engineer

1 week ago

Vadodara, Gujarat, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000
Job Title: Big Data Engineer

This job posting is for a highly motivated individual with expertise in PySpark, Python, SQL, and real-time data streaming.

Role Overview:

This is a contract remote role that requires building, optimizing, and maintaining large-scale data processing solutions. The ideal candidate will contribute to the design and implementation of scalable data pipelines in cloud environments such as Azure.

Key Responsibilities:
  • Design, develop, and optimize data pipelines using PySpark, Spark, and Python.
  • Implement ETL/ELT solutions to integrate structured and unstructured data from multiple sources.
  • Develop and maintain Hive queries and ensure seamless data transformations.
  • Write and optimize SQL queries for large datasets.
  • Build, monitor, and support real-time streaming solutions using Apache Kafka.
  • Work on Databricks to create, manage, and optimize data workflows.
  • Deploy and manage solutions in Azure cloud services (e.g., Data Lake, Synapse, Azure Databricks).
  • Debug, troubleshoot, and optimize code for performance and scalability.
  • Collaborate with cross-functional teams to deliver data-driven solutions.
  • Adhere to best practices for coding, testing, and deployment.
Required Skills & Qualifications:
  • 3–4 years of experience in data engineering with a focus on PySpark and Python.
  • Strong knowledge of Spark, Hive, SQL, and Unix shell scripting.
  • Experience with Databricks in a production environment.
  • Hands-on experience with Azure cloud services (preferred: Data Lake, Synapse, Databricks).
  • Experience working with Apache Kafka or other real-time streaming technologies.
  • Strong problem-solving and debugging skills.
  • Excellent written and verbal communication skills.
  • Ability to work independently in a fast-paced environment and adapt to new technologies quickly.
Nice-to-Have Skills:
  • Exposure to CI/CD pipelines for data solutions.
  • Knowledge of data governance and security best practices.
  • Experience with Agile/Scrum methodology.

  • Big Data Innovator

    1 week ago

    Vadodara, Gujarat, India beBeeData Full time US$ 15,00,000 - US$ 20,00,000

    About This RoleWe're seeking a seasoned Big Data Engineer to drive the development of next-generation solutions across various sectors.Our ideal candidate has extensive experience in big data analytics platforms, Python/Java programming, CI/CD, infrastructure & cloud, and Terraform.Main Responsibilities:Develop and implement big data analytics solutions...

  • Senior Technical Specialist

    2 weeks ago

    Vadodara, Gujarat, India beBeeBigData Full time ₹ 25,00,000 - ₹ 30,00,000

    Job Opportunity: Big Data DeveloperWe are seeking a skilled Big Data Developer to join our delivery team. This role is ideal for a developer who thrives in a technical environment and is passionate about coaching and developing the next generation of talent.The successful candidate will work on live projects, provide technical support, contribute to solution...

  • Senior Big Data Engineer

    2 weeks ago

    Vadodara, Gujarat, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Description:We are seeking a highly skilled Senior Data Engineer to join our data engineering team. The ideal candidate will have deep expertise in building scalable data pipelines using Apache Spark, PySpark, SQL, and Python, along with hands-on experience in the AWS ecosystem.Key Responsibilities:Design and Develop Robust ETL/ELT PipelinesOptimize...

  • Lead Big Data Architect

    2 weeks ago

    Vadodara, Gujarat, India beBeeDataEngineer Full time ₹ 2,05,43,257 - ₹ 2,51,52,929

    Job Title: Lead Big Data ArchitectThis role involves designing and implementing highly scalable, distributed SaaS-based security risk prioritization products.The ideal candidate will have expertise in big data solutions using Spark and experience with engineering resources for innovation.They will be responsible for delivering roadmap features of Enterprise...

  • Big Data Developer

    2 weeks ago

    Vadodara, Gujarat, India beBeeDataEngineering Full time ₹ 20,00,000 - ₹ 25,00,000

    Data Engineering SpecialistAbout the Role:Job Credentials: Experience - 5 to 12 years.Responsibilities:Skill Requirements: Big data spark, SQL, Hive, Scala.Our Expertise:Solutions: Data Aggregation & Visualization.Industries: Capital Markets, Asset & Wealth Management, Financial Services, Life Sciences, Manufacturing & Logistics, Software Product...

  • Big Data Processing Expert

    2 weeks ago

    Vadodara, Gujarat, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Data EngineerWe are seeking a skilled Data Engineer to join our organization.Key Responsibilities:Create and maintain high-performance data processing pipelines using big data technologies like HIVE, HBase, Impala, and Parquet.Analyze complex data sets and design efficient data storage and retrieval systems.Collaborate with cross-functional teams to...

  • Big Data Administrator

    2 weeks ago

    Vadodara, Gujarat, India beBeebigdata Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    About UsWe are a leading organization in the field of big data and cloud technology. Our mission is to empower businesses to innovate and succeed through our expertise.

  • Big Data Specialist

    1 week ago

    Vadodara, Gujarat, India beBeeDataSpecialist Full time ₹ 6,00,000 - ₹ 12,00,000

    We are seeking a seasoned Big Data Specialist to join our team. As a key player in our data engineering department, you will be responsible for designing and implementing large-scale data pipelines using modern tools and technologies.Key ResponsibilitiesDesign and Implement CI/CD Pipelines: Use GitHub Actions or equivalent tools to create and manage...

  • Chief Big Data Tester

    2 weeks ago

    Vadodara, Gujarat, India beBeeData Full time ₹ 50,00,000 - ₹ 1,00,00,000

    Job Overview:As a key player in the development of cutting-edge intelligence solutions, we are seeking a seasoned professional to spearhead our ETL/Big Data testing initiatives.This role offers an unparalleled opportunity to leverage your expertise in big data ecosystems and drive innovation in lawful interception & communication analytics.Responsibilities...

  • Senior Big Data Data Scientist

    2 weeks ago

    Vadodara, Gujarat, India beBeeData Full time ₹ 15,63,179 - ₹ 21,14,286

    Job DescriptionWe are seeking an experienced data analytics professional with a strong background in Big Data technologies and Google Cloud Platform services to design and implement scalable data solutions.The ideal candidate will have extensive experience in designing, developing, and optimizing data pipelines using GCP BigQuery, Dataflow, and Apache...