Big Data Engineer

3 days ago

India beBeeIngestion Full time ₹ 12,00,000 - ₹ 20,00,000
Job Title

We are seeking an Ingestion Engineer to design, develop and optimize data ingestion pipelines for integrating multiple sources into Databricks.

The ideal candidate will have hands-on experience with data pipelines, automation and infrastructure management to support scalable and efficient ingestion workflows.

Key Responsibilities:
  1. Design, Develop and Optimize Data Ingestion Pipelines: Integrate multiple data sources into Databricks using a scalable and efficient approach.
  2. Implement and Maintain CI/CD Pipelines: Automate data workflows using GitHub Actions or similar tools.
  3. Deploy and Manage Containerized Applications: Use Kubernetes to deploy and manage containerized applications.
  4. Ensure Data Integrity, Security and Compliance: Guarantee the integrity, security and compliance of ingested data throughout the process.
Required Skills & Qualifications:
  • Proven experience in data ingestion and pipeline development.
  • Hands-on experience with CI/CD tools (GitHub Actions, Jenkins, Azure DevOps etc).
  • Strong knowledge of Kubernetes and container orchestration.
  • Experience with Databricks, Spark and data lake architectures.
  • Proficiency in Python, Scala or SQL for data processing.
  • Familiarity with cloud platforms such as AWS, Azure or GCP.
Benefits:

This role requires strong problem-solving and analytical skills. The ideal candidate will be able to troubleshoot and optimize ingestion pipelines for performance and scalability.

Others:

Preferred qualifications include experience with Infrastructure as Code (Terraform, Helm etc), background in streaming data ingestion (Kafka, Kinesis etc) and knowledge of data governance and security best practices.

  • Big Data Engineer

    3 weeks ago

    India Amazon Music Full time

    Job DescriptionDESCRIPTIONProfit Intelligence team in Amazon Retail is seeking a seasoned and talented Senior Data Engineer to join the Historical Contribution Profit (HCP) team. HCP is a fast growing team with a mandate to build tools to automate profit-and-loss forecasting and planning for the Physical Consumer business. We are building the next generation...

  • Big Data Developer

    4 weeks ago

    India Kresta Softech Private Limited Full time

    Role Highlights: Position: Big Data Engineer Experience: 4+ years Location: All India-Remote, Hyderabad- Hybrid Notice Period: Immediate/7 days joiners mandate Job Overview: Must have skills- Big Data, Scala, AWS and Python or Java

  • Big Data

    4 weeks ago

    India Technomech Placements Full time

    Job Title Big Data DeveloperExperience 4 to 10 yearsLocation Pune Bangalore Gurgaon Develop optimize and maintain big data solutions Work with Hadoop Spark Hive and other big data technologies Design and implement scalable data pipelines Collaborate with cross-functional teams for data-driven solutions Strong programming skills in Java Scala...

  • Big Data Developer

    3 weeks ago

    India DigiHelic Solutions Pvt. Ltd. Full time

    Job descriptionKey Responsibilities:Big Data Architecture: Design, develop, and maintain scalable and distributed data architectures capable of processing large volumes of data.Data Storage Solutions: Implement and optimize data storage solutions using technologies such as Hadoop, Spark, and PySpark.PySpark Development: Develop and implement efficient ETL...

  • Big Data Architect

    6 days ago

    India beBeeDataEngineering Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    Senior Big Data EngineerWe are seeking an experienced data professional to design, build and maintain large-scale data pipelines. The ideal candidate will have a strong understanding of big data technologies, including Spark, PySpark and SQL.Main Responsibilities:Create efficient data pipelines using Apache Spark and ETL workflowsProcess large datasets using...

  • Engineering Manager – Big Data, Azure

    6 days ago

    India DataE2E Technologies Full time

    Technical Manager – AzureLocation: 100% RemoteEmployment Type: Full-timeShift Timing : 3 pm – 11 pm ISTJob Summary:About the Role:We are seeking a highly experienced and strategic Engineering Manager with deep expertise in Big Data platforms, Microsoft Azure, and Databricks. The ideal candidate will lead a team of talented data engineers to design and...

  • Scala Big Data Lead Engineer

    1 day ago

    India UST Full time

    If you arehighlyinterested and availableimmediately , please submit your resume along with your total experience, current CTC, notice period, and current location details toNitin.patil@ust.comKey Responsibilities:Design, develop, and optimizedata pipelines and ETL workflows .Work withApache Hadoop, Airflow, Kubernetes, and Containersto streamline data...

  • Engineering Manager – Big Data, Azure

    4 days ago

    India DataE2E Technologies Full time

    Technical Manager – Azure Location: 100% Remote Employment Type: Full-time Shift Timing : 3 pm – 11 pm IST Job Summary: About the Role: We are seeking a highly experienced and strategic Engineering Manager with deep expertise in Big Data platforms , Microsoft Azure , and Databricks . The ideal candidate will lead a team of talented data engineers...

  • Big Data Solutions Architect

    15 hours ago

    India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job OverviewThe Scala Big Data Engineer will design, develop and optimize data pipelines and ETL workflows. They will work with Apache Hadoop, Airflow, Kubernetes and Containers to streamline data processing.Key Responsibilities:Design and develop data pipelines and ETL workflowsWork with Apache Hadoop, Airflow, Kubernetes and Containers to streamline data...

  • Big Data Revolutionist

    23 hours ago

    India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    The Travel Industry is Evolving. Be a Part of It.In today's digital age, the travel industry is witnessing a paradigm shift. The traditional approach to booking travel arrangements is being revolutionized by innovative technologies and forward-thinking companies. As a Data Engineer, you will play a crucial role in this transformation.Job DescriptionWe are...