Big Data Engineer

3 days ago

Pune, Maharashtra, India beBeeData Full time ₹ 1,04,000 - ₹ 1,30,878
Data Engineer Position

We are seeking a highly skilled data engineer to spearhead the development of robust data solutions using Azure Databricks. The ideal candidate will possess a strong background in data engineering and be proficient in leveraging Apache Spark to build efficient data pipelines.

Responsibilities include:

  • Data Pipeline Development: Design, implement, and optimize data pipelines using Azure Databricks to process large datasets efficiently.
  • Collaboration: Work closely with data engineers and scientists to build scalable and maintainable data solutions.
  • Workflow Optimization: Optimize existing Databricks workflows for performance and reliability.
  • Troubleshooting: Monitor and troubleshoot Azure Databricks jobs to ensure data integrity and availability.
  • Documentation: Develop and maintain comprehensive documentation for data processes and architectures.

Required Skills and Qualifications

The following skills and qualifications are essential for this position:

  • 6-8 years of experience in data engineering or a related field.
  • Strong proficiency in Apache Spark and Databricks platform.
  • Hands-on experience with Azure services, particularly Azure Data Lake, Azure SQL Database, and Azure Data Factory.
  • Proficient in programming languages such as Python, Scala, or SQL.
  • Experience with data modeling, ETL processes, and data warehousing concepts.
  • Familiarity with DevOps practices and CI/CD pipelines in a cloud environment.
  • Ability to analyze and visualize data using tools like Power BI or Tableau.
About This Role

This is an exciting opportunity for a talented data engineer to join our team and contribute to the development of innovative data solutions using Azure Databricks.

Azure Databricks is a powerful platform for big data analytics and machine learning workloads. As a data engineer, you will play a key role in designing, building, and maintaining data pipelines that enable real-time insights and business decision-making.

This is a great chance to grow your career and work on challenging projects with a talented team of professionals.

  • Big Data Engineer

    2 days ago

    Pune, Maharashtra, India Nice Software Solutions Pvt. Ltd. Full time

    Big Data Engineer (PySpark) Location: Pune/Nagpur (WFO) Experience: 8 - 12 Years Employment Type: Full-time Job Overview We are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high...

  • Big Data Engineer

    2 weeks ago

    Pune, Maharashtra, India LION AND ELEPHANTS CONSULTANCY PRIVATE LIMITED Full time

    Job Description : We are looking for skilled Big Data Engineers using Java Spark with 5-10 years of experience in Big Data / legacy platforms, who can join immediately. Desired candidates should have design, development and optimization of real-time & batch data pipelines experience in a Big Data environment at an enterprise scale application. You will work...

  • Big Data Engineer

    4 days ago

    Pune, Maharashtra, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Big Data Engineer">We are seeking a highly skilled Big Data Engineer to drive the design and implementation of scalable data pipelines using Apache Spark, Databricks, or equivalent big data frameworks. This is an excellent opportunity for experienced professionals to join our organization and leverage their expertise to drive business outcomes through...

  • Big Data Engineer

    4 weeks ago

    Pune, Maharashtra, India WorkiFicient Full time

    Roles and Responsibility :- Collaborate closely with Product Management and Engineering leadership to devise and build the right solution.- Participate in Design discussions and brainstorming sessions to select, integrate, and maintain Big Data tools and frameworks required to solve Big Data problems at scale.- Design and implement systems to cleanse,...

  • Senior Big Data Engineer

    4 weeks ago

    Pune, Maharashtra, India Devlats Pvt Ltd Full time

    About the Role : We are seeking a highly skilled and experienced Big Data Engineer to join our team. The ideal candidate will have strong expertise in designing, building, and optimizing data pipelines and architectures, as well as a solid understanding of big data tools and Responsibilities : - Design, develop, and maintain scalable and high-performance...

  • Senior Big Data Engineer

    1 day ago

    Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 30,00,000

    Our company is seeking a highly skilled Data Architect to design, develop and implement scalable data solutions using cutting-edge technologies.Key Responsibilities:Data Ingestion and Processing: Develop and maintain high-quality code, adhering to industry standards and best practices, for efficient data ingestion from disparate sources onto our Big Data...

  • Big Data Engineering Leadership

    2 days ago

    Pune, Maharashtra, India beBeeCloud Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job OverviewWe are seeking ambitious professionals for the role of Lead Apache Hadoop Engineer in our Technology, Data, and Innovation (TDI) team.Key ResponsibilitiesDevelop robust architectures and designs for big data platforms and applications within the Apache Hadoop ecosystem.Implement and deploy big data platforms and solutions on-premises and in...

  • Big Data Engineering Specialist

    2 hours ago

    Pune, Maharashtra, India beBeeBigData Full time ₹ 1,04,000 - ₹ 1,30,878

    We are seeking a seasoned professional to fill the role of Big Data Engineering Specialist. This is an exceptional opportunity for individuals with expertise in designing and developing large-scale data processing systems.The ideal candidate will have extensive knowledge of big data technologies and a strong track record of delivering projects on time and...

  • Lead Big Data Engineer

    6 hours ago

    Pune, Maharashtra, India beBeeDataScientist Full time ₹ 10,00,000 - ₹ 12,00,000

    Big Data Architect PositionWe are seeking a skilled Big Data Architect to lead our team's technical efforts.The ideal candidate will have hands-on experience in designing and developing applications using Big Data, DWH, Cloud technologies, ETL, SQL.Expertise in Spark, Hadoop ecosystem components, and cloud platforms (AWS/Azure/GCP)Good knowledge of Shell...

  • Big Data Engineering Lead

    7 minutes ago

    Pune, Maharashtra, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Senior Data Engineering Leader">We are looking for a seasoned Senior Data Engineering Leader to oversee the development and implementation of scalable, distributed data platforms that drive business value.">As a key member of our engineering team, you will be responsible for:">">Designing and architecting high-performance data pipelines that integrate with...