Senior Data Engineer

4 days ago

Bengaluru, Karnataka, India beBeeData Full time
Big Data Developer Job

We're seeking an experienced Big Data Developer to join our team. As a key member of our data engineering team, you will be responsible for designing and implementing large-scale data processing pipelines using Databricks and PySpark.

This role is ideal for someone with strong hands-on experience in big data processing and analytics, as well as a proven track record of delivering high-quality solutions under tight deadlines.

Key Responsibilities:
  • Data Ingestion & Transformation: Design and develop efficient PySpark code to extract, transform, and load large volumes of data from various sources into Databricks.
  • Data Lake/Warehouse Management: Work with data lake and data warehouse concepts, ensuring data quality, consistency, and efficient storage within Databricks (e.g., Delta Lake).
  • Performance Optimization: Optimize PySpark jobs and Databricks notebooks for performance, cost-efficiency, and scalability, addressing bottlenecks in data processing.
  • Analysis & Insights: Perform complex data analysis using PySpark to uncover insights, build data models, and support business intelligence and machine learning initiatives.
  • Troubleshooting: Perform in-depth troubleshooting, debugging, and issue resolution for PySpark jobs, Databricks environments, and data pipeline failures.
Requirements:
  • PySpark Expertise: Strong hands-on development experience with PySpark for big data processing and analytics.
  • Databricks Platform: Proven experience working with the Databricks platform, including Databricks notebooks, clusters, Delta Lake, and related services.
  • SQL Proficiency: Excellent proficiency in SQL for data manipulation and querying.
  • Python Programming: Strong programming skills in Python for data engineering tasks.
  • Cloud Platforms: Familiarity with cloud services from AWS, Azure, or GCP, particularly those related to data storage and processing.
  • Problem-Solving: Excellent analytical and problem-solving skills with a methodical approach to complex data challenges.
  • Communication: Strong verbal and written communication skills to articulate technical solutions and collaborate effectively within a team.

  • Senior Associate Data Engineer

    2 days ago

    Bengaluru, Karnataka, India NTT DATA, Inc. Full time

    Job DescriptionMake an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion its a place where you can grow, belong and thrive.Your day at NTT DATAThe...

  • senior hydraulic systems design expert

    1 day ago

    Bengaluru, Karnataka, India beBeehydraulic engineer Full time ₹ 12,00,000 - ₹ 20,00,000

    Job Title:">Senior Hydraulic Systems Design Expert"> ">Job Summary:"]}We are seeking an experienced Senior Hydraulic Systems Design Expert to join our team. As a key member of the engineering department, you will be responsible for designing, developing, and improving hydraulic systems, components, and handling ECNs and ECRs for power equipment and...

  • Senior Data Engineer

    4 days ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 37,50,000 - ₹ 45,00,000

    Job Opportunity: Senior Data Engineer">We are seeking a highly experienced and skilled Senior Data Engineer to contribute to the design, development, and deployment of large-scale data systems.">This role involves designing, developing, and deploying large-scale data systems that support our business needs. A high degree of technical expertise is required to...

  • Senior Quality Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT Data Full time

    Job DescriptionReq ID:332649NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior Quality Engineer - Remote to join our team in Bangalore, Karntaka (IN-KA), India (IN).Senior Quality...

  • Senior Data Engineer

    2 days ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 1,80,00,000 - ₹ 2,40,00,000

    Senior Data Engineer - MumbaiThe role of Senior Data Engineer is an exciting opportunity for a seasoned professional to drive the development and implementation of our data pipelines, leveraging their expertise in PySpark and Spark APIs.This position requires a strong background in data engineering, with experience in designing, developing, and maintaining...

  • Senior Data Engineer

    4 days ago

    Bengaluru, Karnataka, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 20,00,000

    Senior Data Engineer PositionWe are seeking an experienced Senior Data Engineer to join our team. The ideal candidate will have a strong background in data engineering, with expertise in designing and implementing large-scale data systems.The Senior Data Engineer will be responsible for owning end-to-end delivery of large features and systems, defining...

  • Senior Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India One Click AI Full time

    Senior Data EngineerJob Description : Senior Data EngineerCompany : Think Right Type : : Type : HybridSenior Data Engineer at Think Right Technologies will be responsible for building and deploying scalable data driven solutions. Data Engineers are responsible for working with both SQL and NoSQL technologies and collaborating with other product...

  • Senior Software Engineer

    3 days ago

    Bengaluru, Karnataka, India beBeeSoftware Full time ₹ 1,00,00,000 - ₹ 1,50,00,000

    Senior Software Engineer - Data Engineering\

  • Senior Cloud Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT Data Full time

    Job DescriptionNTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior Cloud Engineer to join our team in Bangalore, Karntaka (IN-KA), India (IN).Request you to share these Screening...

  • Senior Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India UNP Consultancy & Training PVT. LTD. Full time

    Senior Data EngineerJob Title: Senior Data EngineerLocation: Bengaluru Job Type: Full-Time Experience Required: 5–10 Years Department: Data Engineering / AnalyticsAbout the Role:We are seeking a highly skilled and experienced Senior Data Engineer to lead the design, development, and maintenance of scalable and resilient data infrastructure and pipelines...