Expert Large Scale Data Pipeline Developer

2 days ago


Vijayawada, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000
Job Description

We are seeking an experienced Data Engineer to design, build, and optimize large-scale data pipelines. The ideal candidate will have expertise in Python, SQL, Linux/Bash, PySpark, AWS, and Databricks.

The primary focus of this role is to develop, maintain, and optimize ETL/ELT pipelines using PySpark, AWS, and Databricks. Additionally, the candidate will be responsible for performing data profiling, validation, and transformation using Python and SQL, as well as optimizing and monitoring workflows for performance, scalability, and cost efficiency.

Key Responsibilities
  • Design, develop, and maintain ETL/ELT pipelines using PySpark, AWS, and Databricks.
  • Perform data profiling, validation, and transformation using Python and SQL.
  • Optimize and monitor workflows for performance, scalability, and cost efficiency.
  • Build automation scripts using Linux and Bash for data pipeline orchestration.
  • Collaborate with analytics teams to surface trustworthy KPIs and metrics.
  • Troubleshoot and resolve data inconsistencies, configuration issues, and pipeline errors.
Qualifications:
  • 5+ years of data engineering experience with a strong background in Python and SQL.
  • Proven expertise in PySpark, AWS services, and Databricks.
  • Proficiency in Linux/Bash scripting for automation and data operations.
  • Hands-on experience with ETL/ELT pipelines and large-scale integrations.
  • Familiarity with Agile and CI/CD practices.
  • Strong problem-solving and communication skills, with experience in client-facing roles.


  • Vijayawada, Andhra Pradesh, India beBeeDataCenter Full time ₹ 20,00,000 - ₹ 25,00,000

    Optimize Data Center Cooling SystemsWe are seeking a highly skilled data center cooling expert with extensive experience in the design and engineering of large-scale, hyperscale data centers or mission-critical facilities.The ideal candidate will have a deep understanding of thermal management, HVAC systems, and energy-efficient cooling technologies critical...


  • Vijayawada, Andhra Pradesh, India beBeeTechnical Full time ₹ 1,80,00,000 - ₹ 2,20,00,000

    Job Overview:We are seeking an experienced Technical Project Manager to lead large-scale projects within big data and analytics environments, particularly on Databricks.Deliver complex technical projects efficiently and effectivelyEnsure seamless execution and client satisfactionManage large-scale projects with expertise in big data and analyticsKey...


  • Vijayawada, Andhra Pradesh, India beBeeAdvertising Full time ₹ 15,00,000 - ₹ 17,00,000

    Job DescriptionUnlock Your Potential as a Large Scale Advertising ManagerWe are seeking experienced professionals to join our team and take ownership of advertising accounts, driving performance growth through effective campaign management.Key Responsibilities:Deep knowledge of Google Ads campaign managementExperience working with monthly invoiced accountsA...


  • Vijayawada, Andhra Pradesh, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Big Data ExpertWe are seeking an experienced professional to join our high-impact team building robust, scalable data solutions on Microsoft Fabric and Azure platforms. As a key contributor, you will have the opportunity to work with cutting-edge technologies and collaborate with stakeholders to design and implement innovative data pipelines.Key...


  • Vijayawada, Andhra Pradesh, India beBeeData Full time ₹ 12,00,000 - ₹ 18,00,000

    Job OpportunityWe are seeking a seasoned Data Engineering Expert to design and implement scalable data pipelines, ensuring seamless data flow for analytics and operations.This role presents a unique chance to collaborate with talented professionals, tackle real-world data challenges, and drive business decisions.Main Responsibilities:Design and optimize...


  • Vijayawada, Andhra Pradesh, India beBeeEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job DescriptionWe are seeking a seasoned Python Platform Engineer to lead the design and development of high-performance data pipelines. The ideal candidate will have extensive experience with Apache Beam, Databricks, and other related technologies.The successful candidate will be responsible for designing and building shared component libraries,...


  • Vijayawada, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    We are seeking a highly skilled Senior Data Engineer to join our team in implementing an application for a large retailer.The ideal candidate will have hands-on coding expertise in Python and Pyspark, with strong experience in Telemetry, Big Data, and backend engineering.Key Responsibilities:Design and implement data processing pipelines using Python and...


  • Vijayawada, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job OverviewWe are seeking an experienced Data Engineer to join our team. This role involves designing, developing, and maintaining large-scale data pipelines using AWS Cloud services.


  • Vijayawada, Andhra Pradesh, India beBeeCloud Full time ₹ 15,00,000 - ₹ 25,00,000

    Cloud Migration ExpertAt the forefront of cloud migration, we're seeking a seasoned Cloud Migration Expert to drive large-scale data engineering projects forward.Key Responsibilities:Data Pipeline Development: Design and deploy robust and scalable data pipelines for migrating on-premise data systems to cloud platforms.Technical Expertise: Serve as a subject...


  • Vijayawada, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job OverviewWe're seeking a skilled professional to design, develop, and maintain large-scale data pipelines. Main Responsibilities:Data Pipeline Development: Design and develop ETL/ELT data pipelines to ingest data from various sources into our data warehouse.Cloud Services: Utilize AWS services for data ingestion and transformation processes.Database...