Senior Data Pipeline Developer

4 days ago


Ellore, Andhra Pradesh, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,00,00,000
Job Opportunity: Data Engineering Specialist

We are seeking a highly skilled Data Engineering Specialist to lead the design, development, and deployment of scalable data pipelines and ETL processes using Databricks.

Responsibilities:

  • Collaborate with stakeholders to understand data requirements and develop technical solutions that meet their needs.
  • Design, develop, and deploy complex data pipelines and ETL processes using Databricks, ensuring efficient data processing and storage.
  • Optimize Spark jobs to ensure high performance and reliability.
  • Develop and maintain Databricks notebooks, workflows, and dashboards, providing real-time insights to stakeholders.
  • Troubleshoot and resolve data pipeline issues, identifying and implementing process improvements.
  • Monitor and manage data processing jobs, ensuring high availability and reliability.
  • Stay current with the latest Databricks and big data technologies, applying knowledge to drive innovation and growth.

Requirements:

  • 6+ years of experience in data engineering, with expertise in developing and maintaining large-scale data systems.
  • Proficient in writing complex SQL queries, including analytical functions and joins.
  • Programming skills in languages such as Python and Scala, with experience in cloud-based platforms like AWS, Azure, or Google Cloud.
  • Excellent communication and collaboration skills, with the ability to work effectively with cross-functional teams.


  • Ellore, Andhra Pradesh, India beBeeBackend Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Opportunity:Senior Backend EngineerWe are seeking a skilled professional to design and develop data pipelines, crawl/fetch data from external systems, and connect interfaces.Pipelines development using API-first approach with playwright/requestsData parsing/normalization, deduplication/delta logic, and embeddings/similarity searchIntegrations with...


  • Ellore, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job SummaryWe are seeking an experienced Data Engineer to design, develop, and manage scalable data pipelines and transformation frameworks.Key Responsibilities:Design and build large-scale ETL/ELT pipelines using Azure Data Factory (ADF), Azure Databricks (Spark), and Azure Synapse Analytics.Develop and optimize PySpark/Spark SQL jobs for batch and...


  • Ellore, Andhra Pradesh, India beBeeData Full time ₹ 12,60,000 - ₹ 20,16,000

    Job Role: Senior Data EngineerThe role of a Senior Data Engineer is to design and implement data pipeline architecture that is optimal.Key Responsibilities:Create and maintain data pipeline architecture that is efficient and scalable.Design and implement events/streaming based architecture/design.Identify and implement internal process improvements.Build the...


  • Ellore, Andhra Pradesh, India beBeeData Full time ₹ 20,00,000 - ₹ 24,00,000

    Job SummaryWe are seeking a Senior Data Engineer with experience in developing and refining data pipelines for high-volume data processing.Your primary responsibilities will include:Ensuring precision, coherence, and system performance in our data systems.Collaborating with architects, analysts, and stakeholders to drive impactful solutions.Implementing...


  • Ellore, Andhra Pradesh, India beBeeMachineLearning Full time ₹ 8,00,000 - ₹ 15,00,000

    Job Summary:We are seeking a skilled MLOps Engineer with 7+ years of experience to join our team.Key Responsibilities:Pipeline Development: Develop and maintain scalable ML pipelines and backend services.Application Deployment: Build, deploy, and manage applications using Python and frameworks like Tornado and FastAPI.Data Storage: Work with RDBMS models in...


  • Ellore, Andhra Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job OverviewWe are seeking a skilled Data Transformation Specialist to join our team.The ideal candidate will have a strong background in developing and maintaining data pipelines using various frameworks.They will be responsible for implementing client-specific data transformations, supporting data ingestion and quality checks during onboarding, and...


  • Ellore, Andhra Pradesh, India beBeeData Full time ₹ 12,00,000 - ₹ 20,00,000

    Job Title: Data Pipeline ArchitectAs a key member of our data team, you will be responsible for designing and implementing scalable data pipelines to ensure seamless data flow and accessibility.Create interactive dashboards using Bold BI, Power BI, or Superset to provide actionable insights to stakeholders.Develop data pipelines and automation scripts using...


  • Ellore, Andhra Pradesh, India beBeeDataFlowConsultant Full time ₹ 15,00,000 - ₹ 25,50,000

    Data Flow ConsultantAs a Data Flow Consultant, you will design and develop scalable data pipelines using Oracle Data Flow or Apache Spark (PySpark) with expertise in data patterns, Python, and SQL.The ideal candidate will have 3-12 years of experience in building and managing Data Lakes, Data Warehouses, and Data Pipelines for seamless integration and...


  • Ellore, Andhra Pradesh, India beBeeExpertise Full time ₹ 15,00,000 - ₹ 25,00,000

    Cloud Data SpecialistDesign and implement scalable data pipelines using Azure services such as Data Factory, Databricks, and Data Lake to deliver high-quality business insights and visualization solutions.The ideal candidate will have a strong expertise in building end-to-end data pipelines and delivering efficient data solutions on Azure.Key...


  • Ellore, Andhra Pradesh, India beBeeDataEngineer Full time US$ 1,00,000 - US$ 1,40,000

    Job Overview:As a seasoned Data Engineer, you will play a pivotal role in designing and developing scalable data pipelines that fuel our analytics and reporting capabilities.Key Responsibilities:Develop robust, high-quality data pipelines that support business needs.Integrate diverse data sources into data lakes and warehouses.Collaborate with...