Senior Data Pipeline Developer

2 days ago

Mangalore, Karnataka, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

Job Role Overview

As a Data Engineer, you will play a pivotal role in designing and developing data pipelines and infrastructure. Your responsibilities will involve working closely with data scientists and analysts to ensure accurate, timely, and accessible data.


Key Responsibilities:


  • • Develop and maintain data pipelines using Azure Data Factory and Databricks to support the ingestion, transformation, and movement of data.

  • • Design, develop, and optimize ETL (Extract, Transform, Load) processes to ensure efficient data flow and transformation.

  • • Manage and optimize databases (SQL and NoSQL) to support high-performance queries and data storage requirements.

  • • Implement data quality checks and monitoring to ensure accuracy and consistency of data.

  • • Collaborate with stakeholders to understand data requirements and deliver actionable insights.

  • • Create and maintain comprehensive documentation for data processes, pipelines, and infrastructure.


Qualifications:



  • • Experience: 2-4 years of experience in data engineering or a related field.

  • • Technical Skills:
  • o Proficiency with Azure Data Factory, Azure Synapse Analytics, Databricks, and Azure Data Lake
  • o Strong SQL skills and experience with data warehousing concepts
  • o Knowledge of data modeling, ETL processes, and data integration
  • o Experience with relational databases
  • o Hands-on experience with ETL tools and frameworks
  • o Familiarity with big data technologies


Skills and Abilities:


  • • Strong problem-solving abilities and attention to detail.

  • • Excellent verbal and written communication skills, with the ability to explain technical concepts to non-technical stakeholders.


Education:


Bachelor's degree in computer science, Engineering, Mathematics, or a related field. Advanced degrees or certifications are a plus.

  • Data Pipeline Architect

    7 days ago

    Mangalore, Karnataka, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 30,00,000

    Senior Data Engineer">">Our company is seeking an experienced Senior Data Engineer to join our team. This role will play a critical part in building scalable data pipelines and integrating healthcare data.">

  • Chief Data Pipeline Architect

    2 days ago

    Mangalore, Karnataka, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Data Engineer PositionWe are seeking a highly skilled Senior Data Engineer to join our team at a cutting-edge organization.This role is critical to the development of new data products and the modernization of existing platforms in cloud environments, especially AWS.Key ResponsibilitiesDesign, build, and maintain scalable data pipelines and data...

  • Senior Data Pipeline Engineer

    2 days ago

    Mangalore, Karnataka, India beBeeDataSolutions Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Job Title: Data Solutions ArchitectAs a highly skilled data solutions architect, you will play a pivotal role in designing and developing scalable data pipelines and ETL processes on Databricks. Your expertise will be instrumental in leading the architecture and implementation of secure, high-performance data solutions.Key Responsibilities:Develop and...

  • Data Pipeline Architect

    5 days ago

    Mangalore, Karnataka, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Data EngineerAbout the Role:We are seeking a highly skilled Senior Data Engineer to join our team. As a key member of our engineering group, you will be responsible for designing, developing, and implementing large-scale data processing systems.Key Responsibilities:Design and develop scalable ETL workflows using Python and Pandas.Implement and manage...

  • Senior Data Pipeline Specialist

    1 week ago

    Mangalore, Karnataka, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job SummaryAs a seasoned Data Engineer, you will be responsible for designing and building scalable data pipelines to support business intelligence initiatives. Your expertise in Spark, PySpark, and SQL will enable the efficient processing of large datasets.Key ResponsibilitiesDesign and Build Scalable Pipelines: Develop and maintain data pipelines using...

  • Senior Data Pipeline Architect

    4 days ago

    Mangalore, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Key ResponsibilitiesWe're looking for a skilled professional to lead the development of scalable data pipelines.Design and Implement Efficient ETL ProcessesCollaborate with Data TeamsImplement Data Governance and Security StandardsOptimize Data Workflows for Scalability and PerformanceRequired Skills & QualificationsTo succeed in this role, you'll...

  • Senior Python Pipeline Architect

    6 days ago

    Mangalore, Karnataka, India beBeeEngineer Full time ₹ 1,80,00,000 - ₹ 2,40,00,000

    Senior Python Engineer PositionAbout This Role:We are seeking a highly skilled Senior Python Engineer to lead our team in developing scalable data pipelines using Apache Beam and Databricks.Key ResponsibilitiesPipeline ArchitectureCreate a shared component library/SDK for ingestion, parsing/extraction, validation, enrichment, and publishing.Define standard...

  • Data Pipeline Specialist

    1 week ago

    Mangalore, Karnataka, India beBeeDataEngineering Full time ₹ 14,10,000 - ₹ 20,30,000

    Data Engineer Role Summary:">We are seeking a skilled Data Engineer to join our team. As a Data Engineer, you will be responsible for designing and implementing scalable data pipelines that provide real-time insights for our maritime operations.">Key Responsibilities:">Build & Optimize Data Systems:Create and maintain data highways that fuel AI/ML...

  • Data Systems Developer

    7 hours ago

    Mangalore, Karnataka, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 25,00,000

    Job Title: Senior Data EngineerDescription:This is a senior-level position for a data engineer to design, develop, and maintain large-scale data systems.The ideal candidate will have experience with big data technologies such as pySpark, Hadoop, and Hive, as well as strong analytical and problem-solving skills.Responsibilities:Develop efficient ETL pipelines...

  • Cloud Data Pipeline Architect

    4 days ago

    Mangalore, Karnataka, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Opportunity: Data Engineer">We are seeking a skilled professional to lead our data engineering efforts.Our team requires someone who can design, develop and maintain robust data pipelines using cloud services such as GCP, AWS or Azure.">Main Responsibilities:Develop scalable data pipelines for real-time data processing.Implement data ingestion,...