Senior Data Pipeline Developer

2 days ago

Bengaluru, Karnataka, India beBeeData Full time US$ 90,000 - US$ 1,20,000

Job Description:

">

We are seeking a skilled and experienced Data Engineer to help build, optimize, and maintain data pipelines and architectures. The ideal candidate will have deep expertise in the Microsoft data engineering ecosystem, particularly leveraging tools such as Azure Data Factory, Databricks, Alteryx, Synapse Analytics, and a strong command of SQL, Python, and Apache Spark.

">

Key Responsibilities:

">
  • ">
  • Design, develop, and optimize scalable data pipelines and workflows using Azure Data Factory, Synapse Pipelines, and Microsoft Fabric.">
  • Responsible for transforming and migrating existing Alteryx workflows into scalable Azure Databricks pipelines, ensuring optimized performance and maintainability.">
  • Collaborate with data engineering and analytics teams to redesign Alteryx pipelines into Spark-based solutions on Azure Databricks, leveraging Delta Lake and Azure services for automation and efficiency.">
  • Build and maintain ETL/ELT processes for ingesting structured and unstructured data from various sources.">
  • Develop and manage data transformation logic using Databricks (PySpark/Spark SQL) and Python.">
  • Collaborate with data analysts, architects, and business stakeholders to understand requirements and deliver high-quality data solutions.">
  • Ensure data quality, integrity, and governance across the data lifecycle.">
  • Implement monitoring and alerting for data pipelines to ensure reliability and performance.">
  • Work with Azure Synapse Analytics to build data models and enable analytics and reporting.">
  • Utilize SQL for querying and managing large datasets efficiently.">
  • Participate in data architecture discussions and contribute to technical design decisions.">
">

Required Skills and Qualifications:

">
  • ">
  • 3+ years of experience in data engineering or a related field.">
  • Strong experience in Alteryx workflows and data preparation/ETL processes.">
  • Strong proficiency in the Microsoft Azure data ecosystem including:">
  1. ">
  2. Azure Data Factory (ADF)">
  3. Azure Synapse Analytics">
  4. Microsoft Fabric">
  5. Azure Databricks">
">Solid experience with Python and Apache Spark (including PySpark).">Advanced skills in SQL for data manipulation and transformation.">Experience in designing and implementing data lakes and data warehouses.">Familiarity with data governance, security, and compliance standards.">Strong analytical and problem-solving skills.">Excellent communication and collaboration abilities.">">

Preferred Qualifications:

">
  • ">
  • Microsoft Azure certifications (e.g., Azure Data Engineer Associate).">
  • Experience with DevOps tools and CI/CD practices in data workflows.">
  • Knowledge of REST APIs and integration techniques.">
  • Background in agile methodologies and working in cross-functional teams.">

  • Senior Data Pipeline Specialist

    6 days ago

    Bengaluru, Karnataka, India beBeeAutomatedTesting Full time

    Job Opportunity:The role of Senior Consultant involves designing, developing and implementing automated test cases for complex data pipelines.Ability to design, develop, and implement automated test cases for complex data pipelines is a mustAWS experience is essentialPharma expertise is also requiredProficiency in programming languages such as Python, R, or...

  • Data Pipeline Engineer

    8 hours ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    The Role of Senior Data Integration SpecialistJob Overview:We are seeking an experienced Senior Data Integration Specialist to join our team. This is a key position that will play a vital role in the smooth operation of our enterprise data integration platform.As a Senior Data Integration Specialist, you will be responsible for designing, implementing, and...

  • Position: Data Pipeline Lead

    4 days ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 9,00,000 - ₹ 12,00,000

    Senior Data Engineer Position">We are looking for a seasoned Senior Data Engineer to join our team and lead the design, development, and maintenance of scalable data pipelines using Google Cloud Platform (GCP), BigQuery, DataProc (PySpark), and Informatica.]"The ideal candidate will have hands-on expertise in GCP, including BigQuery, DataProc, Cloud Storage,...

  • Data Pipeline Architect

    8 hours ago

    Bengaluru, Karnataka, India beBeeETLDeveloper Full time ₹ 1,20,00,000 - ₹ 1,50,00,000

    About the RoleWe are seeking a skilled Senior ETL Developer to join our team.The successful candidate will be responsible for evaluating and understanding current-state Informatica ETL jobs, mappings, workflows, and dependencies.Key responsibilities include:Evaluating and redesigning data pipelines using Databricks (PySpark/Scala/SQL) to optimize performance...

  • Senior Data Pipeline Developer

    5 hours ago

    Bengaluru, Karnataka, India beBeeIntegration Full time US$ 80,000 - US$ 1,50,000

    Job DescriptionWe are seeking a skilled Data Integration Specialist to join our team. The successful candidate will have deep expertise and proven success using analytical thinking and iterative problem-solving to manage programs and processes strategically and tactically.The role involves designing, developing, and maintaining complex data pipelines,...

  • Data Pipeline Specialist

    4 days ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 9,00,000 - ₹ 12,00,000

    Job TitleSenior Data EngineerJob DescriptionAs a Senior Data Engineer, you will design, implement, and optimize data pipelines for large-scale data processing. You will work with data scientists, analysts, and engineers to ensure data availability and quality.Key responsibilities include developing and maintaining ETL/ELT workflows using Spark, Hadoop, Hive,...

  • Senior Data Pipeline Architect

    6 hours ago

    Bengaluru, Karnataka, India beBeeDataEngineering Full time ₹ 1,04,000 - ₹ 1,30,878

    Lead Data EngineeringThe Lead Data Engineer is a key member of our engineering team, responsible for leading and mentoring data engineers to ensure alignment with technical and business objectives.As a senior data engineer, you will architect, design, and implement scalable and high-performance data pipelines using AWS technologies. You will collaborate with...

  • Senior Data Engineer

    2 days ago

    Bengaluru, Karnataka, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 20,00,000

    Data Engineering Professional RoleWe are seeking a highly skilled Data Engineering professional to design, develop, and maintain scalable and reliable data pipelines using Google Cloud Platform (GCP) tools.

  • Lead Data Pipeline Developer

    1 day ago

    Bengaluru, Karnataka, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data EngineerWe are seeking a skilled Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing, developing and maintaining ETL/ELT data pipelines that ingest data from various sources into our data warehouse.Pipeline Development: Design and develop efficient data pipelines using AWS...

  • Principal Data Pipeline Architect

    1 day ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time US$ 1,25,000 - US$ 1,75,000

    About the RoleWe are seeking a Senior Data Engineer with expertise in building scalable data pipelines and integrating healthcare data. Reporting to the Head of AI and Engineering, you will design, build, and operate the backbone of our AI-powered applications.You will work closely with software engineers, ML specialists, and clinical partners to bring our...