Expert Data Pipeline Specialist

2 weeks ago


Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 80,00,000 - ₹ 2,00,00,000
Job Role

    We seek an expert in building and maintaining scalable, high-performance data pipelines using Python and PySpark. The ideal candidate will have hands-on experience with Google Cloud Platform (GCP) services including BigQuery, Dataflow, and Cloud Functions.

Data Responsibilities:

    
  • Design and build efficient data pipelines using Python and PySpark.
  • Develop and optimize ETL/ELT workflows for large-scale datasets.
  • Work extensively with GCP services including BigQuery, Dataflow, and Cloud Functions.
  • Implement containerized solutions using Docker, and manage code through Git and CI/CD pipelines.
  • Collaborate with data scientists, analysts, and other engineers to deliver high-quality data solutions.
  • Monitor, troubleshoot, and improve the performance of data pipelines.


Qualifications

    
  • Proficiency in Python, PySpark, and Big Data technologies.
  • Strong experience in ETL/ELT, data modeling, distributed computing, and performance tuning.
  • Hands-on expertise in GCP and its services.
  • Working knowledge of workflow orchestration tools such as Apache Airflow or Cloud Composer.
  • GCP certification is a plus.
  • Experience with Docker, CI/CD practices, and version control tools like Git.


  • Pune, Maharashtra, India beBeeData Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    As a Technical Specialist, you will play a pivotal role in designing, implementing, and managing data pipelines using Azure Data Factory.You will work closely with clients to understand their data needs and requirements, providing support and training on data tools and solutions.The ideal candidate will have 5-7 years of experience in data warehousing and...


  • Pune, Maharashtra, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Big Data Specialist - Data Pipeline EngineerAs a Big Data Specialist - Data Pipeline Engineer, you will be responsible for designing, building, and maintaining robust data pipelines to efficiently curate and ingest data.Our ideal candidate will have strong SQL skills, experience with Fabric, and proficiency in programming languages such as Python (PySpark)....


  • Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    About the Role : The Senior Data Engineer will be responsible for designing, developing and maintaining ETL pipelines using Azure Data Factory. The ideal candidate should have strong hands-on experience with Azure Data Factory (pipelines, triggers, linked services, datasets) as well as proficiency in T-SQL and SQL Server.Key Responsibilities :Design, develop...


  • Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 25,00,000 - ₹ 35,00,000

    Job DescriptionPosition Overview:We're seeking a skilled Data Engineer to join our team. The successful candidate will be responsible for maintaining and supporting data transformation pipelines from Landing Zone to Consumption Layer, ensuring seamless operations in GCP and SAP BW environments.Key Responsibilities:Maintain and support data transformation...

  • Data + AI Pipeline

    2 weeks ago


    Pune, Maharashtra, India VME Vhire Solutions Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    You will be a key member of the Data + AI Pipeline team, leading to the integration ofKubeflow, Kubernetes, Docker, Keda, and Python technologies. Your role will involvedeveloping and maintaining AI pipelines that support various projects, ensuring seamlessand efficient data processing, model training, and deployment. As part of a dynamic...


  • Pune, Maharashtra, India beBeePipeline Full time ₹ 1,04,000 - ₹ 1,30,878

    We are seeking an experienced Data Pipeline Quality Assurance specialist to join our data team.The ideal candidate will be proficient in analyzing source data, understanding data structures and distributions, and identifying anomalies to ensure data quality and readiness for processing.Key Responsibilities:Analyzing data pipelines and identifying areas for...


  • Pune, Maharashtra, India beBeeETL Full time ₹ 15,00,000 - ₹ 25,00,000

    We are seeking a seasoned ETL professional to spearhead data pipeline testing and validation. The ideal candidate will possess extensive experience in designing, developing, and implementing complex ETL solutions.Key Responsibilities:Conduct thorough testing of ETL processes to ensure seamless data flow and accuracy.Develop advanced SQL queries to validate...


  • Pune, Maharashtra, India beBeeETL Full time ₹ 20,00,000 - ₹ 25,00,000

    Job TitleWe are seeking a seasoned ETL Developer to spearhead the implementation of data pipelines in Azure Data Factory (ADF). The ideal candidate will design and develop comprehensive test plans, ensuring seamless integration and optimal performance of complex data workflows.Key Responsibilities:Design and develop manual and automated test cases for ADF...


  • Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    As a key member of our team, you will play a pivotal role in designing, developing and maintaining data pipelines that extract data from Oracle Symphony via APIs, process and store it in the Databricks Lakehouse platform.The successful candidate will have deep expertise in data integration, ETL/ELT, APIs and Databricks. You will work closely with business...


  • Pune, Maharashtra, India beBeeCommercial Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Description:The Senior Specialist, Global Commercial Pharma Pipeline Analytics role is a pivotal position within the organization. The successful candidate will be responsible for developing and delivering data and analytics solutions to support strategic business decisions.This critical role requires a strong analytical mindset, excellent communication...