Data Pipelines Specialist

2 days ago

Solapur, Maharashtra, India beBeeData Full time ₹ 1,20,00,000 - ₹ 2,00,00,000
Senior Data Engineering Specialist

We are seeking an experienced Senior Data Engineer to join our team.

This is a critical role in designing, developing, and maintaining data pipelines and infrastructure.

Your Key Responsibilities:

  • Designing and implementing scalable data pipelines using Azure Data Factory and Databricks for efficient data ingestion, transformation, and movement.
  • Developing and optimizing Extract, Transform, Load (ETL) processes for streamlined data flow and transformation.
  • Maintaining large datasets with optimal storage solutions on Azure Data Lake.
  • Collaborating with stakeholders to design and manage scalable data warehousing solutions using Azure Synapse Analytics.
  • Ensuring seamless integration of various data sources into MS-Fabric.
  • Optimizing data processing workflows and storage solutions for improved performance and reduced costs.
  • Managing databases (SQL and NoSQL) to support high-performance queries and data storage requirements.
  • Implementing robust data quality checks and monitoring to ensure accurate and consistent data.
  • Collaborating with cross-functional teams to understand data requirements and deliver actionable insights.
  • Creating comprehensive documentation for data processes, pipelines, and infrastructure architecture.

Requirements:

  • Minimum 2-4 years of experience in data engineering or a related field.
  • Strong proficiency with Azure Data Factory, Azure Synapse Analytics, Databricks, and Azure Data Lake.
  • Familiarity with Microsoft Fabric is a plus.
  • Excellent SQL skills and experience with data warehousing concepts.
  • Knowledge of data modeling, ETL processes, and data integration.
  • Experience with relational databases (e.g., MS-SQL, PostgreSQL, MySQL).
  • Hands-on experience with ETL tools and frameworks.
  • Knowledge of big data technologies is a plus.
  • Familiarity with cloud platforms and associated data services.
  • Familiarity with data visualization tools and programming languages like Python, Java, or Scala.
  • Experience with schema design and dimensional data modeling.
  • Strong analytical and problem-solving abilities.
  • Effective communication skills, with the ability to explain technical concepts to non-technical stakeholders.

About the Role:

This is an exciting opportunity to work with cutting-edge technology and collaborate with a talented team of professionals.

  • Azure Data Pipeline Specialist

    6 days ago

    Solapur, Maharashtra, India beBeeData Full time US$ 9,00,000 - US$ 12,00,000

    Job Description:We are seeking a skilled Azure Data Engineer to design, build, and maintain robust data pipelines that efficiently curate and ingest data. This role involves working with cross-functional teams to ensure data integrity, security, and accessibility.Key Responsibilities:Design, build, and maintain data pipelines for efficient data curation and...

  • Senior Data Pipelining Specialist

    1 week ago

    Solapur, Maharashtra, India beBeeTalendDateveloper Full time ₹ 15,00,000 - ₹ 20,00,000

    We are seeking a skilled professional to fill the role of a Talend ETL Developer with expertise in BigQuery, PostgreSQL, and Python for an immediate contract position.Job Summary:Talend ETL Developers design, optimize, and deploy complex data pipelines using Talend Open Studio. The ideal candidate will have experience with partitioning, clustering, federated...

  • Senior Data Pipeline Specialist

    6 days ago

    Solapur, Maharashtra, India beBeeDataEngineer Full time US$ 1,00,000 - US$ 1,20,000

    Key Responsibilities:Design and maintain complex data pipelines utilizing Python, AWS (S3, Redshift, Lambda, Glue), and other relevant technologies.Implement and optimize Extract, Transform, Load (ETL) processes for large-scale datasets to ensure efficient data processing.Guarantee the integrity, security, and accessibility of data assets while adhering to...

  • High-Performance Data Pipelines Specialist

    1 week ago

    Solapur, Maharashtra, India beBeeData Full time US$ 1,80,000 - US$ 2,40,000

    Big Data Engineer Job DescriptionWe're seeking a skilled Big Data Engineer to help us scale and optimize our data processing capabilities. Our platform processes massive amounts of data to provide actionable insights to clients.The ideal candidate will design, build, and maintain high-performance data pipelines that process terabytes of data. This includes...

  • Data Ingestion Specialist

    1 week ago

    Solapur, Maharashtra, India beBeeDataIngestion Full time US$ 80,000 - US$ 1,50,000

    Job Title: Azure Databricks EngineerWe are seeking a skilled Data Ingestion Specialist to design, develop and optimize data ingestion pipelines for integrating multiple sources into Databricks.Design, develop, and optimize data ingestion pipelines using modern technologies and methodologies.Implement and maintain Continuous Integration/Continuous Deployment...

  • Data Pipeline Quality Assurance Specialist

    6 days ago

    Solapur, Maharashtra, India beBeeTest Full time ₹ 80,00,000 - ₹ 1,50,00,000

    About this opportunity :We are seeking an experienced professional to join our team as a Test Engineer, specializing in ETL and Power BI.This role will involve ensuring the accuracy, integrity, and performance of ETL Pipelines deployed into Azure cloud environments.The ideal candidate will have experience with Python scripting and Pytest framework,...

  • Advanced Data Pipeline Validator

    9 hours ago

    Solapur, Maharashtra, India beBeeDataValidator Full time ₹ 15,00,000 - ₹ 20,00,000

    ETL Tester Job DescriptionWe are seeking a skilled ETL Tester to validate data pipelines and transformations.Expertise in ETL testing methodologies, debugging data issues, and working with large datasets in cloud environments is required.The ideal candidate will have 6 years of experience and be able to identify and resolve data discrepancies efficiently.Key...

  • Lead Data Pipeline Developer

    7 days ago

    Solapur, Maharashtra, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 20,00,000

    Data Engineering ExpertWe are seeking an experienced Data Engineer to join our team. The ideal candidate will have a strong background in designing, developing, and maintaining data pipelines, as well as expertise in working with cloud-based services.Key Responsibilities:Develop and maintain ETL/ELT data pipelines to ingest data from various sources into our...

  • Data Quality Specialist

    2 days ago

    Solapur, Maharashtra, India beBeeDataQuality Full time ₹ 15,00,000 - ₹ 25,00,000

    Data Quality SpecialistAbout this role:We are seeking a highly skilled data quality specialist to join our team.The ideal candidate will have hands-on experience in developing and executing tests using tools such as SQL, Tricentis, Python (PySpark), and data quality frameworks, specifically for Data Pipeline, ETL, and ingestion...

  • Data Pipeline Specialist

    9 hours ago

    Solapur, Maharashtra, India beBeeEtloperator Full time ₹ 90,00,000 - ₹ 1,20,00,000

    ETL Testing JobWe are seeking a skilled ETL Tester to join our team.Job Responsibilities:Develop in-depth understanding of technical requirements from high-level and low-level design documents.Configure development and user acceptance testing environments in Teradata, Unix, and Hadoop.Allocate space for Teradata databases in system integration testing and...