Senior Data Pipeline Specialist

3 days ago

Hyderabad, Telangana, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000
Job Opportunity

We are seeking a skilled data engineer to design, build, and optimize robust data pipelines for our platform.

The ideal candidate will ingest, transform, and manage vast amounts of operational data from various devices, ensuring high data quality and availability for analytics and AI/ML applications.

Key Responsibilities:

  1. Design scalable and efficient data ingestion pipelines using IBM StreamSets, Azure Data Factory, Apache Spark, Talend Apache Flink, and Kafka.
  2. Implement robust data transformation and processing logic to clean, enrich, and structure raw data into formats suitable for analysis and machine learning models.
  3. Manage and optimize data storage solutions in MongoDB, ensuring efficient schema design, data indexing, and query performance for large datasets.
  4. Collaborate with data scientists to understand their data needs, provide high-quality, reliable datasets, and assist in deploying data-driven solutions.
  5. Ensure data quality, consistency, and integrity across all data pipelines and storage systems, implementing monitoring and alerting mechanisms for data anomalies.
  6. Work with cross-functional teams to integrate data solutions with the React frontend and Node.js backend applications.
  7. Contribute to the continuous improvement of data architecture, tooling, and best practices, advocating for scalable and maintainable data solutions.
  8. Troubleshoot and resolve complex data-related issues, optimizing pipeline performance and ensuring data availability.
Requirements and Qualifications
  • 5-8 years of professional experience in data engineering or a related field.
  • Proven hands-on experience with data pipeline tools such as IBM StreamSets, Azure Data Factory, Apache Spark, Talend Apache Flink, and Apache Kafka.
  • Strong expertise in database management, particularly with MongoDB, including schema design, data ingestion pipelines, and data aggregation.
  • Proficiency in at least one programming language commonly used in data engineering, such as Python or Java/Scala.
  • Experience with big data technologies and distributed processing frameworks is highly desirable.
  • Familiarity with cloud platforms (Azure, AWS, or GCP) and their data services.
  • Solid understanding of data warehousing concepts, ETL/ELT processes, and data modeling.
  • Experience with DevOps practices for data pipelines (CI/CD, monitoring, logging).
  • Knowledge of Node.js and React environments to facilitate seamless integration with existing applications.
Qualifications
  • Demonstrated expertise in written and verbal communication, adept at simplifying complex technical concepts for both technical and non-technical audiences.
  • Strong problem-solving and analytical skills with a meticulous approach to data quality.
  • Experienced in collaborating and communicating seamlessly with diverse technology roles, including development, support, and product management.
  • Highly motivated to acquire new skills, explore emerging technologies, and stay updated on the latest trends in data engineering and business needs.

  • Data Pipeline Specialist

    6 days ago

    Hyderabad, Telangana, India beBeeDataPipeline Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job OverviewWe are seeking an accomplished Data Pipeline Specialist to join our team. The successful candidate will be responsible for creating and maintaining data pipelines that drive business growth.Main ResponsibilitiesCreate and implement new data pipelines to meet business needsEssential Skills and Qualifications4-6 years of experience in designing and...

  • Data Pipelines Expert

    1 week ago

    Hyderabad, Telangana, India beBeeDataSpecialist Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job DescriptionWe are seeking a skilled Technical Data Specialist with expertise in designing, developing, and maintaining robust and scalable data pipelines. The ideal candidate will be responsible for implementing efficient data solutions, collaborating with cross-functional teams, and ensuring high-quality data delivery.The Technical Data Specialist will...

  • Data Pipeline Architect

    4 days ago

    Hyderabad, Telangana, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Senior Data EngineerWe are seeking a seasoned professional to fill the role of Senior Data Engineer. The successful candidate will design and develop end-to-end data integration solutions, optimize workflows, and collaborate with stakeholders to deliver scalable and high-performing data pipelines.Develop ETL/ELT workflows using Informatica...

  • Lead Data Pipeline Specialist

    5 days ago

    Hyderabad, Telangana, India beBeeData Full time ₹ 1,55,52,000 - ₹ 2,59,20,000

    Senior Data EngineerWe seek an expert in designing and maintaining large-scale data pipelines on Azure using Databricks, ADF, and PySpark. This individual will ensure high-quality, secure, and reliable datasets for analytics and customer-facing applications.Key ResponsibilitiesDesign, build, and optimise large-scale data pipelines using Databricks, Delta...

  • Senior Data Pipeline Specialist

    3 days ago

    Hyderabad, Telangana, India beBeeDataEngineer Full time ₹ 6,00,000 - ₹ 10,00,000

    Job Description">We are seeking a highly skilled data professional to lead our efforts in designing and maintaining efficient, scalable data pipelines and analytics platforms. The ideal candidate will have hands-on experience in Azure Data Factory, Azure Databricks, PySpark, and .NET Framework. "The selected individual will be responsible for application...

  • Data Pipeline Specialist

    3 days ago

    Hyderabad, Telangana, India beBeeDataPipelineManager Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    Job Title: Data Team LeadData Pipeline ManagerWe are seeking a seasoned data pipeline manager with experience in designing and developing robust data pipelines using Azure Databricks, Data Factory, and Datalake.The role involves overseeing the development of complex ETL solutions, managing large volumes of data, and working closely with business teams to...

  • Senior Data Operations Specialist

    5 days ago

    Hyderabad, Telangana, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Data Operations SpecialistWe are seeking an experienced Senior Data Operations Specialist to support our data organization. This role involves maintaining data pipelines and corresponding platforms while collaborating with global teams on DataOps initiatives.Maintain day-to-day operations of data pipelines, ensuring data governance, reliability, and...

  • Lead/Senior Data Engineer

    2 days ago

    Hyderabad, Telangana, India Data Economy Full time US$ 1,50,000 - US$ 2,00,000 per year

    We are seeking a Lead/Senior Data Engineer with 7-12 years of experience to architect, develop, and optimize data solutions in a cloud-native environment. The role requires strong expertise in AWS Glue, PySpark, and Python with a proven ability to design scalable data pipelines and frameworks for large-scale enterprise systems. Prior exposure to financial...

  • Senior Data Architecture Specialist

    7 days ago

    Hyderabad, Telangana, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000

    **Job Title:** Senior Data Architecture Specialist**Job Description:** We are seeking an experienced Senior Data Architecture Specialist to lead our data engineering team. The ideal candidate will have strong experience in designing, developing, and maintaining robust and scalable data pipelines using Python.The successful candidate will be responsible for...

  • Data Pipeline Specialist

    7 days ago

    Hyderabad, Telangana, India beBeeBusiness Full time ₹ 12,00,000 - ₹ 20,00,000

    Job Title: Business Intelligence ArchitectWe are seeking an experienced Business Intelligence Architect with a strong background in data engineering and a passion for designing, building, and optimizing scalable data pipelines.The ideal candidate will have hands-on experience in Snowflake, AWS Lambda, Glue, DBT, and SQL. They will be responsible for...