Data Pipeline Specialist

2 days ago

Kanpur, Uttar Pradesh, India beBeeMachine Full time ₹ 25,00,000 - ₹ 28,00,000
Senior Machine Learning Engineer

Key Responsibilities:

  • Develop and optimize complex data pipelines, applying machine learning engineering principles to enhance efficiency and scalability.
  • Integrate and optimize data and model pipelines within production environments, diagnosing data inconsistencies and documenting assumptions.
  • Employ experimental methodologies, statistics, and machine learning concepts to create self-running AI systems for predictive modeling.
  • Collaborate with data science teams to review model-ready datasets and feature documentation, ensuring completeness and accuracy.
  • Perform data discovery and analysis of raw data sources, applying business context to meet model development needs.
  • Comfort with exploratory data exploration and tracking data lineage during inception or root cause analysis.
  • Engage with internal stakeholders to understand business processes and translate requirements into analytical approaches.
  • Write and maintain model monitoring scripts, diagnosing issues and coordinating resolutions based on alerts.
  • Serve as a domain expert in machine learning engineering on cross-functional teams for significant initiatives.
  • Stay updated with the latest advancements in AI/ML and apply them to real-world challenges.

Qualifications:

  • Undergraduate degree or equivalent experience; a graduate degree is preferred.
  • Minimum of 8 years of relevant work experience.
  • At least 3 years of hands-on experience designing ETL pipelines using AWS services (e.g., Glue, SageMaker).
  • Proficiency in programming languages, particularly Python (including PySpark, PySQL) and familiarity with machine learning libraries and frameworks.
  • Strong understanding of cloud technologies, including AWS and Azure, and experience with NoSQL databases.
  • Familiarity with Feature Store usage, LLMs, GenAI, RAG, Prompt Engineering, and Model Evaluation.
  • Experience with API design and development is a plus. Docker Kubernetes
  • Solid understanding of software engineering principles, including design patterns, testing, security, and version control.
  • Knowledge of Machine Learning Development Lifecycle (MDLC) best practices and protocols.
  • Understanding of solution architecture for building end-to-end machine learning data pipelines.

Benefits:

For more information please refer to our company's website

  • Data Pipeline Specialist

    6 days ago

    Kanpur, Uttar Pradesh, India beBeeData Full time ₹ 10,00,000 - ₹ 20,00,000

    Big Data EngineerWe are seeking a highly skilled professional to join our team as a Big Data Engineer. In this role, you will be responsible for designing, building, and maintaining robust data pipelines that handle high-volume financial data.Key Responsibilities:Design, develop, and manage end-to-end data pipelines for stocks, crypto, and other financial...

  • Data Pipeline Specialist

    2 days ago

    Kanpur, Uttar Pradesh, India beBeeBusiness Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Title: Business Intelligence Developer","Role Summary:","We are seeking an experienced professional to design and implement scalable data pipelines using industry-standard tools.","Key Responsibilities:","Design and implement scalable data pipelines.","Develop ETL/ELT solutions within a cloud-based environment.","Manage and optimize data storage and...

  • Data Pipeline Specialist

    7 days ago

    Kanpur, Uttar Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 17,50,000

    About the RoleWe're seeking a highly skilled Data Engineer to build robust pipelines that transform large, messy technical datasets into high-quality structured data.

  • Financial Data Pipeline Specialist

    3 days ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    We are seeking a highly skilled data engineer to take on the challenge of designing and building robust data pipelines that handle high-volume financial data.Key Responsibilities:Design and develop end-to-end data pipelines for stocks, cryptocurrencies, and other financial datasets.Integrate third-party APIs and data feeds into internal systems.Build and...

  • Data Pipeline Specialist

    7 days ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    Job OpportunityWe are seeking a skilled professional to take on the role of Data Engineer, responsible for designing, developing, and maintaining data pipelines that ingest data from various sources into our enterprise-wide data warehouse system.Utilize cutting-edge technologies like AWS services (AWS Lambda, AWS DMS, and AWS Glue) for data ingestion and...

  • Data Pipeline Architect

    2 days ago

    Kanpur, Uttar Pradesh, India beBeeDataPipeline Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data Pipeline ArchitectWe are seeking an experienced professional to design and build scalable data pipelines for processing channel activity data from multiple sources.The ideal candidate will possess strong knowledge of AWS cloud infrastructure, particularly services like Pyspark, Glue, EMR, lambda, Redshift, Athena, and other tools like...

  • Data Pipeline Optimization Specialist

    16 hours ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,90,00,000 - ₹ 2,15,00,000

    About the RoleWe are looking for a seasoned professional to join our Data Engineering team in Mumbai. This role requires working at least three days a week in the office, with a focus on optimizing data pipelines and developing a deep understanding of digital trading platforms.

  • Pipeline Intervention Specialist

    1 week ago

    Kanpur, Uttar Pradesh, India beBeePipeline Full time ₹ 80,00,000 - ₹ 1,50,00,000

    Job Title: Pipeline Intervention SpecialistThe pipeline industry is seeking a highly skilled professional to conduct hot tapping and pipeline isolation services on-site.Key Responsibilities:Conduct hot tapping and pipeline isolation services in accordance with industry standards and regulations.Collaborate with project teams to ensure safe and efficient...

  • Data Pipeline Engineer

    6 days ago

    Kanpur, Uttar Pradesh, India beBeeETL Full time ₹ 10,00,000 - ₹ 15,00,000

    Senior ETL DeveloperWe are seeking an accomplished Senior ETL Developer to join our Data Engineering team. The ideal candidate will have a proven track record of designing, developing, and maintaining scalable and efficient data pipelines using IBM DataStage, AWS Glue, and Snowflake.The role involves collaborating with architects, business analysts, and data...

  • Data Pipeline Architect

    2 weeks ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    About the Role:The ideal candidate will have a solid foundation in data engineering with a minimum of 7 years of experience in development on data-centric projects.Mandatory Skills:Expertise in working with real-time data and Kafka framework, including kSQL/Mirror Maker etc.Proficiency in at least one programming language: Groovy or Java.Thorough...