Big Data Expertise

3 days ago

Kottayam, Kerala, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 25,00,000

Job Title: Data Engineering Expertise

Overview

We are seeking a skilled Data Engineer to design and build high-performance data pipelines that process massive amounts of data. The ideal candidate will have expertise in big data technologies, Python programming skills, and experience with database optimization.

Key Responsibilities

  • Design and implement scalable batch processing systems using Python and big data technologies.
  • Optimize database performance, focusing on slow-running queries and latency improvements.
  • Use Python profilers and performance monitoring tools to identify bottlenecks.
  • Reduce P95 and P99 latency metrics across our data platform.
  • Build efficient ETL pipelines that can handle large-scale data processing.
  • Collaborate with data scientists and product teams to understand data requirements.
  • Monitor and troubleshoot data pipeline issues in production.
  • Implement data quality checks and validation mechanisms.
  • Document data architecture and engineering processes.
Qualifications

  • Bachelor's degree in Computer Science, Engineering, or related technical field.
  • 4+ years of experience in data engineering roles.
  • Strong Python programming skills with focus on data processing libraries.
  • Experience with big data technologies (Spark, Hadoop, etc.).
  • Proven experience optimizing database performance (SQL or NoSQL).
  • Knowledge of data pipeline orchestration tools (Airflow, Luigi, etc.).
  • Understanding of performance optimization techniques and profiling tools.
  • Experience with cloud platforms (AWS, GCP, or Azure).
  • Ability to translate business requirements into technical specifications.
  • Ability to work effectively in a collaborative team environment.
Preferred Qualifications

  • Master's degree in Computer Science or related field.
  • Experience with SEO data or web crawling systems.
  • Experience with Clickhouse Database.
  • Knowledge of distributed systems and microservices architecture.
  • Familiarity with container orchestration (Kubernetes, Docker).
  • Experience with real-time data processing.
  • Contributions to open-source projects.
  • Experience with machine learning operations.
Skills and Abilities

  • Strong analytical thinking and problem-solving skills.
  • Excellent attention to detail and commitment to data quality.
  • Good communication skills to explain complex technical concepts.
  • Self-motivated with the ability to work independently.
  • Passion for performance optimization and efficiency.

  • Big Data Specialist

    1 day ago

    Kottayam, Kerala, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Big Data ProfessionalWe are seeking a highly skilled professional to work with large datasets and possess expertise in data analysis, visualization, and processing.The ideal candidate will have experience working with big data technologies and skills in:PySpark: To analyze and process large datasetsHive: For storing and querying large datasetsIn addition to...

  • Big Data Architect

    1 day ago

    Kottayam, Kerala, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Role: Big Data ArchitectWe are seeking a seasoned data expert to drive the development and implementation of our big data infrastructure.The ideal candidate will have a strong background in designing, building, and maintaining large-scale data systems, including data warehousing, ETL/ELT pipelines, and data governance.Key Responsibilities:* Design and...

  • Senior Data Scientist

    24 hours ago

    Kottayam, Kerala, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Big Data Engineer PositionWe are seeking a skilled Big Data Engineer to lead the design, development, and maintenance of scalable data pipelines and ETL processes.About the RoleThe ideal candidate will have a strong background in data engineering and experience working with big data ecosystems. They will collaborate with stakeholders to gather requirements...

  • Building Scalable Big Data Platforms

    1 day ago

    Kottayam, Kerala, India beBeeTechnical Full time ₹ 20,00,000 - ₹ 30,00,000

    Big Data Platform EngineerSigmoid empowers enterprises to make data-driven decisions by integrating advanced data engineering with AI consulting. We collaborate with leading organizations across sectors such as CPG-retail, BFSI, life sciences, manufacturing, and more to solve complex business challenges.As a Technical Lead, you will build a highly scalable...

  • Senior Big Data Specialist

    3 days ago

    Kottayam, Kerala, India beBeeData Full time ₹ 1,68,00,000 - ₹ 2,03,20,000

    Job OverviewDevelop high-performance data pipelines using AWS and PythonCollaborate with cross-functional teams to implement scalable big data solutionsRequired Skills & QualificationsProficiency in Python and PySparkHands-on experience with version control tools like GitFamiliarity with Amazon's Analytics services such as AWS Glue, Amazon Athena, and...

  • Principal Big Data Specialist

    1 day ago

    Kottayam, Kerala, India beBeeDataEngineering Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Data EngineerOverview:We are looking for a skilled professional to design, develop and maintain large-scale data systems.About the Role:This is a key position within our organization, responsible for creating, implementing and maintaining scalable data solutions.The ideal candidate will have strong technical skills, excellent problem-solving...

  • Data Architect

    1 day ago

    Kottayam, Kerala, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Big Data Specialist PositionWe are seeking a highly skilled Big Data Specialist with extensive expertise in data engineering, Scala, Spark, Python, NiFi, and Apache Kafka.The ideal candidate will have a strong track record of designing, developing, and optimizing large-scale data pipelines and distributed data processing systems.Design and develop scalable...

  • Senior Data Developer

    1 day ago

    Kottayam, Kerala, India beBeeDataAnalyst Full time ₹ 15,00,000 - ₹ 25,00,000

    Seeking a skilled Data Analyst with expertise in software development and data analysis. The ideal candidate will design, implement, and optimize software solutions using Python.About the RoleThis position involves working with large datasets, developing insights, and collaborating with cross-functional teams to drive business growth.Key...

  • Large Scale Data Architect

    6 days ago

    Kottayam, Kerala, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    **Job Title:** Senior Data Engineer – GCPAbout the RoleWe are seeking a seasoned data engineer to lead our team in building large-scale data platforms on Google Cloud. As a key member of our team, you will design and implement robust data pipelines for ingestion, transformation, and analytics.Your expertise in the GCP ecosystem, combined with your...

  • Big Data Technical Lead

    3 days ago

    Kottayam, Kerala, India beBeeData Full time ₹ 90,00,000 - ₹ 1,50,00,000

    Senior Data ArchitectJob Title: Senior Data ArchitectThe role involves designing scalable data pipelines, managing large datasets and collaborating with stakeholders to deliver high-quality solutions.Key Responsibilities:Design, build and optimize complex data systems.Manage large datasets using Spark, PySpark and SQL.Build and maintain real-time...