High Performance Big Data Expert

2 days ago

Vijayawada, Andhra Pradesh, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

Job Title: High Performance Data Engineer

">

We are seeking a highly skilled Data Engineer to join our team and help us optimize our data processing capabilities. Our ideal candidate will have a strong background in designing, building, and maintaining high-performance data pipelines that process large amounts of data.

">

Key Responsibilities include:

">
  • Design and implement scalable batch processing systems using Python and big data technologies">
  • Optimize database performance, focusing on slow-running queries and latency improvements">
  • Use Python profilers and performance monitoring tools to identify bottlenecks">
  • Reduce P95 and P99 latency metrics across our data platform">
  • Build efficient ETL pipelines that can handle large-scale data processing">
  • Collaborate with data scientists and product teams to understand data requirements">
  • Monitor and troubleshoot data pipeline issues in production">
  • Implement data quality checks and validation mechanisms">
  • Document data architecture and engineering processes">
  • Stay current with emerging big data technologies and best practices
">

Qualifications Required:

">
  • Bachelor's degree in Computer Science, Engineering, or related technical field">
  • 4+ years of experience in data engineering roles">
  • Strong Python programming skills with focus on data processing libraries">
  • Experience with big data technologies (Spark, Hadoop, etc.)">
  • Proven experience optimizing database performance (SQL or NoSQL)">
  • Knowledge of data pipeline orchestration tools (Airflow, Luigi, etc.)">
  • Understanding of performance optimization techniques and profiling tools">
  • Experience with cloud platforms (AWS, GCP, or Azure)">
  • Ability to translate business requirements into technical specifications">
  • Ability to work in UK timezone (12 PM IST to 9:30 PM IST)
">

Preferred Qualifications:

">
  • Master's degree in Computer Science or related field">
  • Experience with SEO data or web crawling systems">
  • Experience with Clickhouse Database">
  • Knowledge of distributed systems and microservices architecture">
  • Familiarity with container orchestration (Kubernetes, Docker)">
  • Experience with real-time data processing">
  • Contributions to open-source projects">
  • Experience with machine learning operations
">

Skills and Abilities:

">
  • Strong analytical thinking and problem-solving skills">
  • Excellent attention to detail and commitment to data quality">
  • Ability to work effectively in a collaborative team environment">
  • Good communication skills to explain complex technical concepts">
  • Self-motivated with the ability to work independently">
  • Passion for performance optimization and efficiency
"] ,

  • Big Data Engineering Expert

    2 weeks ago

    Vijayawada, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 60,00,000 - ₹ 1,20,00,000

    Job Title: Senior Big Data EngineerAbout the RoleWe are seeking a highly skilled Senior Big Data Engineer to join our team. The ideal candidate will have hands-on experience with AWS, Python, and PySpark.Key ResponsibilitiesDesign, develop, and deploy scalable big data architectures using AWS services such as Amazon EMR, Amazon Athena, and AWS Glue.Work...

  • High-Performance Data Specialist

    1 week ago

    Vijayawada, Andhra Pradesh, India beBeeBigData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Big Data Expertise SoughtJob OverviewWe are seeking an accomplished Big Data professional to drive the development of robust data pipelines and contribute to our organization's growth.The ideal candidate will be responsible for designing, building, and maintaining high-volume data pipelines that handle financial data from various sources.Key...

  • Big Data Hadoop Scala Expert

    6 days ago

    Vijayawada, Andhra Pradesh, India beBeeData Full time ₹ 8,00,000 - ₹ 12,50,000

    Big Data Hadoop Scala ExpertWe are seeking an experienced Big Data Hadoop Scala professional to join our team. The ideal candidate will have a strong background in data analysis and a solid understanding of big data ecosystems.About the Role:This is a challenging opportunity for someone who wants to work with large datasets and drive business growth through...

  • Big Data Professional

    3 days ago

    Vijayawada, Andhra Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Big Data Developer OpportunityWe're seeking an experienced professional to design and implement scalable big data solutions using cutting-edge technologies.Main Responsibilities:Create high-performance big data architectures leveraging Spark, SQL, Hive, and Scala.Collaborate with cross-functional teams to drive data-driven decision making across the...

  • A Seasoned Big Data Expert Wanted

    6 days ago

    Vijayawada, Andhra Pradesh, India beBeeData Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Big Data SpecialistWe are seeking a talented and experienced Big Data Specialist to join our team. The ideal candidate will have a strong background in big data technologies, including PySpark/Scala Spark/Java Spark SQL.The successful applicant will possess excellent analytical and problem-solving skills, with the ability to design and implement effective...

  • Senior Big Data Systems Specialist

    1 week ago

    Vijayawada, Andhra Pradesh, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Senior Big Data Systems SpecialistWe are seeking an experienced Senior Big Data Systems Specialist to join our team.About the Role:Design, develop and maintain large-scale big data systems.Ensure high-performance and scalability of data systems.Analyze complex problems and implement effective solutions.This role requires a strong foundation in computer...

  • Senior Big Data Engineer

    8 hours ago

    Vijayawada, Andhra Pradesh, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Opportunity:Big Data ExpertA seasoned professional with a strong background in big data technologies is required to implement infrastructure via code and provide strategic advice on Microsoft Azure technologies.Responsibilities:Lead the implementation of infrastructure via code, ensuring efficient development and deployment of Microsoft Azure...

  • Big Data Specialist

    1 week ago

    Vijayawada, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 1,20,00,000 - ₹ 2,50,00,000

    Key ResponsibilitiesWe design, develop and maintain data pipelines for ingestion, transformation and storage of large-scale data.Our team works with Big Data ecosystems to process structured and unstructured data.We develop efficient ETL processes to integrate data from multiple sources into data warehouses/lakes.We write and optimize SQL queries for data...

  • Big Data Platform Manager

    2 days ago

    Vijayawada, Andhra Pradesh, India beBeeDataAdministrator Full time ₹ 15,00,000 - ₹ 25,00,000

    About the Role:We are seeking a skilled Data Administrator to manage our big data platform.Responsibilities include:Handling day-to-day operations, administering and monitoring Big Data Platform components including Hadoop, Hive, HBase, BigSQL, etc.Supporting Technical and Application team requests for data copies across environments, data cleanup, query...

  • Senior Data Analyst

    1 week ago

    Vijayawada, Andhra Pradesh, India beBeeBigData Full time ₹ 80,00,000 - ₹ 1,50,00,000

    Job OverviewAs a seasoned professional, you will be responsible for managing and analyzing large datasets to extract valuable insights.Key Responsibilities:Develop and implement data management strategies to optimize data quality and integrityAnalyze complex data sets to identify trends and patternsTechnical Requirements:Expertise in PySpark and Hive...