Senior Big Data Engineer

2 days ago

Palakkad, Kerala, India beBeeData Full time ₹ 19,00,800 - ₹ 24,24,000
Key Responsibilities

As a Senior Data Engineer, you will be responsible for driving the design and development of large-scale data processing systems. Your key responsibilities include:

  • Studying the existing technology landscape to identify areas for improvement.
  • Developing and enhancing Spark jobs using Scala to meet new project needs.
  • Performing performance tuning of Spark jobs, stress testing, and other optimization techniques.
  • Creating new data pipelines for developed/enhanced Spark jobs using AWS Lambda or Apache Airflow.
  • Designing databases, including logical and physical design, star schema, and snowflake schema.
  • Analyzing data processing, integration, modeling, and reporting requirements to define data loading strategies.
  • Evaluating the optimal balance between cost and performance.
  • Managing the build phase and ensuring code quality meets requirements and adheres to Cloud architecture standards.
Technical Requirements

The following technical skills are required for this position:

  • 7-9 years of IT experience.
  • 5+ years of experience in AWS-related projects.
  • Associate Level and Professional Level AWS Certification (good to have).
  • In-depth knowledge of the following AWS Services: S3, EC2, EMR, Serverless, Athena, AWS Glue, Lambda, Step Functions.
  • Cloud Databases: AWS Aurora, Singlestore, RedShift, Snowflake.
  • Big Data: Hadoop, Hive, Spark, YARN.
  • Programming Languages: Scala, Python, Shell Scripts, PySpark.
  • Operating Systems: Any flavor of Linux, Windows.
  • Strong SQL skills.
  • Orchestration Tools: Apache Airflow.
  • Expertise in developing ETL workflows with complex transformations.
  • Thorough conceptual understanding of AWS VPC, Subnets, Security Groups, and Route Tables.
Qualifications and Benefits

The following qualifications and benefits are offered for this position:

  • Bachelor's degree in Computer Science, Engineering, or related field (Master's degree is a plus).
  • Demonstrated continued learning through technical certifications or related methods.
  • Minimum 5 years of experience on Cloud-related projects.
  • Opportunities for professional growth and development.
  • A competitive salary and benefits package.
Working Style and Qualities

The ideal candidate will possess the following working style and qualities:

  • Hold strong technical knowledge and experience.
  • Capability to deep dive and research in various technical fields.
  • Self-motivated and focused on delivering outcomes for a fast-growing team and firm.
  • Able to communicate persuasively through speaking, writing, and client presentations.
  • Able to consult, write, and present persuasively.
  • Able to work in a self-organized and cross-functional team.
  • Able to iterate based on new information, peer reviews, and feedback.

  • Big Data Visionary

    2 days ago

    Palakkad, Kerala, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job DescriptionWe are seeking a seasoned Big Data professional to lead our data engineering efforts. As a Senior Big Data Engineer, you will be responsible for designing and implementing large-scale data processing systems that drive business growth.Team Leadership: Proven experience in leading high-performing data engineering teams across multiple...

  • Big Data Engineer

    2 days ago

    Palakkad, Kerala, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000

    Job OverviewAs a skilled professional in Big Data engineering, you will be responsible for developing and implementing scalable data processing solutions using cutting-edge technologies.With 2+ years of experience in the field, you will have a solid understanding of Big Data concepts, Scala programming language, AWS cloud platform, and either Python or...

  • Senior Big Data Specialist

    2 days ago

    Palakkad, Kerala, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Senior Big Data SpecialistDescriptionWe are seeking a highly skilled Senior Big Data Specialist to join our team. The ideal candidate will have a minimum of 3-5 years of experience in designing and implementing data pipelines using various technologies.The successful candidate will be responsible for developing and maintaining large-scale data...

  • Big Data Solutions Architect

    1 week ago

    Palakkad, Kerala, India beBeeData Full time ₹ 10,00,000 - ₹ 15,00,000

    Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer with hands-on expertise in Python and Pyspark. This role requires someone who can implement a large-scale data engineering application for a prominent retailer.Key Responsibilities:Design, develop, and deploy scalable data processing pipelines using Pyspark.Collaborate with...

  • Principal Big Data Engineer

    6 days ago

    Palakkad, Kerala, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data Engineer- Apache DruidThis role is a great opportunity to utilize your technical skills and experience in big data technologies. We are seeking a skilled engineer to join our team.Apache Druid Experience:We are looking for someone with strong hands-on experience working with Apache Druid in production environments.Required Skills and...

  • Principal Big Data Architect

    2 days ago

    Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Data EngineerWe are seeking a skilled professional to design, build, and maintain robust data pipelines for processing channel activity data. The ideal candidate will have experience in digital channel activities, data engineering, and cloud technologies.Key Responsibilities:Design and develop scalable data architectures using AWS cloud infrastructure and...

  • Big Data Solutions Lead

    17 hours ago

    Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 24,00,000

    Job Title: Data EngineerAbout the RoleWe are seeking a highly skilled professional to lead the design and implementation of big data solutions.The ideal candidate will have 5-12 years of experience in Big Data & Data related technology, with expertise in distributed computing principles and Apache Spark.Responsibilities:Designing and implementing big data...

  • Big Data Specialist

    6 days ago

    Palakkad, Kerala, India beBeeData Full time ₹ 12,00,000 - ₹ 20,00,000

    We are seeking a Big Data professional to join our team.Key Responsibilities:Analyzing large datasets using PySpark and Hive.Desirable Skills:Proficiency in Spark and HBase for efficient data processing.Data Quality tools expertise for data accuracy and reliability.Awareness of Agile methodologies for effective collaboration.Familiarity with data ingestion...

  • Big Data Architect

    2 days ago

    Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    Role OverviewThe Data Engineer II will design, execute, and manage large-scale distributed data systems to drive business insights and growth.Key Responsibilities:Design and execute complex distributed data systems.Monitor and optimize existing projects for performance.Research and integrate new big data tools and frameworks as required.Implement scalable...

  • Big Data Solutions Architect

    15 hours ago

    Palakkad, Kerala, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Position OverviewOur organization is seeking a seasoned Data Engineer to spearhead the development of scalable data solutions.The ideal candidate will possess a deep understanding of big data technologies, including HDFS, Hive, and Spark architecture. Proficiency in working with Python and PySpark programming languages is also essential.Key...