Big Data Professional with Distributed Systems Expertise

1 day ago

Anantapur, Andhra Pradesh, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

Job Summary

Our organization is seeking a highly skilled Big Data Professional to join our data engineering team. The ideal candidate will have strong expertise in designing, developing, and optimizing large-scale data pipelines and distributed data processing systems.

Key Responsibilities

We are looking for a professional who can work extensively with Scala, Spark (PySpark), and Python for data processing and transformation. Additionally, the candidate should have hands-on experience with Apache Kafka and orchestration tools like NiFi / Airflow. Other key responsibilities include writing efficient queries, performing data analysis using Jupyter Notebooks and SQL, and collaborating with cross-functional teams to design scalable cloud-based data architectures. Furthermore, we require delivery of high-quality code through code reviews, performance tuning, and best practices. Building monitoring and alerting systems leveraging Splunk or equivalent tools is also essential. Participation in CI/CD workflows using Git, Jenkins, and other DevOps tools is mandatory. Finally, contributing to product development with a focus on scalability, maintainability, and performance is crucial.

Mandatory Skills

To be considered for this role, the candidate must have the following skills: Scala – Minimum 3+ years of hands-on experience, Strong expertise in Spark (PySpark) and Python, Hands-on experience with Apache Kafka, Knowledge of NiFi / Airflow for orchestration, Strong experience in Distributed Data Systems (5+ years), Proficiency in SQL and query optimization, and Good understanding of Cloud Architecture.

Benefits

Our organization offers endless opportunities for personal and professional growth in a collaborative environment. This is the perfect destination for individuals who want to grow their skills and career in the world of data engineering.

About Us

As a company, we provide a commitment to excellence and strive for exceptional results. Our culture emphasizes innovation, teamwork, and continuous improvement. Join us and take advantage of our vast resources and opportunities to advance your career.

  • Big data engineer

    7 days ago

    Anantapur, Andhra Pradesh, India Idyllic Services Full time

    Job Title: Big Data Engineer – ScalaLocation: Bangalore, Chennai, Gurgaon, Pune, Mumbai.Experience: 7–10 Years (Minimum 3+ years in Scala)Notice Period: Immediate to 30 DaysMode of Work: Hybrid ???? Click the link below to learn more about the role and take the AI Interview to begin your application journey: Pe NRole OverviewWe are looking for a highly...

  • Big Data Developer

    3 weeks ago

    Anantapur, Andhra Pradesh, India DigiHelic Solutions Pvt. Ltd. Full time

    Job description Key Responsibilities: Big Data Architecture: Design, develop, and maintain scalable and distributed data architectures capable of processing large volumes of data. Data Storage Solutions: Implement and optimize data storage solutions using technologies such as Hadoop , Spark , and PySpark . PySpark Development: Develop and implement...

  • Big Data Architect

    5 days ago

    Anantapur, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job PostingWe are seeking a skilled Data Engineer to design and implement scalable data architectures.The ideal candidate will have 6-12 years of experience in big data technologies, including PySpark, Scala Spark, and Java Spark SQL.More than 5.8 years of experience in big data projects.Strong understanding of data structures and algorithms.Experience in...

  • Big Data Solution Architect

    5 days ago

    Anantapur, Andhra Pradesh, India beBeeCloudEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Overview">A highly skilled Cloud Engineer is required to design and implement scalable big data solutions.">Key Responsibilities">Develop and deploy data pipelines for batch and real-time processing.Work with distributed data systems and ensure high availability.Collaborate with cross-functional teams to deliver business outcomes.">Requirements">5-8...

  • Cloud Era Big Data Administrator

    2 days ago

    Anantapur, Andhra Pradesh, India beBeeDataiku Full time ₹ 1,80,00,000 - ₹ 2,10,00,000

    Job Title: Data Analyst - Cloud Era Big Data AdministratorAre you looking for a challenging role as a Cloud Era Big Data Administrator? We are seeking an experienced professional to manage and maintain our big data platform.This position involves handling day-to-day operations, administering and monitoring big data platform components, and supporting...

  • Distributed Systems Architect

    5 days ago

    Anantapur, Andhra Pradesh, India beBeeEngineer Full time ₹ 1,20,00,000 - ₹ 2,00,00,000

    Job OpportunityWe are seeking a highly experienced Principal Engineer to play a crucial role in architecting and building the core distributed systems that provide a reliable and scalable foundation.This is an opportunity for hands-on technical leaders who want to solve complex infrastructure challenges and build a platform trusted to make critical business...

  • Senior Big Data Specialist

    3 days ago

    Anantapur, Andhra Pradesh, India beBeeDataProcessor Full time ₹ 75,00,000 - ₹ 1,50,00,000

    Big Data ExpertA seasoned professional in Big Data with a proven track record of delivering large-scale data processing systems using Big Data technologies.Key Responsibilities:Design and implement efficient data pipelines using tools like Apache Hadoop, Spark, and Kafka.Collaborate with cross-functional teams to ensure seamless integration of data-driven...

  • Senior Big Data Developer

    1 day ago

    Anantapur, Andhra Pradesh, India beBeeBig Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Transform Your Career with a Data Engineering RoleJob Overview:This is an exciting opportunity to take your skills in Big Data development to the next level by joining our team. As a seasoned developer, you will have the chance to work on high-impact projects, share your knowledge and expertise with junior engineers, and contribute to solution delivery.You...

  • Data Architect

    4 days ago

    Anantapur, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    Big Data Engineer RoleWe are seeking a highly skilled Big Data Engineer to join our data engineering team.The ideal candidate will have a proven track record in building, scaling, and optimizing big data pipelines, as well as hands-on experience in distributed data systems and cloud-based solutions.Main Responsibilities:Design, develop, and optimize...

  • Big Data Specialist

    4 days ago

    Anantapur, Andhra Pradesh, India beBeeData Full time ₹ 18,00,000 - ₹ 25,00,000

    Job TitleKey ResponsibilitiesAnalyze business requirements and understand technical specifications to deliver high-quality results.Develop software solutions using Spark and Scala, perform unit testing, and deploy changes in higher environments to ensure seamless integration.Identify and rectify bugs/errors to optimize system performance.Collaborate with...