Big data engineer

2 days ago

Anantapur, Andhra Pradesh, India Idyllic Services Full time

Job Title: Big Data Engineer – ScalaLocation: Bangalore, Chennai, Gurgaon, Pune, Mumbai.Experience: 7–10 Years (Minimum 3+ years in Scala)Notice Period: Immediate to 30 DaysMode of Work: Hybrid ???? Click the link below to learn more about the role and take the AI Interview to begin your application journey: Pe NRole OverviewWe are looking for a highly skilled Big Data Engineer (Scala) with strong expertise in Scala, Spark, Python, Ni Fi, and Apache Kafka to join our data engineering team. The ideal candidate will have a proven track record in building, scaling, and optimizing big data pipelines, and hands-on experience in distributed data systems and cloud-based solutions.Key Responsibilities- Design, develop, and optimize large-scale data pipelines and distributed data processing systems.- Work extensively with Scala, Spark (Py Spark), and Python for data processing and transformation.- Develop and integrate streaming solutions using Apache Kafka and orchestration tools like Ni Fi / Airflow.- Write efficient queries and perform data analysis using Jupyter Notebooks and SQL.- Collaborate with cross-functional teams to design scalable cloud-based data architectures.- Ensure delivery of high-quality code through code reviews, performance tuning, and best practices.- Build monitoring and alerting systems leveraging Splunk or equivalent tools.- Participate in CI/CD workflows using Git, Jenkins, and other Dev Ops tools.- Contribute to product development with a focus on scalability, maintainability, and performance.Mandatory Skills- Scala – Minimum 3+ years of hands-on experience.- Strong expertise in Spark (Py Spark) and Python.- Hands-on experience with Apache Kafka.- Knowledge of Ni Fi / Airflow for orchestration.- Strong experience in Distributed Data Systems (5+ years).- Proficiency in SQL and query optimization.- Good understanding of Cloud Architecture.Preferred Skills- Exposure to messaging technologies like Apache Kafka or equivalent.- Experience in designing intuitive, responsive UIs for data analytics visualization.- Familiarity with Splunk or other monitoring/alerting solutions.- Hands-on experience with CI/CD tools (Git, Jenkins).- Strong grasp of software engineering concepts, data modeling, and optimization techniques.

  • Big Data Engineering Lead

    4 days ago

    Anantapur, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Senior Data EngineerAbout the RoleWe are seeking a highly skilled and experienced Senior Data Engineer to lead our data engineering team. The successful candidate will be responsible for designing, developing, and maintaining large-scale data processing systems.The ideal candidate will have a strong technical background in big data engineering,...

  • Big Data Developer

    2 weeks ago

    Anantapur, Andhra Pradesh, India DigiHelic Solutions Pvt. Ltd. Full time

    Job description Key Responsibilities: Big Data Architecture: Design, develop, and maintain scalable and distributed data architectures capable of processing large volumes of data. Data Storage Solutions: Implement and optimize data storage solutions using technologies such as Hadoop , Spark , and PySpark . PySpark Development: Develop and implement...

  • Big Data Developer Position

    6 days ago

    Anantapur, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 12,00,000 - ₹ 48,00,000

    Job SummaryWe are seeking a highly skilled Big Data Developer to join our team. This is an exciting opportunity for a talented individual to work on challenging projects and contribute to the growth of our organization.About the RoleThe ideal candidate will have experience in developing big data solutions using Azure Databricks, PySpark, and other relevant...

  • Senior Big Data Expert

    4 days ago

    Anantapur, Andhra Pradesh, India beBeeDataEngineer Full time US$ 90,000 - US$ 1,80,000

    Job Opportunity:We are seeking a highly skilled Data Engineer with strong experience in PySpark and hands-on expertise in building scalable data pipelines. Main Responsibilities:Design, develop, and maintain large-scale data processing systems using PySparkCollaborate with cross-functional teams to identify business needs and develop data-driven...

  • Big Data Specialist in BFSI

    3 days ago

    Anantapur, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job DescriptionWe seek a seasoned Data Engineer to lead our project in the BFSI domain.Key ResponsibilitiesCollaborate with data science teams to implement complex algorithms and analytical solutions.Develop scalable data pipelines using agile development practices.Model front-end and back-end data flows to empower business decision-making through advanced...

  • Data Engineering Specialist

    6 days ago

    Anantapur, Andhra Pradesh, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data Engineering SpecialistWe are seeking a skilled Data Engineer to help implement an application for large retailers. The ideal candidate will have hands-on expertise in Python and PySpark, as well as experience with Telemetry, Big Data, and backend engineering.Key Responsibilities:Design, develop, and deploy data processing pipelines using...

  • High-Performance Data Engineer Expert Wanted

    6 days ago

    Anantapur, Andhra Pradesh, India beBeeData Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    Expert Big Data Engineer Wanted.Our team is looking for a highly skilled data engineer to join our professional services. As a big data engineer at our company, you will be responsible for designing and building high-performance data pipelines that process large volumes of data.The ideal candidate will have experience with Python, big data technologies, and...

  • Senior Data Engineering Professional

    4 days ago

    Anantapur, Andhra Pradesh, India beBeeDataEngineer Full time US$ 1,00,000 - US$ 2,00,000

    Data EngineerJob Summary:We are seeking an experienced Data Engineer to design, build and maintain robust data pipelines and ETL/ELT processes for our marketing data ingestion and processing. The ideal candidate will have a strong background in cloud data platforms, big data technologies and programming languages.Responsibilities:Design and implement...

  • Data Engineering Specialist

    1 week ago

    Anantapur, Andhra Pradesh, India beBeeDataEngineering Full time ₹ 10,00,000 - ₹ 12,00,000

    Job Title: Data EngineerWe are seeking a skilled professional to join our team as a Data Engineer.The ideal candidate will have experience working with data platforms, designing and developing efficient and scalable data systems, and collaborating with cross-functional teams to drive business outcomes.Key Responsibilities:Design and develop data pipelines...

  • Data Engineer

    4 days ago

    Anantapur, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job DescriptionAt our organization, we are seeking an experienced ETL Developer to join our team. The successful candidate will be responsible for designing and developing scalable ETL workflows using Informatica BDM/DEI.This is a full-time on-site role based in Bengaluru for one of our esteemed overseas clients. The role involves understanding requirements,...