iKrux - Big Data Engineer - Python/Scala

1 week ago

Pune, Maharashtra, India Ikrux Engineeering Full time

We are seeking a highly skilled and motivated Big Data Engineer with strong experience in building, optimizing, and managing large-scale data processing pipelines. The ideal candidate will have hands-on expertise in Scala, Python, Apache Spark, Kafka, and/or Apache NiFi, and will work closely with cross-functional teams to design and implement scalable data solutions that drive business intelligence and analytics.

Key Responsibilities :

- Design, build, and maintain large-scale distributed data processing systems using Spark, Kafka, and/or NiFi.

- Write efficient and reusable code in Scala and Python for data transformation, ingestion, and enrichment.

- Collaborate with data scientists, analysts, and engineers to ensure data quality, consistency, and availability.

- Optimize data pipelines for performance, reliability, and scalability.

- Integrate diverse data sources, ensuring data governance and security compliance.

- Monitor and troubleshoot data workflows, resolving bottlenecks and issues.

- Participate in code reviews, architecture discussions, and best practice development.

- Stay updated on emerging big data technologies and frameworks to continuously improve system efficiency.

Required Skills and Experience :

- Strong programming experience in Scala and Python for data engineering.

- Extensive experience with Apache Spark, including optimization, tuning, and resource management.

- Hands-on experience with Kafka for real-time data streaming and messaging systems.

- Experience with Apache NiFi for data flow automation and integration.

- Working knowledge of Hadoop ecosystem components is a plus.

- Understanding of cloud-based data solutions (AWS, Azure, or GCP) is desirable.

- Familiarity with relational and NoSQL databases (e.g., PostgreSQL, MongoDB, Cassandra).

- Experience in working with data warehousing solutions and ETL pipelines.

- Good analytical, problem-solving, and debugging skills.

- Excellent communication and teamwork abilities.

(ref:hirist.tech)

  • iKrux - Data Engineer - Java/ETL

    2 weeks ago

    Pune, Maharashtra, India Ikrux Engineeering Full time

    Role : Data EngineerWe are looking for an experienced Data Engineer with a strong foundation in Java, AWS, SQL, and Apache Spark to join our growing data engineering team. The ideal candidate will be responsible for building scalable data pipelines, ensuring data quality and reliability, and contributing to the overall data infrastructure.Key...

  • Big Data Architect

    2 weeks ago

    Pune, Maharashtra, India beBeeBigDataEngineer Full time US$ 1,20,000 - US$ 1,75,000

    Senior Big Data EngineerAs a seasoned Big Data Engineer, you will be responsible for architecting and building large-scale data pipelines using Apache Spark and Databricks.Key Responsibilities:Design and implement complex data pipelines to process and analyze large datasets.Develop expertise with streaming systems such as Apache Kafka and Confluent...

  • Big Data Expert

    2 weeks ago

    Pune, Maharashtra, India beBeeData Full time ₹ 80,00,000 - ₹ 1,20,00,000

    Big Data DeveloperThis role is ideal for a skilled and ambitious professional with expertise in big data technologies. As a Big Data Developer, you will play a key part in designing, developing, and maintaining large-scale data processing systems using Hadoop, Spark, and NoSQL databases.Key Responsibilities:Develop scalable data pipelines using Apache Kafka,...

  • Big Data Engineer

    2 weeks ago

    Pune, Maharashtra, India Merito Full time

    Job Description :We are looking for a Big Data Engineer who will work on building, and managing Big Data Pipelines for us to deal with the huge structured data sets that we use as an input to accurately generate analytics at scale for our valued Customers. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining,...

  • Big Data Engineer

    2 weeks ago

    Pune, Maharashtra, India WorkiFicient Full time

    To be successful in this role, you should possess : - Collaborate closely with Product Management and Engineering leadership to devise and build the right solution.- Participate in Design discussions and brainstorming sessions to select, integrate, and maintain Big Data toolsand frameworks required to solve Big Data problems at scale.- Design and implement...

  • Big Data Architecture Expert

    1 week ago

    Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 35,00,000

    Senior Big Data EngineerWe are seeking a highly skilled and experienced Senior Big Data Engineer to fill this role.Design, develop, and maintain scalable and high-performance data processing systems for maximum efficiency.Build robust ETL/ELT workflows for structured and unstructured data using big data technologies like Hadoop, Spark, Hive, HBase, Kafka,...

  • Big Data Specialist

    2 weeks ago

    Pune, Maharashtra, India beBeeBigData Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Data Expert - Big Data SpecialistDescription:We are seeking a highly skilled professional to join our team. The ideal candidate will have extensive experience with big data technologies and a strong background in developing and optimizing data integration frameworks and applications.Main Responsibilities:Data Solutions: Design and implement robust...

  • Big Data Engineer

    1 week ago

    Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Microsoft Azure Data Engineer OpportunityWe are seeking an experienced Microsoft Azure Data Engineer to join our team.This role requires a deep understanding of data engineering principles, experience with cloud-based platforms, and the ability to design and implement scalable data architectures.Key Responsibilities:Design and implement data pipelines using...

  • Big Data Solutions Specialist

    2 weeks ago

    Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 25,00,000

    Job Overview:We are seeking a highly skilled Data Engineer with strong expertise in big data processing, data analytics, and cloud architecture to join our team.Key Responsibilities:Design, develop, and optimize large-scale data pipelines and distributed data processing systems.Work extensively with Scala, Spark (PySpark), and Python for data processing and...

  • Big Data Developer

    2 weeks ago

    Pune, Maharashtra, India MRO Full time

    Analyst - Big Data DeveloperWe are seeking a highly skilled Analyst -Big Data Developer to join our dynamic team.The ideal candidate will have extensive experience with big data technologies and a strong background in developing and optimizing data integration frameworks and applications.You will be responsible for designing, implementing, and maintaining...