Distributed Data Engineer Position

1 day ago

Chennai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

Job Title:

Senior Distributed Data Engineer

Job Description:

We are seeking a seasoned expert in distributed data processing with extensive experience in designing and implementing scalable solutions using PySpark. The ideal candidate will have expertise in Apache Spark, Reltio MDM, and big data ecosystems.

Key Responsibilities:

  • Design and implement PySpark data pipelines on platforms like AWS EMR or Databricks for efficient data processing and analysis.
  • Develop and maintain complex data transformation, cleansing, and enrichment logic to ensure data quality and integrity.
  • Collaborate with architects and analysts to design effective data models and optimize data workflows.
  • Build and manage API-based integrations between Reltio and upstream/downstream systems for seamless data exchange.
  • Optimize PySpark jobs for performance, scalability, and cost-efficiency to meet business requirements.

Required Skills & Qualifications:

  • 8+ years of hands-on experience in PySpark, Apache Spark, and distributed data processing.
  • Strong command of data integration techniques, REST APIs, and JSON data formats.
  • Deep expertise in Reltio MDM (entity modeling, survivorship rules, match & merge configuration).
  • Proficiency in ETL workflows, data warehousing, and data modeling principles.
  • Excellent problem-solving, communication, and collaboration skills.

  • Data Engineer

    23 hours ago

    Chennai, Tamil Nadu, India NTT DATA Full time US$ 80,000 - US$ 1,20,000 per year

    Req ID: 336026NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer - Azure to join our team in Chennai, Tamil Nādu (IN-TN), India (IN). "Job Duties: Key Responsibilities:...

  • Distributed Systems Engineer

    2 days ago

    Chennai, Tamil Nadu, India beBeeSoftware Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Overview:We are seeking a highly skilled software engineer to join our team. The ideal candidate will have experience in developing distributed applications in a multi-tenanted cloud environment.Key Responsibilities:Design and implement large-scale distributed data processing frameworks like Spark, Hadoop, and YARN.Develop techniques for cluster...

  • Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India NTT DATA North America Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer - Azure to join our team in Chennai, Tamil Nādu (IN-TN), India (IN)."Job Duties: Key Responsibilities:•  ...

  • Data Engineer

    22 hours ago

    Chennai, Tamil Nadu, India NTT DATA North America Full time US$ 80,000 - US$ 1,20,000 per year

    Req ID:336025NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer - Azure to join our team in Chennai, Tamil Nādu (IN-TN), India (IN)."Job Duties: Key...

  • Data Engineer

    3 weeks ago

    Chennai, Tamil Nadu, India Quantrail Data Full time

    Location: Chennai, India (On-site)Duration: 3 monthsStipend: UnpaidEligibility: 2025 graduates onlyConversion: Opportunity for a full-time role based on performance Note: Please apply only if you are serious about joining us. This is a learning-focused internship, and we expect interns to be committed to upskilling themselves and contributing to real-world...

  • Data Engineer

    24 hours ago

    Chennai, Tamil Nadu, India Quantrail Data Full time US$ 60,000 - US$ 1,00,000 per year

    Location: Chennai, India (On-site)Duration: 3 monthsStipend: UnpaidEligibility:2025 graduates onlyConversion: Opportunity for a full-time role based on performance Note: Please apply only if you are serious about joining us. This is a learning-focused internship, and we expect interns to be committed to upskilling themselves and contributing to real-world...

  • Data Architect Position

    3 days ago

    Chennai, Tamil Nadu, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Data Architect PositionAbout the RoleWe are seeking a highly skilled data professional to oversee end-to-end data pipeline development and management for our financial datasets.Design, develop, and manage large-scale data pipelines for stocks, crypto, and other financial data sources.Integrate third-party APIs and data feeds into our internal systems.Create...

  • Career Opportunity

    4 days ago

    Chennai, Tamil Nadu, India beBeeData Engineer Full time ₹ 1,04,000 - ₹ 1,30,878

    Job Summary">This role is responsible for participating as an individual contributor in project teams, troubleshooting operational issues, and providing technical solutions to operational problems.The ideal candidate will have experience with Python, SQL, and Bash Scripting, as well as knowledge of data pipeline fundamentals and general understanding of...

  • Automation Engineer for Distributed Systems

    3 days ago

    Chennai, Tamil Nadu, India beBeeAutomationEngineer Full time ₹ 1,08,62,100 - ₹ 1,73,35,100

    Job Title:Automation Engineer for Distributed SystemsAs an Automation Engineer for Distributed Systems, you will play a pivotal role in ensuring the quality and reliability of our distributed database management platform. Your expertise will be instrumental in designing and implementing robust automation frameworks, validating complex workflows across hybrid...

  • Spark Developer

    1 day ago

    Chennai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 18,12,500 - ₹ 23,40,000

    Senior Data Processing SpecialistElevate your career in data engineering and drive business growth as a Senior Data Processing Specialist. This exciting role offers the opportunity to design, implement, and optimize distributed data processing jobs to handle large-scale data in Hadoop Distributed File System (HDFS) using Apache Spark and Python.This position...