High-Performance Data Engineer

3 days ago

Chennai, Tamil Nadu, India beBeePySparkDeveloper Full time ₹ 5,00,000 - ₹ 8,00,000

We are seeking a seasoned Data Engineer with expertise in PySpark and distributed data processing to lead our team in designing and implementing scalable data solutions.

Key Responsibilities:
  • Design, implement and maintain high-performance PySpark data pipelines on cloud-based platforms like AWS EMR or Databricks.
  • Integrate and synchronize data between enterprise applications and master data management systems using REST APIs and data integration techniques.
  • Create and manage data transformation, cleansing and enrichment logic to ensure data quality and integrity throughout the pipeline lifecycle.
  • Collaborate with architects and analysts to develop effective data models and ETL workflows for data warehousing and business intelligence.
  • Build and manage API-based integrations between data sources and targets for seamless data exchange.
  • Optimize PySpark jobs for performance, scalability and cost-efficiency while ensuring compliance with data governance policies.
  • Troubleshoot and resolve complex data-related and performance issues to minimize downtime and maximize system availability.
Required Skills & Qualifications:
  • At least 7 years of hands-on experience in PySpark, Apache Spark, Spark SQL and DataFrames.
  • Deep expertise in entity modeling, survivorship rules, match and merge configuration for master data management systems.
  • Strong proficiency in data integration techniques, REST APIs, JSON and data formats.
  • Experience with cloud-based services like AWS S3, Lambda and Step Functions.
  • Solid understanding of ETL workflows, data warehousing, data modeling and data governance principles.
  • Familiarity with CI/CD pipelines, version control systems like Git and agile development methodologies.
  • Excellent problem-solving, communication and collaboration skills to work effectively in a team environment.
Bonus Points:
  • Prior experience with Reltio MDM or similar master data management systems.
  • Knowledge of big data processing frameworks like Hadoop and NoSQL databases like Cassandra.
  • Familiarity with containerization technologies like Docker and Kubernetes.

This is an exciting opportunity for a skilled data engineer to take their career to the next level and contribute to the success of our organization. If you have the passion and expertise to succeed in this role, we encourage you to apply.

  • Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Quantrail Data Full time US$ 60,000 - US$ 1,00,000 per year

    Location: Chennai, India (On-site)Duration: 3 monthsStipend: UnpaidEligibility:2025 graduates onlyConversion: Opportunity for a full-time role based on performance Note: Please apply only if you are serious about joining us. This is a learning-focused internship, and we expect interns to be committed to upskilling themselves and contributing to real-world...

  • High-Performance Data Specialist

    4 days ago

    Chennai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 30,00,000 - ₹ 42,00,000

    Key Data Engineer RoleWe are seeking a highly skilled data engineer to join our professional services team. As a key member of our engineering team, you will design, build, and maintain high-performance data pipelines that process large volumes of data.This role is critical in ensuring our data infrastructure can handle increasing volumes of data while...

  • High Performance Data Leader

    2 weeks ago

    Chennai, Tamil Nadu, India beBeeDataLeader Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job DescriptionWe are seeking an experienced professional to lead our high-performance data team. As a key member of our organization, you will be responsible for driving the technical vision for our real-time streaming platform that powers critical trading infrastructure.Your Role- Lead and scale a specialized data & platform engineering team of talented...

  • High-Performance Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job DescriptionWe are seeking an experienced Data Engineer to join our team. As a Senior Data Engineer, you will play a key role in designing and implementing large-scale data pipelines using cloud-based technologies.You will be responsible for analyzing complex data sets from multiple sources, integrating data from various systems, and modeling data to...

  • High Performance Data Architect

    2 weeks ago

    Chennai, Tamil Nadu, India beBeeData Full time ₹ 1,20,00,000 - ₹ 2,00,00,000

    Senior Data EngineerOur organization seeks an accomplished Senior Data Engineer to lead the development and implementation of large-scale data processing systems.The ideal candidate will have a strong background in designing, building, and maintaining high-performance data solutions using Hadoop ecosystem, Spark, and other distributed frameworks.Design,...

  • High-Performance Backend Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India beBeeSoftware Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Advantest, a leading company in the semiconductor industry, is seeking a skilled Staff Software Engineer to join their team. This individual will design and implement reliable backend software solutions that interact with numerous services.Key Responsibilities:Design high-quality, scalable backend software solutions that solve critical customer...

  • High-Performance Data Engineering Specialist

    1 week ago

    Chennai, Tamil Nadu, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 25,00,000

    Senior Data Engineer    Overview of the Role We are seeking an experienced Senior Data Engineer to lead our team in designing, deploying and maintaining data platforms. As a key member of our engineering team, you will work closely with clients to understand their data-related needs, provide recommendations and deliver solutions that meet their...

  • High Performance Data Applications Specialist

    1 week ago

    Chennai, Tamil Nadu, India beBeeDataScience Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Senior Data Engineering PositionWe are seeking an experienced Senior Data Engineer to join our team. As a key member of our data engineering group, you will be responsible for designing and implementing scalable systems, developing real-time data engineering pipelines, and building high-performance applications.You will architect and build petabyte-scale...

  • Senior Lead Data Engineer

    1 week ago

    Chennai, Tamil Nadu, India beBeeDataEngineering Full time ₹ 1,72,10,000 - ₹ 2,54,15,000

    Senior Data Engineering Leader Talent500 seeks a visionary Senior Lead Data Engineer to join their Enterprise Data & Analytics team.This role involves leading the design and implementation of scalable, high-performance data solutions that support various business functions across a world-class manufacturing organization.The successful candidate will drive...

  • Senior Data Engineer

    1 week ago

    Chennai, Tamil Nadu, India Eucloid Data Solutions Full time US$ 1,50,000 - US$ 2,00,000 per year

    About The Role:Eucloid is looking for a senior Data Engineer with hands-on expertise in Databricks to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure on Databricks for our clients by participating in activities which may include starting from up- stream and down-stream...