Senior Data Processing Engineer

1 week ago

Mumbai Kolkata Delhi, India beBeeData Full time ₹ 18,00,000 - ₹ 30,00,000

Job Title: Senior Data Processing Engineer


About the Role:

We are seeking a highly skilled and experienced Senior Data Processing Engineer to join our team. As a key member of our engineering group, you will be responsible for designing and implementing high-performance data processing solutions using Apache Spark and Scala.

The ideal candidate will have a strong background in distributed data processing, experience with Cloudera Hadoop ecosystem, and excellent problem-solving skills. You will work closely with our data scientists, business analysts, and other engineers to deliver solutions that meet our business requirements.


Key Responsibilities:

  • Design and implement high-performance data processing solutions using Apache Spark and Scala.
  • Develop, test, and deploy data processing applications using Apache Spark and Scala.
  • Optimize and tune Spark applications for better performance on large-scale data sets.
  • Work with the Cloudera Hadoop ecosystem (e.g., HDFS, Hive, Impala, HBase, Kafka) to build data pipelines and storage solutions.
  • Collaborate with data scientists, business analysts, and other developers to understand data requirements and deliver solutions.
  • Ensure data integrity, accuracy, and security across all processing tasks.
  • Troubleshoot and resolve performance issues in Spark, Cloudera, and related technologies.
  • Implement version control and CI/CD pipelines for Spark applications.


Requirements:

  • Minimum 8 years of experience in application development.
  • Strong hands-on experience in Apache Spark, Scala, and Spark SQL for distributed data processing.
  • Hands-on experience with Cloudera Hadoop (CDH) components such as HDFS, Hive, Impala, HBase, Kafka, and Sqoop.
  • Familiarity with other Big Data technologies, including Apache Kafka, Flume, Oozie, and Nifi.
  • Experience building and optimizing ETL pipelines using Spark and working with structured and unstructured data.
  • Experience with SQL and NoSQL databases such as HBase, Hive, and PostgreSQL.
  • Knowledge of data warehousing concepts, dimensional modeling, and data lakes.
  • Ability to troubleshoot and optimize Spark and Cloudera platform performance.
  • Familiarity with version control tools like Git and CI/CD tools (e.g., Jenkins, GitLab).


Bonus Points:

  • Certifications in Hadoop, Spark, or related technologies.
  • Experience with cloud-based big data platforms (e.g., AWS EMR, GCP Dataproc).
  • Participation in open-source projects related to big data and analytics.


About Us:

We are a dynamic company that values innovation, collaboration, and continuous learning. We offer a competitive salary, comprehensive benefits package, and opportunities for professional growth and development.


What We Offer:

  • Competitive salary.
  • Comprehensive benefits package.
  • Opportunities for professional growth and development.
],

  • Senior Data Engineer

    2 weeks ago

    Delhi, Delhi, India Aptus Data Labs Full time

    Job Title : Senior Data EngineerLocation : : 58 YearsEmployment Type : Full-TimeAbout the Role :Aptus Data Labs is looking for a talented and proactive Senior Data Engineer to help build the backbone of our enterprise data and AI initiatives. Youll work on modern data lake architectures and high-performance pipelines in AWS, enabling real-time insights and...

  • Senior Data Processing Engineer

    2 weeks ago

    Delhi, Mumbai, Kolkata, India beBeeDataEngineering Full time ₹ 12,00,000 - ₹ 30,00,000

    Job TitleWe are seeking a skilled Data Engineering Expert to join our team. The ideal candidate will have experience in designing and implementing high-performance data processing and analytics solutions using Apache Spark, Scala, and Cloudera Hadoop ecosystem.About the RoleDevelop and optimize data processing applications using Apache Spark and Scala for...

  • Senior Data Processing Engineer

    2 weeks ago

    Kolkata, Delhi, Mumbai, India beBeeData Full time ₹ 1,04,000 - ₹ 1,30,878

    Job DescriptionWe are seeking a seasoned data processing professional to join our team. The ideal candidate will have extensive experience in developing, testing, and deploying large-scale data processing applications using Apache Spark and Scala.The successful candidate will be responsible for optimizing and tuning Spark applications for better performance...

  • Senior Data Platform Engineer

    2 weeks ago

    Kolkata, Delhi, Mumbai, India beBeeDataEngineering Full time ₹ 1,04,000 - ₹ 1,30,878

    Job Title: Senior Data EngineerA senior data engineer role is available in our organization. The ideal candidate will have experience working with large-scale data systems and a strong understanding of cloud-based infrastructure.Key Responsibilities:Design, implement, and maintain large-scale data processing systemsDevelop and implement data pipelines to...

  • Senior Data Engineer

    2 weeks ago

    Kolkata, West Bengal, India beBeeDataEngineer Full time ₹ 35,00,000 - ₹ 50,00,000

    About Data Engineering, a key role in driving business innovation.This position requires expertise in developing scalable data systems, automating workflows, and ensuring data governance.As a senior data engineer, you will lead the design and implementation of enterprise data platforms, drive innovation and adoption of new technologies, mentor junior and...

  • Senior Data Processing Specialist

    2 weeks ago

    Delhi, Mumbai, Kolkata, India beBeeData Full time ₹ 15,00,000 - ₹ 28,00,000

    Job DescriptionWe are seeking an experienced data processing professional to join our team. As a key member of our engineering group, you will be responsible for designing and implementing high-performance data processing solutions using Apache Spark and Scala.You will work closely with data scientists, business analysts, and other developers to understand...

  • Senior Process Engineer

    2 weeks ago

    Kolkata, West Bengal, India TRS Staffing Solutions Full time

    You are subscribed to our push notifications but not currently for jobs like this Would you like to receive notifications for jobs like this as well Notify me of jobs like thisGreat news You are subscribed to receive alerts for jobs similar to this one TRS Staffing Solutions has an exciting opportunity to work on the permanent roles of a European EPC...

  • Senior Data Engineer

    2 weeks ago

    Delhi, Delhi, India Vikash Technologies Full time

    Senior Data EngineerSenior Data Engineer with strong expertise in SQL, Python, Azure Synapse, Azure Data Factory, Snowflake, and Databricks. The ideal candidate should have a solid understanding of SQL (DDL, DML, query optimization) and ETL pipelines while demonstrating a learning mindset to adapt to evolving Responsibilities : - Design and implement...

  • Senior data engineer

    2 days ago

    Mumbai, India EClerx Full time

    Job Title: Senior Data Engineering Lead (Databricks) Next Step: If this role excites you and matches your expertise, kindly fill out the Google Form with the required details so we can reach out to you and make the process seamless: Role Overview: We are seeking a highly experienced Senior Data Engineering Lead with a strong focus on Databricks and...

  • Data Processing Systems Specialist

    1 week ago

    Delhi, Delhi, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Senior Data Engineer RoleThe Senior Data Engineer position is a key role in our organization's data engineering team. As a senior-level data engineer, you will be responsible for designing, developing, and maintaining large-scale data processing systems using Python and Pyspark.Key responsibilities include collaborating with cross-functional teams to...