Highly Skilled Data Engineer Wanted

6 days ago

Mumbai Kolkata, India beBeeData Full time ₹ 9,00,000 - ₹ 12,00,000
Streaming Data Engineer Job Description

This is a dynamic and challenging role that involves designing, developing, and maintaining large-scale data streaming systems.

Job Responsibilities:
  • Create efficient and scalable code for data processing and analysis.
  • Develop in-depth knowledge of Spark-related queries and optimize performance.
  • Maintain expertise in Flink coding and troubleshooting pipeline-level issues.
  • Craft high-quality Python code for data validation, string comparison, and collections.
  • Implement Spark optimization techniques and submit commands.
  • Design and execute SQL queries with joins and aggregate window functions.
  • Develop excellent communication skills to effectively collaborate with team members.
  • Familiarize yourself with the basics of streaming pipelines, Spark sessions, and data transformation.
  • Stay up-to-date with the latest developments in Spark Streaming and Flink Streaming with Kafka projects.
  • Become proficient in Azure Cloud services and their applications.
Evaluation Questions:
  • Explain the differences between Spark Streaming sessions and batch sessions.
  • Discuss the concepts of Spark Structured Streaming.
  • Describe the use of the readStream method in Spark Streaming Applications.
  • Elaborate on the usage of the writeStream method and its arguments for writing data frames to a sink.
  • Explain how to read data from a Kafka queue using the start() method.
  • Show an example of printing the output of a streaming operation on the terminal using the console format.
About This Opportunity:

We are looking for highly skilled individuals who can thrive in a fast-paced environment and take ownership of complex data engineering challenges. If you have experience with big data technologies and a passion for innovative solutions, we encourage you to apply.

Requirements:
  • Strong background in big data technologies, specifically Spark and Flink.
  • Experience with Python programming and familiarity with libraries like Pandas and NumPy.
  • Knowledge of SQL and query optimization techniques.
  • Excellent communication and collaboration skills.
  • Familiarity with cloud platforms, particularly Azure.

  • Highly Skilled Data Engineer Wanted

    3 days ago

    Noida, Mumbai, Pune, India beBeeDataEngineer Full time ₹ 10,00,000 - ₹ 20,00,000

    Job OpportunityWe seek an experienced data engineer to fill this key role.Primary Responsibilities:Proven experience in working with cloud-based services including EMR, EC2, S3, Athena, RDS, and Redshift.Strong SQL and operational expertise with the ability to thrive in a fast-paced environment.Familiarity with scripting tools like PySpark is highly...

  • Highly Skilled Senior Data Engineer Wanted

    2 days ago

    Navi Mumbai, Maharashtra, India beBeeData Full time ₹ 25,00,000 - ₹ 30,00,000

    Job Opportunity:We are seeking a highly skilled and innovative Senior Data Engineer to join our team. As a member of the Enterprise Data Platform, you will be part of a group of talented individuals who strive to improve customer experiences through product innovation.The successful candidate will be responsible for designing, developing, and maintaining...

  • Highly Skilled Data Engineer Wanted

    5 days ago

    Bengaluru / Bangalore, Pune, Mumbai, India beBeeDataEngineer Full time ₹ 8,00,000 - ₹ 25,00,000

    Job Position: Data Engineer The ideal candidate for this role will possess expertise in data processing, with a strong foundation in Scala and Spark.We are seeking an experienced professional to lead our team's data engineering efforts, ensuring the highest quality of output.The key qualifications for this position include:

  • Highly Skilled Software Developer Wanted

    4 days ago

    Mumbai, Maharashtra, India beBeeSoftware Full time US$ 1,80,000 - US$ 2,50,000

    Job Title: Software Engineer, BackendWe are seeking a highly skilled and passionate software engineer to join our team. This is an exceptional opportunity to work on building a trusted, scalable, and compliant platform for financial data.

  • Highly Skilled Backend Developer Wanted

    3 days ago

    Kolkata, West Bengal, India beBeeSoftware Full time ₹ 8,00,000 - ₹ 15,00,000

    Backend Software EngineerWe are seeking a talented Backend Software Engineer to join our team and contribute to the development of scalable backend systems using Golang, Python, and JavaScript/TypeScript.This is an exciting opportunity for a highly skilled engineer to design, build, and maintain efficient and maintainable code that adheres to best practices....

  • Highly Skilled Data Platform Engineers Wanted

    3 days ago

    Mumbai, Maharashtra, India beBeeDataPlatform Full time ₹ 1,80,00,000 - ₹ 2,00,00,000

    Job Title: Data Platform EngineerKey Responsibilities:Lead the development of data platforms as a Subject Matter Expert (SME)Mentor and manage team performance to ensure seamless integration between systems and data modelsMake strategic decisions for the teamCollaborate with multiple teams to drive key decisionsProvide solutions to problems across...

  • Highly Skilled Data Architect

    4 days ago

    Mumbai, Maharashtra, India beBeeDataEngineer Full time ₹ 10,00,000 - ₹ 20,00,000

    Key RolesWe are seeking a highly skilled Data Engineer to drive the development and maintenance of large-scale data processing systems using Apache Druid.Main Responsibilities:Design, build, and maintain complex data pipelines using Apache Druid in production environments for improved scalability and efficiency.Collaborate with cross-functional teams to...

  • Highly Skilled Data Expert Wanted

    7 days ago

    Mumbai, Maharashtra, India beBeeData Full time ₹ 1,20,00,000 - ₹ 1,50,00,000

    We are seeking a highly skilled Data Expert to join our team in developing and implementing cutting-edge data solutions.About the Role:The successful candidate will be responsible for leading end-to-end initiatives, analyzing complex data sets, and drawing insightful conclusions. They will work closely with cross-functional teams to design and deploy...

  • Highly Skilled Senior Software Engineer Wanted

    3 days ago

    Delhi, Mumbai, Kolkata, India beBeeSoftwareDeveloper Full time ₹ 12,00,000 - ₹ 30,00,000

    Senior Software Engineer Role OverviewThe position of Senior Software Engineer involves the design, development, and maintenance of high-quality software applications. Key responsibilities include analyzing system requirements, designing software solutions, coding, testing, debugging, and ensuring compliance with industry standards.Key Skills and...

  • Highly Skilled Golang Developer Wanted

    2 days ago

    Kolkata, West Bengal, India beBeeGolang Full time ₹ 18,00,000 - ₹ 24,00,000

    Job Title:Golang Developer Job Description:We are seeking a highly skilled Golang developer to join our team. As a Golang developer, you will be responsible for designing, developing, and maintaining scalable applications using Golang.Your primary focus will be on application development, ensuring that the code is optimized for performance, scalability, and...