Data Engineer

Found in: Talent IN 2A C2 - 3 weeks ago

Pune, India gethyr Full time

We are seeking a talented and motivated Data Engineer stream processing, particularly utilizing Flink and Kafka, alongside expertise in Spark and AWS technologies.

As a Data Engineer, you will play a crucial role in designing, implementing, and maintaining robust stream processing solutions to handle large volumes of data in real-time, ensuring high performance, scalability, and reliability.

Responsibilities :

  • Stream Processing Development: Design, develop, and optimize stream processing pipelines using Apache Flink and Kafka to process real-time data streams efficiently.
  • Data Ingestion: Implement robust data ingestion pipelines to collect, process, and distribute streaming data from various sources into the Flink and Kafka ecosystems.
  • Data Transformation: Perform data transformation and enrichment operations on streaming data using Spark Streaming and other relevant technologies to derive actionable insights.
  • Performance Optimization: Continuously optimize stream processing pipelines for performance, scalability, and reliability, ensuring low-latency and high-throughput data processing.
  • Monitoring and Troubleshooting: Monitor stream processing jobs, troubleshoot issues, and implement necessary optimizations to ensure smooth operation and minimal downtime.
  • Integration with AWS Services: Leverage AWS technologies such as Amazon Kinesis, AWS Lambda, Amazon EMR, and others to build end-to-end stream processing solutions in the cloud environment.
  • Data Governance and Security: Implement data governance and security measures to ensure compliance with regulatory requirements and protect sensitive data in streaming pipelines.
  • Collaboration: Collaborate closely with cross-functional teams including data scientists, software engineers, and business stakeholders to understand requirements and deliver impactful solutions.
  • Documentation: Create and maintain comprehensive documentation for stream processing pipelines, including design specifications, deployment instructions, and operational procedures.
  • Continuous Learning: Stay updated with the latest advancements in stream processing technologies, tools, and best practices, and incorporate them into the development process as or Master's degree in Computer Science, Engineering, or a related field.
  • 2-5 years of professional experience in data engineering roles with a focus on stream processing.
  • Strong proficiency in Apache Flink and Kafka for building real-time stream processing applications.
  • Hands-on experience with Spark and Spark Streaming for batch and stream processing.
  • Solid understanding of cloud computing platforms, particularly AWS services such as Amazon, Kinesis, AWS Lambda, Amazon EMR, etc.
  • Proficiency in programming languages such as Java, Scala, or Python.
  • Experience with containerization and orchestration tools like Docker and Kubernetes is a plus.
  • Excellent problem-solving skills and the ability to troubleshoot complex distributed systems.
  • Strong communication skills and the ability to work effectively in a collaborative team (ref:hirist.tech)

  • Senior Data Engineer

    Found in: Talent IN C2 - 3 weeks ago

    Pune, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology...

  • Senior Data Engineer

    Found in: Talent IN C2 - 3 days ago

    Pune, India Optimum Data Analytics Full time

    About Optimum DataAnalytics:Optimum Data Analytics operatesin the IT Services and IT Consulting industry specializing inproviding advanced data solutions to its clients. Our team isdedicated to driving innovation and excellence in the field of dataanalytics and engineering.Roleand Responsibilities:We are seeking atalented Staff Engineer (Data / Platform) to...

  • Senior Data Scientist

    Found in: Talent IN C2 - 3 weeks ago

    Pune, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the USA. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology...

  • Software Engineer

    Found in: Whatjobs IN C2 - 11 hours ago

    Pune, India Data Axle Full time

    Job DescriptionAbout Data Axle:Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for over 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology platform and...

  • Software Engineer

    Found in: Whatjobs IN C2 - 2 weeks ago

    Pune, India Data Axle Full time

    Job DescriptionAbout Data Axle:Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology platform and...

  • Software Engineer

    Found in: Talent IN C2 - 2 weeks ago

    Pune, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology...

  • Software Engineer

    Found in: Whatjobs IN C2 - 2 weeks ago

    pune, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology...

  • Software Engineer

    Found in: Talent IN C2 - 3 days ago

    Pune, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for over 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology...

  • Software Engineer

    Found in: Whatjobs IN C2 - 14 hours ago

    pune, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for over 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based...

  • Digital Engineering Staff Engineer

    Found in: beBee S IN - 4 weeks ago

    Pune, India NTT DATA Services Full time

    Req ID: 260538  NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Digital Engineering Staff Engineer to join our team in Pune, Mahārāshtra (IN-MH), India...

  • Data Engineer

    Found in: Talent IN C2 - 3 weeks ago

    Pune, India Kiash Solutions LLP Full time

    Role- Data EngineerExperience Level: 5+ Should have the following Mandatory skills: Good working experience in SOLR and Elastic Search Good in Machine learning & NLP techniques with work experience. Excellent experience in data processing using Python Experience with relational SQL, NoSQL DB,...

  • Data Engineer

    Found in: Whatjobs IN C2 - 2 weeks ago

    Pune, India Kiash Solutions LLP Full time

    Role-Data EngineerExperience Level: 5+Should have the following Mandatory skills:Good working experience in SOLR and Elastic SearchGood in Machine learning & NLP techniques with work experience.Excellent experience in data processing using PythonExperience with relational SQL, NoSQL DB, Streaming and ETL.Experience with integration of data from multiple data...

  • Senior Software Engineer – Fullstack

    Found in: Whatjobs IN C2 - 2 weeks ago

    Pune, India Crest Data Systems Full time

    Job Description:We are looking for software engineers to join our engineering team in Ahmedabad & Pune. The primary responsibility will be to help design and develop Enterprise Software for our Global Fortune 500 clients in Data Analytics, Security, and Cloud segments. The job requires an understanding of backend & frontend programming languages such as Core...

  • Data Engineer

    Found in: Whatjobs IN C2 - 4 days ago

    pune, India Synechron Full time

    Good day,We have immediate opportunity for Data Analytics Engineer.Job Role: Data Analytics EngineerJob Location: Kharadi, PuneExperience- 8 Years to 12 Years.Notice Period: upto 30 days.About Company:At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and innovative...

  • Data Engineer

    Found in: Whatjobs IN C2 - 3 weeks ago

    Pune, India The Citco Group Limited Full time

    About CitcoSince the 1940s Citco has provided specialist financial services to alternative investment funds, investors, multinationals and private clients worldwide. With over 6,000 employees in 45 countries we pioneer innovative solutions that meet our clients’ evolving needs and deliver exceptional service.About the Role:The Data Engineering will be a...

  • Data Engineer

    Found in: Whatjobs IN C2 - 4 days ago

    Pune, India Synechron Full time

    Good day,We have immediate opportunity for Data Analytics Engineer.Job Role:Data Analytics EngineerJob Location:Kharadi, PuneExperience-8 Years to 12 Years.Notice Period:upto 30 days.About Company:At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and innovative technology...

  • Data Engineer

    Found in: Whatjobs IN C2 - 4 days ago

    Pune, India Synechron Full time

    Good day,We have immediate opportunity for Data Analytics Engineer.Job Role:Data Analytics EngineerJob Location:Kharadi, PuneExperience-8 Years to 12 Years.Notice Period:upto 30 days.About Company:At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and innovative technology...

  • Data Engineer

    Found in: Whatjobs IN C2 - 13 hours ago

    pune, India Tech Mahindra Full time

    Title: Reltio GCP Data EngineerLocation: PuneExperience: 6+yearsNP: Immediate to 15daysCompany Profile:Tech Mahindra represents the connected world, offering innovative and customer-centric information technology experiences. We #Rise together to create sustainable businesses that can bring about lasting change in our communities – to create an equal...

  • Data Engineer

    Found in: Whatjobs IN C2 - 5 days ago

    Pune, India Synechron Full time

    Good day, We have immediate opportunity for Data Analytics Engineer. Job Role: Data Analytics Engineer Job Location: Kharadi, Pune Experience- 8 Years to 12 Years. Notice Period: upto 30 days. About Company: At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and...

  • Data Engineer

    Found in: Whatjobs IN C2 - 4 days ago

    pune, India Synechron Full time

    Good day, We have immediate opportunity for Data Analytics Engineer. Job Role: Data Analytics Engineer Job Location: Kharadi, Pune Experience- 8 Years to 12 Years. Notice Period: upto 30 days. About Company: At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and...