Adobe - Big Data Engineer - Python/Java/Scala

3 weeks ago


Bangalore, Karnataka, India Adobe Systems India Pvt. Ltd. Full time

What You'll Do :


- Build the underneath data platform and maintain data processing pipelines using best in class technologies.


- Special focus on R&D to challenge status-quo and build the next generation big data platform that is efficient and cost effective.

- Translate complex technical and functional requirements into detailed designs.

- Collaborate with Data Scientists and Product Managers to bring AI-based Assistive experiences to life.


- Create the roadmap and socialize among team members and stake holders.

- Create and instill a team culture that focuses on sound scientific processes and encourages deep engagement with our customers.

- Handle project scope and risks with data, analytics, and creative problem-solving.

- Help define data governance policies and implement security and privacy guard rails.

What you require :

- Hands-on engineer with extensive experience ingesting, storing, processing, and analysing large datasets

- Understanding of data-warehousing and data-modelling techniques

- Experience working (deploying and operationalising) with DNN frameworks like TensorFlow or PyTorch on large-scale data sets will be a great plus.-

Proficient in some of the following tools and technologies :

1. Languages: Python, Java/Scala, SQL

2. Dev-ops pipelines

3. Container orchestration: Docker, Kubernetes

4. Hadoop ecosystem: Hadoop, Spark, Hive etc.

5. Stream processing: Storm, Flink etc.

6. Workflow: Airflow, Flyte, etc.

7. Data Engineering ecosystem: DeltaLake, DataHub etc.

8. General understanding of data structures, algorithms, multi-threaded programming, and distributed computing concepts

9. Ability to be a self-starter and work closely with other data scientists and software engineers to design, test, and build production-ready ML and optimization models and distributed algorithms running on large-scale data sets

Ideal Candidate Profile :

- A total of 10+ years of experience in technical roles involving Big Data Engineering, implementation of Machine Learning models.

- Masters or B.Tech in Engineering.

- Deep knowledge of Big Data Architectures, Platforms, Frameworks.

- Experience with Real Time Bidding / Ad Tech background preferred.

- Comfort with ambiguity, adaptability to dynamic environment, and the ability to lead a team while working autonomously.

- Demonstrated ability to influence technical and non-technical stakeholders.

- Track record of delivering cloud-scale, data-intensive applications and services that are widely adopted with large customer bases.

- An ability to think strategically, look around corners, and create a vision for the current quarter, the year, and five years down the road.

- A relentless pursuit of great customer experiences and continuous improvements to the product.

- Excellent Verbal, Written and interpersonal communication skills.

(ref:hirist.tech)
  • Big Data Engineer

    3 weeks ago


    Bangalore, Karnataka, India randstad Full time

    Primary Responsibilities :- Design, develop, and implement big data solutions using Java, Python, or Scala.- Utilize SQL and Spark Scala for data processing and analysis.- Implement and manage data storage solutions using AWS services like S3, EC2, and Lambda.- Work with various databases such as Postgres, MySQL, Oracle, etc., for data storage and...

  • Big Data Consultant

    3 weeks ago


    Bangalore, Karnataka, India Nilasu consulting Full time

    Summary :We are seeking a highly skilled consultant with expertise in Spark/Scala and Python to join our team. You will play a key role in developing and maintaining big data applications using Spark and Scala with a focus on functional programming principles. Additionally, your experience in Python and microservices architecture will be valuable in building...

  • Big Data Engineer

    3 weeks ago


    Bangalore, Karnataka, India TalentPulse Consultant Full time

    Role : Big Data EngineerJob Description :We are currently seeking a highly skilled Big Data Engineer to join our team on a remote basis.The ideal candidate will have at least 5 years of experience in Big Data technologies and a strong proficiency in Python, Spark, and Scala.This is an active position with a high priority, offering a competitive rate and the...


  • Bangalore, Karnataka, India Edge in Asia Recruitment Private Limited Full time

    Job Description :Our client is a Limited is a platform in the financial services sector in India set up primarily with the mission of catering to the diverse credit requirements of under-served households and businesses, who is growing their presence in India and is searching for Associate Data Engineers to join their team.Some of the key responsibilities...

  • Senior Data Engineer

    3 weeks ago


    Bangalore, Karnataka, India TalentPulse Consultant Full time

    Job Description:We are looking for a talented Senior Data Engineer to join our team in Bangalore. As a Senior Data Engineer, you will be responsible for designing, building, and maintaining scalable data pipelines and infrastructure to support our data-driven initiatives. This position offers a hybrid work model, allowing a combination of remote and onsite...

  • Senior Data Engineer

    3 weeks ago


    Bangalore, Karnataka, India Dashhire Full time

    Responsibilities : Design and build parts of our data pipeline architecture for extraction, transformation, and loading of data from a wide variety of data sources using the latest Big Data technologies. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for...


  • Bangalore, Karnataka, India TecHHire Global Full time

    Job Description :Function : Technical Management - Engineering Management, Software Engineering - Backend Development, Big Data / DWH / ETLResponsibilities :- Design and implement large-scale data processing systems using the Databricks platform.- Work closely with clients to understand their requirements and design solutions that meet their needs.-...

  • Staff Data Engineer

    3 weeks ago


    Bangalore, Karnataka, India People Nest Full time

    Responsibilities :You will be responsible for product features related to data transformations, enrichment, and analytics.You will discuss with product managers, UX designers, and other backend teams to understand requirements and translate the same into functional specifications.Lead and provide technical direction and mentorship to junior engineers and...

  • Big Data Engineer

    3 weeks ago


    Bangalore/Hyderabad, Karnataka, India VIDPRO CONSULTANCY SERVICES Full time

    About the job:- This position will be part of a growing team working towards building world class large scale Big Data architectures.- This individual should have a sound understanding of programming principles, experience in programming in Java, Python or similar languages and can expect to spend a majority of their time coding.Location - Bangalore and...


  • Bangalore, Karnataka, India Notus Full time

    Job Title : GCP (Google Cloud Platform) SpecialistNumber of Positions : 6 (1 Senior Role, 5 Lead Roles)Shift : 12 : 00 PM to 9 : 00 PM OR 1 : 00 PM to 10 : 00 PMLocation : TechM India locationsNotice Period : Early/Immediate/30 days (Positions need to be filled by May)Budget : Senior Role : 4 to 5/6 years of experience - Depending upon hike % and candidate...

  • Big Data Engineer

    3 weeks ago


    Hyderabad/Bangalore, Karnataka, India HRwing Full time

    Role & Responsibilities:Good development practices:Hands on coder with good experience in programming languages like Java or Python.Hands-on experience on the Big Data stack like PySpark, Hbase, Hadoop, Mapreduce and ElasticSearch.Good understanding of programming principles and development practices like checkin policy, unit testing, code deploymentSelf...

  • Big Data Engineer

    3 weeks ago


    Bangalore/Hyderabad, Karnataka, India VIDPRO CONSULTANCY SERVICES Full time

    This position will be part of a growing team working towards building world class large scale Big Data architectures. This individual should have a sound understanding of programming principles, experience in programming in Java, Python or similar languages and can expect to spend a majority of their time coding.Location : Bangalore and HyderabadWork...

  • Senior Data Engineer

    3 weeks ago


    Bangalore, Karnataka, India Infimetrics Info Solutions Full time

    Position : Senior Java (Data Engineer). Reports to : Technical Lead. Responsible for : Contributing to software development as per business requirements.Purpose of Role :- We are a data driven asset management firm with a very wide variety of datasets with varying frequency of updates that essentially drive our research models which ultimately determine the...

  • Software Engineer

    3 weeks ago


    Bangalore, Karnataka, India symphoni HR Full time

    Skill: Java (Data structures/Algorithims) & Spark - Design, develop and implement, scalable, maintainable, and efficient software solutions using best practices and modern technologies. - Write clean, well designed, and testable code, following coding standards and guidelines. - Conduct code reviews and provide constructive feedback to team members to ensure...

  • Engineering Manager

    3 weeks ago


    Bangalore, Karnataka, India FX Consulting Full time

    Roles & Responsibilities : - Minimum 14+ years of professional hands-on experience in data engineering, data science, or a similar role.- Minimum of 2+ years of people management experience, hiring, developing, and promoting engineering talent.- Significant experience leading data architecture projects (medallion architecture pattern) and driving decisions...

  • AWS Data Engineer

    3 weeks ago


    Bangalore, Karnataka, India Quess Corp Ltd Full time

    Exp Range : 6 - 10 years. Primary Skills :- Combination of Data Engineering (60%) and Database (40%) Developer.- 3-4 years relevant exp in Data Engineering in AWS & DataBases in MongoDB, SQL Server.- data engineering and DB developer.- IDBI Good to have. Shift Timing : 9AM to 6PM. Working Model ( WFH / WFO / Hybrid )Skills Required : -Data Engineering (60%)...

  • Data Engineer

    3 weeks ago


    Bangalore, Karnataka, India LOGIC PLANET IT SERVICES Full time

    About the job :We are hiring for a global technology company, home to more than 224,000 people across 60 countries.Role : Data EngineerExp : 5+ years Mode of Work : HybridMode of Hire : FTEAvailability : Immediate joinee or serving maximum of 30 days notice period.Job Description :- 5+ years of experience operating and deploying solutions using AWS Service...

  • Databricks Engineer

    2 days ago


    Bangalore, Karnataka, India Hunarstreet Technologies Full time

    Databricks Role Overview :As a Data Engineer specializing in Databricks, you will play a pivotal role in collaborating with our clients to leverage the Databricks platform for extracting actionable insights from their data. From designing and implementing scalable data pipelines to optimizing workflows and fostering cross-functional collaboration, you will...

  • Data Engineer

    3 weeks ago


    Bangalore, Karnataka, India ShopDeck Full time

    What You'll Do :- Solely own your product; Build the product from scratch- Collaborate with the business team to understand business requirements and define tech roadmap- Working on complex architectural problems- Have an understanding of loopholes in current system/architecture which can potentially break in future and push towards solving them with...

  • AWS Data Engineer

    6 days ago


    Bangalore/Hyderabad, Karnataka, India Sureminds Solutions Pvt. Ltd. Full time

    Job Description :Experience in architecting and delivering highly scalable, distributed, cloud-based enterprise data solutionsStrong expertise in end-to-end implementation of Cloud data engineering solutions like Enterprise Data Lake, Data hub in AWSProficient in Lambda or Kappa ArchitecturesShould be aware of Data Management concepts and Data...