Big Data Engineer

3 days ago


bangalore, India NetApp Full time

Title: Big Data Engineer

Location:

Bangalore, Karnataka, IN, 560071

Requisition ID: 126725

Job Summary

As a Software Engineer at NetApp India’s R&D division, you will be responsible for the design, development and validation of software for Big Data Engineering across both cloud and on-premises environments. You will be part of a highly skilled technical team named NetApp Active IQ. 
The Active IQ DataHub platform processes over 10 trillion data points per month that feeds a multi-Petabyte DataLake. The platform is built using Kafka, a serverless platform running on Kubernetes, Spark and various NoSQL databases. This platform enables the use of advanced AI and ML techniques to uncover opportunities to proactively protect and optimize NetApp storage, and then provides the insights and actions to make it happen. We call this “actionable intelligence”
You will be working closely with a team of senior software developers and a technical director. You will be responsible for contributing to the design, and development and testing of code. The software applications you build will be used by our internal product teams, partners, and customers.
We are looking for a hands-on lead engineer who is familiar with Spark and Scala, Java and/or Python. Any cloud experience is a plus. You should be passionate about learning, be creative and have the ability to work with and mentor junior engineers.

Job Requirements

Your Responsibility 
• Design and build our Big Data Platform, and understand scale, performance and fault-tolerance
• Interact with Active IQ engineering teams across geographies to leverage expertise and contribute to the tech community. 
• Identify the right tools to deliver product features by performing research, POCs and interacting with various open-source forums 
• Build and deploy products both on-premises and in the cloud
• Work on technologies related to NoSQL, SQL and in-memory databases
• Develop and implement best-in-class monitoring processes to enable data applications meet SLAs 
• Should be able to mentor junior engineers technically. 
• Conduct code reviews to ensure code quality, consistency and best practices adherence. 

Our Ideal Candidate 
• You have a deep interest and passion for technology
• You love to code. An ideal candidate has a github repo that demonstrates coding proficiency
• You have strong problem solving, and excellent communication skills
• You are self-driven and motivated with the desire to work in a fast-paced, results-driven agile environment with varied responsibilities

Education

• 5+ years of Big Data hands-on development experience 
• Demonstrate up-to-date expertise in Data Engineering, complex data pipeline development. 
• Design, develop, implement and tune distributed data processing pipelines that process large volumes of data; focusing on scalability, low -latency, and fault-tolerance in every system built
• Awareness of Data Governance (Data Quality, Metadata Management, Security, etc.) 
• Experience with one or more of Python/Java/Scala 
• Proven, working expertise with Big Data Technologies Hadoop, HDFS, Hive, Spark Scala/Spark, and SQL 
• Knowledge and experience with Kafka, Storm, Druid, Cassandra or Presto is an added advantage


Job Segment: Test Engineer, Cloud, Testing, Software Engineer, Developer, Engineering, Technology


  • Big Data Engineer

    4 weeks ago


    bangalore, India Awign Expert Full time

    This is a remote position. Job Title: Big Data Solutions Engineer Reporting to: Director, Product Management Location: India (Remote) Our Ideal Candidate Our company is seeking a talented Big Data Solutions Engineer to join our team. The ideal candidate will be responsible for designing, developing, and implementing our proprietary...

  • Big data engineer

    2 days ago


    bangalore, India Virtusa Full time

    Big data engineer - CREQ191311 Description Position: Big data engineer Primary skills: Big data, hadoop, pyspark Location: Hyderabad &BLR Experience in building SparkStreaming process. Proficient in understanding distributed computing principles. Experience in managing Hadoop cluster with all services. Experience with Nosql Databases and Messaging...

  • Big Data Engineer

    7 days ago


    bangalore, India Adobe Full time

    Our Company Changing the world through digital experiences is what Adobe’s all about. We give everyone—from emerging artists to global brands—everything they need to design and deliver exceptional digital experiences! We’re passionate about empowering people to create beautiful and powerful images, videos, and apps, and transform how companies...

  • Big Data Engineer

    2 weeks ago


    bangalore, India Alp Consulting Limited Full time

    Big Data Engineer Duties & Responsibilities To write an effective big data engineer job description, begin by listing detailed duties, responsibilities and expectations. We have included big data engineer job description templates that you can modify and use. Sample responsibilities for this position include: You are responsible for Hadoop...

  • Big Data Engineer

    7 days ago


    bangalore, India Alp Consulting Limited Full time

    Senior Big Data Engineer Duties & Responsibilities To write an effective senior big data engineer job description, begin by listing detailed duties, responsibilities and expectations. We have included senior big data engineer job description templates that you can modify and use. Sample responsibilities for this position include: Performing...

  • Big Data Engineer

    1 week ago


    bangalore, India shreeniwas Full time

    Big Data Engineer Job Description: Data Scientist We are looking for a candidate with 6+ years of experience as a DATA ENGINEER, with education (Bachelors degree or higher) in Computer Science, Mathematics, or a related technical field, or equivalent practical experience. They should also have the following abilities: Required 1. Relevant Experience...

  • Solution Engineer

    5 days ago


    Bangalore, India Coders Brain Technology & Private Limited Full time

    Big Data Solutions Engineer to join our team.The ideal candidate will be responsible for designing, developing, and implementing our proprietary Big Data-powered Fin Crime solution for our clients. The candidate will work closely with the software development team, data scientists, and product managers to understand requirements, configureand implement our...

  • Solution Engineer

    1 week ago


    bangalore, India Coders Brain Technology & Private Limited Full time

    Big Data Solutions Engineer to join our team.The ideal candidate will be responsible for designing, developing, and implementing our proprietary Big Data-powered Fin Crime solution for our clients. The candidate will work closely with the software development team, data scientists, and product managers to understand requirements, configureand implement our...

  • Solution Engineer

    6 days ago


    Bangalore, Karnataka, India Coders Brain Technology & Private Limited Full time

    Big Data Solutions Engineer to join our team.The ideal candidate will be responsible for designing, developing, and implementing our proprietary Big Data-powered Fin Crime solution for our clients. The candidate will work closely with the software development team, data scientists, and product managers to understand requirements, configureand implement our...

  • Big Data Engineer

    2 weeks ago


    Mumbai/Pune/Bangalore, India Techno Wise Full time

    Job Description :We are seeking a talented Big Data Engineer proficient in PySpark to join our dynamic team. The ideal candidate will play a pivotal role in designing, implementing, and maintaining scalable data solutions leveraging the Big Data technologies like PySpark. This role requires a strong understanding of big data technologies, data engineering...

  • Big Data Engineer

    4 weeks ago


    bangalore, India Hexaware Technologies Full time

    Description Data Engineer: Primary : ADF, SQL Seconday : Databricks, Pyspark / Scala, CICD, Data Security, encryption Responsibilities: •Work closely with scrum master, technical leads and provide daily progress updates on the tasks assigned. •Provide support in defining the scope and sizing of the work. •Working closely with...

  • Big Data Engineer

    1 month ago


    bangalore, India Global Pharma Tek Full time

    Auto req ID BR Title Big Data Engineer Your Role and Responsibilities Understand a data warehousing solution and able to work independently in such an environment Responsible in Project development and delivery experience of a few good size projects Posting Country India State / Province Haryāna City / Township / Village Gurgaon ...

  • Big Data Engineer

    2 weeks ago


    Bangalore/Gurgaon/Gurugram, India HR BIZ HUB Full time

    Job Title : Big Data DeveloperJob Description : Bachelor's degree in Engineering or Computer Science or equivalent OR Master's in Computer Applications or equivalent. Solid Experience of software development experience and leading teams of engineers and scrum teams. 4+ years of handson experience of working with Map-Reduce, Hive, Spark (core, SQL and...

  • Big data Manager

    1 day ago


    Bengaluru/ Bangalore, India timesjobs Full time

    JOB DETAILS1.Experience in Big Data technologies and working exposure in at least one cloud platform on related data services (AWS / Azure / GCP)2.Hands-on experience with the Hadoop stack HDFS, sqoop, kafka, Pulsar, NiFi, Spark, Spark Streaming, Flink, Storm, hive, oozie, airflow and other components required in building end to end data...

  • Big Data Engineer

    4 weeks ago


    bangalore, India Infogain Full time

    Big Data Engineer (Lead) with skills Big Data, Data Warehouse, Azure Datalake for location Any Infogain Base Location (Noida, Gurugram, Bangalore, Mumbai, Pune) Posted on: June 04, Share on Linkedin Share on Twitter Share on Facebook ROLES & RESPONSIBILITIES Responsibilities: · Strong experience of minimum 5+ years in Snowflake &...

  • Big Data Engineer

    3 days ago


    bangalore, India Infogain Full time

    Big Data Engineer (Lead) with skills Big Data, Data Warehouse, Azure Datalake for location Any Infogain Base Location (Noida, Gurugram, Bangalore, Mumbai, Pune) Posted on: July 02, Share on Linkedin Share on Twitter Share on Facebook ROLES & RESPONSIBILITIES Responsibilities: · Strong experience of minimum 5+ years in Snowflake &...

  • Big Data Engineer

    4 weeks ago


    Bangalore, India TalentPulse Consultant Full time

    Role : Big Data Engineer Job Description : We are currently seeking a highly skilled Big Data Engineer to join our team on a remote basis. The ideal candidate will have at least 5 years of experience in Big Data technologies and a strong proficiency in Python, Spark, and Scala. This is an active position with a high priority, offering a competitive rate...

  • Big Data Engineer

    1 week ago


    Bangalore, India Adobe Full time

    JOB LEVELP50EMPLOYEE ROLEIndividual ContributorP50 (ESDEP50)  Big Data Engineer   What You’ll Do Build the underneath data platform and maintain data processing pipelines using best in class technologies. Special focus on R&D to challenge status-quo and build the next generation big data platform that is efficient and cost effective....

  • Manager - Big Data

    2 weeks ago


    bangalore, India Magnus India Consulting Services Full time

    About the Company:Opportunity with one of the leading Media & Entertainment players for a role in Big Data.About the Role:The incumbent will direct the development and implementation of Big data and analytics solutions.Experience - 9+ yearsIndustry - Product based companiesLocation - Bangalore (Work from Office)Responsibilities:Hands-on implementation role...


  • bangalore, India Tiger Analytics Full time

    Job Description Tiger Analytics is a global AI and analytics consulting firm. With data and technology at the core of our solutions, our + tribe is solving problems that eventually impact the lives of millions globally. Our culture is modeled around expertise and respect with a team-first mindset. Headquartered in Silicon Valley, you’ll find our...