Data Engineer for Large-Scale Data Curation

2 weeks ago

Bengaluru, Karnataka, India Infoblox Full time

About the Role

As a Staff Data Engineer at Infoblox, you will play a key role in developing platforms and products for our SaaS product line, delivering next-level networking for our customers.

This is an exciting opportunity to work closely with data scientists and product teams to curate and refine data powering our latest cloud products.

Key Responsibilities

  1. Curate large-scale data from various sources into appropriate sets for research and development across the company.
  2. Design, test, and implement storage solutions for data consumers, including data warehouses like ClickHouse and OpenSearch.
  3. Design and implement mechanisms to monitor data sources over time for changes using summarization, monitoring, and statistical methods.
  4. Develop and maintain APIs for seamless data integration and retrieval processes for internal and external applications, ensuring scalability, security, and efficiency.
  5. Leverage computer science algorithms and constructs to distill large data into sources of insight and enable future analytics.
  6. Convert prototypes into production data engineering solutions through disciplined software engineering practices and modern deployment pipelines.
  7. Collaborate on design, implementation, and deployment of applications with the software engineering team.
  8. Support data scientists and product teams in building, debugging, and deploying Spark applications that best leverage data.
  9. Build and maintain tools for automation, deployment, monitoring, and operations.
  10. Create test plans, test cases, and run tests with automated tools.

Requirements

  1. 12+ years of experience in software development with programming languages such as Golang, C, C++, C#, or Java.
  2. Expertise in Big Data, including MapReduce, Spark Streaming, Kafka, Pub-Sub, and In-memory Database.
  3. Experience with NoSQL databases such as OpenSearch/Clickhouse.
  4. Good exposure in application performance tuning, memory management, and scalability.
  5. Ability to design highly scalable distributed systems using different open-source technologies.
  6. Experience in microservices development and container-based software using Docker/Kubernetes and other container technologies is a plus.
  7. Experience with AWS, GCP, or Azure is a plus.
  8. Experience building high-performance algorithms.
  9. Bachelor's degree in Computer Science, Computer Engineering, or Electrical Engineering is required; a master's degree is preferred.

Success Metrics

  1. After six months, you will complete onboarding by demonstrating knowledge of the Data Lake and associated technologies and CI/CD processes by deploying ETL pipelines for curation and warehousing to production.
  2. Complete rotations on support/duty where you will gain experience with the different systems, tools, and processes in the Data Lake by resolving reported issues.
  3. Contribute to the team's velocity by participating in Scrum and driving stories to completion.

We Offer

A competitive salary and benefits package, including a Provident Fund with company matches and generous paid time off to help you balance your life.

A strong culture that emphasizes transparency, curiosity, respect, and having fun while delighting our customers.

Perks such as delicious and healthy snacks and beverages, electric vehicle charging stations, a courtyard with amenities, and newly remodeled offices with state-of-the-art amenities.

  • Data Engineer

    4 weeks ago

    Bengaluru, Karnataka, India EXL Full time

    Job SummaryEXL is seeking a highly skilled Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, developing, and deploying large-scale data processing systems and pipelines. You will work closely with cross-functional teams to identify business needs and develop solutions that meet those needs.Key ResponsibilitiesDesign...

  • KDB Developer for Large-Scale Systems

    1 week ago

    Bengaluru, Karnataka, India NTT DATA Services Full time

    Job DescriptionWe are seeking an experienced KDB developer to join our team in Bangalore, India. As a key member of our development team, you will be responsible for designing and implementing large-scale systems using KDB+, Linux, Shell, and CI/CD tools.About the Role:You will work on developing complex algorithms and engineering large systems with...

  • Data Engineering Expert for Large-Scale Analytics

    5 days ago

    Bengaluru, Karnataka, India Talent500 Full time

    Talent500 Seeks Data Engineer for Large-Scale AnalyticsEstimated salary range: $150,000 - $200,000 per year.About the Role:We are seeking an experienced Data Engineer to join our team at Talent500. As a key member of our analytics department, you will be responsible for designing and developing large-scale analytics solutions using Snowflake, dbt Cloud, and...

  • Data Engineer for Large Scale Distributed Systems

    3 weeks ago

    Bengaluru, Karnataka, India exl Full time

    Job OverviewAs a seasoned Data Engineer at EXL, you will play a critical role in designing, developing, and maintaining large-scale distributed systems and data pipelines to support our analytics environment.Key ResponsibilitiesCollaborate with project stakeholders to identify product and technical requirements, conducting analysis to determine integration...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job DescriptionKey Skills and Qualifications:Cloud Platforms: Proficient in AWS.Programming Languages: Expert in SQL and Python.ETL Tools: Experience with ETL tools and frameworks.Requirements:Education: Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.Skills:Proven experience in using Snowflake for data...

  • Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India One Degree North HR Services Full time

    **Job Description**We are seeking an experienced Data Engineer to join our team at One Degree North HR Services. The ideal candidate will have a strong foundation in Python programming and a proven track record in delivering end-to-end data solutions.The successful candidate will be responsible for designing, developing, and maintaining robust data pipelines...

  • Senior Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    About NTT DATANTT DATA is a $30+ billion global innovator of business and technology services. We help clients innovate, optimize, and transform for success. As a Global Top Employer, we have diverse experts in more than 50 countries and a robust partner ecosystem. Our services include business and technology consulting, data and artificial intelligence,...

  • Data Engineer for Large Scale Data Warehousing Projects

    3 days ago

    Bengaluru, Karnataka, India Altimetrik Full time

    About the RoleWe are seeking a skilled Data Engineer to join our team at Altimetrik in Bangalore. As a Data Engineer, you will play a critical role in designing and developing large-scale data warehousing solutions.ResponsibilitiesDesign and develop ETL jobs across multiple platforms using AWS services like S3, Redshift, EMR, Lambda, Athena, and Glue.Develop...

  • Snowflake Data Engineer and Architect

    6 days ago

    Bengaluru, Karnataka, India NTT DATA Full time

    About the RoleWe are seeking a highly skilled Snowflake Data Engineer and Architect to join our team in Bangalore, India. This is a senior-level position that requires expertise in designing, developing, and maintaining large-scale data warehousing solutions using Snowflake.Key ResponsibilitiesDesign and implement efficient data models and architectures for...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Services Full time

    Job DescriptionWe are seeking a highly skilled Data Engineer to join our team at NTT DATA Services.About the Role:The ideal candidate will have a strong background in data engineering, with expertise in designing and implementing automated data pipelines, analyzing complex datasets, and providing actionable insights.Key Responsibilities:Develop and maintain...

  • Data Engineer

    1 month ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job Title: Data EngineerJob Summary:As a Data Engineer at NTT DATA, you will be responsible for designing, building, and maintaining large-scale data processing systems using Azure data factory and Azure databricks. You will work closely with cross-functional teams to ensure data quality, integrity, and security.Key Responsibilities:Design and implement data...

  • Data Engineering Leader for Large-Scale Data Platforms

    5 days ago

    Bengaluru, Karnataka, India ADCI HYD 13 SEZ Full time

    About UsADCI HYD 13 SEZ is a leading provider of innovative data solutions, enabling teams across Amazon to effectively manage and analyze massive datasets. Our mission is to empower businesses with cutting-edge technologies that simplify complex data challenges.Job DescriptionWe are seeking an experienced Sr. Software Development Engineer to join our team...

  • Data Architect for Large Scale eCommerce Solutions

    3 days ago

    Bengaluru, Karnataka, India ADCI - Karnataka - A66 Full time

    Amazon Dev Center India - Hyderabad is seeking a highly skilled Data Engineer to join the eCS Data Engineering and Analytics team.About the RoleThis is a unique opportunity to own the architecture of DW solutions for the Enterprise using multiple platforms. As a Data Engineer, you will design, create, and manage extremely large datasets working backwards...

  • Digital Engineering Principal | Cloud Data Architect

    3 weeks ago

    Bengaluru, Karnataka, India NTT DATA Services Full time

    {"boldAboutNTTDATATitle":"About NTT DATA Services"}NTT DATA Services is a recognized leader in IT and business services, including cloud, data and applications, headquartered in Texas. As part of NTT DATA, a $30 billion trusted global innovator with a combined global reach of over 80 countries, we help clients transform through business and technology...

  • Data Analyst for Large-Scale Project Initiation

    3 weeks ago

    Bengaluru, Karnataka, India Goldman Sachs Full time

    Data Analysis RoleWe are seeking a skilled Data Analyst to lead large-scale project initiation. The ideal candidate will have expertise in data analysis and interpretation, with the ability to communicate complex findings to stakeholders.Key Responsibilities:Design and implement data collection methodologiesAnalyze and interpret large data setsDevelop and...

  • Data Engineering Specialist

    3 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    About the JobThis position requires a highly skilled Data Engineer to design, build, and maintain scalable data pipelines using Databricks on AWS. The ideal candidate will have strong experience with Databricks, including Apache Spark development, and proficiency in Python or Scala for Spark job development.Key ResponsibilitiesDesign, build, and maintain...

  • Data Engineering Specialist

    3 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job SummaryReq ID: 302206NTT DATA is seeking a highly skilled Senior Data Engineer to join our team in Bangalore, India.Job PurposeThe Senior Data Engineer will play a key role within the Data & Analytics team, designing, building, and maintaining systems and infrastructure for processing, collecting, storing, and analyzing large volumes of data.Key...

  • Data Engineer

    1 month ago

    Bengaluru, Karnataka, India Cardinal Health Full time

    At Cardinal Health, we're building a team of innovators who can drive business success through data-driven solutions. As a Data Engineer, you'll play a critical role in designing, building, and operationalizing large-scale enterprise data solutions and applications using Google Cloud Platform.Key Responsibilities:Design and implement data transformation,...

  • NLP Engineer for Large-Scale Search and Retrieval Systems

    4 weeks ago

    Bengaluru, Karnataka, India alphastream Full time

    Search and Retrieval Systems EngineerWe are seeking a highly skilled NLP Engineer to join our team and contribute to the development and improvement of our large-scale search and retrieval systems. In this role, you will be responsible for researching and developing novel search and retrieval techniques that combine traditional search, embeddings, and LLMs....

  • Data Analyst

    3 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job Title: Data AnalystJob Summary:NTT DATA is seeking a skilled Data Analyst to join our team in Bangalore, India. As a Data Analyst, you will be responsible for extracting, transforming, and loading data from various sources, ensuring data quality, integrity, and accuracy.Key Responsibilities:Perform data cleansing, validation, and preprocessing to prepare...