Data Engineer

3 weeks ago

Bengaluru, India PNR Software Solutions Full time
Job Description:

Data Engineer

We are looking for a highly motivated Data Engineer who will be responsible for designing and developing scalable solutions for a large data infrastructure in a fast-paced environment. You will participate in detailed technical design, development, and implementation of applications using cutting-edge technology stacks.Our main focus is on building and managing a highly scalable Data Platform for our Identity Analytics product that is capable of performing large scale data analytics. The ideal candidate will have a strong engineering background with the ability to tie engineering initiatives to business impact.

This position is within our data science team and the individual will work with data scientists, software engineers and qa engineers in the team to create solutions to improve the scale of ML models.

OUR TEAM MISSIONAt Aquera, we believe we have an important story to tell: companies can transform their operations with a single source of truth of user identities that already exists in their HRIS. Aquera helps companies of all sizes to eliminate security gaps, reduce operational costs, and support day-1 employee productivity with HR-driven IT automationin a click.

RESPONSIBILITIES

Design scalable and reliable data pipelines to consume, integrate and analyze large volumes of complex data from different sources to support the growing needs of our business.Build & manage a DataLake in AWS infrastructure.Build Knowledge graph of user identities for Identity Management and Analytics.Design our data pipeline architecture which collects, processes, streams and analyses very large amounts of data.Analyzing batch and stream data sources and designing scalable data pipelines that transform ETL efficiently and at scale.Designing and implementing the feature store to support current and anticipated data analytics and machine learning workloads.Running and troubleshooting data and ML pipelines running on AWS infrastructure.Implementing CI/CD pipelines that integrate data processing logic and perform validation and logic promotion to production.Design custom workflows to orchestrate the control plane to manage model, data source and pipeline lifecycle.Write code, leverage managed solutions, open-source tools, and industry best practices to ensure our data is reliable, standardised and reusable.

QUALIFICATIONS

3 to 7 years of industry research experience in a large scale data computational area such as data science, computer science, machine learning, artificial intelligence or statistics.Proficiency in Python (or similar) programming language for data analysis and software development.3+ years experience in the data warehouse and data lake space.Experience with building distributed systems.Experience with Graph DatabasesExperience with Machine Learning Life Cycle.Familiarity with data engineering tech stack - SQL, ETL tools, Spark,PySpark, Kafka, Redis, ElasticSearchExperience in Data Lakes & Analytics on AWS:Storage- (S3/Opensearch); Data Transformation Pipeline (Lake formation, Glue, data catalogue); Data Analytics Pipeline (Athena / Redshift).Experience with CI/CD tools and procedures.Experience with json, parquet/avro file formats.Good to have:- Experience in MLops, MLflow, Airflow, Databricks.Proven ability to work creatively and analytically in a problem-solving environment.Exceptional written and oral communication skills.Fundamental belief in the importance of Integrity, good vibes and good work ethic.Poised under pressure and a sense of humor.Preferred:-

Experience with Product based companies.Bachelors or Masters degree in a computational, mathematical, engineering or scientific field.

  • Aptus Data Labs

    2 weeks ago

    Bengaluru, India Aptus Data Labs Full time

    Aptus Data Labs is an innovative Data Engineering and Advanced Analytics company offering analytical solutions and consulting services.Role - Machine Learning EngineerExp- 3-5 YrsLocation - RemoteSkill - AWS Sagemaker , ML Pipeline, Machine Learning, AI, Python, NLP, , CI/CD , AWS Services - Lambda, Step functions and other AWS ServicesThe ideal...

  • Big Data Developer

    4 weeks ago

    Bengaluru, India Aptus Data Labs Full time

    Job Title: Big Data DevelopersExperience: Minimum 8 yearsLocation: Remote / core location - BangaloreMandatory Skills ( Hadoop , Hive, HiveQL , Cloudera , ETL, Pyspark/Python, Shell Scripting, Automation )Strong proficiency in Hadoop ecosystem includingHDFS, Hive, and HSQLExtensive experience withPython and PySparkfor data processing and...

  • Data Engineer

    1 day ago

    Bengaluru, India JRD Systems Full time

    Role : Data Engineer - AWS and Oracle ExpertJob Summary:We are seeking a highly skilled and experienced Data Engineer with expertise in data development in AWS and Oracle. The ideal candidate will have a minimum of 5 years of experience in data engineering, with a strong focus on AWS and Oracle technologies. The Data Engineer will be responsible for...

  • Data Engineer

    8 hours ago

    Bengaluru, India JRD Systems Full time

    Role : Data Engineer - AWS and Oracle ExpertJob Summary:We are seeking a highly skilled and experienced Data Engineer with expertise in data development in AWS and Oracle. The ideal candidate will have a minimum of 5 years of experience in data engineering, with a strong focus on AWS and Oracle technologies. The Data Engineer will be responsible for...

  • Data Engineer

    5 days ago

    Bengaluru, India ANZ Full time

    About the roleData Engineer –Role Type – FTEJob Grade: Group 4.3At ANZ our purpose is to shape a world where people and communities thrive and to achieve this, we need a talented Data Engineer to join our Australia Commercial tribe. The Australia Data tribe sits within our Australia Commercial division and our mission is to combine business & technology...

  • Data Engineer

    2 weeks ago

    Bengaluru, India Joveo Full time

    We are looking for talented Data Engineers who are passionate about data engineering and have strong experience in building scalable data pipelines. As a Data Engineer at Joveo, you will play a leading role in designing and building datasets that power analytics for teams across Joveo. You will also be working closely with Data Analysts, Data Scientists, and...

  • Data Engineer

    5 days ago

    Bengaluru, India Australia and New Zealand Banking Group Limited (ANZ) Full time

    Data EngineerData EngineerReq ID: Department: Commercial Data & Analytics Division: Australia Commercial Location: BengaluruAbout the roleData Engineer –Role Type – FTEJob Grade: Group 4.3At ANZ our purpose is to shape a world where people and communities thrive and to achieve this, we need a talented Data Engineer to join our Australia Commercial tribe....

  • Data Engineer

    2 weeks ago

    Bengaluru, India Ascendion Full time

    Job Title: Data Engineer/ LeadDuration: Full TimeLocation: Bengaluru, Hyderabad, Chennai, Pune (Hyrbrid)NOTE: Looking for Immediate Joiners ( 2-3 weeks notice)Proficient in Azure Cloud services (or GCP is Ok) for data engineering tasks.Familiarity with Kubeflow, with a preference on hands-on experience.Strong proficiency in Python programming for data...

  • Data Engineer

    5 days ago

    Bengaluru, India Aryng Full time

    Welcome! You made it to the job description page!Aryngis looking for aSenior Data Engineerwith experience in developing enterprise-class distributed data engineering solutions on the cloud. We are seeking an entrepreneurial and technology-proficientData Engineerwho is an expert in theimplementation of a large-scale, highly efficient data platform, batch, and...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NatWest Group Full time

    Join us as a Data EngineerThis is an exciting opportunity to use your technical expertise to collaborate with colleagues and build effortless, digital first customer experiencesYou'll be simplifying the bank by developing innovative data driven solutions, using insight to be commercially successful, and keeping our customers' and the bank's data safe and...

  • Data Engineer

    3 weeks ago

    Bengaluru, India Joveo Full time

    We are looking for talented Data Engineers who are passionate about data engineering and have strong experience in building scalable data pipelines. As a Data Engineer at Joveo, you will play a leading role in designing and building datasets that power analytics for teams across Joveo. You will also be working closely with Data Analysts, Data Scientists, and...

  • Data Engineer

    3 weeks ago

    Bengaluru, India Joveo Full time

    We are looking for talented Data Engineers who are passionate about data engineering and have strong experience in building scalable data pipelines. As a Data Engineer at Joveo, you will play a leading role in designing and building datasets that power analytics for teams across Joveo. You will also be working closely with Data Analysts, Data Scientists, and...

  • Data Engineer, IN Data Engineering

    4 weeks ago

    Bengaluru, India ADCI - Karnataka Full time

    IN Data Engineering & Analytics(IDEA) Team is looking to hire a rock star Data Engineer to build and manage the largest petabyte-scale data infrastructure in India for Amazon India businesses. IN Data Engineering & Analytics (IDEA) team is the central Data engineering and Analytics team for all A.in businesses. The team's charter includes 1) Providing...

  • Data Engineer

    2 weeks ago

    Bengaluru, India Ascendion Full time

    Job Title: Data Engineer/ LeadDuration: Full TimeLocation: Bengaluru, Hyderabad, Chennai, Pune (Hyrbrid)NOTE: Looking for Immediate Joiners ( 2-3 weeks notice)- Proficient in Azure Cloud services (or GCP is Ok) for data engineering tasks.- Familiarity with Kubeflow, with a preference on hands-on experience.- Strong proficiency in Python programming for data...

  • Data Engineer

    2 days ago

    Bengaluru, India SMARTBORN SOFTWARE CONSULTING PVT.LTD. Full time

    Role DescriptionThis is a full-time hybrid role as a Data Engineer at SMARTBORN SOFTWARE CONSULTING PVT.LTD. The Data Engineer will be responsible for day-to-day tasks related to data engineering, including data modeling, Extract Transform Load (ETL) development, data warehousing, and data analytics. The role is based in Bengaluru, with flexibility for some...

  • Data Engineer, IN Data Engineering

    3 weeks ago

    Bengaluru, India ADCI - Karnataka Full time

    IN Data Engineering & Analytics(IDEA) Team is looking to hire a rock star Data Engineer to build and manage the largest petabyte-scale data infrastructure in India for Amazon India businesses.IN Data Engineering & Analytics (IDEA) team is the central Data engineering and Analytics team for all A.in businesses. The team's charter includes 1) Providing Unified...

  • Data Engineer

    3 weeks ago

    Bengaluru, India Analog Devices Full time

    Gather business and functional requirements and translate into robust scalable, solutions aligned to the Data Architecture guidelinesHands on development work on all aspects of data analysis, data provisioning, Data modeling, performance tuning and optimizationDesign, implement and operate large scale, high volume, high performance data structures for...

  • Data Engineer

    1 week ago

    Bengaluru, India Australia and New Zealand Banking Group Limited (ANZ) Full time

    Data Engineer Data Engineer Req ID: Department: Commercial Data & Analytics Division: Australia Commercial Location: Bengaluru About the roleData Engineer – Role Type – FTE Job Grade: Group 4.3 At ANZ our purpose is to shape a world where people and communities thrive and to achieve this, we need a talented Data Engineer to join our Australia...

  • Data Engineer

    3 weeks ago

    Bengaluru, India Yara Full time

    ResponsibilitiesParticipate in designing and building data pipelines to collect, manage and store data across different solutions and tools where the data resides for the YAA teams.Monitor, manage and maintain data pipelines as part of the day-to-day data operations.Build data mart tables for various use cases by working with the different data consumers...

  • Data Engineer

    10 hours ago

    Bengaluru, India Joveo Full time

    Role: Data Engineer Location: Hybrid (Bangalore/Hyderabad) About Joveo Joveo was founded in 2017 by a group of visionary entrepreneurs and enthusiastic problem solvers who foresaw the need for greater transparency and predictability in recruitment marketing. The focus was on building a cutting-edge technology platform that uses data and machine...