Data Engineer 2

2 weeks ago


Bengaluru, Karnataka, India Epsilon Full time

Job Description

Organization Objective/Purpose :

This position in the Engineering team under the Digital Integration Services organization. We drive the first mile of the customer experience through personalization of offers and content. We are currently on the lookout for a smart, highly driven software engineer.

You will be part of a team that is focused on building solutions, pipelines using latest software engineering design principles and tech stacks. You will also be expected to Identify, design, and implement improvements including re-designing infrastructure for greater scalability, optimizing data delivery and automate continuous integration and deployment processes/pipelines.

The incumbent is also expected to partner with various stakeholders, bring scientific rigor to design and develop high quality software.

She / He also must have excellent verbal and written communication skills and be comfortable working in an entrepreneurial, 'startup' environment within a larger company.

Brief Description of Role :

Develop solutions for Epsilon that will deliver high quality personalized recommendations across different channels to our customers Working with Data science team to ensure seamless integration and support of machine learning models. Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS 'big data' technologies. Develop end-to-end (Data/Dev/MLOps) pipelines based on in-depth understanding of cloud platforms, AI/ML lifecycle, and business problems to ensure solutions are delivered efficiently and sustainably. Collaborate with other members of the team to ensure high quality deliverables Learning and implementing the latest design patterns in software engineering

Qualifications

Bachelor's Degree in Engineering and related field with 2+ years of similar experience Tech Stack: Python, PySpark, Micro services, Docker, Serverless Frameworks & Databricks. Hands on experience building ETL workflows/pipelines Experience in relational and non-relational databases and SQL (NoSQL is a plus). Experience with Cloud technologies (AWS or Azure) Experience building Data and CI/CD/MLOps pipelines Familiarity with Airflow and MLFlow tools Familiarity with automated unit/integration test frameworks Experience working on AdTech or MarTech technologies is added advantage Knowledge of machine learning algorithms and concepts and implementation will be a plus Good written and spoken communication skills, team player. Strong analytic thought process and ability to interpret findings

Data Management

Experience with both structured and unstructured data, and Hadoop, Apache Spark, or similar technologies Good understanding of Data Modeling, Data Warehouse, Data Catalog concepts and tools Experience with Data Lake architectures, and with combining structured and unstructured data into unified representations Able to identify, join, explore, and examine data from multiple disparate sources and formats Ability to reduce large quantities of unstructured or formless data and get it into a form in which it can be analyzed Ability to deal with data imperfections such as missing values, outliers, inconsistent formatting, etc. Ability to manipulate large datasets, (millions of rows, thousands of variables)

Software Development

Ability to write code in programming languages such as Python, PySpark and shell script on Linux Familiarity with software development methodology such as Agile/Scrum Love to learn new technologies, keep abreast of the latest technologies within the cloud architecture, and drive your organization to adapt to emerging best practices

Architecture and Infrastructure

Architectural design experience on AWS. Experience in delivering software with AWS EC2, S3, EMR/Glue, Lambda, Data Pipeline, CloudFormation, Redshift etc. Good knowledge of working in UNIX/LINUX systems

In addition, the candidate should have strong business acumen, and interpersonal and communication skills, yet also be able to work independently. He/she should be able to communicate findings and the way techniques work in a manner that all stakeholders, both technical and non-technical, will understand.


  • Data Engineer

    2 weeks ago


    Bengaluru, Karnataka, India NTT DATA Services Full time

    Data Engineer - Snowflake, Oracle/MySQL, NoSQL, Python • 8 to 13 years of experience in Data engineering• Extensive experience in different database platforms both RDBMS (Oracle, MySQL) and NoSQL (MongoDB)• Hands-on experience in Snowflake• Proficient in Data Pipeline Orchestration• Proficient in any one of the programing languages (python or go)...

  • Data Engineer 2

    2 weeks ago


    Bengaluru, Karnataka, India Amadeus Full time

    Job Title Data Engineer 2Key responsibilities •Design, implement, deploy, and maintain data marts, ETL (extract, transform, load) processes and systems. A.k.a. as our assets. •Ensure the reliability, scalability, and performance of our data assets. •Design and maintain quality indicators as well as the corresponding monitoring and alerting means....

  • Data Engineer 2

    2 weeks ago


    Bengaluru, Karnataka, India Microsoft Full time

    Overview Data Engineer - Office Platform Microsoft IDC - Bengaluru, India Profession: Software Engineering Discipline: Data Engineering Microsoft is a global leader in software, services, devices and solutions that empower every person and every organization on the planet to achieve more. Our mission is to enable people and businesses throughout...

  • Data Engineer 2

    2 weeks ago


    Bengaluru, Karnataka, India Microsoft Full time

    OverviewData Engineer - Office PlatformMicrosoft IDC - Bengaluru, IndiaProfession: Software EngineeringDiscipline: Data EngineeringMicrosoft is a global leader in software, services, devices and solutions that empower every person and every organization on the planet to achieve more. Our mission is to enable people and businesses throughout the world to...

  • Data Engineer

    2 weeks ago


    Bengaluru, Karnataka, India NTT DATA Full time

    Job DescriptionData Engineer - Snowflake, Oracle/MySQL, NoSQL, Python8 to 13 years of experience in Data engineeringExtensive experience in different database platforms both RDBMS (Oracle, MySQL) and NoSQL (MongoDB)Hands-on experience in SnowflakeProficient in Data Pipeline OrchestrationProficient in any one of the programing languages (python or...

  • Data Engineer-2

    2 weeks ago


    Bengaluru, Karnataka, India Deutsche Bank Full time

    Data Engineer-2:Job ID:R0333521Full/Part-Time:Full-timeRegular/Temporary:RegularListed: Location:BangalorePosition Overview:Job Title - Data EngineerLocation - Bangalore, IndiaRole DescriptionRole is an Data Engineer to manage the databases/data warehouses/ data marts within the Asset Management Fund Accounting/Investment Business. Has good understanding of...

  • Senior Data Engineer

    2 weeks ago


    Bengaluru, Karnataka, India NTT DATA Full time

    Job Description Lead Data Engineering Project Team, Expertise in Databricks,Expertise in Data Management & Data Governance,Expertise in Azure Cloud and its services, Experienced in Data Modelling and Data Pipelines,Experienced in Healthcare projects, Experience implementing FHIR Data model and FHIR Server Apply from local Career site Listen to the story of...

  • Senior Data Engineer

    2 weeks ago


    Bengaluru, Karnataka, India NTT DATA Full time

    Job Description Lead Data Engineering Project Team, Expertise in Databricks, Expertise in Data Management & Data Governance,Expertise in Azure Cloud and its services, Experienced in Data Modelling and Data Pipelines,Experienced in Healthcare projects, Experience implementing FHIR Data model and FHIR Server Apply from local Career site Listen to the story...

  • Data Engineer 2

    2 weeks ago


    Bengaluru, Karnataka, India JLL Full time

    Job DescriptionAs aData Engineerat JLL Technologies, you will be directly responsible forWorking with our leaders to find new and creative ways to use data to create value for our businessDesigning, developing, deploying and maintaining data warehouses, ETL workflows, reporting cubes and reports on the Microsoft SQL Server and Azure Business Intelligence...


  • Bengaluru, Karnataka, India NTT DATA Full time

    Job Description Azure Data Engineer Requirements4+ years' overall experience in data domain (data analysis, database developer)Minimum 3 years' experience as a cloud-based Data EngineerMinimum 3 years' experience and strong knowledge of Azure Data Services such as Azure Data Lake Storage, Blob Storage, Azure Data Factory, Minimum 3 years' experience on...


  • Bengaluru, Karnataka, India NTT DATA Full time

    Job Description Azure Data Engineer Requirements 4+ years' overall experience in data domain (data analysis, database developer) Minimum 3 years' experience as a cloud-based Data Engineer Minimum 3 years' experience and strong knowledge of Azure Data Services such as Azure Data Lake Storage, Blob Storage, Azure Data Factory, Minimum 3 years' experience on...

  • Senior Data Engineer

    2 weeks ago


    Bengaluru, Karnataka, India NTT DATA Services Full time

    Lead Data Engineering Project Team, Expertise in Databricks, Expertise in Data Management & Data Governance,Expertise in Azure Cloud and its services, Experienced in Data Modelling and Data Pipelines,Experienced in Healthcare projects, Experience implementing FHIR Data model and FHIR Server


  • Bengaluru, Karnataka, India NTT DATA Services Full time

    Azure Data Engineer Requirements 4+ years' overall experience in data domain (data analysis, database developer) Minimum 3 years' experience as a cloud-based Data Engineer Minimum 3 years' experience and strong knowledge of Azure Data Services such as Azure Data Lake Storage, Blob Storage, Azure Data Factory, Minimum 3 years' experience on Apache Spark or...

  • Data Engineer-2

    2 weeks ago


    Bengaluru, Karnataka, India E901 DWS India Private Limited, Bangalore Branch Full time

    Role is an Data Engineer to manage the databases/data warehouses/ data marts within the Asset Management Fund Accounting/Investment Business. Has good understanding of Bafin/FCA/BundesBank regulatory reporting framework and reporting experiences. Can handle the day-to-day BAU of several Datawarehouse and Database Applications as part of the ADC Team. Perform...


  • Bengaluru, Karnataka, India NTT DATA Services Full time

    Azure Data Engineer Requirements4+ years' overall experience in data domain (data analysis, database developer)Minimum 3 years' experience as a cloud-based Data EngineerMinimum 3 years' experience and strong knowledge of Azure Data Services such as Azure Data Lake Storage, Blob Storage, Azure Data Factory,Minimum 3 years' experience on Apache Spark or Azure...

  • Data Analyst 2

    2 weeks ago


    Bengaluru, Karnataka, India Oracle Full time

    Building off our Cloud momentum, Oracle has formed a new organization - Oracle Health Applications & Infrastructure. This team will focus on product development and product strategy for Oracle Health, while building out a complete platform supporting modernized, automated healthcare. This is a net new line of business, constructed with an entrepreneurial...

  • Data Analyst 2

    2 weeks ago


    Bengaluru, Karnataka, India Oracle Full time

    Building off our Cloud momentum, Oracle has formed a new organization - Oracle Health Applications & Infrastructure. This team will focus on product development and product strategy for Oracle Health, while building out a complete platform supporting modernized, automated healthcare. This is a net new line of business, constructed with an entrepreneurial...

  • Data Architect

    2 weeks ago


    Bengaluru, Karnataka, India NTT DATA Full time

    Job Description: Data Modeler (8 to 12 years of experience)Summary:Do you thrive in the nexus of business needs and complex data landscapes? Do you possessthe vision to translate strategic goals into elegant and scalable data architectures? If so, thenstep into the role of Data Modeler and shape the future of data at SalesforceWe are seeking a Data Modeler...


  • Bengaluru, Karnataka, India Unisys Full time

    What success looks like in this role: The key responsibilities of an Architect position can vary depending on the specific industry, organization, and project requirements. However, some common responsibilities typically associated with an Architect role include: 1. Architecting Data Solutions : Designing and architecting end-to-end data science...

  • Software Engineer 2

    2 weeks ago


    Bengaluru, Karnataka, India Dell International Services India Pvt Ltd (7451) Full time

    Software Engineer 2 (I6) Global Operations - Data, Analytics and AutomationThe Global Operations - Data Analytics and Automation team delivers next-generation analytics and new products for digital supply chain world. Working at the cutting edge, we design and develop products— all with the most advanced technologies, tools, software engineering...