Data Engineer

2 weeks ago

Maharashtra, India RIA Advisory Full time

Job Summary

Seeking a Data Engineer who will be working closely with data scientists and developers to architect data transformations, analyze, develop, maintain, and test data infrastructures. This role requires deep hands-on production expertise in Apache Spark and data engineering, along with a variety of knowledge of the big data ecosystem.


Job Responsibilities

● Work with large data sets, develop programs to create, parse, process, analyze complex datasets used throughout our organization

● Consolidate, join datasets to create easily consumable, consistent, holistic information and create visualizations for the same

● Create and define performance metrics; ideate, innovate, and hack through the existing systems to improve performance

● Analyze third party and client data to understand its overlap with different data sources and how best to integrate it

● Guide strategic customers as they implement transformational big data projects, including end-to-end development and deployment of industry-leading big data and AI applications


Job Specifications

● Education, qualifications: solid foundation in Computer Science and strong competencies in data structure, algorithms and software architecture

● Experience: 5+ years


Required Skills

● Deep hands-on expertise in Apache Spark (Python, Java, or Kotlin)

● 3+ years’ experience in design and implementation of big data technologies (Apache Spark, Apache Delta lake, Apache Kafka, NoSQL database) and familiarity with data architecture patterns (data Lakehouse, data warehouse, data lake, and data streaming)

● Familiarity with Kubernetes as the resource manager for Spark jobs is a plus

● Minimum 2 year of programming experience with Python in similar, relevant data engineering role

● Experience with query tuning, performance tuning, troubleshooting, and debugging Spark and other big data solutions

● Experience working with libraries like pandas, numpy, scipy and comfortable with ETL processes

● Experience with creation, orchestration and monitoring of job pipelining with either one of frameworks like airflow, etc.

● Familiarity with Restful API's is a plus

● Working knowledge of relational databases such as Snowflake, Postgres, No SQL databases like, Elastic, Mongo is desirable

● Experience with source code-controlled environments like git

● Ability to communicate complex technical topics concisely

● Demonstrated experience working with diverse data sets across multiple domains

● Good understanding of DBT, Iceberg, Delta Lake, Hive, Hadoop

● Experience of data processing architectures (both streaming and batch) is highly preferable


Soft Skills

● Excellent and proven communication, presentation, facilitation and organizational skills

● Proven ability to work well with multi-disciplinary teams

● Critical thinking and problem-solving


About RIA Advisory:

  • RIA Advisory LLC (RIA) is a business advisory and technology company that specializes in the field of Revenue Management and Billing for Banking, Payments, Capital Markets, Exchanges, Utilities, Healthcare, and Insurance industry verticals. With a highly experienced team in the field of Pricing, Billing & Revenue Management, RIA prioritizes understanding client needs and industry best practices to approach any problem with insight and careful strategic planning. Each one of RIA Advisory's Managing Partners have over 20 years of industry expertise and experience, our leadership and consulting team demonstrate our continued efficiency to serve our clients as a strategic partner especially for transforming ORMB and CC&B space.


  • Services Offered: • Business Process Advisory for Revenue management processes • Technology Consulting & Implementation • Help clients transition to latest technology suite and overcome business problems. • Managed Services • Quality Assurance • Cloud Services


  • Product Offered: • Data Migration and Integration Hub • Data Analytics Platform • Test Automation • Document Fulfilment • Customer Self Service


  • Top Industries/Verticals: • Financial Services • Healthcare • Energy and Utilities • Public Sector Revenue Management


  • We recognize the impact of technologies on the process & people that drive them, innovate scalable processes & accelerate the path to revenue realization. We value our people and are “Great Place to Work” – Certified.

  • Data Scientist

    1 week ago

    Pune, Maharashtra, India Data Dynamics Full time

    **Objectives of this role**: - Collaborate with product design and engineering teams to develop an understanding of needs. - Research and devise innovative statistical models for data analysis. - Build innovative models and integrate them in our product. - Communicate findings to all stakeholders. - Keep current with technical and industry...

  • Associate Data Scientist

    3 weeks ago

    Pune, Maharashtra, India Data Axle Full time

    Data Axle Inc. has been an industry leader in data, marketing solutions, sales, and research for over 50 years in the USA. Data Axle now has an established strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology platform and by leveraging...

  • Associate Data Scientist

    3 weeks ago

    Pune, Maharashtra, India Data Axle Full time

    Data Axle Inc. has been an industry leader in data, marketing solutions, sales, and research for over 50 years in the USA. Data Axle now has an established strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology platform and by leveraging...

  • Data Engineer

    1 week ago

    Maharashtra, India RIA Advisory Full time

    Job Summary Seeking a Data Engineer who will be working closely with data scientists and developers to architect data transformations, analyze, develop, maintain, and test data infrastructures. This role requires deep hands-on production expertise in Apache Spark and data engineering, along with a variety of knowledge of the big data ecosystem. Job...

  • Data Engineer

    4 weeks ago

    Maharashtra, India Perfict Full time

    Title: Data EngineerJob Location: 100% Remote (PAN India)Role: Contract RoleTiming: 09 - 05 ISTLooking candidate who can join in 15 -30 days after job offerData Engineer - Financial DomainJob Description: Based on the requirements of the project, need a strong resource who can translate the business requirements to logical SQLs by their own.Must have SQL (4+...

  • Data Engineer

    4 weeks ago

    Maharashtra, India Perfict Full time

    Title: Data EngineerJob Location: 100% Remote (PAN India)Role: Contract RoleTiming: 09 - 05 ISTLooking candidate who can join in 15 -30 days after job offerData Engineer - Financial DomainJob Description: Based on the requirements of the project, need a strong resource who can translate the business requirements to logical SQLs by their own.Must have SQL (4+...

  • Data Engineer

    4 weeks ago

    Maharashtra, India Perfict Full time

    Title: Data Engineer Job Location: 100% Remote (PAN India) Role: Contract Role Timing: 09 - 05 IST Looking candidate who can join in 15 -30 days after job offer Data Engineer - Financial Domain Job Description: Based on the requirements of the project, need a strong resource who can translate the business requirements to logical SQLs by their own....

  • Data Enginer

    3 weeks ago

    Pune, Maharashtra, India Deutsche Bank Full time

    **Data Enginer**: **Job ID**:R0316159**Full/Part-Time**:Full-time**Regular/Temporary**:Regular**Listed**:2024-03-20**Location**:Pune**Position Overview**: **Job Title - Cloud Data Engineer, Surveillance Analytics** **Location - Pune, India** **Role Description** Surveillance & Regulatory: We are responsible for delivering solutions that protect Deutsche...

  • Data Engineer

    2 months ago

    Maharashtra, India XL Advisors Full time

    Job Description :Qualification :- Bachelors in Engineering (CS/IT) and above with a bent towards analytics.- Cloud certifications (AWS/Azure) will be a plusSkills/ Experience :- Ability to engage with stakeholders to understand business requirements and convert the same into technical problems for solution development - Knowledge of creating notebooks using...

  • Data Engineer

    3 months ago

    Maharashtra, India XL Advisors Full time

    Job Description : Qualification :- Bachelors in Engineering (CS/IT) and above with a bent towards analytics. - Cloud certifications (AWS/Azure) will be a plusSkills/ Experience :- Ability to engage with stakeholders to understand business requirements and convert the same into technical problems for solution development - Knowledge of creating notebooks...

  • Regional Solutions Architect

    7 days ago

    Pune, Maharashtra, India Data Dynamics Full time

    At Data Dynamics, we take a holistic approach to managing the explosive growth of unstructured data! We care about delivering storage optimization and data optimization. Our Unified Unstructured Data Management Platform is an industry leader in Unstructured Data Management and our journey has just begun. With over 300 clients operating in 6 continents, we...

  • Data / Ml Engineer-4

    6 days ago

    Pune, Maharashtra, India Mastercard Full time

    Our Purpose We work to connect and power an inclusive, digital economy that benefits everyone, everywhere by making transactions safe, simple, smart and accessible. Using secure data and networks, partnerships and passion, our innovations and solutions help individuals, financial institutions, governments and businesses realize their greatest potential. Our...

  • Enterprise Resource Planning Advisor

    2 months ago

    Pune, Maharashtra, India NTT DATA Full time

    Cloud Engineer will be responsible for providing technical solutions to support and forward Independent Health's cloud strategy **Mandatory**: Python and Data engineering skills Needs to have Power Platform environment or tenant level experience. We need someone that has experience provisioning environments, managing the Dataverse, deploying gateways and...

  • Data Engineer-1

    2 weeks ago

    Pune, Maharashtra, India Deutsche Bank Full time

    **Data Engineer-1**: **Job ID**:R0333522**Full/Part-Time**:Full-time**Regular/Temporary**:Regular**Listed**:2024-05-20**Location**:Pune**Position Overview**: **Job Title - Data Engineer** **Location - Pune, India** **Role Description** Role is an Data Engineer to manage the databases/data warehouses/ data marts within the Asset Management Fund...

  • Associate Solution Architect

    6 days ago

    Pune, Maharashtra, India NTT DATA Full time

    **Req ID**: 272090 We are currently seeking a Associate Solution Architect to join our team in Pune, Mahārāshtra (IN-MH), India (IN). "Qualifications Required: Minimum of 8+ years' project experience with PeopleSoft HCM. Strongly prefer PeopleSoft HCM, Application designer, Application engine, component interface, File layout, SQR, PS Query, Integration...

  • data Engineer-3

    2 weeks ago

    Pune, Maharashtra, India Talent21 Full time

    Pune, Maharashtra, India **Job Description**: Data Engineer Role 2: - Power bI -Intermediate - Pyspark /Python - ADF - ADB Azure fundamentals Budget : till 25 LPA NP:0-30 days Exp : 6 years + Loc : Pune (Kharadi) **Required Skills**: PowerBI, Python, Azure ADF, Azure Databricks

  • Aws Redshift Admin

    4 weeks ago

    Pune, Maharashtra, India ITI Data Full time

    **Requisition No** - VN313 **Employment Type** - Permanent **Location Country** - India **Location Region** **Location City** - Pune **Job Details** **Locations**: India - **Type**: Full-time - **Experience**: 6-8 Years - **Functions**: Consulting, Finance, Information Technology, Data Governance - **Industries**: Capital Markets, Investment Banking,...

  • Systems Integration Advisor

    2 weeks ago

    Pune, Maharashtra, India NTT DATA Full time

    **Req ID**: 283887 We are currently seeking a Systems Integration Advisor to join our team in Chennai, Tamil Nādu (IN-TN), India (IN). **Role Title** Authentication Infrastructure Operations Engineer Key Responsibilities - Deploy and manage day-to-day operations of critical security infrastructure - Function as an Infrastructure Subject Matter Expert...

  • Principal Architect

    4 days ago

    Pune, Maharashtra, Maharashtra, India Johnson Controls International Full time

    Introduction The future is being built today and Johnson Controls is making that future more productive, more secure and more sustainable. We are harnessing the power of cloud, AI/ML and Data analytics, the Internet of Things (IoT), and user design thinking to deliver on the promise of intelligent buildings and smart cities that connect communities in ways...

  • Data Scientist

    3 weeks ago

    Pune, Maharashtra, India Serendipity Full time

    Role - Data Scientist (Isometric) Location - Pune Notice period - 15days Key Responsibilities: Utilize data science techniques to analyze and interpret complex data from isometric drawings, P&IDs, and other engineering documents. Develop algorithms and predictive models to forecast system behaviors and optimize engineering designs. Collaborate with...