Senior Data Engineer

7 days ago


Pune, Maharashtra, India Sutherland Full time ₹ 8,00,000 - ₹ 24,00,000 per year

The BIA Senior Data Engineer designs and leads the implementation of data engineering solutions in the Business Intelligence and Analytics team.

Responsible for design, development, optimization, and maintenance of robust and scalable data engineering pipelines. Ensure data and analytical solutions meet Business Requirements and best practices. Collaborate with data architects, data scientists and other stakeholders to provide scalable analytical solutions for the organization. Continuously improve and optimize existing ETL processes adhering to data privacy and best practices. Document all data engineering processes and architecture.

The position also requires staying abreast with changes in technology, programming languages, and software development tools.

Key Responsibilities & Duties:

Data Modeling/Designing Datasets (20%): Designs, implements, and maintains complex data engineering solutions to acquire and prepare data to support reporting, BI, analytics and data science use cases. Plans and drives the development of data engineering solutions ensuring that solutions balance functional and non-functional requirements.

Data Architecture and Technical Infrastructure (20%): Plans and drives the development of data engineering solutions ensuring that solutions balance functional and non-functional requirements. Monitors application of data standards and architectures including security and compliance.

Data Pipeline/ETL (15%): Develops various functional components of the data warehouse including ETL processes, data cleansing, system management, load automation, data acquisitions, and exception handling. Creates and maintains data pipelines to connect data within and between data stores, applications and organizations.

SDLC Methodology & Project Management (10%): Plans and contributes to technical transitions between development, testing, and production phases of solutions' lifecycle, and the facilitation of the change control, problem management, and communication processes.

Innovation, Continuous Improvement & Optimization (10%): Provides advice and guidance to others using the data structures and associated components.

Support & Operations (5%): Manages monitoring, job control and production support. Works with other team members, DBA team and other teams to resolve issue.

Data Governance and Data Quality (5%): Carries out complex data quality checking and remediation and Explore ways to enhance data quality and reliability. Makes recommendations for improving data management and development of data policies.

Metadata Management & Documentation (5%): Contributes to organizational policies, standards, and guidelines for data engineering. Develops metadata content to document data assets and data pipelines.

End-User Support, Education and Enablement (5%): Plans and facilitates training and Data Literacy initiatives within the team and End user community.

Partnership and Community Building (5%): Collaborates with other IT teams, business community, data scientists and other architects to meet business requirements. Interact with DBAs on data designs optimal for data engineering solutions performance

Skills and Requirements:

  1. Development, maintenance, and enhancement of Data Pipelines (ETL/ELT) and processes with thorough knowledge of star/snowflake schemas
  2. Developing complex SQL queries and SQL optimization
  3. Development experience must be full Life Cycle experience including business requirements gathering, data sourcing, testing/data reconciliation, and deployment within Business Intelligence/Data Warehousing Architecture.
  4. Designing and implementing data security
  5. Monitoring and optimizing data storage and data processing
  6. Use of Object function/object-oriented scripting languages including Scala, C++, Java and Python
  7. Cloud data warehouse and data lake platforms (e.g., Snowflake, Databricks, Redshift, BigQuery, MS Synapse, MS ADLS, Apache Haddop, etc.)

Specific Skills and/or Business Competencies

  • Developing data services that are fit for purpose, resilient, scalable, and future proof to meet user needs
  • Advanced SQL knowledge and experience working with relational databases and working familiarity with various databases.
  • Demonstrated understanding and experience using software and tools including ETL, relational SQL and NoSQL databases, big data tools like Kafka, Spark, and Hadoop
  • Experience building and optimizing big data pipelines, architecture, and data sets.
  • Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
  • Strong analytic skills related to working with unstructured datasets.
  • Experience and knowledge of project management best practices and agile software development methodologies
  • A successful history of manipulating, processing, and extracting value from large, disconnected datasets.
  • Working knowledge of message queuing, stream processing, and highly scalable big data data stores.
  • Strong project management and organizational skills.
  • Experience supporting and working with cross-functional teams in a dynamic environment.
  • Understanding of data science concepts

Familiarity with AI/ML frameworks and libraries


  • Data Engineer

    5 days ago


    Pune, Maharashtra, India Data Axle Full time ₹ 15,00,000 - ₹ 28,00,000 per year

    About Data Axle:Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology platform and leveraging proprietary...

  • Senior QE Engineer

    1 week ago


    Pune, Maharashtra, India NTT DATA Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Req ID: 330974NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Senior QE Engineer to join our team in Pune, Mahārāshtra (IN-MH), India (IN). Senior Quality Engineer I (JavaScript)...

  • Data Engineer

    1 week ago


    Pune, Maharashtra, India Jash Data Sciences Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Do you love solving real-world data problems with the latest and best techniques? And having fun while solving them in a team Then come join our high-energy team of passionate data people. Jash Data Sciences is the right place for you.We are a cutting-edge Data Sciences and Data Engineering startup based in Pune, India.We believe in continuous learning and...


  • Pune, Maharashtra, India Data Dynamics Full time US$ 1,20,000 - US$ 2,00,000 per year

    About The RoleWe are seeking a Senior Software Engineer to join our team in Pune. As a key member of our engineering group, you will play a crucial role in the entire software development lifecycle, from initial design to hands-on development. You will work with cross-functional teams to create innovative solutions that are scalable and reliable. This...


  • Pune, Maharashtra, India NCSI Technologies Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Title: Senior Data Engineer Company Name: NCSi Job Description: NCSi is seeking a highly skilled Senior Data Engineer to join our dynamic team. In this role, you will be responsible for designing, developing, and optimizing data pipelines and architectures to support our data-driven initiatives. You will collaborate with cross-functional teams to...

  • Senior Data Engineer

    2 weeks ago


    Pune, Maharashtra, India BlueCloud Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    ResponsibilitiesWe are looking for a Senior Data Engineer to join their team and contribute to the design and implementation of data engineering solutions. This role requires strong hands-on experience withDBT and Snowflake. The Senior Data Engineer will be responsible for building, maintaining, and optimizing data pipelines, as well as implementing...


  • Pune, Maharashtra, India Vanderlande Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job TitleSenior Data EngineerJob DescriptionIf you are passionate about unlocking the power to gain next level insights into a great variety of data, powering data driven decision making through advanced analytics and data science practises, you might have just found your perfect next career step.In this job you will have versatile responsibilities,...


  • Pune, Maharashtra, India True Talents Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Job Title :Senior Data EngineerLocation :Pune / NoidaExperience :4 - 8 YearsEmployment Type :Full-timeNotice Period :Up to 30 DaysAbout The RoleWe are looking for an experienced Senior Data Engineer to design, build, and optimize scalable data pipelines and data platforms. The ideal candidate should have strong expertise in modern data engineering tools,...


  • Pune, Maharashtra, India NCS Group Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Title: Senior Data EngineerCompany Name: NCSiJob Description:NCSi is seeking a highly skilled Senior Data Engineer to join our dynamic team. In this role, you will be responsible for designing, developing, and optimizing data pipelines and architectures to support our data-driven initiatives. You will collaborate with cross-functional teams to understand...

  • Senior Data Engineer

    4 weeks ago


    Pune, Maharashtra, India SpeedMart Full time

    Company Profile :Our client is a global IT services company that helps businesses with digital transformation with offices in India and the United States.It helps businesses with digital transformation, provide IT collaborations and uses technology, innovation, and enterprise to have a positive impact on the world of business.With expertise is in the fields...