Staff Data Engineer

2 months ago

Bengaluru, Karnataka, India Leading-Biotechnology-Company Full time

JOB DUTIES:

  • Translate business requirements into data requirements, data warehouse design and sustaining data management strategies on enterprise data platforms like Snowflake.
  • Work with project leads, stakeholders, and business SMEs to define technical specifications to develop data modeling requirements and maintain data infrastructure to provide business users with the tools and data needed.
  • Solution architect, design and develop large scale and optimized analytics solutions.
  • Requirements gathering and analysis, development planning and coordination in collaboration with stakeholder teams.
  • Understand data architecture & solution design, design & develop dimensional models in an enterprise data warehouse environment.
  • Development and automation of enterprise data transformation pipelines, ELT/ ETL processes.
  • Work with cross functional teams and process owners on the development of test cases and scripts, test models and solutions to verify that requirements are met and ensuring high levels of data quality.
  • Develop and apply quality assurance best practices.
  • Design and apply data engineering best practices for an enterprise data warehouse.
  • Analyze data and data behaviors to support business user queries.
  • Excellent understanding of impact due to changes in data platforms, data models and data behaviors.
  • Excellent problem-solving skills and ability to troubleshoot complex data engineering issues.
  • Benchmark application operational performance periodically, track (metrics) and fix issues.
  • Understand and comply with data governance and compliance practices as defined for risk management. This includes data encryption practices, RBAC and security policies.
  • Promote and apply metadata management best practices supporting enterprise data catalogs.
  • Support change and release management processes.
  • Support incident and response management including problem solving and root cause analysis, documentation.
  • Support automation and on-call processes (Tier 1 / Tier 2).

Specific Skills Or Other Requirements:

  • Requires 8 + years of experience with:

Primary Experience

  • Data Warehouses and Data Transformations

Data Platforms:

Required: Snowflake cloud data platform

Preferred: Denodo data virtualization, SAP HANA analytics

Data Engineering:

  • Snowflake: 5+ years of required expertise with Snowflake SnowSQL, Snowpipe (integrated with AWS S3), Streams and Tasks, Stored Procedures, Merge statements, Functions, RBAC, Security Policies, Compute and Storage usage optimization techniques, Performance optimization techniques.
  • Dbt Cloud:3+ years of expertise with dbt cloud platform, very good understanding of data models (views, data materializations, incremental data loads, snapshots), cross functional references, DAGs and its impact, job scheduling, audit and monitoring, working with code repositories, deployments.
  • SnapLogic: Data integrations with SnapLogic integration platform is a plus.
  • Certifications:Snowflake and dbt Cloud data engineering certifications is a plus.

Data Orchestration:

  • Required: Control-M, Apache Airflow
  • Cloud Storage Platforms:
  • Required: Amazon Web Services
  • Preferred: Microsoft Azure

Programming/ Scripting:

  • Snowflake: SQL Scripting (Snowpipes, Tasks, Streams, Merge Statements, Stored Procedures, Functions, Security Policies DDM, Row-Access), SQL Scripting, Python

Code Management:

  • Required:Excellent understanding of working with code repositories like GitHub, GitLab ,code version management, branching and merging patterns in a central repository managing cross functional code deployments.
  • Data Operations:Excellent understanding of Data Ops practices for data management.
  • Solution Design:Good understanding of end-to-end solution architecture and design practices, ability to document solutions (maintain diagrams)
  • Stakeholder Engagement: Ability to take the lead and drive project activities collaborating with Analytics stakeholders and ensure the requirement is completed.
  • Data Warehousing: Excellent on fundamental concepts of dimensional modeling, experience working on data warehouse solutions, requirement gathering, design & build, data analysis, data quality. data validations, developing data transformations using ELT/ ETL patterns.

Source Systems:

  • Required: Knowledge and experience maintaining dimensional data model built on data from source systems like SAP ERP (On premises, Cloud), Salesforce CRM, Workday, Planisware, Team Center PLM. Other source systems are a plus.
  • Data As-A-Product: Preferred knowledge and experience working with data treated as data products. Illumina is following a hybrid data mesh architecture which promotes data as a product for data lifecycle management.
  • Governance: Good understanding of working with companies having regulated systems and processes for data. Adherence to data protection practices using tagging, security policies and data security (object-level, column-level, row-level). Promoting and applying best practices for data catalogs, following data classification practices and metadata management for data products within your scope.
  • Operating Systems: Windows, Linux

Education & Experience:

  • Bachelors degree equivalent in Computer Science/Engineering or equivalent degree.

  • Digital Engineering Staff Engineer

    3 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job Title: Digital Engineering Staff EngineerJob Summary:NTT DATA Services is seeking a highly skilled Digital Engineering Staff Engineer to join our team in Bangalore, India. As a key member of our digital transformation team, you will be responsible for designing, developing, and implementing SAP BODS solutions to drive business growth and improve...

  • Digital Engineering Staff Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job OverviewNTT DATA Services is seeking a highly skilled Digital Engineering Staff Engineer to join our team in Bangalore, India. As a key member of our digital transformation team, you will be responsible for designing and implementing SAP BODS solutions to drive business growth and innovation.Key ResponsibilitiesDesign and develop SAP BODS solutions to...

  • Staff Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India Infoblox Full time

    Job Title: Staff Data EngineerInfoblox is seeking a highly skilled Staff Data Engineer to join our Cloud Engineering team in Bangalore, India. As a key member of our team, you will play a critical role in developing platforms and products for Infoblox's SaaS product line, delivering next-level networking for our customers.Key Responsibilities:Curate...

  • Cloud Engineering Staff Developer

    6 hours ago

    Bengaluru, Karnataka, India NTT DATA Services Full time

    Description: We are seeking a talented Cloud Engineering Staff Developer to join our team in India.About the Job:We are looking for a skilled professional with expertise in Databricks on AWS to design, build, and maintain scalable data pipelines.Responsibilities:Design, build, and maintain scalable data pipelines using Databricks on AWS.Ingest data from...

  • Data Centre Support Engineer

    6 hours ago

    Bengaluru, Karnataka, India NTT DATA Full time

    About the RoleThe Data Centre Remote Technical Support Engineer is an exciting entry-level engineering position at NTT DATA, where you will play a crucial role in delivering exceptional technical support services to our clients.Key ResponsibilitiesProvide professional remote technical support to clients, identifying and resolving technical incidents and...

  • Staff Data Engineer

    3 months ago

    Bengaluru, Karnataka, India Leading-Biotechnology-Company Full time

    We are seeking a highly experienced Staff Data Engineer with 10+ years of experience to join our talented team in Bangalore. This position will lead our data engineering efforts in Bangalore. In this strategic role, you will be responsible for designing, implementing, and optimizing complex data solutions using Python, Spark, SQL, Snowflake, dbt, machine...

  • Staff Data Architect

    3 weeks ago

    Bengaluru, Karnataka, India Wabtec Full time

    Job Title: Staff Data ArchitectWabtec Corporation is a leading global provider of equipment, systems, and digital solutions for the rail and transit industries. We are seeking a highly skilled Staff Data Architect to join our team in Bangalore, India.Job Summary:The Staff Data Architect will be responsible for developing and implementing database-specific...

  • Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job Title: Data EngineerJob Summary:We are seeking a highly skilled Data Engineer to join our team at NTT DATA. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data systems and pipelines using Azure data factory and Azure databricks.Key Responsibilities:Design and develop innovative data pipelines using Azure...

  • Staff Software Engineer

    3 weeks ago

    Bengaluru, Karnataka, India Databricks Full time

    About DatabricksDatabricks is a leading data and AI platform that enables organizations to solve complex problems and drive business growth. Our platform is built by engineers, for engineers, and we're passionate about empowering data teams to make a meaningful impact.Job SummaryWe're seeking a highly skilled Staff Software Engineer to join our team in...

  • Data Engineer

    4 days ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job DescriptionData Engineer - Diverse Data Engineering Activities on Azure Data PlatformMain Responsibilities:Design and develop big data processing notebooks using Azure DatabricksLead the data identification, data analysis efforts for data sources and work directly with data owner teamsDevelop data requirements for new data sourcesDesign and development...

  • Staff Engineer

    3 weeks ago

    Bengaluru, Karnataka, India American Express Full time

    About the RoleWe are seeking a highly skilled Staff Engineer to join our Finance Data Engineering team at American Express. As a Staff Engineer, you will play a key role in driving our technology transformation, delivering innovative products and services, and leading a team of talented engineers.Key ResponsibilitiesLead the development of complex software...

  • Senior Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job Title: Senior Data EngineerJob Summary:We are seeking a highly skilled Senior Data Engineer to join our team in Bangalore, India. The successful candidate will have a strong background in AWS cloud services, expertise in SQL and relational databases, and proficiency in Python programming.Key Responsibilities:Design and implement data engineering...

  • Data Visualization Engineer

    1 week ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job Title: Data Visualization EngineerJob Summary:Data Visualization Engineer - NTT DATAData visualization engineers utilize expertise in information visualization and design to create and deploy new platforms and services for visualization initiatives. Visualization specialists build interactive and complex data visualizations and analysis tools. They are...

  • Staff Software Engineer

    3 weeks ago

    Bengaluru, Karnataka, India Rippling Full time

    About RipplingRippling is a cutting-edge technology company that provides a comprehensive platform for HR, IT, and Finance. Our mission is to simplify the complexities of workforce management and empower businesses to thrive.The RoleWe are seeking a highly skilled Staff Software Engineer to join our Analytics Platform team. As a key member of our team, you...

  • Data Engineer

    5 hours ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job DescriptionReq ID: 301470NTT DATA is seeking a skilled Data Engineer to partner with stakeholders across business units to develop analyses and documentation in a collaborative way.The ideal candidate will have three or more years of experience with Python, SQL, and data visualization/exploration tools, as well as familiarity with the AWS ecosystem.About...

  • Data Engineering Specialist

    4 days ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job SummaryReq ID: 302206NTT DATA is seeking a highly skilled Senior Data Engineer to join our team in Bangalore, India.Job PurposeThe Senior Data Engineer will play a key role within the Data & Analytics team, designing, building, and maintaining systems and infrastructure for processing, collecting, storing, and analyzing large volumes of data.Key...

  • Staff Software Engineer

    6 days ago

    Bengaluru, Karnataka, India 6sense Full time

    Mission and Values: At 6sense, we envision a future where our technology empowers businesses to create revenue and foster a culture built on accountability, growth mindset, integrity, fun, and teamwork. As a Staff Software Engineer - Data Architect, you will play a pivotal role in shaping our technology and driving our mission forward. Key...

  • Staff Software Engineer

    3 weeks ago

    Bengaluru, Karnataka, India Databricks Full time

    Unlock the Power of Data with DatabricksWe're passionate about empowering data teams to tackle the world's most complex challenges. Our cutting-edge data and AI infrastructure platform enables customers to harness deep data insights and drive business growth.As a Staff Software Engineer (Data Platform) at Databricks, you'll have the opportunity to work on a...

  • Data Engineer

    1 week ago

    Bengaluru, Karnataka, India NTT DATA Services Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at NTT DATA Services. As a Data Engineer, you will be responsible for designing and developing data pipelines utilizing Cloud technologies Azure/AWS/GCP.Key ResponsibilitiesDesign and develop data pipelines using Cloud technologies Azure/AWS/GCPLead a team of Data Engineers and act...

  • Data Engineer

    3 days ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job DescriptionKey Skills and Qualifications:Cloud Platforms: Proficient in AWS.Programming Languages: Expert in SQL and Python.ETL Tools: Experience with ETL tools and frameworks.Requirements:Education: Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.Skills:Proven experience in using Snowflake for data...