Senior Data Engineer

4 months ago

Greater Noida, India Coforge Full time

Job Description

? Must be extremely proficient in Data Warehouse ETL Design/Architecture, dimensional/relational data modelling. ? Experience in atleast one ETL development project, writing/analyzing complex stored procedures. ? Should have entry level/intermediate experience in Python/PySpark ? working knowledge on spark/pandas dataframe, spark multi-threading, exception handling, familiarity with different boto3 libraries, data transformation and ingestion methods, ability to write UDF. ? Snowflake ? Familiarity with stages and external tables, commands in snowflake like copy, unload data to/from S3, working knowledge of variant data type, flattening nested structure thru SQL, familiarity with marketplace integrations, role-based masking, pipes, data cloning, logs, user and role management is nice to have. ? Familiarity with Coalesce/dbt is an added advantage for this job. ? Collibra integration experience for Data Quality and Governance in ETL pipeline development is nice to have. ? AWS ? Should have hands-on experience with S3, Glue (jobs, triggers, workflow, catalog, connectors, crawlers), CloudWatch, RDS and secrets manager. ? AWS - VPC, IAM, Lambda, SNS, SQS, MWAA is nice to have. ? Should have hands-on experience with version controlling tools like github, working knowledge on configuring, setting up CI/CD pipelines using yaml, pip files. ? Streaming Services ? Familiarity with Confluent Kafka or spark streaming, or Kinesis (or equivalent) is nice to have. ? Data Vault 2.0 (hubs satellite links) experience will be a PLUS. Interpersonal ? Highly proficient in Publisher, PowerPoint, SharePoint, Visio, Confluence and Azure DevOps ? Working knowledge of best practices in value-driven development (requirements management, prototyping, hypothesis-driven development, usability testing) ? Good communicator with problem solving mindset and focus on process improvement ? Consistently demonstrates clear and concise written and verbal communication skills ? Good interpersonal skills, ability to interact with Senior Management ? Highly self-motivated with a strong sense of initiative ? Excellent multitasking skills and task management strategies ? Ability to work well in a team environment, meet deadlines, demonstrate good time management, and multi-task in a fast-paced project environment. Responsibilities ? Partner with business stakeholders to gather requirements and translate them into technical specifications and process documentation for IT counterparts (onshore and of

Posted On

? Must be extremely proficient in Data Warehouse ETL Design/Architecture, dimensional/relational data modelling. ? Experience in atleast one ETL development project, writing/analyzing complex stored procedures. ? Should have entry level/intermediate experience in Python/PySpark ? working knowledge on spark/pandas dataframe, spark multi-threading, exception handling, familiarity with different boto3 libraries, data transformation and ingestion methods, ability to write UDF. ? Snowflake ? Familiarity with stages and external tables, commands in snowflake like copy, unload data to/from S3, working knowledge of variant data type, flattening nested structure thru SQL, familiarity with marketplace integrations, role-based masking, pipes, data cloning, logs, user and role management is nice to have. ? Familiarity with Coalesce/dbt is an added advantage for this job. ? Collibra integration experience for Data Quality and Governance in ETL pipeline development is nice to have. ? AWS ? Should have hands-on experience with S3, Glue (jobs, triggers, workflow, catalog, connectors, crawlers), CloudWatch, RDS and secrets manager. ? AWS - VPC, IAM, Lambda, SNS, SQS, MWAA is nice to have. ? Should have hands-on experience with version controlling tools like github, working knowledge on configuring, setting up CI/CD pipelines using yaml, pip files. ? Streaming Services ? Familiarity with Confluent Kafka or spark streaming, or Kinesis (or equivalent) is nice to have. ? Data Vault 2.0 (hubs satellite links) experience will be a PLUS. Interpersonal ? Highly proficient in Publisher, PowerPoint, SharePoint, Visio, Confluence and Azure DevOps ? Working knowledge of best practices in value-driven development (requirements management, prototyping, hypothesis-driven development, usability testing) ? Good communicator with problem solving mindset and focus on process improvement ? Consistently demonstrates clear and concise written and verbal communication skills ? Good interpersonal skills, ability to interact with Senior Management ? Highly self-motivated with a strong sense of initiative ? Excellent multitasking skills and task management strategies ? Ability to work well in a team environment, meet deadlines, demonstrate good time management, and multi-task in a fast-paced project environment. Responsibilities ? Partner with business stakeholders to gather requirements and translate them into technical specifications and process documentation for IT counterparts (onshore and of

Skills Required

AZURE DATA FACTORY

Posted On AZURE DATA FACTORY Location

Greater Noida

Posted On Greater Noida Desirable Skills

PYTHON, AZURE DATA FACTORY, SNOWFLAKE

Posted On PYTHON, AZURE DATA FACTORY, SNOWFLAKE

  • Senior Data Engineer

    4 months ago

    Greater Noida, India Coforge Full time

    Job Description Position: Senior Data Engineer (7-10 Years) Location: Gr. Noida Duration: Full Time MUST HAVE SKILLS: 1: Python 2: Spark 3: SQL 4: Azure Cloud (Databricks) 5: AI (GPT-4, OpenAI Codex) RESPONSIBILITIES: Design, develop, and maintain data ingestion and processing pipelines for various data sources and...

  • Senior Data Engineer

    3 months ago

    Greater Noida, India Coforge Ltd. Full time

    ? Must be extremely proficient in Data Warehouse ETL Design/Architecture, dimensional/relational data modelling. ? Experience in atleast one ETL development project, writing/analyzing complex stored procedures. ? Should have entry level/intermediate experience in Python/PySpark ? working knowledge on spark/pandas dataframe, spark multi-threading, exception...

  • Senior Software Engineer Data Engineering-Noida/ Greater Noida-TO THE NEW-4-7

    2 days ago

    Noida/ Greater Noida, India TO THE NEW Full time

    Apply for Senior Software Engineer Data Engineering, Career Progress Consultants in Noida/ Greater Noida for 4 - 7 Year of Experience on TimesJobs.com.

  • Data Engineer

    1 month ago

    Noida, India NTT DATA Services Full time

    Req ID: 287747  NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Noida, Uttar Pradesh (IN-UP), India (IN). Mandatory Skills...

  • Data engineer

    2 days ago

    Greater Noida, India Dev Full time

    Overview:The data engineer plays a crucial role in our organization responsible for designing developing and maintaining our data architecture and infrastructure. They will collaborate with data scientists analysts and other stakeholders to ensure the efficient flow and storage of data across our systems. The data engineer will also be instrumental in...

  • Senior Data Engineer

    1 week ago

    Noida, India Sopra Steria Full time

    Senior Data EngineerLocation: Noida/ ChennaiExperience: 4-8 yearsSkillset: Spark, Scala and HadoopRequired Skills:Strong expertise of programming languages –ScalaHands-on experience in Spark including Data frame core functions SparkSQL, Spark Streaming and with HIVE, Hadoop, Kafka, YARN.Good Experience in big data cloud platform, preferably Azure,...

  • Senior Data Engineer

    1 week ago

    Noida, India Sopra Steria Full time

    Senior Data Engineer Location: Noida/ Chennai Experience: 4-8 years Skillset: Spark, Scala and Hadoop Required Skills: Strong expertise of programming languages –Scala Hands-on experience in Spark including Data frame core functions SparkSQL, Spark Streaming and with HIVE, Hadoop, Kafka, YARN. Good Experience in big data cloud platform, preferably Azure,...

  • Senior Data Engineer

    3 days ago

    noida, India Sopra Steria Full time

    Senior Data Engineer Location: Noida/ Chennai Experience: 4-8 years Skillset: Spark, Scala and Hadoop Required Skills: Strong expertise of programming languages –Scala Hands-on experience in Spark including Data frame core functions SparkSQL, Spark Streaming and with HIVE, Hadoop, Kafka, YARN. Good Experience in big data cloud platform, preferably Azure,...

  • Senior Data Engineer

    3 days ago

    noida, India Sopra Steria Full time

    Senior Data EngineerLocation: Noida/ ChennaiExperience: 4-8 yearsSkillset: Spark, Scala and HadoopRequired Skills:Strong expertise of programming languages –ScalaHands-on experience in Spark including Data frame core functions SparkSQL, Spark Streaming and with HIVE, Hadoop, Kafka, YARN.Good Experience in big data cloud platform, preferably Azure,...

  • Routing and Switching-Cisco Network Engineering Senior Specialist.

    4 weeks ago

    Noida, India NTT DATA Services Full time

    Req ID: 291641 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Routing and Switching-Cisco Network Engineering Senior Specialist. to join our team in Noida, Uttar Pradesh (IN-UP),...

  • Routing and Switching-Cisco Network Engineering Senior Specialist.

    4 weeks ago

    Noida, India NTT DATA Services Full time

    Req ID: 291641  NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Routing and Switching-Cisco Network Engineering Senior Specialist. to join our team in Noida, Uttar...

  • Senior Data Engineer

    3 months ago

    Noida, India InfoStride Full time

    As aSenior Data Platform Engineer , you will lead the design, development, and management across investments to platform and data pipelines. You will identify, monitor, and lead initiatives to ensure our data platform remains scalable, reliable, and efficient in light of evolving data requirements of our products and services. You will work closely with...

  • Training Senior Associate- Noida/ Greater Noida-NTT Data Vertex Software Inc.-1-4

    3 weeks ago

    Noida/ Greater Noida, India NTT Data Vertex Software Inc. Full time

    Apply Training Senior Associate, NTT Data Vertex Software Inc. in Noida/ Greater Noida for 1 - 4 year of Experience on TimesJobs.com

  • Senior Data Engineer

    4 months ago

    Noida, India RateGain Full time

    About the Organization: RateGain is a leading provider of SaaS products, which help travel and hospitality companies with cognitive revenue management, smart e-distribution, and brand engagement to make more revenue every day. RateGain is proud to support 250,000+ hotel properties globally by providing 240 billion rate and availability updates & powering...

  • Data engineer

    2 days ago

    Greater Noida, India Dev Full time

    Overview:The dataengineer plays a crucial role in our organization responsible fordesigning developing and maintaining our data architecture andinfrastructure. They will collaborate with data scientists analystsand other stakeholders to ensure the efficient flow and storage ofdata across our systems. The data engineer will also beinstrumental in implementing...

  • Systems Integration Senior Specialist_278293- Noida/ Greater Noida-NTT Data Vertex Software Inc.-5-9

    2 weeks ago

    Noida/ Greater Noida, India NTT Data Vertex Software Inc. Full time

    Apply Systems Integration Senior Specialist_278293, NTT Data Vertex Software Inc. in Noida/ Greater Noida for 5 - 9 year of Experience on TimesJobs.com

  • Senior Data Engineer

    3 months ago

    Noida, India InfoStride Full time

    As a Senior Data Platform Engineer, you will lead the design, development, and management across investments to platform and data pipelines. You will identify, monitor, and lead initiatives to ensure our data platform remains scalable, reliable, and efficient in light of evolving data requirements of our products and services. You will work closely with...

  • Senior Data Engineer

    4 weeks ago

    Noida, India InfoStride Full time

    As a Senior Data Platform Engineer, you will lead the design, development, and management across investments to platform and data pipelines. You will identify, monitor, and lead initiatives to ensure our data platform remains scalable, reliable, and efficient in light of evolving data requirements of our products and services. You will work closely with...

  • Senior Data Engineer

    3 months ago

    noida, India InfoStride Full time

    As a Senior Data Platform Engineer , you will lead the design, development, and management across investments to platform and data pipelines. You will identify, monitor, and lead initiatives to ensure our data platform remains scalable, reliable, and efficient in light of evolving data requirements of our products and services. You will work closely with...

  • Helpdesk Senior Associate

    2 days ago

    Noida/ Greater Noida, India NTT Data Vertex Software Inc. Full time

    Apply Helpdesk Senior Associate - Technical Troubleshooting, NTT Data Vertex Software Inc. in Noida/ Greater Noida for 3 - 5 year of Experience on TimesJobs.com