Senior Data Engineer

3 months ago

GurgaonGurugram, India Muoro Full time

Job Description :


Key Responsibilities :

Data Engineering & Development :

- Design, develop, and maintain scalable data pipelines using Azure Databricks.

- Write and optimize PySpark code for processing large datasets.

- Implement ETL processes to ingest and transform data from various sources.

Data Management :

- Develop and maintain Delta Lake tables for efficient data storage and retrieval.

- Ensure data quality and integrity through rigorous testing and validation processes.

- Manage and optimize data workflows in Databricks for performance and cost-efficiency.

Collaboration & Communication :

- Work closely with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions.

- Participate in design and code reviews to maintain high development standards.

- Document data processes, pipeline designs, and best practices.

Azure Integration :

- Integrate Databricks workflows with other Azure services such as Azure Data Lake Storage, Azure SQL Database, and Azure Data Factory.

- Monitor and troubleshoot data pipelines, ensuring reliable and consistent data availability.

Continuous Improvement :

- Stay up-to-date with the latest developments in Azure Databricks and PySpark.

- Identify opportunities to improve data processing efficiency and reduce operational costs.

- Implement best practices for data engineering and contribute to the team's knowledge base.

Required Skills and Qualifications :

Technical Skills :

- 5+ years of experience in data engineering or a related field.

- Extensive experience with Azure Databricks and PySpark.

- Strong proficiency in SQL for data manipulation and querying.

- Experience with Delta Lake and data versioning.

- Knowledge of Azure Data Lake Storage, Azure Data Factory, and other Azure services.

- Proficiency in programming languages such as Python or Scala.

Professional Skills :

- Excellent problem-solving and analytical skills.

- Strong communication and collaboration skills.

- Ability to work in a fast-paced, dynamic environment.

Preferred Qualifications :

- Experience with Databricks MLflow for machine learning workflows.

- Knowledge of data warehousing concepts and solutions such as Snowflake or Azure Synapse Analytics.

- Familiarity with CI/CD pipelines and version control systems like Git.

- Experience in Agile development methodologies.

(ref:hirist.tech)

  • Senior Data Engineer

    1 week ago

    gurgaon, India True Tech Professionals Full time

    Job Title: Senior Data EngineerLocation: Gurgaon (Hybrid)Experience Required: 6 to 9 yearsJob Summary:We are seeking a highly skilled Senior Data Engineer with expertise in PySpark, Python, ETL processes, and data pipeline development. The ideal candidate will have a strong background in Snowflake and hands-on experience working with AWS cloud. You will be...

  • Senior Data Engineer

    5 days ago

    Gurugram, India INFONET Full time

    Company DescriptionInfoNet provides customized software solutions for businesses of all sizes, from startups to Forbes 500 enterprises. Since 1996, we've been dedicated to helping companies streamline operations, enhance customer experiences, and drive growth with cutting-edge technology. Our team of developers, engineers, and consultants delivers tailored...

  • Senior Data Engineer

    3 weeks ago

    Gurugram, India Coforge Full time

    Job Title: Data EngineerSkills: Scala, Airflow, Spark, Scala, Hive, Hadoop & Data Engineering ConceptsExperience:Location: GurugramMode of Work: Hybrid We at Coforge are hiring Senior Data Engineers with the following skillset:Worked in a data engineering team environment of at least 5 engineers (agile, version control, etc.).Built and maintained at least...

  • Senior Data Engineer

    3 weeks ago

    Gurugram, India Coforge Full time

    Job Title: Data EngineerSkills: Scala, Airflow, Spark, Scala, Hive, Hadoop & Data Engineering ConceptsExperience:Location: GurugramMode of Work: Hybrid We at Coforge are hiring Senior Data Engineers with the following skillset:Worked in a data engineering team environment of at least 5 engineers (agile, version control, etc.).Built and maintained at least...

  • Senior Data Engineer

    3 weeks ago

    Gurugram, India Zupee Full time

    Senior Data Engineer We are the biggest online gaming company with largest market share in the Indian gaming sector’s largest segment — Casual & Boardgame. We make skill-based games that spark joy in the everyday lives of people by engaging, entertaining, and enabling earning while at play. In the three plus years of existence, Zupee has been on a...

  • Senior Data Engineer

    3 weeks ago

    Gurugram, India Zupee Full time

    Senior Data EngineerWe are the biggest online gaming company with largest market share in the Indian gaming sector’s largest segment — Casual & Boardgame. We make skill-based games that spark joy in the everyday lives of people by engaging, entertaining, and enabling earning while at play.In the three plus years of existence, Zupee has been on a mission...

  • Senior Data Engineer

    4 weeks ago

    Gurugram, India Zupee Full time

    Senior Data EngineerWe are the biggest online gaming company with largest market share in the Indian gaming sector’s largest segment — Casual & Boardgame. We make skill-based games that spark joy in the everyday lives of people by engaging, entertaining, and enabling earning while at play.In the three plus years of existence, Zupee has been on a mission...

  • Senior Data Engineer

    2 weeks ago

    Gurugram, India True Tech Professionals Full time

    Job Title: Senior Data EngineerLocation: Gurgaon (Hybrid)Experience Required: 6 to 9 yearsJob Summary:We are seeking a highly skilled Senior Data Engineer with expertise in PySpark, Python, ETL processes, and data pipeline development. The ideal candidate will have a strong background in Snowflake and hands-on experience working with AWS cloud. You will be...

  • Senior Data Engineer

    2 weeks ago

    Gurugram, India True Tech Professionals Full time

    Job Title: Senior Data EngineerLocation: Gurgaon (Hybrid)Experience Required: 6 to 9 yearsJob Summary:We are seeking a highly skilled Senior Data Engineer with expertise in PySpark, Python, ETL processes, and data pipeline development. The ideal candidate will have a strong background in Snowflake and hands-on experience working with AWS cloud. You will be...

  • Senior Data Engineer

    1 week ago

    Gurugram, India True Tech Professionals Full time

    Job Title: Senior Data Engineer Location: Gurgaon (Hybrid) Experience Required: 6 to 9 years Job Summary: We are seeking a highly skilled Senior Data Engineer with expertise in PySpark, Python, ETL processes, and data pipeline development. The ideal candidate will have a strong background in Snowflake and hands-on experience working with AWS cloud. You will...

  • Data Engineer

    3 weeks ago

    Gurugram, India NTT DATA Services Full time

    Req ID: 282638 NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer to join our team in Gurgaon, Haryāna (IN-HR), India (IN).Data EngineerJob SummaryWe are...

  • Senior Data Engineer

    1 week ago

    gurgaon, India Coforge Full time

    Job Title: Data EngineerSkills: Scala, Airflow, Spark, Scala, Hive, Hadoop & Data Engineering ConceptsExperience:Location: GurugramMode of Work: HybridWe at Coforge are hiring Senior Data Engineers with the following skillset:Worked in a data engineering team environment of at least 5 engineers (agile, version control, etc.).Built and maintained at least...

  • Senior Data Engineer

    6 days ago

    Gurgaon, India Coforge Full time

    Job Title: Data Engineer Skills: Scala, Airflow, Spark, Scala, Hive, Hadoop & Data Engineering Concepts Experience: Location: Gurugram Mode of Work: Hybrid We at Coforge are hiring Senior Data Engineers with the following skillset: Worked in a data engineering team environment of at least 5 engineers (agile, version control, etc.). ...

  • Senior Software Engineer

    1 week ago

    gurgaon, India Real Time Data Services Full time

    We are seeking a talented and motivated Senior Software Development Engineer to join our dynamic engineering team. As a Senior Software Development Engineer, you will play a crucial role in the development of web applications and services using technologies & frameworks such as Node.js, React.js, Next.js, Nest.js, PHP, Mysql, MongoDB, Microservices pattern....

  • Senior Data Engineer

    1 week ago

    Gurugram, India Coforge Full time

    Job Title: Data Engineer Skills: Scala, Airflow, Spark, Scala, Hive, Hadoop & Data Engineering Concepts Experience: Location: Gurugram Mode of Work: Hybrid We at Coforge are hiring Senior Data Engineers with the following skillset: Worked in a data engineering team environment of at least 5 engineers (agile, version control, etc.). Built and maintained...

  • Senior Software Engineer

    6 days ago

    Gurgaon, India Real Time Data Services Full time

    We are seeking a talented and motivated Senior Software Development Engineer to join our dynamic engineering team. As a Senior Software Development Engineer, you will play a crucial role in the development of web applications and services using technologies & frameworks such as Node.js, React.js, Next.js, Nest.js, PHP, Mysql, MongoDB, Microservices pattern....

  • Senior Data Engineer

    3 months ago

    Gurugram, India The Economist Full time

    Introduction Job Title: Senior Data Engineer Business Unit: Technology / Engineering Location: London Direct Reports:  0 Reporting Manager Title: Bogdan Vrusias Date Prepared: 01 / 05 / 2024 Who we are We are an organisation that exists to drive progress. That's the “red thread” that connects everyone at The Economist Group...

  • Senior Data Engineer

    4 weeks ago

    Gurugram, India Invokhr Full time

    Desired Experience: 3-8 years Salary: Best-in-industry Location: Gurgaon ( 5 days onsite) Overview: You will act as a key member of the Data consulting team, working directly with the partners and senior stakeholders of the clients designing and implementing big data and analytics solutions. Communication and organisation skills are keys for this position,...

  • Senior Software Engineer

    3 weeks ago

    Gurugram, India Real Time Data Services Full time

    We are seeking a talented and motivated Senior Software Development Engineer to join our dynamic engineering team. As a Senior Software Development Engineer, you will play a crucial role in the development of web applications and services using technologies & frameworks such as Node.js, React.js, Next.js, Nest.js, PHP, Mysql, MongoDB, Microservices pattern....

  • Senior Software Engineer

    4 weeks ago

    Gurugram, India Real Time Data Services Full time

    We are seeking a talented and motivated Senior Software Development Engineer to join our dynamic engineering team. As a Senior Software Development Engineer, you will play a crucial role in the development of web applications and services using technologies & frameworks such as Node.js, React.js, Next.js, Nest.js, PHP, Mysql, MongoDB, Microservices pattern....