Senior Data Engineer

6 days ago

GurgaonGurugram, India Steerlean Consulting Services Pvt Ltd. Full time

Job Description :

We are looking for a highly skilled Databricks PySpark Developer to join our data platform implementation team. In this role, you will be instrumental in designing, developing, and maintaining ETL processes to ensure efficient extraction, transformation, and loading of data from various sources into data lake and data warehouse.

You will work closely with data engineers, data scientists, and business intelligence teams to build and optimize data workflows that support the project's analytics and reporting needs.

Must-Have Skills :

- AWS Glue (Crawler, Data Catalog).

- Python/Pyspark.

- Cloud Skills :

- Experience with Snowflake and its architecture (internal/external tables, stages, masking policies).

- Knowledge of AWS Services like SNS, S3, Lambda, Secret Manager, and Athena.

- Familiarity with Jira, GitHub, and Agile methodology.

Key Responsibilities :

1. ETL Development :


- Design and develop ETL processes using Databricks PySpark to extract, transform, and load data from heterogeneous sources into our data lake and data warehouse.

- Optimize ETL workflows for performance and scalability, leveraging Databricks PySpark and Spark SQL to efficiently process large data volumes.

- Implement robust error handling and monitoring mechanisms to proactively detect and resolve issues within ETL processes.

- Design and implement data solutions following the Medallion Architecture principles, organizing data into Bronze, Silver, and Gold layers.

- Ensure data is appropriately cleansed, enriched, and optimized at each stage to support robust analytics and reporting.

2. Data Pipeline Management :


- Hands On experience in creating advanced data pipelines using databricks workflows Develop and maintain data pipelines using Databricks PySpark, ensuring data quality, integrity, and reliability throughout the ETL lifecycle.

- Collaborate with data engineering, data science, and business intelligence teams to translate data requirements into efficient ETL workflows and pipelines.

3. Data Analysis and Query Optimization :


- Write and optimize complex SQL queries for data manipulation, aggregation, and analysis within Databricks PySpark applications.

4. Project Coordination and Continuous Improvement :


- Participate in project planning and coordination activities to ensure timely delivery of ETL solutions.

- Stay updated on the latest developments in Databricks PySpark, Spark SQL, and related technologies, recommending and implementing best practices and optimizations.

- Document ETL processes, data lineage, and metadata to facilitate knowledge sharing and ensure compliance with data governance standards.

5. Cloud Platform Expertise :


- Utilize cloud platforms (e.g. , AWS, Azure, or GCP) to design and deploy scalable and reliable SaaS solutions.

- Optimize infrastructure for performance, security, and cost efficiency

(ref:hirist.tech)

  • Senior Data Engineer

    1 month ago

    Gurgaon, India True Tech Professionals Full time

    Senior Data EngineerLocation: Gurgaon (Hybrid Mode)Experience: 5 years and above.Immediate Joiners Preferred (Notice Period: 7 to 10 days)We are seeking a skilled Senior Data Engineer to join our dynamic team. The ideal candidate should have a strong background in data engineering with expertise in Python, Pyspark, AWS Glue, Athena, Redshift, and SQL. The...

  • Petonic Infotech

    1 month ago

    Gurgaon/Gurugram, India Petonic Infotech Pvt. Ltd. Full time

    Job Title : Senior Data EngineerLocation : Gurgaon (Work from Office)Company Overview :Petonic is an innovative leader in harnessing AI-driven solutions to solve complex challenges. We empower our clients by enabling data-driven insights and advanced analytics. Join us as we build transformative AI systems and revolutionize industries.Job Summary :We are...

  • Senior data engineer

    8 hours ago

    Gurgaon, India Infinite Locus Full time

    Job description: Job Title: Senior Data Engineer (Azure Focus) Role Summary: We are seeking a Senior Data Engineer with expertise in Azure and Snowflake to design, implement, and maintain robust data solutions within our Data and Analytics (Dn A) Team. You will ensure high-quality data pipelines, optimize cloud-based architectures, and drive...

  • Senior Data Engineer

    1 month ago

    Gurugram, India True Tech Professionals Full time

    Senior Data Engineer Location: Gurgaon (Hybrid Mode) Experience: 5 years and above. Immediate Joiners Preferred (Notice Period: 7 to 10 days) We are seeking a skilled Senior Data Engineer to join our dynamic team. The ideal candidate should have a strong background in data engineering with expertise in Python, Pyspark, AWS Glue, Athena, Redshift, and SQL....

  • Senior Data Engineer

    1 month ago

    Gurugram, India True Tech Professionals Full time

    Senior Data EngineerLocation: Gurgaon (Hybrid Mode)Experience: 5 years and above. Immediate Joiners Preferred (Notice Period: 7 to 10 days)We are seeking a skilled Senior Data Engineer to join our dynamic team. The ideal candidate should have a strong background in data engineering with expertise in Python, Pyspark, AWS Glue, Athena, Redshift, and SQL. The...

  • Senior Data Engineer

    1 month ago

    Gurugram, India True Tech Professionals Full time

    Senior Data EngineerLocation: Gurgaon (Hybrid Mode)Experience: 5 years and above. Immediate Joiners Preferred (Notice Period: 7 to 10 days)We are seeking a skilled Senior Data Engineer to join our dynamic team. The ideal candidate should have a strong background in data engineering with expertise in Python, Pyspark, AWS Glue, Athena, Redshift, and SQL. The...

  • Senior Data Engineer

    4 days ago

    Gurugram, India Infinite Locus Full time

    Job description:Job Title: Senior Data Engineer (Azure Focus)Role Summary:We are seeking a Senior Data Engineer with expertise in Azure and Snowflake to design, implement, and maintain robust data solutions within our Data and Analytics (DnA) Team. You will ensure high-quality data pipelines, optimize cloud-based architectures, and drive efficiency...

  • Senior Data Engineer

    3 days ago

    Gurugram, India Infinite Locus Full time

    Job description:Job Title: Senior Data Engineer (Azure Focus)Role Summary:We are seeking a Senior Data Engineer with expertise in Azure and Snowflake to design, implement, and maintain robust data solutions within our Data and Analytics (DnA) Team. You will ensure high-quality data pipelines, optimize cloud-based architectures, and drive efficiency...

  • Senior Data Engineer

    1 month ago

    Gurgaon, Haryana, India Petonic Infotech Pvt. Ltd. Full time

    Job Title : Senior Data EngineerAt Petonic Infotech Pvt. Ltd., we are seeking a skilled Senior Data Engineer to play a pivotal role in our AI-driven projects. As a key member of our data team, you will be responsible for designing, developing, and maintaining robust data pipelines, managing APIs, and collaborating with data scientists to ensure high-quality...

  • Senior Data Engineering Manager

    2 weeks ago

    Gurgaon, Haryana, India Renovate Career Management Services Full time

    Job Title: Senior Data Engineering ManagerOverview:Renovate Career Management Services is seeking a highly skilled Senior Data Engineering Manager to lead our team of Azure data engineers. The successful candidate will be responsible for designing, implementing, and optimizing scalable data solutions using Azure data services.Key Responsibilities:- Design...

  • Senior Data Engineer

    1 week ago

    Gurgaon/Gurugram, India Global KPO Full time

    Role : Senior Data Engineer Exp : 5-8yrsRequired Skills : SQL queries,ETL tools and frameworks,Big data technologies,Cloud platforms (AWS, Azure, GCP),Data warehousingYour Mission, Should You Choose to Apply :- Data Architecture : Design, implement, and optimize scalable and efficient data architectures and pipelines to support various data use cases.- Data...

  • Senior Data Engineering Manager

    4 weeks ago

    Gurgaon, Haryana, India Impetus Full time

    Job Title:Senior Data Engineering ManagerCompany Overview:Impetus is a leading organization in the field of data engineering, dedicated to delivering innovative solutions that drive business growth. Our team of experts is passionate about harnessing the power of big data to create value for our clients.Salary:$120,000 - $180,000 per yearJob Description:We...

  • Senior Data Engineer

    6 days ago

    Gurgaon/Gurugram, India MINDTEL GLOBAL PRIVATE LIMITED Full time

    Job Title : Senior Data EngineerLocation : Gurgaon, IndiaExperience : 4 to 8 YearsWork Mode : Hybrid (3 days Work From Office)Key Responsibilities :- Design, develop, and maintain robust data pipelines using Azure Data Factory (ADF) to ingest, transform, and load data into data warehouses and data lakes.- Optimize data pipelines for performance and...

  • Senior Data Engineer Position

    2 weeks ago

    Gurgaon, Haryana, India Worksconsultancy Full time

    Data Architecture and EngineeringWe are seeking a highly skilled Senior Data Engineer to join our team at Worksconsultancy. This role is responsible for designing, implementing, and optimizing scalable and efficient data architectures and pipelines to support various data use cases.As a Senior Data Engineer, you will create and maintain data pipelines that...

  • Senior Data Engineering Expert

    4 weeks ago

    Gurgaon, Haryana, India True Tech Professionals Full time

    Senior Data Engineer Job DescriptionWe are seeking a skilled Senior Data Engineer to join our dynamic team at True Tech Professionals.Key Responsibilities:Data Pipeline Design: Collaborate with cross-functional teams to understand data requirements and design efficient data pipelines.ETL Process Development: Develop and maintain ETL processes using Python,...

  • Senior Data Engineer

    2 days ago

    Gurgaon/Gurugram, India ParaHR Full time

    ROLES AND RESPONSIBILITIES : Data Architecture : Design, implement, and optimize scalable and efficient data architectures and pipelines to support various data use cases.- Data Pipeline Development : Create and maintain data pipelines that ingest, transform, and store data from various sources (e.g., databases, APIs, logs, and external data feeds). Data...

  • Senior Data Engineering Lead

    3 weeks ago

    Gurgaon, Haryana, India Global KPO Full time

    Overview of the JobWe are currently seeking an experienced Senior Data Engineer to join our team at Global KPO. As a Senior Data Engineer, you will be responsible for designing, implementing, and optimizing scalable and efficient data architectures and pipelines to support various data use cases.Your MissionYou will be responsible for creating and...

  • Senior Azure Data Engineer

    4 weeks ago

    Gurgaon, Haryana, India ENGRO TECHNOLOGIES L.L.C Full time

    Job Title: Senior Azure Data Engineer About Us:Engro Technologies L.L.C is a leading provider of innovative technology solutions. We are dedicated to delivering high-quality services and products that meet the evolving needs of our clients. Salary Range:We offer a competitive salary range of $120,000 - $180,000 per annum, depending on experience and...

  • Senior AWS Data Engineer

    3 weeks ago

    Gurgaon, Haryana, India Max life insurance co ltd Full time

    Job OverviewWe are seeking a skilled Senior AWS Data Engineer to join our team at Max Life Insurance Co Ltd.

  • Senior Data Engineer Position in Gurgaon

    4 days ago

    Gurgaon, Haryana, India True Tech Professionals Full time

    Are you a highly skilled data engineer looking for a challenging opportunity? We are seeking an experienced Senior Data Engineer to join our team at True Tech Professionals.Key Responsibilities:Collaborate with cross-functional teams to design and develop efficient data pipelines using Python, Pyspark, and AWS Glue.Develop and maintain ETL processes for data...