Data Engineer

2 months ago

Gurugram, India Drimlite Full time

Job Title : Data Engineer

Location : Gurugram, Delhi

Experience : 3-5 Years

Budget : 24 LPA

Work Mode : Work From Office (WFO)

Interview Mode : Virtual

Must-Have Skills : PySpark, Databricks, ML Flow, Azure DevOps, SQL


Job Description :

We are seeking a skilled Data Engineer to design, build, and maintain data pipelines, data architectures, and ETL processes using advanced tools and technologies. The ideal candidate will have hands-on experience with PySpark, Databricks, and Azure DevOps, and will be proficient in building scalable solutions for data processing and analytics.

As a Data Engineer, you will collaborate with global teams, including data science and business intelligence, to ensure efficient data handling and gain insights that drive business decisions

Key Responsibilities :

- Data Pipeline Development: Design and build data pipelines using Spark-SQL and PySpark within Azure Databricks to handle large datasets efficiently.

- ETL Pipeline Design: Develop ETL pipelines using Azure Data Factory (ADF) for robust data transformation and integration.

- Lakehouse Architecture: Build and maintain a Lakehouse architecture in ADLS/Databricks, ensuring scalable and efficient data storage and processing.

- Data Preparation: Perform data preparation tasks including cleaning, normalization, deduplication, and type conversion for optimized data usability.

- Collaboration with DevOps: Work closely with the DevOps team to deploy data solutions in production environments using Azure DevOps.

- Error Handling: Monitor data processes, identify errors, and execute corrective actions, ensuring data integrity. This includes troubleshooting and resolving root causes of data errors.

- Global Team Collaboration: Participate as a member of the global analytics team, collaborating on projects, sharing insights, and adopting best practices from data science and business intelligence colleagues.

- Project Leadership: Lead and participate in projects, applying change management tools to manage upgrades, changes, and data migrations. This includes training, documentation, and communication efforts.

Qualifications :

- 3 to 5 years of experience in data engineering, with a strong focus on PySpark, Databricks, and Azure DevOps.

- Expertise in designing and building ETL pipelines using ADF.

- Strong knowledge of Lakehouse architecture and ADLS/Databricks.

- Solid experience with SQL for database queries and data management.

- Proven ability to collaborate with global teams and lead data projects.

- Strong problem-solving skills and attention to detail in managing data processes and errors.

(ref:hirist.tech)

  • Digital Engineering Engineer

    3 weeks ago

    Gurugram, India NTT DATA Services Full time

    Req ID: 295081 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Digital Engineering Engineer to join our team in Gurgaon, Haryāna (IN-HR), India (IN).Job Duties: Must have:...

  • Data Engineer

    1 month ago

    gurugram, India Turnberry Solutions Full time

    Experienced data engineer with hands on experience on GCP offerings Experienced in BigQuery/ BigTable/ PysparkWorked on prior data engineering projects leveraging GCP product offeringsStrong SQL backgroundPrior Amex Bigdata experience is big PlusData engineering pipeline build on big data on GCPSomeone who has worked with GCPStrong SQLStrong big...

  • Eucloid - Data Engineer - Azure Databricks

    6 months ago

    Gurgaon/Gurugram/Chennai, India Eucloid Data Solutions Pvt Ltd Full time

    Job Description - Data Engineer (Databricks)About Eucloid : Eucloid Data Solutions is a fast-growing AI-consulting firm that enables business growth for its clients through advanced Data Analytics, Machine Learning and Data engineering. Eucloid offers bespoke, full stackand datadriven approach to solve highly complex and large-scale business problems for...

  • Senior Data Engineer

    4 weeks ago

    Chennai/Gurgaon/Gurugram/Delhi NCR/Hyderabad, India Eucloid Data Solutions Pvt Ltd Full time

    About Eucloid Data SolutionsEucloid Data Solutions is a fast-growing AI-consulting firm that enables business growth for its clients through advanced Data Analytics, Machine Learning and Data engineering. We offer bespoke, full stack and data-driven approach to solve highly complex and large-scale business problems for some of the biggest Fortune 500...

  • Data Engineer

    1 week ago

    Gurugram, India StatusNeo Full time

    Job descriptionJob Title: Data EngineerCompany: StatusNeoAboutAt StatusNeo, we are on a mission to redefine the future of data-driven decision-making. We are looking for a passionate and talented Junior Data Engineer to join our dynamic team. If you have up to one year of experience or are fresh out of college with a strong academic background in data...

  • Data Engineer

    7 days ago

    Gurugram, India StatusNeo Full time

    Job descriptionJob Title: Data EngineerCompany: StatusNeoAboutAt StatusNeo, we are on a mission to redefine the future of data-driven decision-making. We are looking for a passionate and talented Junior Data Engineer to join our dynamic team. If you have up to one year of experience or are fresh out of college with a strong academic background in data...

  • Data Engineer

    7 days ago

    Gurugram, India StatusNeo Full time

    Job description Job Title: Data Engineer Company: StatusNeo About At StatusNeo, we are on a mission to redefine the future of data-driven decision-making. We are looking for a passionate and talented Junior Data Engineer to join our dynamic team. If you have up to one year of experience or are fresh out of college with a strong academic background in data...

  • Data Engineer

    1 month ago

    gurugram, India Nutrabay Full time

    We are seeking a skilled Data Engineer to join our team. The ideal candidate will be responsible for handling the movement of data from various sources to our data warehouses and data lakes. The candidate should have a strong understanding of data engineering concepts and experience working with databases such as Redshift, MySQL, and technologies like Athena...

  • Data Engineer

    2 months ago

    Gurugram, India Turnberry Solutions Full time

    Experienced data engineer with hands on experience on GCP offerings Experienced in BigQuery/ BigTable/ PysparkWorked on prior data engineering projects leveraging GCP product offeringsStrong SQL backgroundPrior Amex Bigdata experience is big PlusData engineering pipeline build on big data on GCPSomeone who has worked with GCPStrong SQLStrong big...

  • Data Engineer

    2 months ago

    gurugram, India Turnberry Solutions Full time

    Experienced data engineer with hands on experience on GCP offerings Experienced in BigQuery/ BigTable/ PysparkWorked on prior data engineering projects leveraging GCP product offeringsStrong SQL backgroundPrior Amex Bigdata experience is big PlusData engineering pipeline build on big data on GCPSomeone who has worked with GCPStrong SQLStrong big...

  • Data Engineer

    2 months ago

    Gurugram, India Turnberry Solutions Full time

    Experienced data engineer with hands on experience on GCP offerings Experienced in BigQuery/ BigTable/ Pyspark Worked on prior data engineering projects leveraging GCP product offerings Strong SQL background Prior Amex Bigdata experience is big Plus Data engineering pipeline build on big data on GCP Someone who has worked with GCP Strong SQL Strong big query...

  • Data Engineer

    2 months ago

    Gurugram, India Turnberry Solutions Full time

    Experienced data engineer with hands on experience on GCP offerings Experienced in BigQuery/ BigTable/ PysparkWorked on prior data engineering projects leveraging GCP product offeringsStrong SQL backgroundPrior Amex Bigdata experience is big PlusData engineering pipeline build on big data on GCPSomeone who has worked with GCPStrong SQLStrong big...

  • Data Engineer

    2 months ago

    gurugram, India Turnberry Solutions Full time

    Experienced data engineer with hands on experience on GCP offerings Experienced in BigQuery/ BigTable/ Pyspark Worked on prior data engineering projects leveraging GCP product offerings Strong SQL background Prior Amex Bigdata experience is big Plus Data engineering pipeline build on big data on GCP Someone who has worked with GCP Strong SQL Strong big query...

  • Data Engineer/Senior Data Engineer

    6 days ago

    Gurugram, India Cognitio Analytics Inc Full time

    Experience: 1 to 8 years - DE/ Sr. DE/ AM /Manager Must Have: SQL Python/ Pyspark Azure Databricks, ADLS Gen 2 Blob Storage, Key Vaults, BitBucket, Version control, Azure DevOps ETL, Building Data Pipelines, Data Warehousing, Data Modelling and Governance, Data Security Agile Practices, SDLC Data Exploration, Data Quality metrics, Historical data...

  • Data Engineer/Senior Data Engineer

    7 days ago

    Gurugram, India Cognitio Analytics Inc Full time

    Experience:1 to 8 years - DE/ Sr. DE/ AM /ManagerMust Have:SQLPython/ Pyspark Azure Databricks, ADLS Gen 2 Blob Storage, Key Vaults, BitBucket, Version control, Azure DevOpsETL, Building Data Pipelines, Data Warehousing, Data Modelling and Governance, Data SecurityAgile Practices, SDLCData Exploration, Data Quality metrics, Historical data Management, Good...

  • Data Engineer

    2 days ago

    Gurugram, India Aays Full time

    Position: Senior Data Engineer / Data Engineer Desired Experience: 3+ years You will act as a key member of the Data consulting team, working directly with the partners and senior stakeholders of the clients designing and implementing big data and analytics solutions. Communication and organisation skills are keys for this position, along with a...

  • Data Engineer

    20 hours ago

    Gurugram, India Aays Full time

    Position : Senior Data Engineer / Data Engineer Desired Experience : 3+ years You will act as a key member of the Data consulting team, working directly with the partners and senior stakeholders of the clients designing and implementing big data and analytics solutions. Communication and organisation skills are keys for this position, along with a...

  • Data Engineer

    2 days ago

    Gurugram, India Aays Full time

    Position: Senior Data Engineer / Data Engineer Desired Experience: 3+ years You will act as a key member of the Data consulting team, working directly with the partners and senior stakeholders of the clients designing and implementing big data and analytics solutions. Communication and organisation skills are keys for this position, along with a...

  • Data Engineer

    6 months ago

    Gurugram, India Nutrabay Full time

    We are seeking a skilled Data Engineer to join our team. The ideal candidate will be responsible for handling the movement of data from various sources to our data warehouses and data lakes. The candidate should have a strong understanding of data engineering concepts and experience working with databases such as Redshift, MySQL, and technologies like Athena...

  • Data Engineer

    3 weeks ago

    Gurugram, India Markovate Full time

    Job Summary We are seeking a skilled and motivated Data Engineer with a strong background in data integration, ETL processes, and data warehousing. The ideal candidate will have over 2 years of experience in data engineering, with a good understanding of data architecture, pipeline creation, and big data technologies. This role requires a proactive...