Data Engineer

4 weeks ago

PuneHyderabadRemote, India HARP Technologies and Services Full time

Job Location : Pune & Hyderabad (Initial 1 -2 months hybrid and then complete remote job. Client will offer accommodation(stay), food & travel exp. )

Exp range : 6+ years

Shift timings : General 9am - 6pm IST or 10am - 7pm IST

Mandatory skills : Data engineering (4+years), Databricks (3.5+ years), Pyspark (3+ years), Python (preferred) or bash, Data pipeline, Azure, ETL.

Job Type : C2H long term (within 6 months based on your performance/cultural fit you might be onboarded by the end client as a permanent employee)

We are seeking a skilled Data Engineer with 5+ years of experience specializing in DataBricks. The ideal candidate should have a robust understanding and hands-on expertise in PySpark and various components within DataBricks.


As a crucial member of our data team, you will play a pivotal role in developing, optimizing, and maintaining our data infrastructure, ensuring seamless and efficient data processing.

Responsibilities :

- Design, develop, and maintain data pipelines using DataBricks and PySpark to process and manipulate large scale datasets.

- Proven experience in optimizing Apache Spark batch processing workflows.

- Extensive experience in building and maintaining streaming data pipelines.

- Optimize and finetune existing DataBricks jobs and PySpark scripts for enhanced performance and reliability.

- Troubleshoot issues related to data pipelines, identify bottlenecks, and implement effective solutions.

- Implement best practices for data governance, security, and compliance within DataBricks environments.

- Work closely with Data Scientists and Analysts to support their data requirements and enable efficient access to relevant datasets.

- Stay updated with industry trends and advancements in DataBricks and PySpark technologies to propose and implement innovative solutions.

- Demonstrated expertise in optimizing systems for low-latency and high-throughput performance.

- Proficiency in using Spark SQL and DataFrame API for dynamic data transformations.

- Experience with using programming languages such as Python or Scala to implement advanced filtering logic in Databricks notebooks or scripts.

- Familiarity with the principles of distributed systems and their application in message broking.

- Collaborate with cross functional teams to gather requirements, understand data needs, and implement scalable solutions.

Requirements :

- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

- 4 to 5 years of proven experience as a Data Engineer with a strong emphasis on DataBricks.

- Proficiency in PySpark and extensive hands on experience in building and optimizing data pipelines using DataBricks.

- Solid understanding of different components within DataBricks such as clusters, notebooks, jobs, and libraries.

- Strong knowledge of SQL, data modeling, and ETL processes.

- Ability to analyzxcellent communication skills with the ability to collaborate with cross functional teams.

(ref:hirist.tech)

  • Data Engineer

    5 days ago

    hyderabad, India NTT DATA Full time

    Job Description Req ID:  280008  NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN). "Data...

  • Data Engineer

    5 days ago

    Hyderabad, India NTT DATA Full time

    Job Description Req ID: 280024 NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN).Data Engineer...

  • Data Engineer

    4 days ago

    hyderabad, India NTT DATA Full time

    Job Description Req ID:  280024  NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN). Data...

  • Data Engineer

    3 days ago

    hyderabad, India NTT DATA Full time

    Job Description Req ID:  280024  NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN). Data...

  • Data Engineer

    4 days ago

    Hyderabad, India NTT DATA Full time

    Job Description Req ID: 280024 NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN).Data Engineer...

  • Data Engineer

    5 days ago

    hyderabad, India NTT DATA Services Full time

    Req ID:  280024  NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN). Data Engineer...

  • Senior Data Engineer

    2 months ago

    Pune, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology...

  • Senior Data Engineer

    3 weeks ago

    pune, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology...

  • Data Engineer

    2 days ago

    Hyderabad, India NTT DATA Services Full time

    Req ID: 280024  NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN). Data Engineer...

  • Data Engineer

    5 days ago

    Hyderabad, India NTT DATA Services Full time

    Req ID: 280024 NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN).Data Engineer...

  • Data Engineer

    4 days ago

    Pune, India NTT DATA Full time

    Job DescriptionReq ID:278258NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer (ETL/SQL) to join our team in Pune, Mahārāshtra (IN-MH), India (IN)."Job Duties:...

  • Lead Engineer

    5 days ago

    Remote, India NTT DATA Services Full time

    Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Lead Engineer to join our team in Remote, Mahārāshtra (IN-MH), India (IN).esponsibilities:1. Salesforce...

  • Lead Engineer

    2 days ago

    Remote, India NTT DATA Services Full time

    Req ID: 280075  NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Lead Engineer to join our team in Remote, Mahārāshtra (IN-MH), India...

  • Lead Engineer

    2 days ago

    Remote, India NTT DATA Services Full time

    Req ID: 280074  NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Lead Engineer to join our team in Remote, Mahārāshtra (IN-MH), India...

  • Lead Engineer

    2 days ago

    Remote, India NTT DATA Services Full time

    Req ID: 280076  NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Lead Engineer to join our team in Remote, Mahārāshtra (IN-MH), India...

  • Digital Engineering Staff Engineer

    5 days ago

    Remote, India NTT DATA Services Full time

    Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Digital Engineering Staff Engineer to join our team in Remote, Telangana (IN-TG), India (IN).Who we are:NTT DATA...

  • Data Engineer

    5 days ago

    Pune, India NTT DATA Full time

    Job Description Req ID: 278258 NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer (ETL/SQL) to join our team in Pune, Mahārāshtra (IN-MH), India (IN)."Job...

  • Data Engineer

    5 days ago

    pune, India NTT DATA Full time

    Job Description Req ID:  278258  NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer (ETL/SQL) to join our team in Pune, Mahārāshtra (IN-MH), India (IN)....

  • Data Engineer

    3 days ago

    pune, India NTT DATA Full time

    Job Description Req ID:  278258  NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer (ETL/SQL) to join our team in Pune, Mahārāshtra (IN-MH), India (IN)....

  • Data Engineer

    4 days ago

    Pune, India NTT DATA Full time

    Job Description Req ID: 278258 NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer (ETL/SQL) to join our team in Pune, Mahārāshtra (IN-MH), India (IN)."Job...