Data Engineer

3 weeks ago

PuneHyderabadRemote, India HARP Technologies and Services Full time

Job Location : Pune & Hyderabad (Initial 1 -2 months hybrid and then complete remote job. Client will offer accommodation(stay), food & travel exp. )

Exp range : 6+ years

Shift timings : General 9am - 6pm IST or 10am - 7pm IST

Mandatory skills : Data engineering (4+years), Databricks (3.5+ years), Pyspark (3+ years), Python (preferred) or bash, Data pipeline, Azure, ETL.

Job Type : C2H long term (within 6 months based on your performance/cultural fit you might be onboarded by the end client as a permanent employee)

We are seeking a skilled Data Engineer with 5+ years of experience specializing in DataBricks. The ideal candidate should have a robust understanding and hands-on expertise in PySpark and various components within DataBricks.


As a crucial member of our data team, you will play a pivotal role in developing, optimizing, and maintaining our data infrastructure, ensuring seamless and efficient data processing.

Responsibilities :

- Design, develop, and maintain data pipelines using DataBricks and PySpark to process and manipulate large scale datasets.

- Proven experience in optimizing Apache Spark batch processing workflows.

- Extensive experience in building and maintaining streaming data pipelines.

- Optimize and finetune existing DataBricks jobs and PySpark scripts for enhanced performance and reliability.

- Troubleshoot issues related to data pipelines, identify bottlenecks, and implement effective solutions.

- Implement best practices for data governance, security, and compliance within DataBricks environments.

- Work closely with Data Scientists and Analysts to support their data requirements and enable efficient access to relevant datasets.

- Stay updated with industry trends and advancements in DataBricks and PySpark technologies to propose and implement innovative solutions.

- Demonstrated expertise in optimizing systems for low-latency and high-throughput performance.

- Proficiency in using Spark SQL and DataFrame API for dynamic data transformations.

- Experience with using programming languages such as Python or Scala to implement advanced filtering logic in Databricks notebooks or scripts.

- Familiarity with the principles of distributed systems and their application in message broking.

- Collaborate with cross functional teams to gather requirements, understand data needs, and implement scalable solutions.

Requirements :

- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

- 4 to 5 years of proven experience as a Data Engineer with a strong emphasis on DataBricks.

- Proficiency in PySpark and extensive hands on experience in building and optimizing data pipelines using DataBricks.

- Solid understanding of different components within DataBricks such as clusters, notebooks, jobs, and libraries.

- Strong knowledge of SQL, data modeling, and ETL processes.

- Ability to analyzxcellent communication skills with the ability to collaborate with cross functional teams.

(ref:hirist.tech)

  • Big Data Engineer, Data Modeling

    1 week ago

    Hyderabad, Telangana, India data Full time

    What can you tell your friends when they ask you what you do? We're looking for an experienced Big Data Engineer who can create innovative new products in the analytics and data space. You will participate in the development that creates the world's #1 mobile app analytics service. Together with the team, you will build out new product features and...

  • Big Data Engineer, Data Modeling

    1 week ago

    Hyderabad, Telangana, India data Full time

    What can you tell your friendswhen they ask you what you do?We're looking for an experienced Big Data Engineer who can create innovative new products in the analytics and data space. You will participate in the development that creates the world's #1 mobile app analytics service. Together with the team, you will build out new product features and...

  • Big Data Engineer, Data Analysis

    1 week ago

    Hyderabad, Telangana, India data Full time

    What can you tell your friends when they ask you what you do? "As a Data Analyst at , I help analyze, monitor, and deliver comprehensive and high-quality app market data to app professionals. I am empowered to work independently with the support of team members spread across APAC, AMER, and EMEA. I am the key contributor to the new and existing product...

  • Azure Data Architect

    3 weeks ago

    Remote, India Data PlatformExperts Full time

    Job Description: Azure Data Engineer (Architect Level) We are seeking an experienced and highly skilled Azure Data Engineer at the Architect level to lead the design and implementation of complex data solutions on the Microsoft Azure platform. As an Azure Data Engineer Architect, you will be responsible for providing technical expertise, guiding the...

  • Data Engineer

    1 week ago

    Hyderabad, Telangana, India NTT DATA Full time

    Job Description Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN). "Data Engineer...

  • Data Engineer

    1 month ago

    Hyderabad, India NTT DATA Full time

    Job Description Req ID: 280024 NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN).Data Engineer...

  • Data Engineer

    1 month ago

    hyderabad, India NTT DATA Full time

    Job Description Req ID:  280024  NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN). Data...

  • Azure Data Architect

    2 weeks ago

    Remote, India Data PlatformExperts Full time

    **Job Description: Azure Data Architect** **Key Responsibilities**: - **Design and Implementation**: - Design and implement end-to-end data solutions (data models, pipelines, data lakes, data warehouses) in Azure. - Optimize and maintain existing Azure data structures and integration processes. - Ensure architectural solutions are scalable, maintainable,...

  • Data Engineer

    1 week ago

    Hyderabad, Telangana, India NTT DATA Services Full time

    Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN). Data Engineer Responsibilities-Create...

  • Data Engineer

    1 month ago

    hyderabad, India NTT DATA Services Full time

    Req ID:  280024  NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN). Data Engineer...

  • Senior Data Engineer

    2 months ago

    Pune, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology...

  • Senior Data Engineer

    1 week ago

    Pune, Maharashtra, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology...

  • Senior Data Engineer

    3 weeks ago

    Pune, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology...

  • Senior Data Engineer

    2 months ago

    pune, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology...

  • Senior Data Engineer

    3 weeks ago

    pune, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology...

  • Data Engineer

    1 month ago

    Hyderabad, India NTT DATA Services Full time

    Req ID: 280024 NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN).Data Engineer...

  • Data Engineer

    1 month ago

    Hyderabad, India NTT DATA Services Full time

    Req ID: 280024  NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN). Data Engineer...

  • Data Engineer

    4 weeks ago

    Hyderabad, India NTT DATA Services Full time

    Req ID: 280024  NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN). Data Engineer...

  • Data Engineer

    1 week ago

    Pune, Maharashtra, India NTT DATA Full time

    Job Description Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer (ETL/SQL) to join our team in Pune, Mahārāshtra (IN-MH), India (IN)."Job Duties: ●...

  • Data Engineer

    1 week ago

    Pune, Maharashtra, India NTT DATA Full time

    Job DescriptionReq ID:278258NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer (ETL/SQL) to join our team in Pune, Mahārāshtra (IN-MH), India (IN)."Job Duties:...