See more Collapse

Data Engineering Lead

1 month ago

Delhi, India Kanerika Software Full time
Job DescriptionRole

This position will be engaged in design and deploy of large-scale data processing frameworks and pipelines on Data bricks to support enterprise-wide BI and data science initiatives. It will also provide user training and data coaching for effective Data bricks usage.

Locations

We are located at Austin (USA), Singapore, Guru-gram (India), Ahmedabad (India) and Hyderabad (India).

Job Location – Hyderabad, Indore, Ahmadabad, Pune, Gurgaon

We are looking for you

Are you passionate about leveraging data to drive business insights and innovation? Do you thrive in a dynamic, collaborative environment where you can contribute your expertise to building scalable data solutions? Data Bricks is seeking a talented Data Engineer to join our team and play a pivotal role in designing and implementing data pipelines and infrastructure for our cloud-based analytics platform.

Requirements

What you will do

Create ML-Ops frameworks for model development, training, deployment, and monitoring.

Optimize Data-bricks clusters for performance, scalability, and resource efficiency.

Monitor cluster health, performance metrics, and resource usage, adjusting cluster sizes as needed.

Design efficient data processing systems and pipelines using Data bricks, APIs, and other cloud services.

Act as the Data bricks expert and uphold platform standards.

Collaborate with Data and Platform architects on defining the product roadmap.

Build and maintain scalable infrastructure on cloud platforms such as AWS, Azure, or Google Cloud to support data processing and analytics workloads.

Implement data governance best practices to ensure data quality, reliability, and security throughout the data lifecycle.

Monitor and troubleshoot data pipelines to identify and resolve performance bottlenecks, data integrity issues, and other operational challenges.

What you’ll bring

Bachelor’s degree in computer science or related field.

7+ years of IT experience, including 5+ years in Data bricks development and administration, Spark, and cloud platforms (preferably Azure).

Proficiency in Databricks admin and development, with expertise in Resource and Performance Management.

Expertise in test automation, continuous integration, Docker, GitHub, Jenkins, and Unix/Linux.

Leadership in large project initiatives, willingness to mentor, and support for team members from different time zones.

You will have advantage if you bring -

Experience with Kafka and Spark, and strong in Python and Pyspark.

Hands-on experience in machine learning pipelines and scripting languages.

Familiarity with Java, Python, SQL/NoSQL databases, and Big Data tools.

Passion for new tech, continuous improvement, and collaborative problem-solving.

Proactive, strong work ethic, and team player.

Benefits

1. Culture:

Open Door Policy: Encourages open communication and accessibility to management.

Open Office Floor Plan: Fosters a collaborative and interactive work environment.

Flexible Working Hours: Allows employees to have flexibility in their work schedules.

Employee Referral Bonus: Rewards employees for referring qualified candidates.

Appraisal Process Twice a Year: Provides regular performance evaluations and feedback.

2. Inclusivity and Diversity:

Hiring practices that promote diversity: Ensures a diverse and inclusive workforce.

Mandatory POSH training: Promotes a safe and respectful work environment.

3. Health Insurance and Wellness Benefits:

GMC and Term Insurance: Offers medical coverage and financial protection.

Health Insurance: Provides coverage for medical expenses.

Disability Insurance: Offers financial support in case of disability.

RequirementsWhat you will do Create ML-Ops frameworks for model development, training, deployment, and monitoring. Optimize Data-bricks clusters for performance, scalability, and resource efficiency. Monitor cluster health, performance metrics, and resource usage, adjusting cluster sizes as needed. Design efficient data processing systems and pipelines using Data bricks, APIs, and other cloud services. Act as the Data bricks expert and uphold platform standards. Collaborate with Data and Platform architects on defining the product roadmap. Build and maintain scalable infrastructure on cloud platforms such as AWS, Azure, or Google Cloud to support data processing and analytics workloads. Implement data governance best practices to ensure data quality, reliability, and security throughout the data lifecycle. Monitor and troubleshoot data pipelines to identify and resolve performance bottlenecks, data integrity issues, and other operational challenges. What you’ll bring Bachelor’s degree in computer science or related field. 7+ years of IT experience, including 5+ years in Data bricks development and administration, Spark, and cloud platforms (preferably Azure). Proficiency in Databricks admin and development, with expertise in Resource and Performance Management. Expertise in test automation, continuous integration, Docker, GitHub, Jenkins, and Unix/Linux. Leadership in large project initiatives, willingness to mentor, and support for team members from different time zones. You will have advantage if you bring - Experience with Kafka and Spark, and strong in Python and Pyspark. Hands-on experience in machine learning pipelines and scripting languages. Familiarity with Java, Python, SQL/NoSQL databases, and Big Data tools. Passion for new tech, continuous improvement, and collaborative problem-solving. Proactive, strong work ethic, and team player.

We have other current jobs related to this field that you can find below

  • Lead Data Engineer

    1 week ago

    Delhi, Delhi, India Akivna Technologies Pvt. Ltd. Full time

    Company DescriptionAkivna Technologies Pvt. Ltd. is an IT company based in Gurugram. We specialize in Data Engineering, Data Science, Data Analytics, DevOps, and Product Development. Our team is passionate about technology and we strive to deliver innovative and high-quality solutions to our clients.Job Description:We are seeking a skilled and experienced...

  • Senior Lead Engineer

    1 week ago

    Delhi, Delhi, India NTT DATA Full time

    Job DescriptionReq ID:282068NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior Lead Engineer to join our team in Delhi, Delhi (IN-DL), India (IN)."Strongly prefer...

  • Lead Engineer

    1 week ago

    Delhi, Delhi, India NTT DATA Full time

    Job DescriptionReq ID:282070NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Lead Engineer to join our team in Delhi, Delhi (IN-DL), India (IN)."RPA Consultant (C)Once You Are...

  • Lead Engineer

    3 weeks ago

    New Delhi, India NTT DATA Full time

    Job DescriptionReq ID:282070NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Lead Engineer to join our team in Delhi, Delhi (IN-DL), India (IN)."RPA Consultant (C)Once You Are...

  • Lead data engineer

    4 weeks ago

    delhi, India USEReady Full time

    Useready is hiring for Lead Data Engineer(Bangalore/Mohali/Gurugram)Job DescriptionMentor a team of developers in building and maintaining high-quality, scalable, and secure data pipelines and analytics applications on the Databricks platformWork closely with data engineers, business stakeholders to understand their needs and design and develop solutions...

  • Senior Lead Engineer

    3 weeks ago

    New Delhi, India NTT DATA Full time

    Job DescriptionReq ID:282068NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior Lead Engineer to join our team in Delhi, Delhi (IN-DL), India (IN)."Strongly prefer...

  • Lead Engineer

    1 week ago

    Delhi, Delhi, India NTT DATA Services Full time

    Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Lead Engineer to join our team in Delhi, Delhi (IN-DL), India (IN). "RPA Consultant (C) Once You Are Here, You Will:...

  • Software Test Engineer

    8 hours ago

    Delhi, India Lead Masters AI Full time

    Company DescriptionLeadMasters.ai is an innovative AI-driven lead generation and ad optimization platform that empowers businesses to maximize their marketing potential through intelligent lead generation and data-driven ad optimization. Our platform leverages advanced artificial intelligence algorithms to analyze top-performing ads across various social...

  • Lead Data Engineer

    3 weeks ago

    Delhi, India Orion Innovation Full time

    As a Lead Data Engineer, you will play a crucial role in designing, developing, and maintaining our data infrastructure. You will lead a team of data engineers to ensure the successful delivery of projects and contribute to the overall data strategy. The ideal candidate will have extensive experience in cloud-based data technologies, particularly in Azure,...

  • Lead Data Engineer

    4 weeks ago

    delhi, India Digitas India Full time

    Role- Lead Data EngineerExperience - 6 +YearsLocations- Mumbai/Pune/AhmedabadOverview: As a Lead Data Engineer, you will play a pivotal role in designing, implementing, and managing complex data pipelines within Google Cloud Platform (GCP) & AWS . Your expertise in integrating SAP, Salesforce and other cloud databases and marketing platforms , will be...

  • Lead Data Engineer

    1 week ago

    Delhi, Delhi, India Orion Innovation Full time

    As a Lead Data Engineer, you will play a crucial role in designing, developing, and maintaining our data infrastructure. You will lead a team of data engineers to ensure the successful delivery of projects and contribute to the overall data strategy. The ideal candidate will have extensive experience in cloud-based data technologies, particularly in Azure,...

  • Lead Data Engineer

    8 hours ago

    Delhi, India Advance Data Analytics Ltd. Full time

    *Immediate Joiners Preferred* We are hiring for a UK based client Experience: 6 to 10 years Location: Noida (Hybrid) Your main duties & responsibilities: Design, build and unit test of aspects data services, from ingestion, through harmonization and transformation into business usable analytics. Input into High level design and responsibility for low level...

  • Lead Data Engineer

    8 hours ago

    Delhi, India Advance Data Analytics Ltd. Full time

    *Immediate Joiners Preferred*We are hiring for a UK based clientExperience: 6 to 10 yearsLocation: Noida (Hybrid)Your main duties & responsibilities:Design, build and unit test of aspects data services, from ingestion, through harmonization and transformation into business usable analytics.Input into High level design and responsibility for low level...

  • Senior Lead Engineer

    1 week ago

    Delhi, Delhi, India NTT DATA Services Full time

    Req ID:283992NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior Lead Engineer to join our team in Delhi, Delhi (IN-DL), India (IN).Work closely with Enterprise...

  • Senior Lead Engineer

    1 week ago

    Delhi, Delhi, India NTT DATA Services Full time

    Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior Lead Engineer to join our team in Delhi, Delhi (IN-DL), India (IN)."Strongly prefer MuleSoft Developer –...

  • Data Engineer

    2 months ago

    Delhi, India dentsu Full time

    The purpose of this role is to maintain, improve, clean and manipulate data in the business’s operational and analytics databases. The Data Engineer works with the business’s software engineers, data analytics teams, data scientists and data warehouse engineers in order to understand and aid in the implementation of database requirements, analyse...

  • Senior Lead Engineer

    3 weeks ago

    new delhi, India NTT DATA Full time

    Job Description Req ID:  282068  NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Senior Lead Engineer to join our team in Delhi, Delhi (IN-DL), India (IN). "Strongly...

  • Senior Lead Engineer

    3 weeks ago

    New Delhi, India NTT DATA Full time

    Job Description Req ID: 282068 NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior Lead Engineer to join our team in Delhi, Delhi (IN-DL), India (IN)."Strongly prefer...

  • Lead Data Engineer

    3 weeks ago

    delhi, India Helius Technologies Full time

    Work Location: HyderabadWork Mode : 5 days Work from OfficeWork Timings : 9 am to 6 pmClient: Singapore basedJob Type: 1 year renewable contract (Long Term)Kindly revert with below details in addition to updated resume.Full Name as per the Passport or Aadhar Card :Current SalaryExp SalaryNotice period (availability)Reason for ChangeOk for 1 year renewable...

  • Lead Data Engineer

    1 week ago

    Delhi, Delhi, India Helius Technologies Full time

    Work Location: HyderabadWork Mode : 5 days Work from OfficeWork Timings : 9 am to 6 pmClient: Singapore basedJob Type: 1 year renewable contract (Long Term)Kindly revert with below details in addition to updated resume.Full Name as per the Passport or Aadhar Card :Current SalaryExp SalaryNotice period (availability)Reason for ChangeOk for 1 year renewable...