Data Engineering Lead

1 week ago

Delhi, Delhi, India Kanerika Software Full time
Job DescriptionDatabricks Lead

About Kanerika

Who we are.

Kanerika Inc.

is a premier global software products and services firm that specializes in providing innovative solutions and services for data-driven enterprises.

Confounded by Warton Business School Aluminous, our focus is to empower businesses to achieve their digital transformation goals and maximize their business impact through the effective use of data.

We leverage cutting-edge technologies and industry best practices to deliver custom solutions that help organizations optimize their operations, enhance customer experiences, and drive growth.

Awards and Recognitions

Kanerika has won several awards over the years, including:

  • Best Place to Work 2022 by Great Place to Work
  • Top 10 Most Recommended RPA Start-Ups in 2022 by RPA Today
  • NASSCOM Emerge 50 Award in 2014
  • Frost & Sullivan India 2021 Technology Innovation Award for its Kompas composable solution architecture.
  • Kanerika has also been recognized for its commitment to customer privacy and data security, having achieved ISO 27701, SOC2, and GDPR compliances.
Working for usKanerika is rated 4.5/5 on Glassdoor, for many good reasons. We truly value our employees' growth, well-being, and diversity, and people's experiences bear this out.

At Kanerika, we offer a host of enticing benefits that create an environment where you can thrive both personally and professionally.

From our inclusive hiring practices and mandatory training on creating a safe work environment to our flexible working hours and generous parental leave, we prioritize the well-being and success of our employees.

Our commitment to professional development is evident through our mentorship programs, job training initiatives, and support for professional certifications. Additionally, our company-sponsored outings and various time-off benefits ensure a healthy work-life balance.

Join us at Kanerika and become part of a vibrant and diverse community where your talents are recognized, your growth is nurtured, and your contributions make a real impact.

See the benefits section below for the perks you'll get while working for Kanerika.


RoleThis position will be engaged in design and deploy of large-scale data processing frameworks and pipelines on Databricks to support enterprise-wide BI and data science initiatives.

It will also provide user training and data coaching for effective Databricks usage.

Locations:
We are located at Austin (USA), Singapore, Gurugram (India), Ahmedabad (India) and Hyderabad (India).Job Location

– Hyderabad, Indore, Ahmadabad, Pune, Gurgaon

We are looking for you


Are you passionate about leveraging data to drive business insights and innovation? Do you thrive in a dynamic, collaborative environment where you can contribute your expertise to building scalable data solutions? Data Bricks is seeking a talented Data Engineer to join our team and play a pivotal role in designing and implementing data pipelines and infrastructure for our cloud-based analytics platform.

Requirements

What you will do

  • Create ML-Ops frameworks for model development, training, deployment, and monitoring.
  • Optimize Databricks clusters for performance, scalability, and resource efficiency.
  • Monitor cluster health, performance metrics, and resource usage, adjusting cluster sizes as needed.
  • Design efficient data processing systems and pipelines using Databricks, APIs, and other cloud services.
  • Act as the Databricks expert and uphold platform standards.
  • Collaborate with Data and Platform architects on defining the product roadmap.

  • Build and maintain scalable infrastructure on cloud platforms such as AWS, Azure, or Google Cloud to support data processing and analytics workloads.

  • Implement data governance best practices to ensure data quality, reliability, and security throughout the data lifecycle.
  • Monitor and troubleshoot data pipelines to identify and resolve performance bottlenecks, data integrity issues, and other operational challenges.
What you'll bring

  • Bachelor's degree in computer science or related field.
  • 7+ years of IT experience, including 5+ years in Databricks development and administration, Spark, and cloud platforms (preferably Azure).
  • Proficiency in Databricks admin and development, with expertise in Resource and Performance Management.
  • Expertise in test automation, continuous integration, Docker, GitHub, Jenkins, and Unix/Linux.
  • Leadership in large project initiatives, willingness to mentor, and support for team members from different time zones.


You will have advantage if you bring -·Experience with Kafka and Spark, and strong in Python and Pyspark.·Hands-on experience in machine learning pipelines and scripting languages.·Familiarity with Java, Python, SQL/NoSQL databases, and Big Data tools.·Passion for new tech, continuous improvement, and collaborative problem-solving.

Proactive, strong work ethic, and team playerBenefitsEmployee Benefits

Culture:

  • Open Door Policy: Encourages open communication and accessibility to management.
  • Open Office Floor Plan: Fosters a collaborative and interactive work environment.
  • Flexible Working Hours: Allows employees to have flexibility in their work schedules.
  • Employee Referral Bonus: Rewards employees for referring qualified candidates.
  • Appraisal Process Twice a Year: Provides regular performance evaluations and feedback.

Inclusivity and Diversity:

  • Hiring practices that promote diversity: Ensures a diverse and inclusive workforce.
  • Mandatory POSH training: Promotes a safe and respectful work environment.

Health Insurance and Wellness Benefits:

  • GMC and Term Insurance: Offers medical coverage and financial protection.
  • Health Insurance: Provides coverage for medical expenses.
  • Disability Insurance: Offers financial support in case of disability.
  • Child Care &

Parental Leave Benefits:

  • Company-sponsored family events: Creates opportunities for employees and their families to bond.
  • Generous Parental Leave: Allows parents to take time off after the birth or adoption of a child.
  • Family Medical Leave: Offers leave for employees to take care of family members' medical needs.

Perks and Time-Off Benefits:

  • Company-sponsored outings: Organizes recreational activities for employees.
  • Gratuity: Provides a monetary benefit as a token of appreciation.
  • Provident Fund: Helps employees save for retirement.
  • Generous

PTO:
Offers more than the industry standard for paid time off.

  • Paid sick days: Allows employees to take paid time off when they are unwell.
  • Paid holidays: Gives employees paid time off for designated holidays.
  • Bereavement Leave: Provides time off for employees to grieve the loss of a loved one.

Professional Development Benefits:

  • L&D with FLEX- Enterprise Learning Repository: Provides access to a learning repository for professional development.
  • Mentorship Program: Offers guidance and support from experienced professionals.
  • Job Training: Provides training to enhance job-related skills.
  • Professional Certification Reimbursements: Assists employees in obtaining professional certifications.
  • Promote from Within: Encourages internal growth and advancement opportunities.
RequirementsWhat you will do Create ML-Ops frameworks for model development, training, deployment, and monitoring. Optimize Data-bricks clusters for performance, scalability, and resource efficiency. Monitor cluster health, performance metrics, and resource usage, adjusting cluster sizes as needed. Design efficient data processing systems and pipelines using Data bricks, APIs, and other cloud services. Act as the Data bricks expert and uphold platform standards. Collaborate with Data and Platform architects on defining the product roadmap.

Build and maintain scalable infrastructure on cloud platforms such as AWS, Azure, or Google Cloud to support data processing and analytics workloads.

Implement data governance best practices to ensure data quality, reliability, and security throughout the data lifecycle. Monitor and troubleshoot data pipelines to identify and resolve performance bottlenecks, data integrity issues, and other operational challenges. What you'll bring Bachelor's degree in computer science or related field.

7+ years of IT experience, including 5+ years in Data bricks development and administration, Spark, and cloud platforms (preferably Azure).

Proficiency in Databricks admin and development, with expertise in Resource and Performance Management. Expertise in test automation, continuous integration, Docker, GitHub, Jenkins, and Unix/Linux. Leadership in large project initiatives, willingness to mentor, and support for team members from different time zones. You will have advantage if you bring - Experience with Kafka and Spark, and strong in Python and Pyspark. Hands-on experience in machine learning pipelines and scripting languages. Familiarity with Java, Python, SQL/NoSQL databases, and Big Data tools. Passion for new tech, continuous improvement, and collaborative problem-solving. Proactive, strong work ethic, and team player.

  • Lead Data Engineer

    1 week ago

    Delhi, Delhi, India Akivna Technologies Pvt. Ltd. Full time

    Company DescriptionAkivna Technologies Pvt. Ltd. is an IT company based in Gurugram. We specialize in Data Engineering, Data Science, Data Analytics, DevOps, and Product Development. Our team is passionate about technology and we strive to deliver innovative and high-quality solutions to our clients.Job Description:We are seeking a skilled and experienced...

  • Senior Lead Engineer

    1 week ago

    Delhi, Delhi, India NTT DATA Full time

    Job DescriptionReq ID:282068NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior Lead Engineer to join our team in Delhi, Delhi (IN-DL), India (IN)."Strongly prefer...

  • Lead Engineer

    1 week ago

    Delhi, Delhi, India NTT DATA Full time

    Job DescriptionReq ID:282070NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Lead Engineer to join our team in Delhi, Delhi (IN-DL), India (IN)."RPA Consultant (C)Once You Are...

  • Lead Engineer

    1 week ago

    Delhi, Delhi, India NTT DATA Services Full time

    Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Lead Engineer to join our team in Delhi, Delhi (IN-DL), India (IN). "RPA Consultant (C) Once You Are Here, You Will:...

  • Lead Data Engineer

    1 week ago

    Delhi, Delhi, India Orion Innovation Full time

    As a Lead Data Engineer, you will play a crucial role in designing, developing, and maintaining our data infrastructure. You will lead a team of data engineers to ensure the successful delivery of projects and contribute to the overall data strategy. The ideal candidate will have extensive experience in cloud-based data technologies, particularly in Azure,...

  • Senior Lead Engineer

    1 week ago

    Delhi, Delhi, India NTT DATA Services Full time

    Req ID:283992NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior Lead Engineer to join our team in Delhi, Delhi (IN-DL), India (IN).Work closely with Enterprise...

  • Senior Lead Engineer

    1 week ago

    Delhi, Delhi, India NTT DATA Services Full time

    Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior Lead Engineer to join our team in Delhi, Delhi (IN-DL), India (IN)."Strongly prefer MuleSoft Developer –...

  • Lead Data Engineer

    1 week ago

    Delhi, Delhi, India Helius Technologies Full time

    Work Location: HyderabadWork Mode : 5 days Work from OfficeWork Timings : 9 am to 6 pmClient: Singapore basedJob Type: 1 year renewable contract (Long Term)Kindly revert with below details in addition to updated resume.Full Name as per the Passport or Aadhar Card :Current SalaryExp SalaryNotice period (availability)Reason for ChangeOk for 1 year renewable...

  • Senior Lead Engineer

    3 weeks ago

    Delhi, Delhi, India NTT DATA Services Full time

    Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior Lead Engineer to join our team in Delhi, Delhi (IN-DL), India (IN).Work closely with Enterprise Integration...

  • Lead Data Engineer

    1 week ago

    Delhi, Delhi, India Helius Technologies Full time

    Work Location:HyderabadWork Mode : 5 days Work from OfficeWork Timings : 9 am to 6 pmClient:Singapore basedJob Type:1 year renewable contract (Long Term)Kindly revert with below details in addition to updated resume.Full Name as per the Passport or Aadhar Card :Current SalaryExp SalaryNotice period (availability)Reason for ChangeOk for 1 year renewable...

  • Lead data engineer

    1 week ago

    Delhi, Delhi, India USEReady Full time

    Userady is currently seeking a Lead Data Engineer for locations in Bangalore, Mohali, and GurugramJob Description:Mentor a team of developers to create and maintain high-quality, scalable, and secure data pipelines and analytics applications on the Databricks platform.Collaborate closely with data engineers and business stakeholders to understand their...

  • Snowflake Data Engineer

    1 week ago

    Delhi, Delhi, India Lapis Data Analytics Full time

    Company DescriptionLapis Data Analytics is a dynamic and innovative data engineering company founded in 2020 and located in Chennai. Our primary focus is on designing, building, and maintaining robust data pipelines that convert raw data into actionable insights. With a team of seasoned experts and cutting-edge technology, we deliver customized data...

  • Senior Lead Engineer

    1 week ago

    Delhi, Delhi, India NTT DATA Services Full time

    Req ID:285016NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior Lead Engineer to join our team in Delhi, Delhi (IN-DL), India (IN)."The preference here is for a good...

  • Data Lead Engineer

    1 week ago

    Delhi, Delhi, India Squareroot Consulting Pvt Ltd Full time

    Job Description : We are seeking a highly skilled and motivated Data Lead with expertise in Spark, Scala, Kafka, Big Data, and Batch Processing. As a Data Lead, you will play a key role in leading and managing the end-to-end data processing pipeline, ensuring the reliability, scalability, and efficiency of our data systems.Experience : 6+ YearsLocation :...

  • Senior Lead Engineer

    1 week ago

    Delhi, Delhi, India NTT DATA Services Full time

    Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior Lead Engineer to join our team in Delhi, Delhi (IN-DL), India (IN).Note- The preference here is for any good...

  • Senior Lead Engineer

    1 month ago

    Delhi, Delhi, India NTT DATA Services Full time

    Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior Lead Engineer to join our team in Delhi, Delhi (IN-DL), India (IN).Note- The preference here is for any good...

  • Senior Lead Engineer

    3 weeks ago

    Delhi, Delhi, India NTT DATA Services Full time

    Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior Lead Engineer to join our team in Delhi, Delhi (IN-DL), India (IN).Note- The preference here is for any good...

  • Lead Data Engineer

    1 week ago

    Delhi, Delhi, India Coders Connect Full time

    Job DescriptionCoders ConnectandSanofiare joining forces to bring an electrifying twist to the bio pharmaceutical world Work with a rhythm that suits your style (2 days remote and 3 days onsite magic).Language:Proficiency in English is required for this role to ensure effective communication within our diverse, global team.About Sanofi:We're not just a...

  • Lead Azure Data Engineer

    1 week ago

    Delhi, Delhi, India EXL Full time

    Position Lead Data EngineerWork Location – Pune/ Hyderabad / Gurgaon / HybridExperience 9+ YearsMust have Skills:Minimum 5+ years on Azure cloud services.Proficiency in using Azure Synapse, Azure Storage, and Azure Data Factory for building data solutions.Strong understanding of metadata-driven approaches and data governance principles.Excellent...

  • Data QA Lead Engineer

    1 week ago

    Delhi, Delhi, India Apolis Consulting Pvt Ltd Full time

    Apolis is actively looking to hire a Data QA Lead & Data QA Engineer role to join our team. This will be 4-6 months contractual role.Location:RemoteJob descriptionData QA Testing (1-Lead and 5- Data QA Tester)PST 3 hours overlaps-Rest are flexible.4-6 months- Contractors.Must have done Data ValidationMust have Data QA experience.Working with high volumes of...