Data Engineering Lead

2 weeks ago

Hyderabad, Telangana, India Kanerika Software Full time

Role

Thisposition will be engaged in design and deploy of largescale dataprocessing frameworks and pipelines on Data bricks to supportenterprisewide BI and data science initiatives. It will alsoprovide user training and data coaching for effective Data bricksusage.

Locations

WearelocatedatAustin(USA)SingaporeGurugram (India) Ahmedabad (India)andHyderabad(India).

JobLocationHyderabad Indore AhmadabadPuneGurgaon

Weare looking foryou

Areyou passionate aboutleveraging datato drive business insights and innovation Do you thrive in adynamic collaborative environment where you can contribute yourexpertise tobuilding scalable data solutions Data Bricks is seeking a talentedData Engineer to join our team and play a pivotal role in designingand implementing data pipelines and infrastructure for ourcloudbased analyticsplatform.

Requirements

Whatyou will do

  • CreateMLOps frameworks for model development training deployment andmonitoring.
  • OptimizeDatabricks clusters for performance scalability and resourceefficiency.
  • Monitorcluster health performance metrics and resource usage adjustingcluster sizes as needed.
  • Designefficient data processing systems and pipelines using Data bricksAPIs and other cloudservices.
  • Actas the Data bricks expert and uphold platformstandards.
  • Collaboratewith Data and Platform architects on defining the productroadmap.
  • Buildand maintainscalable infrastructure on cloud platforms such as AWS Azure orGoogle Cloud to support data processing and analyticsworkloads.
  • Implementdata governance best practices to ensure data quality reliabilityand security throughout the datalifecycle.
  • Monitorand troubleshoot data pipelines toidentify andresolve performance bottlenecks data integrity issues and otheroperational challenges.

Whatyou llbring

  • Bachelors degree in computer science or relatedfield.
  • 7years of IT experience including 5 years in Data bricks developmentand administration Spark and cloud platforms (preferablyAzure).
  • Proficiencyin Databricks admin and development withexpertise inResource and PerformanceManagement.
  • Expertisein test automation continuous integration Docker GitHub Jenkins andUnix/Linux.
  • Leadershipin large project initiatives willingness to mentor and support forteam members from different timezones.

Youwill have advantage if you bring

  • Experiencewith Kafka and Spark and strong in Python andPyspark.
  • Handsonexperience in machine learning pipelines and scriptinglanguages.
  • Familiaritywith Java Python SQL/NoSQL databases and Big Datatools.
  • Passionfor new tech continuous improvement and collaborativeproblemsolving.
  • Proactivestrong work ethic and teamplayer.
Benefits

1.Culture:

  • OpenDoor Policy: Encourages open communication and accessibility tomanagement.
  • OpenOffice Floor Plan: Fosters a collaborative and interactive workenvironment.
  • FlexibleWorking Hours: Allows employees to have flexibility in their workschedules.
  • EmployeeReferral Bonus: Rewards employees for referring qualifiedcandidates.
  • AppraisalProcess Twice a Year: Provides regular performance evaluations andfeedback.

2.Inclusivity and Diversity:

  • Hiringpractices that promote diversity: Ensures a diverse and inclusiveworkforce.
  • MandatoryPOSH training: Promotes a safe and respectful workenvironment.

3.Health Insurance and WellnessBenefits:

  • GMCand Term Insurance: Offers medical coverage and financialprotection.
  • HealthInsurance: Provides coverage for medicalexpenses.
  • DisabilityInsurance: Offers financial support in case ofdisability.
What you will do Create ML-Ops frameworks for model development,training, deployment, and monitoring. Optimize Data-bricks clustersfor performance, scalability, and resource efficiency. Monitorcluster health, performance metrics, and resource usage, adjustingcluster sizes as needed. Design efficient data processing systemsand pipelines using Data bricks, APIs, and other cloud services.Act as the Data bricks expert and uphold platform standards.Collaborate with Data and Platform architects on defining theproduct roadmap. Build and maintain scalable infrastructure oncloud platforms such as AWS, Azure, or Google Cloud to support dataprocessing and analytics workloads. Implement data governance bestpractices to ensure data quality, reliability, and securitythroughout the data lifecycle. Monitor and troubleshoot datapipelines to identify and resolve performance bottlenecks, dataintegrity issues, and other operational challenges. What you llbring Bachelor s degree in computer science or related field. 7+years of IT experience, including 5+ years in Data bricksdevelopment and administration, Spark, and cloud platforms(preferably Azure). Proficiency in Databricks admin anddevelopment, with expertise in Resource and Performance Management.Expertise in test automation, continuous integration, Docker,GitHub, Jenkins, and Unix/Linux. Leadership in large projectinitiatives, willingness to mentor, and support for team membersfrom different time zones. You will have advantage if you bring -Experience with Kafka and Spark, and strong in Python and Pyspark.Hands-on experience in machine learning pipelines and scriptinglanguages. Familiarity with Java, Python, SQL/NoSQL databases, andBig Data tools. Passion for new tech, continuous improvement, andcollaborative problem-solving. Proactive, strong work ethic, andteam player.

  • Big Data Engineer, Data Modeling

    2 weeks ago

    Hyderabad, Telangana, India data Full time

    What can you tell your friends when they ask you what you do? We're looking for an experienced Big Data Engineer who can create innovative new products in the analytics and data space. You will participate in the development that creates the world's #1 mobile app analytics service. Together with the team, you will build out new product features and...

  • Big Data Engineer, Data Modeling

    2 weeks ago

    Hyderabad, Telangana, India data Full time

    What can you tell your friendswhen they ask you what you do?We're looking for an experienced Big Data Engineer who can create innovative new products in the analytics and data space. You will participate in the development that creates the world's #1 mobile app analytics service. Together with the team, you will build out new product features and...

  • Big Data Engineer, Data Analysis

    2 weeks ago

    Hyderabad, Telangana, India data Full time

    What can you tell your friends when they ask you what you do? "As a Data Analyst at , I help analyze, monitor, and deliver comprehensive and high-quality app market data to app professionals. I am empowered to work independently with the support of team members spread across APAC, AMER, and EMEA. I am the key contributor to the new and existing product...

  • Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India NTT DATA Full time

    Job Description Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN). "Data Engineer...

  • Data Lead

    5 days ago

    Hyderabad, Telangana, India NTT DATA Services Full time

    NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Lead – SAP Sales and Distribution to join our team in [[Hyderabad]], [[Telangana]], [[India]].8+ of Overall experience...

  • Digital Engineering Lead Engineer

    2 weeks ago

    Hyderabad, Telangana, India NTT DATA Full time

    Job Description Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Digital Engineering Lead Engineer to join our team in hyderabad, Telangana (IN-TG), India (IN).Who we...

  • Digital Engineering Lead Engineer

    2 weeks ago

    Hyderabad, Telangana, India NTT DATA Full time

    Job DescriptionReq ID:277204NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Digital Engineering Lead Engineer to join our team in Hyderabad, Telangana (IN-TG), India...

  • Digital Engineering Lead Engineer

    2 weeks ago

    Hyderabad, Telangana, India NTT DATA Full time

    Job Description NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Digital Engineering Lead Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN).Once You Are...

  • Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India NTT DATA Services Full time

    Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN). Data Engineer Responsibilities-Create...

  • Data Engineering Lead

    2 weeks ago

    Hyderabad, Telangana, India Chryselys Full time

    Job Description:As a Data Engineering Lead, you will play a critical role in driving the delivery for our clients data projects. Leveraging your expertise in data engineering and data operations, you will work closely with clients to ensure that the project delivery on time with quality and meeting the requirements. This role requires a deep understanding of...

  • Lead Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Epam Full time

    EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most...

  • Digital Engineering Lead Engineer

    2 weeks ago

    Hyderabad, Telangana, India NTT DATA Services Full time

    Req ID:279798NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Digital Engineering Lead Engineer to join our team in hyderabad, Telangana (IN-TG), India (IN).Who we are:NTT...

  • Digital Engineering Lead Engineer

    2 weeks ago

    Hyderabad, Telangana, India NTT DATA Full time

    Job Description Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Digital Engineering Lead Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN). At...

  • Digital Engineering Lead Engineer

    2 weeks ago

    Hyderabad, Telangana, India NTT DATA Services Full time

    Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Digital Engineering Lead Engineer to join our team in hyderabad, Telangana (IN-TG), India (IN).Who we are: NTT DATA...

  • Tableau Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India NTT DATA Full time

    Job DescriptionReq ID:284210NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Tableau Data Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN).Data Engineer...

  • Tableau Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India NTT DATA Full time

    Job Description Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Tableau Data Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN). Data Engineer...

  • Digital Engineering Lead Engineer

    2 weeks ago

    Hyderabad, Telangana, India NTT DATA Services Full time

    Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Digital Engineering Lead Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN). Position Overview ...

  • Digital Engineering Lead Engineer

    19 hours ago

    Hyderabad, Telangana, India NTT DATA Services Full time

    Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Digital Engineering Lead Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN). Preferred...

  • Lead Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Foundation AI Full time

    Designation : Lead Data Engineer (Individual Contributor)Reporting to : Sr Tech LeadKey Responsibilities : Architect and Design Data Systems : Design and develop scalable data pipelines and systems to manage large volumes of structured and unstructured data. Data Pipeline Development : Develop scalable endtoend ETL (Extract, Transform, Load) data pipelines...

  • Lead Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Foundation AI Full time

    Designation : Lead Data Engineer (Individual Contributor)Reporting to : Sr Tech LeadKey Responsibilities : Architect and Design Data Systems : Design and develop scalable data pipelines and systems to manage large volumes of structured and unstructured data. Data Pipeline Development : Develop scalable endtoend ETL (Extract, Transform, Load) data pipelines...