Senior Big Data Engineer

2 weeks ago


Hyderabad, Telangana, India Info Services Full time

Title : Big Data Engineer.

Location : Financial District, Hyderabad Hybrid.

Type : Offshore.

Shift Timings : 1:00pm 10:00pm IST.

Mode : Fulltime Role.

We are looking for someone who can :

- Support large data distributed pipelines that span across Airflow, Databricks and Snowflake.

- Some knowledge with Airflow is good to have to understand how the DAGs work, etc.

- Candidate doesn't have to be super expert in any of these but knows how these products work and can learn along the way.

- Quickly triage any issues in the data pipeline ecosystem and navigate thru the logs from Airflow & Databricks and look at other metrics, go thru the pre-written runbook to decide what to do.

- Sometimes, the runbook says to send an email to someone or fix it in certain way or escalate it somebody else depending on the SLA of the issue and how long its been around.

- Write some PySpark or Spark scala code when they try to upgrade the versions such as Airflow upgrade or Databricks upgrade.

- Ensure all the existing pipelines work as is or identify what to change if there is a backward compatibility break.

- They don't have to super expert in coding but understand and learn.

- Do some interesting PoC as and when required.

- Knowledge : Airflow, Databricks, Snowflake, Spark, ability to triage & troubleshoot, ability to fix stuff.

- Doesn't have to be a super coder.

- The need for a Senior engineer is the experience that they have processing data pipelines and common issues they have come across.

(ref:hirist.tech)

  • Hyderabad, Telangana, India data Remote Work Freelance Full time

    What can you tell your friends when they ask you what you do? We're looking for an experienced Big Data Engineer who can create innovative new products in the analytics and data space. You will participate in the development that creates the world's #1 mobile app analytics service. Together with the team, you will build out new product features and...


  • Hyderabad, Telangana, India data Remote Work Freelance Full time

    What can you tell your friends when they ask you what you do? "As a Data Analyst at data.ai, I help analyze, monitor, and deliver comprehensive and high-quality app market data to app professionals. I am empowered to work independently with the support of team members spread across APAC, AMER, and EMEA. I am the key contributor to the new and existing...

  • AWS Big Data Engineer

    3 weeks ago


    Hyderabad, Telangana, India Tiger Analytics Full time

    Job Description We are looking for a Senior Data Engineer to be based out of our Chennai, Bangalore & Hyderabad offices. This role involves a combination of hands-on contribution, customer engagement, and technical team management. As a Senior Data Engineer, you will Design and build solutions for near real-time stream processing as well as batch...


  • Hyderabad, Telangana, India Tata Consultancy Services Full time

    Welcome to TATA CONSULTANCY SERVICES LIMITEDWe are seeking an experienced Big Data Engineer to join our team.About the Role:This is a senior-level position requiring 5-8 years of experience in designing and implementing big data solutions using Azure Databricks, Azure Data Factory, and Azure Synapse Analytics.Main Responsibilities:Lead the development of...

  • Big Data Engineer

    7 days ago


    Hyderabad, Telangana, India SMARTWORK IT SERVICES Full time

    Job Title : Big Data Engineer (Dynatrace)Location : Chennai, Bangalore, HyderabadExperience : 6-10 YearsRequired skills : Big Data, DynatraceJob Description : We are looking for an experienced Big Data Engineer (Dynatrace) with building scalable data pipelines and ensuring performance optimization through Dynatrace monitoring. This role requires a seasoned...

  • Senior Data Engineer

    3 weeks ago


    Hyderabad, Telangana, India Enable Data Incorporated Full time

    Enable Data Incorporated is seeking a highly skilled Senior Data Engineer with expertise in cloud technologies, Spark, and Databricks to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, we leverage modern solutions to drive increased value across our clients' business ecosystem.As a Senior Data...

  • Big Data Engineer

    3 hours ago


    Hyderabad, Telangana, India Questsphere Private ltd. Full time

    Our client is looking to deploy big data engineer at one of the worlds top product company R&D centre based out of Hyderabad. As a Big Data Engineer, you are going to work with cross-functional collaboration and communication with members from other teams. As an Engineer, you are responsible to analyze large data sets to identify errors . You are responsible...


  • Hyderabad, Telangana, India Info Services Full time

    Job DescriptionTitle: Big Data EngineerLocation: Financial District, Hyderabad - HybridType: OffshoreShift Timings: 1:00pm - 10:00pm ISTMode: Fulltime Role.We are looking for someone who can:- Support large data distributed pipelines that span across Airflow, Databricks and Snowflake. Some knowledge with Airflow is good to have to understand how the DAGs...

  • Big Data Engineer

    2 weeks ago


    Hyderabad, Telangana, India Questsphere Private ltd. Full time

    Our client is looking to deploy big data engineer at one of the worlds top product company R&D centre based out of Hyderabad. As a Big Data Engineer, you are going to work with cross-functional collaboration and communication with members from other teams.As an Engineer, you are responsible to analyze large data sets to identify errors . You are responsible...

  • Big data engineer

    3 weeks ago


    Hyderabad, Telangana, India Virtusa Full time

    Big data engineer - CREQ189284 Description Position: Big data engineer Primary: big data concepts, aws, pyspark Location: HYD Create Trigger based automation framework for Data Migration Identify roles/access needed for data migration from federated bucket to managed bucket and Build APIs for the same Integrate CDMS framework with Lake and Data bridge...


  • Hyderabad, Telangana, India HSBC Full time

    Job Title: Senior Consultant Specialist - Big DataJob Summary:This role involves designing and implementing big data solutions using Pyspark and GCP technologies. The successful candidate will have a strong background in software engineering, data architecture, and cloud computing, with excellent problem-solving and analytical skills.Key...

  • Big Data Engineer

    4 weeks ago


    Hyderabad, Telangana, India Folks Palette LLP Full time

    Job Description : We are seeking an experienced Big Data Engineer with PySpark Developer to join our dynamic team. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining scalable data solutions in a big data ecosystem. The ideal candidate will have a strong background in big data technologies,...


  • Hyderabad, Telangana, India Virtusa Full time

    Big Data Python Engineer - CREQ191701 Description Role GDT WS Big Data Python Engineer Exp 5+ Location Pune/Hyderabad MUST Have Skills Design and implement data processing pipelines using Python + SCALA programming and big data technologies such as Hadoop, MapReduce, Hive, and Spark. Airflow Jenkins Understand business requirements to code up data assets...


  • Hyderabad, Telangana, India Enable Data Incorporated Full time

    Enable Data Incorporated is seeking a highly skilled Senior Data Engineer with expertise in cloud technologies, Spark, and Databricks to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, we leverage modern solutions to drive increased value across our clients' business ecosystem.As a Senior Data...

  • Big Data Engineer

    3 weeks ago


    Hyderabad, Telangana, India TechSophy Full time

    Job Description: Roles&Responsibilities : 1. As an Engineer, you are responsible to analyze large data sets to identify errors in the map. 2. As an Engineer, you are responsible to design and implement a complex algorithm for resolving the issue, review the solution with a team of engineers and analysts, and integrate the resulting solution into the...

  • GCP Big Data Engineer

    3 weeks ago


    Hyderabad, Telangana, India Tiger Analytics Full time

    Job Description .css-ylb{color:var(--chakra-colors-black-);}.css-l0deym h1{margin:0;padding:0;font-size:26px;}.css-l0deym h2{margin:0;padding:0;font-size:20px;}.css-l0deym h3{margin:0;padding:0;font-size:15px;}.css-l0deym h4{margin:0;padding:0;font-size:13px;}.css-l0deym ul{list-style-type:disc;margin:0.5rem;margin-bottom:1.5rem;}.css-l0deym ul...


  • Hyderabad, Telangana, India Talescope Full time

    About the RoleWe are seeking a highly skilled Senior Data Engineer to join our team at Talescope. As a key member of our engineering team, you will be responsible for leading the design and implementation of complex, scalable data pipelines using AWS services. You will work closely with cross-functional teams to understand their data needs and translate them...


  • Hyderabad, Telangana, India ServiceNow Full time

    Company DescriptionAt ServiceNow our technology makes the world work for everyone and our people make it possible We move fast because the world can t wait and we innovate in ways no one else can for our customers and communities By joining ServiceNow you are part of an ambitious team of change makers who have a restless curiosity and a drive for...


  • Hyderabad, Telangana, India Programmers Full time

    Job Summary:We are seeking an experienced Big Data Engineer Lead to join our team. The successful candidate will have a strong background in designing and developing Big Data solutions, leveraging technologies such as Apache Spark and Data Bricks.Responsibilities:Lead the design and development of Big Data solutionsLeverage expertise in Big Data Platforms,...


  • Hyderabad, Telangana, India Experian Full time

    Job DescriptionJob DescriptionWe are seeking an experienced Data Engineer with a robust technical background to join our Technology Team.You will be #LI-hybrid based in Hyderabad and reporting to Director.The candidate would possess analytic & technical skills , have a zero tolerance for inefficiencies in product development & deliveries.The ideal candidate...