Big Data Engineering Lead

4 weeks ago


Hyderabad, India Techstar Software Development India Pvt. Ltd. Full time
Job Title: Big Data Engineer Lead

A highly skilled Big Data Engineer Lead is required to oversee the design, implementation, and management of large-scale data processing systems. This role involves leading a team of data engineers, collaborating with cross-functional teams, and ensuring the organization effectively utilizes big data technologies to derive actionable insights.

Key Responsibilities:
  • Lead and mentor a team of data engineers, fostering a collaborative and innovative work environment.
  • Provide technical guidance and support for team members in project execution.
  • Design and implement scalable big data architectures and solutions.
  • Ensure systems are optimized for performance and scalability.
  • Develop and maintain data pipelines for processing structured and unstructured data.
  • Utilize big data technologies (e.g., Hadoop, Spark, Kafka) to manage large datasets.
  • Work closely with data scientists, analysts, and stakeholders to understand data needs and requirements.
  • Communicate effectively with technical and non-technical teams.
  • Establish data governance and data quality standards.
  • Implement testing and validation processes for data integrity.
Requirements:
  • Bachelor's degree in computer science, Engineering, Data Science, or a related field; a Master's degree is a plus.
  • 5+ years of experience in data engineering or related fields, with at least 2 years in a leadership role.
  • Proven experience with big data technologies (e.g., Hadoop, PySpark, Hive, NoSQL databases).
  • Proficiency in programming languages such as Python or Java, and Scala.
  • Experience with Cloud Platform CI/CD Pipelines.
  • Experience with cloud platforms (e.g., AWS, Azure, Google Cloud) and data warehousing solutions.
Preferred Skills:
  • Familiarity with machine learning frameworks and tools.
  • Knowledge of data visualization tools (e.g., Tableau, Power BI).
  • Experience with DevOps practices and CI/CD pipelines.
Work Environment:

This position may require occasional overtime and the flexibility to adapt to changing project requirements. Remote or hybrid work options may be available.


  • Big Data Engineer

    2 weeks ago


    Hyderabad, Telangana, India data Full time

    Overview: We're seeking a skilled Big Data Engineer to join our team at data.ai. As a key member of our engineering team, you will be responsible for designing and implementing innovative data solutions that drive business growth. If you're passionate about working with large data sets, have experience with big data technologies, and are eager to...

  • Big Data Engineer

    4 weeks ago


    Hyderabad, Telangana, India data Full time

    About the RoleWe are seeking an experienced Big Data Engineer to join our team at data.ai India subsidiary entity. As a key member of our engineering team, you will be responsible for designing and implementing complex data product components based on requirements with possible technical solutions.Key ResponsibilitiesDesign and implement data processing...


  • Hyderabad, Telangana, India NTT DATA Full time

    Job Title: Data EngineerJob Summary:We are seeking a skilled Data Engineer to design, build, and maintain large-scale data systems. The ideal candidate will have expertise in big data ecosystems, data processing, and data storage.Key Responsibilities:Design and develop scalable data systems for business applications.Work closely with data scientists to...


  • Hyderabad, Telangana, India data Full time

    About the RoleWe are seeking an experienced Big Data Engineer to join our team at data.ai. As a key member of our engineering team, you will be responsible for designing and implementing complex data product components based on requirements with possible technical solutions.Key ResponsibilitiesDesign and implement data processing programs using Python (e.g.,...

  • Big Data Engineer

    4 weeks ago


    Hyderabad, Telangana, India data Full time

    About the RoleWe are seeking an experienced Big Data Engineer to join our team in Hyderabad, India. As a key member of our engineering team, you will be responsible for designing and implementing complex data product components based on requirements with possible technical solutions.Key ResponsibilitiesDesign and implement data processing programs using...


  • Hyderabad, India Techstar Software Development India Pvt. Ltd. Full time

    Job Title : Big Data Engineer LeadThe Big Data Engineer Lead is responsible for leading the design, implementation, and management of large-scale data processing systems at Techstar Software Development India Pvt. Ltd. This role involves overseeing a team of data engineers, collaborating with cross-functional teams, and ensuring the organization effectively...


  • Hyderabad, Telangana, India data Full time

    Unlock the Power of Data with UsWe're seeking an experienced Big Data Engineer to join our team and contribute to the development of innovative analytics and data products. As a key member of our engineering team, you will work closely with Data Scientists, Data Engineers, Product Managers, and Software Architects to design and implement complex data product...


  • Hyderabad, India Techstar Software Development India Pvt. Ltd. Full time

    Job Title: Big Data Engineer LeadJob Summary:The Big Data Engineer Lead is responsible for overseeing the design, implementation, and management of large-scale data processing systems. This role involves leading a team of data engineers, collaborating with cross-functional teams, and ensuring the organization effectively utilizes big data technologies to...

  • Big Data Engineer

    1 week ago


    Hyderabad, Telangana, India NTT DATA Services Full time

    Data Engineer RoleData engineers at NTT DATA Services are responsible for building and testing scalable Big Data ecosystems for our clients. They work closely with data scientists to ensure the source data is aggregated and cleansed, and with product managers to understand the business objectives. Data engineers must have expertise in all phases of data...

  • Data Engineer

    3 weeks ago


    Hyderabad, Telangana, India NTT DATA Services Full time

    Job Title: Data Engineer - Big Data SpecialistAt NTT DATA Services, we are seeking a skilled Data Engineer to join our team. As a Data Engineer, you will be responsible for designing and developing scalable Big Data ecosystems for our clients. Your expertise will help us build stable and highly optimized data systems, ensuring seamless data processing and...

  • Big Data Engineer

    2 months ago


    Hyderabad, India NucleusTeq Full time

    Big Data Engineer Responsibilities: Big Data Engineer with 5+ years of experienceExperience in leading a team of engineers and a good attitude toward learning the domain and implementationStrong exposure and expertise in Spark (Primary), Scala/Java (Scala Primary), Airflow Orchestration and AWS.Finalizing the scope of the system and delivering...

  • Big Data Engineer

    4 weeks ago


    Hyderabad, Telangana, India Virtusa Full time

    Job Title: Big Data EngineerWe are seeking a highly skilled Big Data Engineer to join our team at Virtusa. As a Big Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data processing systems using big data technologies such as Hadoop, Spark, and NoSQL databases.Key Responsibilities:Design and develop big data...

  • Big Data Engineer

    4 weeks ago


    Hyderabad, Telangana, India Virtusa Full time

    Job Title: Big Data EngineerWe are seeking a highly skilled Big Data Engineer to join our team at Virtusa. As a Big Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data processing systems using big data technologies such as Hadoop, Spark, and NoSQL databases.Key Responsibilities:Design and develop data pipelines...

  • Big Data Engineer

    3 weeks ago


    Hyderabad, Telangana, India Virtusa Full time

    Job Summary We are seeking a highly skilled Big Data Engineer to join our team at Virtusa. The ideal candidate will have experience in designing and implementing data processing pipelines using Python and big data technologies such as Hadoop, MapReduce, Hive, and Spark. Key ResponsibilitiesDesign and implement data processing pipelines using Python and big...

  • Big data engineer

    5 months ago


    Hyderabad, India Virtusa Full time

    Big data engineer - CREQ189284 Description Position: Big data engineer Primary: big data concepts, aws, pyspark Location: HYD Create Trigger based automation framework for Data Migration Identify roles/access needed for data migration from federated bucket to managed bucket and Build APIs for the same Integrate CDMS framework with Lake and Data bridge...

  • Big Data Engineer

    4 weeks ago


    Hyderabad, Telangana, India Virtusa Full time

    Job Title: Big Data Python EngineerWe are seeking a highly skilled Big Data Python Engineer to join our team at Virtusa. As a key member of our data engineering team, you will be responsible for designing and implementing data processing pipelines using Python and Scala programming languages, as well as big data technologies such as Hadoop, MapReduce, Hive,...

  • Big Data Engineer

    2 weeks ago


    Hyderabad, Telangana, India ANB IT Solutions Pvt Ltd Full time

    Key ResponsibilitiesCollaborate with cross-functional teams to design, develop, and maintain big data solutions using Google Cloud Big Query. Key responsibilities include data ingestion, transformation, and analytics.Data Engineering and DevelopmentDevelop and maintain ETL/ELT processes to ensure data integrity and consistency. Utilize Google Cloud Composer,...

  • Big Data Engineer

    3 weeks ago


    Hyderabad, Telangana, India Virtusa Full time

    Job Title: Big Data EngineerJob Summary: We are seeking a skilled Big Data Engineer to design and implement data processing pipelines using Python and big data technologies such as Hadoop, MapReduce, Hive, and Spark.Key Responsibilities:Design and implement data processing pipelines using Python and big data technologies.Develop and maintain data assets...

  • Big Data Engineer

    3 weeks ago


    Hyderabad, Telangana, India Vidpro Consultancy Services Full time

    About the Role:Vidpro Consultancy Services is seeking a highly skilled Big Data Engineer to join our team. As a key member of our data architecture team, you will be responsible for designing and implementing large-scale Big Data architectures using Java and Python.Key Responsibilities:Design and implement Big Data architectures using Hadoop and SparkDevelop...

  • Big Data Engineer

    4 weeks ago


    Hyderabad, Telangana, India TechSophy Full time

    Job Title: Big Data EngineerJob Summary:We are seeking a highly skilled Big Data Engineer to join our team at TechSophy. As a Big Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data processing systems.Key Responsibilities:Analyze large data sets to identify errors in the map.Design and implement complex...