Big Data Lead

2 months ago


Chennai, India Virtusa Full time
Big Data Lead - CREQ194625 Description Job Description:
5+ years of hands-on experience in designing, building and supporting Data Applications using Spark, Sqoop and Hive
Strong knowledge of working with large data sets and high-capacity big data processing platform
Strong experience in Unix and Shell scripting
Advanced knowledge of the Hadoop ecosystem and its components
In-depth knowledge of Hive, Shell scripting, Python, Spark
Ability to write MapReduce jobs
Experience using Job Schedulers like Autosys
Hands on experience in HiveQL
Good knowledge on Hadoop Architecture and HDFS
Strong knowledge of working with large data sets and high-capacity big data processing platform
Strong experience in Unix and Shell scripting
Experience with jenkins for Continuous Integration
Experience using Source Code and Version Control Systems like Bitbucket, Git
Good to have experience on Agile Development

Responsibilities: Develop components, application interfaces, and solution enablers while ensuring principal architecture integrity is maintained
Ensures solutions are well designed with maintainability/ease of integration and testing built-in from the outset
Participates and guides team in estimating work necessary to realize a story/requirement through the software delivery lifecycle
Responsible for developing and delivering complex software requirements to accomplish business goals
Ensures that software is developed to meet functional, non-functional, and compliance requirements
Codes solutions, Unit testing and ensure the solution can be integrated successfully into the overall application/system with clear robust, and well-tested interfaces

Required Skills: Hadoop, Hive, HDFS, Spark, Python, Unix
Nice to have skills: AML Domain knowledge
Primary Location Chennai, Tamil Nadu, India Job Type Experienced Primary Skills PySpark Years of Experience 5 Qualification

Job Description:
5+ years of hands-on experience in designing, building and supporting Data Applications using Spark, Sqoop and Hive
Strong knowledge of working with large data sets and high-capacity big data processing platform
Strong experience in Unix and Shell scripting
Advanced knowledge of the Hadoop ecosystem and its components
In-depth knowledge of Hive, Shell scripting, Python, Spark
Ability to write MapReduce jobs
Experience using Job Schedulers like Autosys
Hands on experience in HiveQL
Good knowledge on Hadoop Architecture and HDFS
Strong knowledge of working with large data sets and high-capacity big data processing platform
Strong experience in Unix and Shell scripting
Experience with jenkins for Continuous Integration
Experience using Source Code and Version Control Systems like Bitbucket, Git
Good to have experience on Agile Development

Responsibilities: Develop components, application interfaces, and solution enablers while ensuring principal architecture integrity is maintained
Ensures solutions are well designed with maintainability/ease of integration and testing built-in from the outset
Participates and guides team in estimating work necessary to realize a story/requirement through the software delivery lifecycle
Responsible for developing and delivering complex software requirements to accomplish business goals
Ensures that software is developed to meet functional, non-functional, and compliance requirements
Codes solutions, Unit testing and ensure the solution can be integrated successfully into the overall application/system with clear robust, and well-tested interfaces

Required Skills: Hadoop, Hive, HDFS, Spark, Python, Unix
Nice to have skills: AML Domain knowledge

Travel No

  • Chennai, India Databricks Full time

    FEQ225R141As a Migrations Architect (Big data) in our Professional Services team, you will work with customers on short to medium-term customer engagements on their big data challenges using the Databricks platform. You will provide data engineering, data science, and cloud technology projects that require integrating with client systems, training, and...

  • Solutions Architect

    4 days ago


    Chennai, India Databricks Full time

    FEQ225R141As a Solution Architect (Big data) in our Professional Services team, you will work with customers on short to medium-term customer engagements on their big data challenges using the Databricks platform. You will provide data engineering, data science, and cloud technology projects that require integrating with client systems, training, and other...

  • GCP Big Data Engineer

    2 weeks ago


    Chennai, India IntraEdge Full time

    About the Role: We are seeking a highly motivated and experienced Big Data Engineer to join our growing team in Chennai, Bangalore, or Gurgaon. In this role, you will play a key part in designing, developing, and managing our big data infrastructure on Google Cloud Platform (GCP). You will leverage your expertise in big data technologies and GCP services to...

  • GCP Big Data Engineer

    3 weeks ago


    Chennai, India IntraEdge Full time

    About the Role:We are seeking a highly motivated and experienced Big Data Engineer to join our growing team in Chennai, Bangalore, or Gurgaon. In this role, you will play a key part in designing, developing, and managing our big data infrastructure on Google Cloud Platform (GCP). You will leverage your expertise in big data technologies and GCP services to...

  • Big Data Architect

    3 months ago


    Chennai, Tamil Nadu, India RiskInsight Consulting Pvt Ltd Full time

    **Responsibilities**: - Design and architect Big Data solutions to meet business requirements. - Collaborate with stakeholders to understand their data and analytics needs. - Create and maintain data models, data flow diagrams, and data integration strategies. - Develop and implement data ingestion, processing, and storage solutions using Big Data...


  • Chennai, India Sriven Full time

    Job Title: Senior Big Data Engineer Experience Required: 7+ years Educational Qualification: Bachelor's degree from an accredited college or university Job Description: We are looking for a highly skilled Senior Big Data Engineer to join our dynamic team. The ideal candidate will have extensive experience in designing and developing...


  • Chennai, India Sriven Full time

    Job Title: Senior Big Data Engineer Experience Required: 7+ years Educational Qualification: Bachelor's degree from an accredited college or university Job Description: We are looking for a highly skilled Senior Big Data Engineer to join our dynamic team. The ideal candidate will have extensive experience in designing and developing complex Big Data...


  • Chennai, India Sriven Full time

    Job Title: Senior Big Data EngineerExperience Required: 7+ yearsEducational Qualification: Bachelor's degree from an accredited college or universityJob Description:We are looking for a highly skilled Senior Big Data Engineer to join our dynamic team. The ideal candidate will have extensive experience in designing and developing complex Big Data pipelines...


  • Chennai, India Sriven Full time

    Job Title: Senior Big Data EngineerExperience Required: 7+ yearsEducational Qualification: Bachelor's degree from an accredited college or universityJob Description:We are looking for a highly skilled Senior Big Data Engineer to join our dynamic team. The ideal candidate will have extensive experience in designing and developing complex Big Data pipelines...

  • Big Data Testing

    3 months ago


    Chennai, India powerit services pvt ltd Full time

    **ABOUT US** We are providing Recruitment Services and IT Consulting Services for our Tier-1 Multinational Clients covering following major domains, but not limited to Banking, Financial Services and Insurance, Retail, Airline, Automotive, Energy and Consumer Sector **ABOUT CLIENT** Our Client is a leading International technology consulting and services...

  • Big Data Architect

    1 month ago


    Chennai, India Selsoftinc Full time

    Job Description :- Design and implement scalable big data architectures on Microsoft Azure and Azure Fabric to handle large volumes of data efficiently and securely.- Develop and manage data pipelines for ingesting, processing, and analyzing large datasets, ensuring data quality and integrity.- Collaborate with data scientists, data engineers, and business...

  • Big Data Developer

    2 months ago


    Chennai, India Virtusa Full time

    Big Data Developer - CREQ188135 Description Job Description: 5+ years of hands-on experience in designing, building and supporting Data Applications using Spark, Sqoop and Hive Bachelors or masters degree in Computer Science or related field Strong knowledge of working with large data sets and high-capacity big data processing platform Strong experience...

  • Big Data Engineer

    23 hours ago


    Chennai, India Tata Consultancy Services Full time

    Greetings fromTATA Consultancy Services !!We are currently Hiring ForBIG DATA ENGINEERJob Location :CHENNAIExp 5+Interview Mode: FACE TO FACEINTERVIEW DATE : 21-Aug-24Job description3+ years of relevant experience in Pyspark and Hive, HBaseProficiency in integrating, transforming, and consolidating data from various structured and unstructured data...

  • Big Data Developer

    4 months ago


    Chennai, India RiskInsight Consulting Pvt Ltd Full time

    Responsibilities Design, develop, and deploy scalable Big Data applications using Apache Spark. Collaborate with data scientists and business analysts to understand requirements and translate them into technical solutions. Write efficient and optimized code to process and analyze large volumes of data. Implement data ingestion processes from various data...

  • Big Data Engineer

    22 hours ago


    Chennai, India Tata Consultancy Services Full time

    Greetings from TATA Consultancy Services !!We are currently Hiring For BIG DATA ENGINEERJob Location : CHENNAIExp 5+Interview Mode: FACE TO FACEINTERVIEW DATE : 21-Aug-24Job description3+ years of relevant experience in Pyspark and Hive, HBaseProficiency in integrating, transforming, and consolidating data from various structured and unstructured data...

  • Big Data Engineer

    4 weeks ago


    Chennai, India People Prime Worldwide Full time

    About Company: Our client is a global technologyconsulting and digital solutions company that enables enterprisesto reimagine business models and accelerate innovation throughdigital technologies. Powered by more than 84000 entrepreneurialprofessionals across more than 30 countries it caters to over 700clients with its extensive domain and technology...

  • Solution Architect

    2 months ago


    Chennai, India Recro Full time

    Responsibilities :- Work collaboratively with product managers, engineers, and data scientists to understand business requirements and translate them into scalable, secure, and cost-effective big data solutions leveraging AWS services.- Design and document big data architectures that incorporate data ingestion, processing, storage, analytics, and...

  • Data Analyst

    1 month ago


    Chennai, India BSRI Solutions Full time

    Job Description :- Use data mining, machine learning, and statistical techniques to solve business problems - Formulate analytical problems with appropriate assumptions - Interact with internal stakeholders to understand the business problems - Collaborate with business users for manipulating massive data sets and mash-up of data from multiple sources &...


  • Chennai, India Trinity Technology solutions Full time

    We're seeking a highly skilled and experienced Senior Data Engineer to join our team! You will play a pivotal role in designing, developing, and deploying robust data pipelines and data warehouse solutions leveraging Google Cloud Platform (GCP). You will leverage your expertise in big data technologies like Hadoop, Spark, and Airflow to build scalable...

  • Big Data Engineer

    16 hours ago


    Chennai, India Tata Consultancy Services Full time

    Greetings from TATA Consultancy Services!!We are currently Hiring For BIG DATA ENGINEER Job Location : CHENNAIExp 5+ Interview Mode: FACE TO FACEINTERVIEW DATE : 21-Aug-24Job description3+ years of relevant experience in Pyspark and Hive, HBase Proficiency in integrating, transforming, and consolidating data from various structured and unstructured data...