Big Data Lead
2 months ago
5+ years of hands-on experience in designing, building and supporting Data Applications using Spark, Sqoop and Hive
Strong knowledge of working with large data sets and high-capacity big data processing platform
Strong experience in Unix and Shell scripting
Advanced knowledge of the Hadoop ecosystem and its components
In-depth knowledge of Hive, Shell scripting, Python, Spark
Ability to write MapReduce jobs
Experience using Job Schedulers like Autosys
Hands on experience in HiveQL
Good knowledge on Hadoop Architecture and HDFS
Strong knowledge of working with large data sets and high-capacity big data processing platform
Strong experience in Unix and Shell scripting
Experience with jenkins for Continuous Integration
Experience using Source Code and Version Control Systems like Bitbucket, Git
Good to have experience on Agile Development
Responsibilities: Develop components, application interfaces, and solution enablers while ensuring principal architecture integrity is maintained
Ensures solutions are well designed with maintainability/ease of integration and testing built-in from the outset
Participates and guides team in estimating work necessary to realize a story/requirement through the software delivery lifecycle
Responsible for developing and delivering complex software requirements to accomplish business goals
Ensures that software is developed to meet functional, non-functional, and compliance requirements
Codes solutions, Unit testing and ensure the solution can be integrated successfully into the overall application/system with clear robust, and well-tested interfaces
Required Skills: Hadoop, Hive, HDFS, Spark, Python, Unix
Nice to have skills: AML Domain knowledge
Primary Location Chennai, Tamil Nadu, India Job Type Experienced Primary Skills PySpark Years of Experience 5 Qualification
Job Description:
5+ years of hands-on experience in designing, building and supporting Data Applications using Spark, Sqoop and Hive
Strong knowledge of working with large data sets and high-capacity big data processing platform
Strong experience in Unix and Shell scripting
Advanced knowledge of the Hadoop ecosystem and its components
In-depth knowledge of Hive, Shell scripting, Python, Spark
Ability to write MapReduce jobs
Experience using Job Schedulers like Autosys
Hands on experience in HiveQL
Good knowledge on Hadoop Architecture and HDFS
Strong knowledge of working with large data sets and high-capacity big data processing platform
Strong experience in Unix and Shell scripting
Experience with jenkins for Continuous Integration
Experience using Source Code and Version Control Systems like Bitbucket, Git
Good to have experience on Agile Development
Responsibilities: Develop components, application interfaces, and solution enablers while ensuring principal architecture integrity is maintained
Ensures solutions are well designed with maintainability/ease of integration and testing built-in from the outset
Participates and guides team in estimating work necessary to realize a story/requirement through the software delivery lifecycle
Responsible for developing and delivering complex software requirements to accomplish business goals
Ensures that software is developed to meet functional, non-functional, and compliance requirements
Codes solutions, Unit testing and ensure the solution can be integrated successfully into the overall application/system with clear robust, and well-tested interfaces
Required Skills: Hadoop, Hive, HDFS, Spark, Python, Unix
Nice to have skills: AML Domain knowledge
-
Migrations Architect
4 days ago
Chennai, India Databricks Full timeFEQ225R141As a Migrations Architect (Big data) in our Professional Services team, you will work with customers on short to medium-term customer engagements on their big data challenges using the Databricks platform. You will provide data engineering, data science, and cloud technology projects that require integrating with client systems, training, and...
-
Solutions Architect
4 days ago
Chennai, India Databricks Full timeFEQ225R141As a Solution Architect (Big data) in our Professional Services team, you will work with customers on short to medium-term customer engagements on their big data challenges using the Databricks platform. You will provide data engineering, data science, and cloud technology projects that require integrating with client systems, training, and other...
-
GCP Big Data Engineer
2 weeks ago
Chennai, India IntraEdge Full timeAbout the Role: We are seeking a highly motivated and experienced Big Data Engineer to join our growing team in Chennai, Bangalore, or Gurgaon. In this role, you will play a key part in designing, developing, and managing our big data infrastructure on Google Cloud Platform (GCP). You will leverage your expertise in big data technologies and GCP services to...
-
GCP Big Data Engineer
3 weeks ago
Chennai, India IntraEdge Full timeAbout the Role:We are seeking a highly motivated and experienced Big Data Engineer to join our growing team in Chennai, Bangalore, or Gurgaon. In this role, you will play a key part in designing, developing, and managing our big data infrastructure on Google Cloud Platform (GCP). You will leverage your expertise in big data technologies and GCP services to...
-
Big Data Architect
3 months ago
Chennai, Tamil Nadu, India RiskInsight Consulting Pvt Ltd Full time**Responsibilities**: - Design and architect Big Data solutions to meet business requirements. - Collaborate with stakeholders to understand their data and analytics needs. - Create and maintain data models, data flow diagrams, and data integration strategies. - Develop and implement data ingestion, processing, and storage solutions using Big Data...
-
Senior Big Data Engineer
1 week ago
Chennai, India Sriven Full timeJob Title: Senior Big Data Engineer Experience Required: 7+ years Educational Qualification: Bachelor's degree from an accredited college or university Job Description: We are looking for a highly skilled Senior Big Data Engineer to join our dynamic team. The ideal candidate will have extensive experience in designing and developing...
-
Senior Big Data Engineer
2 weeks ago
Chennai, India Sriven Full timeJob Title: Senior Big Data Engineer Experience Required: 7+ years Educational Qualification: Bachelor's degree from an accredited college or university Job Description: We are looking for a highly skilled Senior Big Data Engineer to join our dynamic team. The ideal candidate will have extensive experience in designing and developing complex Big Data...
-
Senior Big Data Engineer
3 weeks ago
Chennai, India Sriven Full timeJob Title: Senior Big Data EngineerExperience Required: 7+ yearsEducational Qualification: Bachelor's degree from an accredited college or universityJob Description:We are looking for a highly skilled Senior Big Data Engineer to join our dynamic team. The ideal candidate will have extensive experience in designing and developing complex Big Data pipelines...
-
Senior Big Data Engineer
3 weeks ago
Chennai, India Sriven Full timeJob Title: Senior Big Data EngineerExperience Required: 7+ yearsEducational Qualification: Bachelor's degree from an accredited college or universityJob Description:We are looking for a highly skilled Senior Big Data Engineer to join our dynamic team. The ideal candidate will have extensive experience in designing and developing complex Big Data pipelines...
-
Big Data Testing
3 months ago
Chennai, India powerit services pvt ltd Full time**ABOUT US** We are providing Recruitment Services and IT Consulting Services for our Tier-1 Multinational Clients covering following major domains, but not limited to Banking, Financial Services and Insurance, Retail, Airline, Automotive, Energy and Consumer Sector **ABOUT CLIENT** Our Client is a leading International technology consulting and services...
-
Big Data Architect
1 month ago
Chennai, India Selsoftinc Full timeJob Description :- Design and implement scalable big data architectures on Microsoft Azure and Azure Fabric to handle large volumes of data efficiently and securely.- Develop and manage data pipelines for ingesting, processing, and analyzing large datasets, ensuring data quality and integrity.- Collaborate with data scientists, data engineers, and business...
-
Big Data Developer
2 months ago
Chennai, India Virtusa Full timeBig Data Developer - CREQ188135 Description Job Description: 5+ years of hands-on experience in designing, building and supporting Data Applications using Spark, Sqoop and Hive Bachelors or masters degree in Computer Science or related field Strong knowledge of working with large data sets and high-capacity big data processing platform Strong experience...
-
Big Data Engineer
23 hours ago
Chennai, India Tata Consultancy Services Full timeGreetings fromTATA Consultancy Services !!We are currently Hiring ForBIG DATA ENGINEERJob Location :CHENNAIExp 5+Interview Mode: FACE TO FACEINTERVIEW DATE : 21-Aug-24Job description3+ years of relevant experience in Pyspark and Hive, HBaseProficiency in integrating, transforming, and consolidating data from various structured and unstructured data...
-
Big Data Developer
4 months ago
Chennai, India RiskInsight Consulting Pvt Ltd Full timeResponsibilities Design, develop, and deploy scalable Big Data applications using Apache Spark. Collaborate with data scientists and business analysts to understand requirements and translate them into technical solutions. Write efficient and optimized code to process and analyze large volumes of data. Implement data ingestion processes from various data...
-
Big Data Engineer
22 hours ago
Chennai, India Tata Consultancy Services Full timeGreetings from TATA Consultancy Services !!We are currently Hiring For BIG DATA ENGINEERJob Location : CHENNAIExp 5+Interview Mode: FACE TO FACEINTERVIEW DATE : 21-Aug-24Job description3+ years of relevant experience in Pyspark and Hive, HBaseProficiency in integrating, transforming, and consolidating data from various structured and unstructured data...
-
Big Data Engineer
4 weeks ago
Chennai, India People Prime Worldwide Full timeAbout Company: Our client is a global technologyconsulting and digital solutions company that enables enterprisesto reimagine business models and accelerate innovation throughdigital technologies. Powered by more than 84000 entrepreneurialprofessionals across more than 30 countries it caters to over 700clients with its extensive domain and technology...
-
Solution Architect
2 months ago
Chennai, India Recro Full timeResponsibilities :- Work collaboratively with product managers, engineers, and data scientists to understand business requirements and translate them into scalable, secure, and cost-effective big data solutions leveraging AWS services.- Design and document big data architectures that incorporate data ingestion, processing, storage, analytics, and...
-
Data Analyst
1 month ago
Chennai, India BSRI Solutions Full timeJob Description :- Use data mining, machine learning, and statistical techniques to solve business problems - Formulate analytical problems with appropriate assumptions - Interact with internal stakeholders to understand the business problems - Collaborate with business users for manipulating massive data sets and mash-up of data from multiple sources &...
-
Senior Data Engineer
3 days ago
Chennai, India Trinity Technology solutions Full timeWe're seeking a highly skilled and experienced Senior Data Engineer to join our team! You will play a pivotal role in designing, developing, and deploying robust data pipelines and data warehouse solutions leveraging Google Cloud Platform (GCP). You will leverage your expertise in big data technologies like Hadoop, Spark, and Airflow to build scalable...
-
Big Data Engineer
16 hours ago
Chennai, India Tata Consultancy Services Full timeGreetings from TATA Consultancy Services!!We are currently Hiring For BIG DATA ENGINEER Job Location : CHENNAIExp 5+ Interview Mode: FACE TO FACEINTERVIEW DATE : 21-Aug-24Job description3+ years of relevant experience in Pyspark and Hive, HBase Proficiency in integrating, transforming, and consolidating data from various structured and unstructured data...