Senior Big Data Engineer
3 days ago
Position : Big Data Engineer
Experience : 59 Years
Job Type : Full-time
Job Summary :
We are seeking a highly skilled Big Data Engineer with 59 years of experience to develop, maintain, and optimize robust, scalable big data solutions. This role requires expertise in designing and implementing solutions using Hadoop-based technologies and AWS. The engineer will be crucial in building various complex Big Data platform components, focusing on optimizing data access, handling high data volumes, and ensuring real-time performance using frameworks like Spark, Kafka, and other cloud technologies.
Key Responsibilities :
Big Data Platform Architecture and Design :
- Lead the design of comprehensive data solutions leveraging Hadoop-based technologies in conjunction with AWS cloud infrastructure.
- Implement and design various critical Big Data platform components, including Batch Processing, Live Stream Processing, In-Memory Cache, Query Layer (SQL), Rule Engine, and Action Framework.
- Design and implement a robust Data Access Layer capable of connecting to diverse data sources.
Performance and Scalability :
- Implement scalable technical solutions to reliably handle ever-increasing data volumes using modern big data and cloud technologies like Spark, Kafka, and Cloud computing services.
- Utilize advanced caching techniques within the Data Access Layer to ensure fast responses to real-time SQL queries against Big Data Technologies.
Data Pipeline and Infrastructure :
- Set up and operate end-to-end data pipelines and the overall infrastructure required for executing ETL jobs.
- Apply proficiency with distributed file systems and computing concepts to optimize data infrastructure.
Qualifications :
- Experience: 59 years of hands-on Data engineering experience.
- Programming Languages: Good hands-on knowledge and proficiency in one of the primary languages: Java, Scala, or Python.
- Data Handling Frameworks: Strong hands-on experience with at least one of the major data handling frameworks: Apache Spark, Apache Beam, or Apache Flink.
- Data Store: Good technical understanding of both SQL and NoSQL databases (e.g., MySQL, Mongo).
- Data Infrastructure: Familiarity with cloud solutions for data infrastructure, with AWS preferably.
- Distributed Concepts: Proficient with distributed file systems and computing concepts (e.g., HDFS, YARN).
Preferred Skills :
- Familiarity with different types of efficient data formats, including Apache Parquet, Avro, and ORC.
- Direct experience integrating streaming technologies like Kafka with processing frameworks like Spark or Flink.
- Experience in designing a Rule Engine or Action Framework for real-time data processing.
- Certifications in AWS Data Analytics or related Big Data technologies.
-
Big Data Engineer
4 weeks ago
Gurgaon, Haryana, India Infogain Full timePosted on August 17 2024ROLES RESPONSIBILITIESBig Data Engineer Senior As a Big Data Engineer Senior you will be responsible for designing and implementing complex big data solutions You will work closely with clients and team members to understand their requirements and develop architectures that meet their needs You will also be responsible for...
-
Big Data Engineer
5 days ago
Gurgaon, Haryana, India Insight Global Full time ₹ 15,00,000 - ₹ 20,00,000 per yearPay Range: LPA (INR)Required Qualifications:3–5 years of hands-on experience in Big Data engineering.Proficiency in Python (Java is a plus).Strong understanding of Google Cloud Platform (GCP) services and architecture. GCP certification is a plus.Solid problem-solving skills and a structured thought process.Experience with data migration and...
-
Big Data Engineer
4 weeks ago
Gurgaon, Haryana, India Idyllic Services Pvt Ltd Full timeJob Title : Big Data Engineer - ScalaExperience : 7 to 10Years (Minimum 3+ years in Scala)Notice Period : Immediate to 30 DaysRole Overview : We are looking for a highly skilled Big Data Engineer (Scala) with strong expertise in Scala, Spark, Python, NiFi, and Apache Kafka to join our data engineering team. The ideal candidate will have a proven track...
-
Data Scientist/Big Data Engineer
3 weeks ago
Gurgaon, Haryana, India Techmora Full timeJob Description : Data Science / Big Data EngineerLocation : Gurgaon / PuneEmployment Type : Full-timeNotice Period : 30 DaysExperience : 4-6 YearsAbout the Role :We are seeking a skilled Data Science / Big Data Engineer with strong expertise in data wrangling, large-scale data processing, and advanced analytics. The ideal candidate will have hands-on...
-
Big Data Developer
5 days ago
Gurgaon, Haryana, India Kezan Inc Full time ₹ 20,00,000 - ₹ 25,00,000 per yearPosition: Big Data Engineer ( Immediate Joiner)Experience: 5+ YearsLocation: ( Gurugram / Bangalore )Joining: Immediate JoinerBudget LPAJob Summary:We're seeking an experienced Senior Big Data Engineer with 5+ years of experience in designing, developing, and implementing large-scale data systems using Redshift, AWS, Spark, and Scala. The ideal candidate...
-
Big Data Developer
5 days ago
Gurgaon, Haryana, India Kezan Inc Full time ₹ 20,00,000 - ₹ 25,00,000 per yearPosition: Big Data EngineerExperience: 5+ YearsLocation: ( Gurugram / Bangalore )Joining: Immediate JoinerBudget=16.5 LPAJob Summary:We're seeking an experienced Senior Big Data Engineer with 5+ years of experience in designing, developing, and implementing large-scale data systems using Redshift, AWS, Spark, and Scala. The ideal candidate will have...
-
Big Data Developer
5 days ago
Gurgaon, Haryana, India Viraaj HR Solutions Private Limited Full time ₹ 20,00,000 - ₹ 25,00,000 per yearAbout The OpportunityA staffing and HR services firm focused on placing talent into Enterprise IT, Analytics, and Big Data engineering projects across Banking, Retail, Telecom, and SaaS sectors. We partner with product and services organisations to deliver scalable data platforms, real-time analytics, and high-throughput ETL solutions.We are hiring an...
-
Big Data Engineer
4 weeks ago
Gurgaon, Haryana, India NITS Solutions Full timeWe have an urgent opening for Big Data Engineer role.Job Location - GurgaonExperience Required - 2 -6 YearsSkills :- Understand business needs & apply analytical concepts to generate actionable business insights/optimization models- Write Java coding for Spark- Write Python/SQL code- Write Scala coding for Spark- Converting hard and complex techniques as...
-
Big Data Tester
5 days ago
Gurgaon, Haryana, India Kezan Inc Full time ₹ 6,00,000 - ₹ 15,00,000 per yearJob Title:Big Data Tester (SQL)Experience:3+YearsLocation:[Gurugram/Bangalore]Employment Type:Full-TimeJob Summary:We are seeking a highly skilledBig Data Tester with strong SQL expertiseto join our team. The ideal candidate will be responsible for validating, testing, and ensuring the quality of large-scale data pipelines, data lakes, and big data...
-
Senior Data Engineer
24 hours ago
Gurgaon, Haryana, India Orange Business Full time ₹ 20,00,000 - ₹ 25,00,000 per yearCompany DescriptionAt Orange Business, our ambition is to become the leading European Network and Digital Integrator by leveraging our proven expertise in next-generation connectivity solutions, the cloud, and cybersecurity. Our 30,000 women and men are present in 65 countries, where every voice counts. Together, we are driven by the determination and team...