Big Data Engineer
13 hours ago
Job Description :
We are looking for a Big Data Engineer to work on collecting, storing, processing, and analyzing large data sets. The primary focus will be on choosing optimal solutions for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating these solutions with the architecture used across our clients.
Roles and Responsibilities :
- Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities.
- Developing and maintaining data pipelines, implementing ETL processes, monitoring performance, and advising on necessary infrastructure changes.
- Translating complex technical and functional requirements into detailed designs.
- Investigating and analyzing alternative solutions for data storage, processing, etc., to ensure the most streamlined approaches are implemented.
- Serving as a mentor to junior staff by conducting technical training sessions and reviewing project outputs.
Skills and Qualifications :
- Strong understanding of data warehousing and data modeling techniques.
- Proficient understanding of distributed computing principles (e.g., Hadoop v2, MapReduce, HDFS).
- Strong data engineering skills on GCP platforms, including Airflow, Cloud Composer, Data Fusion, Data Flow, Data Proc, BigQuery.
- Experience with building stream-processing systems, using solutions such as Storm or Spark Streaming.
- Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala.
- Experience with Spark, SQL, and Linux.
- Knowledge of various ETL techniques and frameworks, such as Flume, Apache NiFi, or DBT.
- Experience with messaging systems like Kafka or RabbitMQ.
- Good understanding of Lambda Architecture, along with its advantages and drawbacks.
-
Big data Engineer
3 weeks ago
Bengaluru / Bangalore, Gurgaon / Gurugram, India Talodyn Networks Private Limited Full timeDescriptionWe are looking for an experienced Big Data Engineer to join our team. The ideal candidate will have 5-8 years of experience in the job market in India and will be responsible for designing, building, and maintaining our big data infrastructure. The candidate should have strong technical skills and experience working with big data technologies such...
-
Big Data Engineer
2 months ago
Gurugram, India Chiselon Technologies Pvt Ltd Full time# Hiring Alert #We are looking for Immediate Joiners Skills:Big Data with GCP GCP; Hive; PySpark; Python GCP Data Engineer with hands-on experience of 3+ years in Big Query, Data Proc, Airflow, Cloud Composer, GCP Hydra Services, Cloud Data Optimization "Join us and become an integral part of our success story. Apply today and let's achieve great things...
-
Technical Lead
13 hours ago
Gurugram, India Coders Brain Technology Private Limited Full timeJob Title : Tech Lead - Big Data, Digital BusinessExperience : 8+ Years (5+ years in Data Technologies, 3+ years in AWS Cloud Services)Budget : 1.6 LPMWork Mode : Work from Office (WFO)Location : GurgaonJob Description :Key Responsibilities :Data Engineering Leadership :- Lead and manage Big Data engineering initiatives, ensuring robust and scalable...
-
Big Data Engineer, Global Finance Technology
5 months ago
Gurugram, India Amazon Development Centre (India) Private Limited Full timeAre you an experienced Big Data Engineer passionate about building scalable and enterprise-level data engineering solutions? This opportunity is a great fit for an experienced Big Data Engineer who enjoys working on large-scale computing leveraging AWS distributed systems. In addition to technical expertise, you will invest time to understand the needs of...
-
Senior Big Data Engineer
2 weeks ago
Gurugram, India dunnhumby Full timedunnhumby is the global leader in Customer Data Science, empowering businesses everywhere to compete and thrive in the modern data-driven economy. We always put the Customer First. Our mission: to enable businesses to grow and reimagine themselves by becoming advocates and champions for their Customers. With deep heritage and expertise in retail – one of...
-
Senior Big Data Engineer
2 weeks ago
Gurugram, India dunnhumby Full timedunnhumby is the global leader in Customer Data Science, empowering businesses everywhere to compete and thrive in the modern data-driven economy. We always put the Customer First. Our mission: to enable businesses to grow and reimagine themselves by becoming advocates and champions for their Customers. With deep heritage and expertise in retail – one of...
-
Senior Big Data Engineer
2 weeks ago
Gurugram, India dunnhumby Full timedunnhumby is the global leader in Customer Data Science, empowering businesses everywhere to compete and thrive in the modern data-driven economy. We always put the Customer First. Our mission: to enable businesses to grow and reimagine themselves by becoming advocates and champions for their Customers. With deep heritage and expertise in retail – one of...
-
Big Data Developer
2 months ago
gurugram, India Shya Workforce Solutions Full time• Collaborate with enterprise architects, data architects, developers & engineers, data scientists, and information designers to lead the identification and definition of required data structures, formats, pipelines, metadata, and workload orchestration capabilities • Experience in service architecture, development, and high performance and scalability....
-
Big Data Developer
2 months ago
Gurugram, India Shya Workforce Solutions Full time• Collaborate with enterprise architects, data architects, developers & engineers, datascientists, and information designers to lead the identification and definition of requireddata structures, formats, pipelines, metadata, and workload orchestration capabilities• Experience in service architecture, development, and high performance and scalability.•...
-
Big Data Developer
2 months ago
gurugram, India Shya Workforce Solutions Full time• Collaborate with enterprise architects, data architects, developers & engineers, datascientists, and information designers to lead the identification and definition of requireddata structures, formats, pipelines, metadata, and workload orchestration capabilities• Experience in service architecture, development, and high performance and scalability.•...
-
Big Data Developer
2 months ago
Gurugram, India Shya Workforce Solutions Full time• Collaborate with enterprise architects, data architects, developers & engineers, datascientists, and information designers to lead the identification and definition of requireddata structures, formats, pipelines, metadata, and workload orchestration capabilities• Experience in service architecture, development, and high performance and scalability.•...
-
Big Data Engineer
3 months ago
Gurugram, India ANRI Solutions HR Services Full timeRole : Big Data Engineer Looking for an experienced Spark and Scala developer able to create an end-to-end data pipeline.Associate should be self-driven, can work with minimal guidance and guide the team technically. Job Details : 1. Expertise in Spark framework with scala language and Pyspark 2. Expertise in ETL and Data Warehousing and cloud concepts 3....
-
Google Cloud Platform Architect
1 month ago
Gurugram, India SMARTWORK IT SERVICES Full timeRole : GCP Architect Big Data Engineering. Location : Bangalore, Chennai, Kolkata, Gurugram, Pune. Experience : 13-17 years. Mandatory Skills : GCP, GCS, BigQuery, SQL, Data Flow, DataProc with PySpark, Pub/Sub, Airflow, Python, Spark. Job Description : We are looking for a Big Data Engineer to work on collecting, storing, processing, and analyzing large...
-
Engineer III- Big Data
5 months ago
Gurugram, India AMEX Full timeYou Lead the Way. Weve Got Your Back. With the right backing, people and businesses have the power to progress in incredible ways. When you join Team Amex, you become part of a global and diverse community of colleagues with an unwavering commitment to back our customers, communities and each other. Here, youll learn and grow as we help you create a...
-
Big Data
3 months ago
Gurugram, India Myananta Technologies Full timeAt Myananta Technologies, we’re looking for a Big Data (Hadoop). The ideal candidate is excited by experimentation and looking for a new challenge that stretches their talents. Job descriptionFull-Time Big Data (Hadoop) DeveloperMyananta Technologies | Gurugram is hiring for “Big Data (Hadoop) Developer” Experience: 3 - 5 Years.Salary: 5.0 - 12.0...
-
Big Data
3 months ago
gurugram, India Myananta Technologies Full timeAt Myananta Technologies, we’re looking for a Big Data (Hadoop). The ideal candidate is excited by experimentation and looking for a new challenge that stretches their talents. Job description Full-Time Big Data (Hadoop) Developer Myananta Technologies | Gurugram is hiring for “Big Data (Hadoop) Developer” Experience : 3 - 5 Years. Salary : 5.0...
-
Big Data Engineer
6 months ago
Gurugram, India dunnhumby Full timeWe’re looking for a Big Data Engineer who expects more from their career. It’s a chance to extend and improve dunnhumby’s Data Engineering Team It’s an opportunity to work with a market-leading business to explore new opportunities for us and influence global retailers. Joining our team, you’ll work with world class and passionate people...
-
Big Data Developer
4 weeks ago
gurugram, India Impetus Full timeLocation: Indore, Pune, Noida, Gurugram Experience: 4- 8 years Description: We are looking for Big Data Developer having minimum experience of 4 year in big data ecosystem, passionate about technology, motivated for continuous learning and an individual who views every client interaction as an opportunity to create an exceptional customer experience....
-
Big Data Developer
4 weeks ago
Gurugram, India Impetus Full timeLocation: Indore, Pune, Noida, GurugramExperience: 4- 8 yearsDescription:We are looking for Big Data Developer having minimum experience of 4 year in big data ecosystem, passionate about technology, motivated for continuous learning and an individual who views every client interaction as an opportunity to create an exceptional customer...
-
Big Data Developer
4 weeks ago
Gurugram, India Impetus Full timeLocation: Indore, Pune, Noida, GurugramExperience: 4- 8 yearsDescription:We are looking for Big Data Developer having minimum experience of 4 year in big data ecosystem, passionate about technology, motivated for continuous learning and an individual who views every client interaction as an opportunity to create an exceptional customer...