Big Data Engineer
4 hours ago
Company:Indian / Global Digital Organization
Key Skills:
Pyspark, AWS, Python, SCALA, ETL
Roles and Responsibilities:
- Develop and deploy ETL and data warehousing solutions using Python libraries and Linux bash scripts on AWS EC2, with data stored in Redshift.
- Collaborate with product and analytics teams to scope business needs, design metrics, and build reports/dashboards.
- Automate and optimize existing data sets and ETL pipelines for efficiency and reliability.
- Work with multi-terabyte data sets and write complex SQL queries to support analytics.
- Design and implement ETL solutions integrating multiple data sources using Pentaho.
- Utilize Linux/Unix scripting for data processing tasks.
- Leverage AWS services (Redshift, S3, EC2) for storage, processing, and pipeline automation.
- Follow software engineering best practices for coding standards, code reviews, source control, testing, and operations.
Skills Required:
Must-Have:
- Hands-on experience with PySpark for big data processing
- Strong knowledge of AWS services (Redshift, S3, EC2)
- Proficiency in Python for data processing and automation
- Strong SQL skills for working with RDBMS and multi-terabyte data sets
Nice-to-Have:
- Experience with SCALA for distributed data processing
- Knowledge of ETL tools such as Pentaho
- Familiarity with Linux/Unix scripting for data operations
- Exposure to data modeling, pipelines, and visualization
Education:
Bachelor's degree in Computer Science, Information Technology, or a related field
-
GCP Big Data Engineer
6 days ago
Gurgaon, Haryana, India Acme Services Full time ₹ 6,00,000 - ₹ 12,00,000 per yearWe are looking for GCP Big Data Engineer who has skill set with Big Query and Airflow
-
Big Data Developer
4 hours ago
Gurgaon, Haryana, India Viraaj HR Solutions Private Limited Full time ₹ 20,00,000 - ₹ 25,00,000 per yearAbout The OpportunityA staffing and HR services firm focused on placing talent into Enterprise IT, Analytics, and Big Data engineering projects across Banking, Retail, Telecom, and SaaS sectors. We partner with product and services organisations to deliver scalable data platforms, real-time analytics, and high-throughput ETL solutions.We are hiring an...
-
Big Data Engineer
10 hours ago
Gurgaon, Haryana, India Qcentrio Full time ₹ 15,00,000 - ₹ 28,00,000 per yearNotice Period : Immediate - 30 days Mandatory Skills : Big Data, Python, SQL, Spark/Pyspark, AWS Cloud JD and required Skills & Responsibilities : Actively participate in all phases of the software development lifecycle, including requirements gathering, functional and technical design, development, testing, roll-out, and support. Solve complex...
-
Engineer - Big Data
2 weeks ago
Gurgaon, Haryana, India American Express Full time ₹ 6,00,000 - ₹ 18,00,000 per yearAt American Express, our culture is built on a 175-year history of innovation, shared values and Leadership Behaviors, and an unwavering commitment to back our customers, communities, and colleagues. As part of Team Amex, you'll experience this powerful backing with comprehensive support for your holistic well-being and many opportunities to learn new...
-
Engineer - Big Data
2 weeks ago
Gurgaon, Haryana, India American Express Full time ₹ 12,00,000 - ₹ 36,00,000 per yearAt American Express, our culture is built on a 175-year history of innovation, shared values and Leadership Behaviors, and an unwavering commitment to back our customers, communities, and colleagues. As part of Team Amex, you'll experience this powerful backing with comprehensive support for your holistic well-being and many opportunities to learn new...
-
Engineer - Data Management / Big Data
2 hours ago
Gurgaon, Haryana, India American Express Full time ₹ 9,00,000 - ₹ 12,00,000 per yearAt American Express, our culture is built on a 175-year history of innovation, shared values and Leadership Behaviors, and an unwavering commitment to back our customers, communities, and colleagues. As part of Team Amex, you'll experience this powerful backing with comprehensive support for your holistic well-being and many opportunities to learn new...
-
Big Data Automation QA
6 hours ago
Gurgaon, Haryana, India Xcel HR Solutions Full time ₹ 20,00,000 - ₹ 25,00,000 per yearResponsibilities: Develop and automate processes for gathering of expected results from data sources and comparing data results from testing.Assist with the development and maintenance of smoke, performance, functional, and regression tests to ensure code is functioning as designed.Work with the team to understand how changes in the DL affect maintenance of...
-
Data Engineer
6 days ago
Gurgaon, Haryana, India CoPoint Data Full time ₹ 12,00,000 - ₹ 36,00,000 per yearSenior Data Engineer Location: India (Gurugram)About CoPointAIAI isn't coming — it's here. And we help enterprises make it real. At CoPointAI, we work inside the enterprise — not just around it — to turn AI potential into practical wins. From hands-on C-suite workshops (our AI Foundations series) to AI-native MVPs built in weeks, our team partners...
-
Senior Big Data Engineer
5 days ago
Gurgaon, Haryana, India dunnhumby Full time ₹ 8,00,000 - ₹ 13,00,000 per yeardunnhumby is the global leader in Customer Data Science, empowering businesses everywhere to compete and thrive in the modern data-driven economy. We always put the Customer First.Our mission: to enable businesses to grow and reimagine themselves by becoming advocates and champions for their Customers. With deep heritage and expertise in retail – one of...
-
Senior Big Data Engineer
5 days ago
Gurgaon, Haryana, India dunnhumby Full time ₹ 12,00,000 - ₹ 24,00,000 per yeardunnhumby is the global leader in Customer Data Science, empowering businesses everywhere to compete and thrive in the modern data-driven economy. We always put the Customer First.Our mission: to enable businesses to grow and reimagine themselves by becoming advocates and champions for their Customers. With deep heritage and expertise in retail – one of...