Big Data Engineer
10 hours ago
Key skills - Hive, SQL, Python, Spark, Hadoop and kafka , apache airflow
Job Title Big Data Engineer
Role Overview
We are seeking a highly skilled Big Data Engineer to join our team. The ideal candidate will have strong experience in building, maintaining, and optimizing large-scale data pipelines and distributed data processing systems. This role involves working closely with cross-functional teams to ensure the reliability, scalability, and performance of data solutions.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines and ETL processes.
- Work with large datasets using Hadoop ecosystem tools (Hive, Spark).
- Build and optimize real-time and batch data processing solutions using Kafka and Spark Streaming.
- Write efficient, high-performance SQL queries to extract, transform, and load data.
- Develop reusable data frameworks and utilities in Python.
- Collaborate with data scientists, analysts, and product teams to deliver reliable data solutions.
- Monitor, troubleshoot, and optimize big data workflows for performance and cost efficiency.
Must-Have Skills
- Strong hands-on experience with Hive and SQL for querying and data transformation.
- Proficiency in Python for data manipulation and automation.
- Expertise in Apache Spark (batch and streaming).
- Experience working with Kafka for streaming data pipelines.
Good-to-Have Skills
- Experience with workflow orchestration tools (Airflow etc.).
- Knowledge of cloud-based big data platforms (AWS EMR)
Familiarity with CI/CD pipelines and version control (Git).
-
Big Data Engineer
2 weeks ago
Bengaluru, Karnataka, India Etaclam Techno Services Full time ₹ 12,00,000 - ₹ 14,40,000 per yearJob Title: Big Data EngineerLocation: BengaluruExperience Required: 5 to 8 yearsJob Description:We are looking for a skilled Big Data Engineer with strong hands-on experience in PySpark and SQL. This is a hybrid role, requiring the candidate to work both remotely and from the office as needed. The ideal candidate should be comfortable working on large-scale...
-
Big Data Engineer
2 days ago
Bengaluru, Karnataka, India Acme Services Full time ₹ 5,00,000 - ₹ 15,00,000 per yearWe are looking for Big Data Engineer who has experience with:- 1) Big Query 2) Pyspark 3) GCP or AWS or Azure Required Candidate profileCandidate should have more than 5 Years of overall experience in Data Engineer. We are not looking for Data Analyst or Snowflake Developer.
-
Big Data Engineer
1 week ago
Bengaluru, Karnataka, India Akshya Patra Services Full time ₹ 5,00,000 - ₹ 25,00,000 per yearWe are looking for a skilled Big Data Engineer with expertise in various Big Data tools and ecosystems. The ideal candidate should have hands-on experience with technologies such as Hadoop, Spark, Hive, and Impala. A strong proficiency in programming languages like Scala, Java, and Python is essential for designing and implementing data processing...
-
Data Engineer
4 days ago
Bengaluru, Karnataka, India NTT DATA Full time ₹ 6,00,000 - ₹ 18,00,000 per yearAs a Data Engineer, your main responsibilities will involve: The project is a migration of a SAS code to DBT code, so it's important that the candidate has experience in this type of projects. Analyze data models and derive logical conclusions. Processes modeling. Hands on development and monitoring of the Azure cloud Platform and various associated...
-
Big Data Engineer
4 days ago
Bengaluru, Karnataka, India Adobe Full time ₹ 20,00,000 - ₹ 40,00,000 per yearOur CompanyChanging the world through digital experiences is what Adobe's all about. We give everyone—from emerging artists to global brands—everything they need to design and deliver exceptional digital experiences We're passionate about empowering people to create beautiful and powerful images, videos, and apps, and transform how companies interact...
-
Big Data Engineer
2 weeks ago
Bengaluru, Karnataka, India Saransh Software Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per yearJob Title: Data EngineerExperience: 8+ yearsLocation: BengaluruNotice Period: Immediate to 20 DaysEmployment Type: Full-timeAbout the RoleWe are looking for an experienced Data Engineer with strong expertise in Python, Spark, and PySpark to design, develop, and optimize large-scale data pipelines and distributed systems. The ideal candidate will possess a...
-
Big Data
2 weeks ago
Bengaluru, Karnataka, India PradeepIT Consulting Services Full time ₹ 12,00,000 - ₹ 36,00,000 per yearAbout the jobAccelerate your career with PradeepITPradeepIT is one of the largest, globally recognized IT Consulting firm to connect Indias deeply vetted talent team to global customer.Were headquartered in Bengaluru, Silicon Valley of India. PradeepITs customers include SAP Lab, Bosch, Rolls-Royce, Daikin, Daimler and J&J and hundreds of other Fortune 500...
-
Big Data Engineer – Azure
2 weeks ago
Bengaluru, Karnataka, India Innova ESI Full time ₹ 20,00,000 - ₹ 25,00,000 per yearRole: Big Data Engineer – AzureExperience: 10+ YearsLocation: BangaloreImmediate Joiners OnlyBig Data :Mandatory Skills:Azure Databricks, Azure Data Factory, Azure Function apps, Apache Spark, Scala, Java, Apache Kafka, event stream and Big DataOptional Skills: Airflow, PythonRoles & ResponsibilitiesOverall, experience in IT industry, including 5 years in...
-
Senior Big Data Engineer
2 weeks ago
Bengaluru, Karnataka, India SK HR SERVICES AND ENTERPRISES Full time ₹ 20,00,000 - ₹ 25,00,000 per yearService Mapping : Application Senior Developer Location : Whitefield, Bangalore Experience : 6 yearsRole Overview : Looking for a Senior Big Data Developer to design, build, and optimize data pipelines and analytics solutions. Candidates with prior exposure to the Ericsson Wallet Platform will be given preference.Key Responsibilities : -...
-
Senior Big Data Engineer
2 weeks ago
Bengaluru, Karnataka, India Allstate Insurance Full time ₹ 20,00,000 - ₹ 25,00,000 per yearAt Allstate, great things happen when our people work together to protect families and their belongings from life's uncertainties. And for more than 90 years our innovative drive has kept us a step ahead of our customers' evolving needs. From advocating for seat belts, air bags and graduated driving laws, to being an industry leader in pricing...