
Big Data Specialist
4 weeks ago
Role Overview
We are seeking a highly skilled Big Data Engineer to join our team. The ideal candidate will have strong experience in building, maintaining, and optimizing large-scale data pipelines and distributed data processing systems. This role involves working closely with cross-functional teams to ensure the reliability, scalability, and performance of data solutions.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines and ETL processes.
- Work with large datasets using Hadoop ecosystem tools (Hive, Spark).
- Build and optimize real-time and batch data processing solutions using Kafka and Spark Streaming.
- Write efficient, high-performance SQL queries to extract, transform, and load data.
- Develop reusable data frameworks and utilities in Python.
- Collaborate with data scientists, analysts, and product teams to deliver reliable data solutions.
- Monitor, troubleshoot, and optimize big data workflows for performance and cost efficiency.
Must-Have Skills
- Strong hands-on experience with Hive and SQL for querying and data transformation.
- Proficiency in Python for data manipulation and automation.
- Expertise in Apache Spark (batch and streaming).
- Experience working with Kafka for streaming data pipelines.
Good-to-Have Skills
- Experience with workflow orchestration tools (Airflow etc.)
- Knowledge of cloud-based big data platforms (AWS EMR, GCP Dataproc, Azure HDInsight).
- Familiarity with CI/CD pipelines and version control (Git).
-
Big Data Developer
1 week ago
Bengaluru, Karnataka, India LION AND ELEPHANTS Full time ₹ 20,00,000 - ₹ 25,00,000 per yearJob Description:We are looking for skilled Big Data Engineers using Java Spark with 5-10 years of experience in Big Data / legacy platforms, who can join immediately. We have openings in Bengaluru, Chennai, Hyderabad, Pune, Delhi.Desired candidates should have design, development and optimization of real-time & batch data pipelines experience in a Big Data...
-
Big Data Engineer
1 week ago
Bengaluru, Karnataka, India Reveille Technologies,Inc Full time ₹ 16,00,000 - ₹ 24,00,000 per yearWe're Hiring: Big Data Engineer – Full-TimeLooking for a skilledBig Data Engineerto join our team inBangaloreRole: Big Data EngineerGrade: G4 |TSRLocation: Bangalore (Onsite/Hybrid – specify if needed)Budget: ₹13,39,200Experience: 4–6 yearsSkills:Azure Databricks (ADB)PySparkSQLSpark (Scala/Python)TerraformAirflow & KubernetesData Warehousing &...
-
Big Data Engineer
1 week ago
Bengaluru, Karnataka, India The IT Firm Full time ₹ 15,00,000 - ₹ 25,00,000 per yearRole OverviewAs a Big Data Engineer, you will be responsible for building and maintaining scalable data pipelines and architectures that support data analytics, reporting, and real-time processing. You will collaborate with data scientists, business analysts, and software engineers to design robust solutions that drive data-driven decisions.Mandatory Skill...
-
Big Data Engineer
1 week ago
Bengaluru, Karnataka, India Fractal Analytics Private Limited Full time ₹ 6,00,000 - ₹ 18,00,000 per yearBigData Engineer It's fun to work in a company where people truly BELIEVE in what they are doing We're committed to bringing passion and customer focus to the business.About the Role:We are seeking highly skilled Big Data Engineers/Senior Engineers with a strong background in building and supporting big data solutions. The ideal candidate will have...
-
Big Data Engineer
1 week ago
Bengaluru, Karnataka, India Tata Consultancy Services Full time ₹ 15,00,000 - ₹ 25,00,000 per yearGreetings from TCS TCS is hiring for Big Data (PySpark & Scala)Location: - Pune/Mumbai/Chennai/BangaloreDesired Experience Range: 4 - 6 YearsMust-HavePySpark HiveGood-to-Have Spark HBase DQ tool Agile Scrum experience Exposure in data ingestion from disparate sources onto Big Data platformThanksAnshika
-
Big Data Developer
1 week ago
Bengaluru, Karnataka, India ITC Infotech Full time ₹ 5,00,000 - ₹ 15,00,000 per yearRequired Skills:Demonstrated ability in designing, building, refactoring and releasing software written in Java, Python.Strong experience with Big Data technologies including Hadoop, Spark, Flink, Hive, HDFS.Ability to handle data integration, bug fixing, performance optimization, and feature enhancements for distributed data pipelines.Having experience with...
-
Big Data Developer
1 week ago
Bengaluru, Karnataka, India Grid Dynamics Full time ₹ 15,00,000 - ₹ 25,00,000 per year7 to 15 Years of experience in Big Data Developer (Mandatory Splunk to ClickHouse and Grafana)Essential FunctionsKey Responsibilities7 to 15 Years of experience in Big Data DeveloperDesign and develop scalable data pipelines to migrate user knowledge objects from Splunk to ClickHouse and Grafana.Implement data ingestion, transformation, and validation...
-
Big Data Lead
1 week ago
Bengaluru, Karnataka, India Persistent Full time ₹ 20,00,000 - ₹ 25,00,000 per yearAbout PersistentWe are a trusted Digital Engineering and Enterprise Modernization partner, combining deep technical expertise and industry experience to help our clients anticipate what's next. Our offerings and proven solutions create a unique competitive advantage for our clients by giving them the power to see beyond and rise above. We work with many...
-
Big Data Developer
4 weeks ago
Bengaluru, Karnataka, India Tata Consultancy Services Full timePosition - Big Data DeveloperExperience: 4 to 10 YearsNotice Period- 0-60 DaysLocation-BengaluruRequired Skill Set_ Hadoop, PySpark, Spark SQL, HIVEJob Description- Hands-on experience of PySpark, Spark, Hive, Hadoop Big Data Eco System Tools.- Strong SQL experience.- Should be able to develop, tweak queries and work on performance enhancement.- Solid...
-
Senior Big Data Engineer
2 weeks ago
Bengaluru, Karnataka, India Allstate Insurance Full time ₹ 20,00,000 - ₹ 25,00,000 per yearAt Allstate, great things happen when our people work together to protect families and their belongings from life's uncertainties. And for more than 90 years our innovative drive has kept us a step ahead of our customers' evolving needs. From advocating for seat belts, air bags and graduated driving laws, to being an industry leader in pricing...