Big Data Engineer – Spark, Kafka

2 weeks ago

Bengaluru, Karnataka, India LTIMindtree Full time

Job description:

Your Experience

5 years of experience in large scale realtime stream processing using Apache Spark with messaging infrastructure like KafkaPulsar

7 years of experience on Data Pipeline development ETL and processing of structured and unstructured data

5 years of experience using NoSQL systems like MongoDB DynamoDB and SnowFlake Athena

Experience with technologies like EMR S3 Lambda AWS Fargate CloudWatch DataDog

Experience owning and implementing technicaldata solutions or pipelines

Excellent written and verbal communication skills in English

Strong work ethic and entrepreneurial spirit

Additional Information Main skills set that are a must are

1 Spark with Scala or Python

2 Apache Kafka

3Snowflake and NoSQL DB experience like MongoDB

4 Experience in AWS cloud services

5 Solid understanding of Medallion architecture Parquet Iceberg

Behavioral Competences

Excellent communication ability to communicate on different levels of seniority

Team player enjoying being part of a crossfunctional setup

Ability to perform well on timecritical endeavors and on multiple fronts at the time

Strong dedication to quality and a client mindset

A passion for learning and continuous improvement

Skills:

Mandatory Skills : Apache Spark,Python,Big Data Hadoop Ecosystem,Data Architecture

  • Big Data Engineer

    6 days ago

    Bengaluru, Karnataka, India Brillio Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Key skills - Hive, SQL, Python, Spark, Hadoop and kafka , apache airflowJob Title Big Data EngineerRole OverviewWe are seeking a highly skilled Big Data Engineer to join our team. The ideal candidate will have strong experience in building, maintaining, and optimizing large-scale data pipelines and distributed data processing systems. This role involves...

  • Scala/Spark Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India JP Techno Park Full time ₹ 25,00,000 - ₹ 30,00,000 per year

    Big Data Orchestration: Airflow, Spark on Kubernetes, Yarn, Oozie Big DataProcessing: Hadoop, Kafka, Spark & Spark Structured Streaming. Experienceon SOLID & DRY principles with Good Software Architecture & Designimplementation experience Advanced Scala experience (e.g. FunctionalProgramming, using Case classes, Complex Data Structures &...

  • Big Data Engineer

    12 hours ago

    Bengaluru, Karnataka, India Talent Worx Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Location – Pune/ChennaiExp - 6 to 9RequirementsTechnology: - Big Data (Hadoop, Hive, Spark, Scala, PLSQL/ No SQL Database)Knowledge/Experience (Must):· 6-9 years of experience in Hadoop/big data technologies.· Experience with Spark/Storm/Kafka or equivalent streaming/batch processing and eventbased messaging.· Relational and NoSQL database integration...

  • Big Data Engineer

    4 days ago

    Bengaluru, Karnataka, India Innova ESI Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Role: Big Data EngineerExperience: 7+ YearsLocation: BangaloreNotice: Immediate Joiners OnlyJob Description:Azure Databricks, Azure Data Factory, Azure Function apps, Apache Spark, Scala, Java, Apache Kafka, event stream and Big DataOptional Skills: Airflow, PythonRoles & ResponsibilitiesOverall, 12 years of experience in IT industry, including 5 years in...

  • Big Data + Azure Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Innova ESI Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Role: Big Data + Azure EngineerLocation: BangaloreExperience: 12+ yearsNotice: ImmediateJob Description:Mandatory Skills:Azure Databricks, Azure Data Factory, Azure Function apps, Apache Spark, Scala, Java, Apache Kafka, event stream and Big DataOptional Skills: Airflow, PythonRoles & ResponsibilitiesOverall, 12 years of experience in IT industry, including...

  • Big Data Developer

    1 week ago

    Bengaluru, Karnataka, India Xebia Full time ₹ 5,00,000 - ₹ 25,00,000 per year

    Job Title : Big Data Developer - Spark/ScalaJob location : BengaluruNotice Period : immediate - 15 daysPosition Overview : We are looking for Scala/Spark developers with strong expertise on Python / Hadoop and Azure cloud ..Big Data Orchestration: Airflow, Spark on Kubernetes, Yarn, OozieBig Data Processing: Hadoop, Kafka, Spark & Spark Structured...

  • Big Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India 3K Softech Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Deep Hadoop ecosystem knowledge: Spark, HDFS, Hive, IcebergBatch and streaming ETL pipeline designscalability and reliabilityPython & Unix proficiency, materialized views, data structures/algorithmsData warehouse security, compliance, and operational excellenceAutomated pipeline design & optimization, platform collaborationMonitoring and logging: Prometheus,...

  • Big Data Engineer

    2 days ago

    Bengaluru, Karnataka, India Deqode Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Profile: Big Data Engineer (System Design)Experience:5+ yearsLocation:BangaloreWork Mode:HybridAbout The RoleWe're looking for an experiencedBig Data Engineerwith system design expertise to architect and build scalable data pipelines and optimize big data solutions.Key ResponsibilitiesDesign, develop, and maintain data pipelines and ETL processes using...

  • Big Data Developer

    6 days ago

    Bengaluru, Karnataka, India Infobell IT Full time ₹ 1,20,000 - ₹ 1,80,000 per year

    Job Title:Big Data DeveloperLocation:Bangalore (Work from Office)Experience:5+ YearsNotice Period:Immediate Joiners PreferredAbout Us:Infobell IT is a technology-driven company specializing inproduct engineering,AI,cloud modernization, anddata analytics. We work with leading enterprises to build scalable, high-performance, and cloud-native solutions.Key...

  • Big Data

    6 days ago

    Bengaluru, Karnataka, India Cognizant Technology Solutions Full time ₹ 1,20,00,000 - ₹ 2,40,00,000 per year

    Job SummaryLocation: AIA-ChennaiWe are seeking Senior Data Engineer resources to work on the migration of applications from our legacy Cloudera environment to the new Kubernetes-based data platform. The role requires strong hands-on development skills & Testing in data engineering, with the ability to deliver high-quality pipelines under guidance from...