Big Data Engineer – Spark, Kafka

3 days ago


Bengaluru, Karnataka, India LTIMindtree Full time

Job description:

Your Experience

5 years of experience in large scale realtime stream processing using Apache Spark with messaging infrastructure like KafkaPulsar

7 years of experience on Data Pipeline development ETL and processing of structured and unstructured data

5 years of experience using NoSQL systems like MongoDB DynamoDB and SnowFlake Athena

Experience with technologies like EMR S3 Lambda AWS Fargate CloudWatch DataDog

Experience owning and implementing technicaldata solutions or pipelines

Excellent written and verbal communication skills in English

Strong work ethic and entrepreneurial spirit

Additional Information Main skills set that are a must are

1 Spark with Scala or Python

2 Apache Kafka

3Snowflake and NoSQL DB experience like MongoDB

4 Experience in AWS cloud services

5 Solid understanding of Medallion architecture Parquet Iceberg

Behavioral Competences

Excellent communication ability to communicate on different levels of seniority

Team player enjoying being part of a crossfunctional setup

Ability to perform well on timecritical endeavors and on multiple fronts at the time

Strong dedication to quality and a client mindset

A passion for learning and continuous improvement

Skills:

Mandatory Skills : Apache Spark,Python,Big Data Hadoop Ecosystem,Data Architecture


  • Big Data Engineer

    6 days ago


    Bengaluru, Karnataka, India Techno Comp Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    JOB RESPONSIBILITY· Experience with Big Data technologies will be a plus (Hadoop, Spark, Kafka, HBase, etc)· Write SQL queries to validate the dashboard output· Working experience with database environment - understanding relational database structure and hands-on SQL knowledge to extract/manipulate data for variance testing.· Performing code reviews and...


  • Bengaluru, Karnataka, India POWER IT SERVICES Full time ₹ 6,00,000 - ₹ 12,00,000 per year

    Big Data Engineer – Spark & AnalyticsLocation- ( Bangalore, Maharashtra)Job Description•Gather operational intel on business processes and policies from multiple sources•Prepare periodical and ad-hoc reports using operational data•Develop semantic core to align data with business processes•Support operations team's work streams for data processing,...

  • Big Data Developer

    3 days ago


    Bengaluru, Karnataka, India LION AND ELEPHANTS Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Description:We are looking for skilled Big Data Engineers using Java Spark with 5-10 years of experience in Big Data / legacy platforms, who can join immediately. We have openings in Bengaluru, Chennai, Hyderabad, Pune, Delhi.Desired candidates should have design, development and optimization of real-time & batch data pipelines experience in a Big Data...


  • Bengaluru, Karnataka, India Innova ESI Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Role: Big Data Engineer – AzureExperience: 10+ YearsLocation: BangaloreImmediate Joiners OnlyBig Data :Mandatory Skills:Azure Databricks, Azure Data Factory, Azure Function apps, Apache Spark, Scala, Java, Apache Kafka, event stream and Big DataOptional Skills: Airflow, PythonRoles & ResponsibilitiesOverall, experience in IT industry, including 5 years in...


  • Bengaluru, Karnataka, India Innova ESI Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Role: Big Data + Azure EngineerLocation: BangaloreExperience: 12+ yearsNotice: ImmediateJob Description:Mandatory Skills:Azure Databricks, Azure Data Factory, Azure Function apps, Apache Spark, Scala, Java, Apache Kafka, event stream and Big DataOptional Skills: Airflow, PythonRoles & ResponsibilitiesOverall, 12 years of experience in IT industry, including...


  • Bengaluru, Karnataka, India Brillio Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Looking for - Hive, SQL, Python, Spark, Hadoop and kafka with Iceberg experienceBachelor's or Master's degree in Computer Science, Information Systems, or a related field.Design, develop, and maintain data pipelines and ETL processes using Python, Hive, Spark, and other relevant technologies.Good Hands on Exp with javaMinimum of 6+ years of experience as a...

  • Big Data Engineer

    3 days ago


    Bengaluru, Karnataka, India The IT Firm Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Role OverviewAs a Big Data Engineer, you will be responsible for building and maintaining scalable data pipelines and architectures that support data analytics, reporting, and real-time processing. You will collaborate with data scientists, business analysts, and software engineers to design robust solutions that drive data-driven decisions.Mandatory Skill...

  • Big Data Engineer

    2 weeks ago


    Bengaluru, Karnataka, India Deqode Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Profile: Big Data Engineer (System Design)Experience:5+ yearsLocation:BangaloreWork Mode:HybridAbout The RoleWe're looking for an experiencedBig Data Engineerwith system design expertise to architect and build scalable data pipelines and optimize big data solutions.Key ResponsibilitiesDesign, develop, and maintain data pipelines and ETL processes using...

  • Kafka Developer

    2 days ago


    Bengaluru, Karnataka, India Appit Software Solutions Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Kafka Developer Experienced Associate (36 Years)Our Analytics & Insights Managed Services team brings a unique combination of industry expertise, technology, data management, and managedservices experience to create sustained outcomes for our clients and improve business performance. We empower companies to transform their approach to analytics and insights...

  • Big Data Engineer

    7 days ago


    Bengaluru, Karnataka, India, Karnataka Tata Consultancy Services Full time

    Job Title :- Data Engineer - PysparkExperience: 5 to 8 YearsLocation: Pune/HyderabadJob DescriptionRequired Skills:5+ years of experience in Big data and pysparkMust-HaveGood work experience on Big Data Platforms like Hadoop, Spark, Scala, Hive, Impala, SQLGood-to-HaveGood Spark, Pyspark,Big Data experienceSpark UI/Optimization/debugging techniquesGood...