Lead Big Data Engineer

4 days ago


Bengaluru, Karnataka, India TalentXO Full time

What You'll Be Doing : - Build highly scalable, available, fault-tolerant distributed data processing systems (batch and streaming systems) processing over 100s of terabytes of data ingested every day and petabyte-sized data warehouse and elasticsearch cluster. - Build quality data solutions and refine existing diverse datasets to simplified models encouraging self-service. - Build data pipelines that optimize on data quality and are resilient to poor-quality data sources. - Own the data mapping, business logic, transformations, and data quality. - Low-level systems debugging, performance measurement & optimization on large production clusters. - Participate in architecture discussions, influence product roadmap, and take ownership and responsibility over new projects. - Maintain and support existing platforms and evolve to newer technology stacks and architectures. Ideal Candidate : - Proficiency in Python and PySpark. - Deep understanding of Apache Spark, Spark tuning, creating RDDs, and building data frames. - Experience in big data technologies like HDFS, YARN, Map-Reduce, Hive, Kafka, Spark, Airflow, Presto, etc. - Experience in building distributed environments using any of Kafka, Spark, Hive, Hadoop, etc. - Good understanding of the architecture and functioning of distributed database systems. - Experience working with various file formats like Parquet, Avro, etc., for large volumes of data. - Experience with one or more NoSQL databases. - Experience with AWS, GCP. - 5 years of professional experience as a data or software engineer. (ref:hirist.tech)



  • Bengaluru, Karnataka, India Andor Tech Full time

    Andor Tech is Hiring: Lead Big Data Engineer Andor Tech is looking for a highly skilled Lead Big Data Engineer to join our team for deployment to a Retail MNC client in Bangalore . Job Details: Role: Lead Big Data Engineer Location: Bangalore Experience: Minimum 5+ Years Relevant Skills Experience: Scala: 3-4 Years SQL & PySpark: 3-4 Years Employment Type:...


  • Bengaluru, Karnataka, India Andor Tech Full time

    Andor Tech is Hiring: Lead Big Data EngineerAndor Tech is looking for a highly skilled Lead Big Data Engineer to join our team for deployment to a Retail MNC client in Bangalore .Job Details:Role: Lead Big Data EngineerLocation: BangaloreExperience: Minimum 5+ YearsRelevant Skills Experience:Scala: 3-4 YearsSQL & PySpark: 3-4 YearsEmployment Type: ...


  • Bengaluru, Karnataka, India Andor Tech Full time

    Andor Tech is Hiring: Lead Big Data Engineer Andor Tech is looking for a highly skilled Lead Big Data Engineer to join our team for deployment to a Retail MNC client in Bangalore.Job Details: Role: Lead Big Data Engineer Location: Bangalore Experience: Minimum 5+ Years Relevant Skills Experience:Scala: 3-4 YearsSQL & PySpark: 3-4 Years Employment...


  • Bengaluru, Karnataka, India Talent Worx Full time

    Job Description:PYSPARK ARCHITECT AND MANAGERA highly skilled PySpark Architect and Manager is required to lead our data engineering initiatives. The ideal candidate will have deep expertise in PySpark, Hadoop, AWS, Teradata, Scala, and data visualization tools. You will be responsible for designing, building, and managing large-scale data processing...


  • Bengaluru, Karnataka, India EXL Service Full time

    About the PositionWe are seeking a highly experienced Senior Manager to lead our Data Engineering team at EXL Service. The ideal candidate will have a strong background in data warehousing, big data engineering, and distributed systems.Main ResponsibilitiesData Warehousing: Design and develop data warehouses using various data warehousing concepts.Big Data...

  • Big Data Engineer

    4 weeks ago


    Bengaluru, Karnataka, India SMARTWORK IT SERVICES Full time

    Job Title : Big Data Engineer (Dynatrace)Location : Chennai, Bangalore, HyderabadExperience : 6-10 YearsRequired skills : Big Data, DynatraceJob Description : We are looking for an experienced Big Data Engineer (Dynatrace) with building scalable data pipelines and ensuring performance optimization through Dynatrace monitoring. This role requires a seasoned...


  • Bengaluru, Karnataka, India Epam Full time

    Description EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that...


  • Bengaluru, Karnataka, India Yogy HR Solutions Full time

    About the RoleYogy HR Solutions is seeking a highly skilled Lead Big Data Engineer to join our team. In this role, you will be responsible for designing, developing, and testing big data solutions using Spark, Scala, and SQL.">Job DescriptionWe are looking for an experienced big data developer with hands-on experience in Spark, Scala, and SQL. The ideal...

  • Big Data Engineer

    4 days ago


    Bengaluru, Karnataka, India Scoop Technologies Pvt Ltd Full time

    Job Title : Big Data Engineer Location : Bangalore Employment Type : Full-time Work Mode : 4 Days Work from Office Notice Period : Immediate to 30 days Job Description : We are looking for a Big Data Engineer with expertise in Hadoop, Hive, SQL, Python, PySpark, and AWS to join our team in Bangalore. The ideal candidate will have hands-on experience in...


  • Bengaluru, Karnataka, India Skilltasy Full time

    Job Description: We are seeking a highly skilled Big Data Analytics Engineer to join our team. As a Big Data Analytics Engineer, you will design, develop, and implement big data analytics solutions using various tools and technologies. Main Responsibilities:Design and develop big data analytics solutions using various tools and technologies.Analyze large...

  • Big Data Engineer

    3 weeks ago


    Bengaluru, Karnataka, India Infogain Full time

    Big Data Engineer (Lead) with skills Data Engineering, Data Warehouse, Big Data, snowflake, Azure Datalake for location Any Infogain Base Location (Noida, Gurugram, Bangalore, Mumbai, Pune) Posted on: July 30, Share on Linkedin Share on Twitter Share on Facebook ROLES & RESPONSIBILITIES Responsibilities: · Strong experience of minimum 5+...

  • Big Data Engineer

    7 days ago


    Bengaluru, Karnataka, India Scoop Technologies Pvt Ltd Full time

    Job Title : Big Data EngineerLocation : BangaloreEmployment Type : Full-timeWork Mode : 4 Days Work from OfficeNotice Period : Immediate to 30 daysJob Description :We are looking for a Big Data Engineer with expertise in Hadoop, Hive, SQL, Python, PySpark, and AWS to join our team in Bangalore. The ideal candidate will have hands-on experience in designing,...

  • Big Data Engineer

    3 days ago


    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Welcome to Tata Consultancy Services, where we are driven by innovation and excellence. We are seeking an experienced Big Data Engineer to join our team.As a Big Data Engineer, you will work on designing and implementing scalable ETL/ELT processes and modeling data for low latency reporting. You will also work with large cloud data lakes and have experience...

  • Big Data Engineer

    3 weeks ago


    Bengaluru, Karnataka, India Human Horizon Full time

    Responsibilities :- Design, develop, and execute test cases for large-scale big data platforms.- Perform testing on various Big Data technologies (e. g., Hadoop, Hive, Spark, NoSQL databases).- Ensure data accuracy, integrity, and quality in data processing and transformations.- Work closely with developers, data engineers, and business analysts to...

  • Big Data Engineer

    2 weeks ago


    Bengaluru, Karnataka, India Human Horizon Full time

    Responsibilities :- Design, develop, and execute test cases for large-scale big data platforms.- Perform testing on various Big Data technologies (e. g., Hadoop, Hive, Spark, NoSQL databases).- Ensure data accuracy, integrity, and quality in data processing and transformations.- Work closely with developers, data engineers, and business analysts to...

  • Big Data Engineer

    3 days ago


    Bengaluru, Karnataka, India Vishanz Business Services Private Limited Full time

    Vishanz Business Services Private Limited is seeking an experienced Big Data Engineer to design and develop scalable data solutions. As a Data Engineer, you will work on creating data pipelines and ETL processes that handle large volumes of data.Key TasksDesign and develop big data solutions using Microsoft Azure services.Collaborate with data scientists and...

  • Big Data Engineer

    3 weeks ago


    Bengaluru, Karnataka, India Telescopp Full time

    Job DescriptionJob OverviewWe are looking for a talented Big Data Engineer to join our team in Bengaluru. This is a mid-level, full-time position requiring a minimum of 4 years to a maximum of 6 years of relevant work experience. As a Big Data Engineer at Telescopp, you will be responsible for developing, testing, and maintaining big data solutions to...

  • Big Data Engineer Lead

    16 hours ago


    Bengaluru, Karnataka, India Amazon Full time

    Key Responsibilities: As a Data Engineer on our DISCO team, you will design and develop big data pipelines, build data models, and deploy data sets using AWS technologies like Redshift, S3, EMR, EC2, DynamoDB, Kinesis Firehose, and Lambda. You will also partner with Product, Marketing, BI, and ML teams to build new behavioral events, pipelines, datasets,...

  • Big Data Engineer

    3 weeks ago


    Bengaluru, Karnataka, India Telescopp Full time

    Job OverviewWe are looking for a talented Big Data Engineer to join our team in Bengaluru. This is a mid-level, full-time position requiring a minimum of 4 years to a maximum of 6 years of relevant work experience. As a Big Data Engineer at Telescopp, you will be responsible for developing, testing, and maintaining big data solutions to support our client's...

  • Big Data Engineer

    3 weeks ago


    Bengaluru, Karnataka, India Adobe Full time

    Our Company Changing the world through digital experiences is what Adobe's all about. We give everyone—from emerging artists to global brands—everything they need to design and deliver exceptional digital experiences We're passionate about empowering people to create beautiful and powerful images, videos, and apps, and transform how companies...