Data Engineer: Scalable Pipeline Expert

2 weeks ago


Bengaluru, Karnataka, India Idexcel Technologies Full time
Job Title: Data Engineer

We are seeking an experienced Data Engineer with expertise in designing, developing and managing scalable data pipelines and streaming solutions.

$120,000 - $180,000 per annum

Mandatory Skills:
  • Apache Spark: Strong expertise in Spark-based data processing.
  • Python: Proficiency in developing and maintaining data pipelines.
  • Databases & SQL: Hands-on experience in working with relational databases and writing optimized SQL queries.
  • GCP Components: Experience with GCP tools like BigQuery, DataProc, Airflow, or Cloud Composer (at least one project).
Responsibilities:
  1. Design, develop and optimize scalable data pipelines using Apache Spark and Python.
  2. Implement workflows and orchestration using Airflow.
  3. Leverage GCP tools to process and analyze large-scale datasets.
  4. Write and optimize complex SQL queries for data extraction and transformation.
  5. Work with real-time data streaming solutions like Kafka or Spark Streaming (if required).
Requirements:
  • Minimum 6+ years of experience in relevant technologies.
Location:

Open to remote or onsite roles depending on the project.


  • Data Engineer

    4 weeks ago


    Bengaluru, Karnataka, India Awign Expert Full time

    We are seeking a skilled Data Engineer to join our Data Science team at Awign Expert, India's largest work-as-a-service platform.About Us:Awign Expert is a division of Awign that connects skilled professionals with exciting project-based opportunities from top companies. Our mission is to empower professionals to focus on their work by managing...


  • Bengaluru, Karnataka, India NTT DATA Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team in Bangalore, India. This role involves designing, building, and maintaining scalable data pipelines using Databricks on AWS.The ideal candidate will have experience with Databricks, including Apache Spark development, and proficiency in Python or Scala for Spark job development....

  • Data Engineer

    1 month ago


    Bengaluru, Karnataka, India Nexthire Full time

    Role SummaryWe are seeking an experienced Data Engineer to design, develop, and maintain scalable and robust data pipelines that support various data processing and analytics needs.This role involves collaborating with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet their needs.About the...


  • Bengaluru, Karnataka, India Hexaware Technologies Full time

    We are seeking a skilled Data Engineer to join our team at Hexaware Technologies. This role will focus on designing and developing scalable data pipelines using Databricks and Apache Spark. The ideal candidate should have 5+ years of experience in data engineering and be proficient in integrating various data sources, including databases, APIs, and cloud...


  • Bengaluru, Karnataka, India Acrocede Technologies Private Limited Full time

    Acrocede Technologies Private Limited is seeking a highly skilled and experienced Data Engineer to join its dynamic team.The ideal candidate will have extensive experience working with Hadoop platforms, expertise in Spark, Python, SQL, and knowledge of cloud-based solutions using AWS services such as S3, Glue, and Lambda.This role requires the ability to...


  • Bengaluru, Karnataka, India HEVO DATA Full time

    Unlock Career Growth as a Data Engineering LeadWe're seeking an exceptional Data Engineering Lead to join our team at Hevo Data. In this role, you'll have the opportunity to own and build scalable platform services that support thousands of customers' data pipelines and ensure load data reliability to cloud warehouses.The Hevo platform core services will...

  • Data Engineer

    1 month ago


    Bengaluru, Karnataka, India Awign Expert Full time

    About Awign ExpertA division of Awign, India's largest work-as-a-service platform, connecting skilled professionals with exciting project-based opportunities from top companies, handling onboarding, feedback, conflict resolution, and payroll.Job SummaryWe are seeking a highly skilled Data Engineer to join our Data Science team. As a Data Engineer, you will...


  • Bengaluru, Karnataka, India WEN Full time

    Big Data Developer Opportunity at WENWe are seeking a highly skilled Big Data Developer to join our dynamic team at WEN. The ideal candidate will have hands-on experience in developing and optimizing data processing pipelines and possess strong expertise in big data technologies.Job Description OverviewIn this role, you will be responsible for designing,...


  • Bengaluru, Karnataka, India Makonis Full time

    About MakonisMakonis is a leading technology firm that specializes in delivering innovative solutions to its clients. We are currently seeking a highly skilled Data Engineer to join our team.Salary InformationThe estimated annual salary for this position is ₹35,00,000 - ₹50,00,000 depending on experience and qualifications.Job DescriptionWe are looking...


  • Bengaluru, Karnataka, India Awign Expert Full time

    About the RoleWe are seeking an experienced Technical Software Engineer to join our team as a Microservices Expert at Awign Expert. This is a great opportunity to work with cutting-edge technologies and contribute to the development of efficient, scalable applications.As a key member of our engineering team, you will design, develop, and maintain...


  • Bengaluru, Karnataka, India TMRW House of Brands Full time

    About TMRW House of BrandsWe're a dynamic company pushing the boundaries in the data engineering space. As a Data Engineer at TMRW House of Brands, you'll play a pivotal role in building and maintaining our robust data infrastructure.Salary RangeThe estimated salary for this position is between $120,000 to $180,000 per year, depending on your level of...


  • Bengaluru, Karnataka, India McLaren Strategic Solutions (MSS) Full time

    Unlock Your Potential as a Data Engineer with McLaren Strategic SolutionsAre you passionate about building efficient, large-scale data pipelines using distributed technologies like Hadoop and Spark? Do you have strong software engineering experience in Python, Scala, Java or equivalent?We are seeking an experienced Data Engineer to join our team at McLaren...

  • Data Engineer Expert

    2 weeks ago


    Bengaluru, Karnataka, India Nuviso Networks Full time

    Nuviso Networks is seeking an experienced Data Engineer Expert to join our team. This role involves designing, developing, and maintaining scalable data pipelines using Azure Databricks. The ideal candidate will have extensive experience with Azure Databricks and strong proficiency in SQL for data manipulation and querying.


  • Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Job SummaryWe are seeking a seasoned Cloud Engineer with extensive experience in designing and building scalable data pipelines on the AWS platform. This is an exciting opportunity to join Tata Consultancy Services and contribute to our mission of delivering innovative solutions to our clients.About the RoleThis position requires a professional with 5-10...


  • Bengaluru, Karnataka, India HEVO DATA Full time

    About Hevo DataWe empower businesses to drive growth by consolidating data from multiple sources, enabling faster analytics and informed decision making.Mission and VisionOur mission is to build technology that's simple to adopt and easy to access. We aim to unlock the potential of data for everyone.A Day in the Life of a Data Engineer at HevoYou'll work on...

  • Data Engineer Expert

    1 month ago


    Bengaluru, Karnataka, India HyrHub Full time

    Data Engineer ExpertHyrHub is seeking a highly skilled Data Engineer Expert to join our team.We are looking for an exceptional professional with expertise in managing model operations on the Dataiku platform, alongside a strong background in data engineering practices.In this role, you will be responsible for building and maintaining robust data pipelines,...

  • Data Engineering Expert

    42 minutes ago


    Bengaluru, Karnataka, India Eximietas Design Full time

    Eximietas Design is seeking a highly skilled Data Engineer to join our team. The ideal candidate will have over 3 years of experience in designing, building, and maintaining robust data pipelines using Java and Python.The successful candidate will work with a wide range of data technologies including Kafka, S3, CDC (Change Data Capture), Spark, and SQL to...


  • Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Tata Consultancy Services is seeking a skilled Data Engineering Expert to join our team. As a key member of our data engineering group, you will play a vital role in designing and building scalable data pipelines and architectures on the AWS platform.About the Role:This is an exciting opportunity for a motivated and detail-oriented individual who can work...


  • Bengaluru, Karnataka, India Tata Consultancy Services Full time

    At Tata Consultancy Services, we are seeking a highly skilled AWS Data Engineer to join our team. This is an exciting opportunity to work with cutting-edge technologies and contribute to the development of innovative data engineering solutions.Job Summary:We are looking for an experienced AWS Data Engineer who can design and build scalable data pipelines on...


  • Bengaluru, Karnataka, India Eximietas Design Full time

    About Eximietas: a technology services and solutions company headquartered in San Jose, CA with a global footprint that extends to Bangalore, Chennai, and Bhubaneswar in India. Specializing in Cloud Computing, Cybersecurity, VLSI, Embedded Software, and Artificial Intelligence, Eximietas is dedicated to empowering businesses with cutting-edge innovations,...