Senior Big Data Developer

1 month ago


Bengaluru, Karnataka, India Guidewire Software Full time

Guidewire Software is seeking a highly skilled Senior Big Data Developer to join its Data Platform and Analytics team. This team owns, builds, and operates the company's cloud-native data platform and analytics SaaS offerings.

The ideal candidate will have a strong background in building large-scale distributed data processing systems, with expertise in big data frameworks like Hadoop, Spark, Hive, and Kafka. Proficiency in data processing languages such as Python, Java, or Scala is also required.

The Senior Big Data Developer will be responsible for designing and developing scalable and high-performance data pipelines for collecting, processing, and storing large datasets. They will also work with cross-functional teams to understand and implement data requirements, and continuously evolve the data architecture to meet the future needs of the company.

Key Responsibilities:

  • Design and develop scalable and high-performance data pipelines for collecting, processing, and storing large datasets.
  • Work with Hadoop, Spark, Kafka, and other big data technologies to manage and process structured and unstructured data.
  • Optimize and troubleshoot data pipelines for speed and efficiency.
  • Collaborate with data scientists and analysts to integrate data-driven insights into business decision-making processes.
  • Implement data security measures and maintain data integrity across the systems.
  • Analyze and improve performance and scalability of data pipelines and storage solutions.
  • Monitor and maintain data workflows, ensuring consistency and reliability.
  • Work with cross-functional teams, including product, engineering, and operations, to understand and implement data requirements.
  • Continuously evolve the data architecture to meet the future needs of the company.

Required Skills and Qualifications:

  • Bachelor's or Master's degree in Computer Science, Engineering, or related field.
  • Minimum 6+ years of work experience.
  • Proven experience in building and managing large-scale distributed data processing systems.
  • Strong experience with big data frameworks like Hadoop, Spark, Hive, Kafka, etc.
  • Proficiency in data processing languages such as Python, Java, or Scala.
  • Hands-on experience with cloud-based big data solutions (AWS, GCP, or Azure).
  • Solid understanding of data modeling, ETL concepts, and data warehousing solutions.
  • Experience with data orchestration tools like Apache Airflow.
  • Familiarity with SQL and NoSQL databases.
  • Strong problem-solving skills and the ability to work in a fast-paced environment.

Preferred Skills:

  • Experience with real-time data processing and streaming (e.g., using Apache Flink, Kafka Streams).
  • Knowledge of containerization technologies like Docker and Kubernetes.

Soft Skills:

  • Excellent communication and collaboration skills.
  • Ability to work independently and as part of a team.
  • Strong analytical mindset and attention to detail.

Next Steps:

  1. Total Experience:
  2. Current CTC:
  3. Expected CTC:
  4. Notice Period:
  5. Current Location:
  6. Reason for Job Change:


  • Bengaluru, Karnataka, India Société Générale Assurances Full time

    Job Title: Senior Big Data Application DeveloperAbout the Role:As a Senior Big Data Application Developer at Société Générale Assurances, you will be responsible for designing, developing, and maintaining large-scale big data applications using Hadoop and Spark. You will work closely with cross-functional teams to deliver high-quality software solutions...


  • Bengaluru, Karnataka, India IntraEdge Full time

    We are seeking a highly skilled Senior Big Data Developer to lead our engineering team in designing, developing, and maintaining large-scale data processing systems. The successful candidate will work closely with cross-functional teams to architect scalable data pipelines and provide valuable insights through advanced data processing and analytics.Key...


  • Bengaluru, Karnataka, India Teva Pharmaceuticals Full time

    About the RoleWe're seeking a highly skilled Senior Big Data Developer to join our team at Teva Pharmaceuticals. As a key member of our data engineering team, you will be responsible for designing, developing, and deploying large-scale data processing systems using cutting-edge technologies like Azure and Python.Key ResponsibilitiesDesign and implement data...

  • Big Data Developer

    1 month ago


    Bengaluru, Karnataka, India ACL Digital Full time

    Job Title: Senior Big Data EngineerWe are seeking a highly skilled Senior Big Data Engineer to join our team at ACL Digital.Key Responsibilities:Create and maintain optimal data pipeline architectureAssemble large complex data sets that meet functional and non-functional business requirementsIdentify design and implement internal process improvements:...


  • Bengaluru, Karnataka, India Visionyle Solutions Full time

    Join Visionyle Solutions as a Senior Big Data Architect and take on the exciting challenge of designing and implementing cutting-edge big data solutions. Expected Salary: $120,000 - $150,000 per year.About the Role:In this role, you will be working with our globally distributed team to develop innovative Hadoop-based solutions that meet business needs and...


  • Bengaluru, Karnataka, India Grid Dynamics Full time

    Job Description:We are seeking a highly skilled Senior Big Data Engineer to join our team at Grid Dynamics. As a key member of our big data team, you will be responsible for designing, developing, and implementing large-scale data processing systems using Hadoop and Spring Boot.Responsibilities:- Design and develop scalable big data architectures using...


  • Bengaluru, Karnataka, India Senior .Net Developer Full time

    Data Warehouse ArchitectWe are seeking a highly skilled Data Warehouse Architect to join our team at Noesys. As a key member of our data team, you will be responsible for designing and implementing data warehouses that meet the needs of our business.About the Role:The Data Warehouse Architect will have a deep understanding of data modeling, SQL, OLAP, and...

  • Big Data Developer

    1 month ago


    Bengaluru, Karnataka, India Unison Consulting Pte Ltd Full time

    Job Title: Big Data DeveloperUnison Consulting Pte Ltd is seeking an experienced Big Data Developer to design, develop, and maintain big data solutions to enable data-driven decision making and analytics capabilities for our clients. As a Big Data Developer, you will work with large volumes of structured and unstructured data using cutting-edge technologies...


  • Bengaluru, Karnataka, India Senior .Net Developer Full time

    About the Role:We are seeking a highly skilled Data Warehouse Architect to join our team. This individual will be responsible for designing and implementing data warehousing systems, developing ETL processes, and maintaining database scripts.Key Responsibilities:Design and implement data warehousing systems to meet business requirements.Develop ETL processes...


  • Bengaluru, Karnataka, India Guidewire Software Full time

    Guidewire Software is seeking a highly skilled Senior Big Data Engineer to join its Data Platform and Analytics team. The ideal candidate will have a strong background in designing and developing scalable and high-performance data pipelines for collecting, processing, and storing large datasets.Key Responsibilities:Design and develop data pipelines using...

  • Big Data Developer

    3 weeks ago


    Bengaluru, Karnataka, India IBCScorp Full time

    Job Title: Big Data Developer About the Role: We are seeking a skilled Big Data Developer to join our team at IBCScorp. As a Big Data Developer, you will be responsible for designing, developing, and implementing data-driven solutions using Hadoop, Spark, and NoSQL databases. Key Responsibilities:Develop and maintain large-scale data processing systems...

  • Senior Data Engineer

    3 weeks ago


    Bengaluru, Karnataka, India Nielsen Full time

    We are seeking a highly skilled Senior Data Engineer (Big Data Analytics) to join our team at Nielsen. This role will focus on designing, building, and maintaining large-scale data processing systems using big data technologies such as Apache Spark.About the Role:Design and develop scalable, reliable, and cost-effective data processing systemsCollaborate...

  • Senior Data Engineer

    4 weeks ago


    Bengaluru, Karnataka, India wm staffing solutions Full time

    Job Description: We are seeking a talented Senior Data Engineer - Big Data Pipelines to join our team at WM Staffing Solutions. In this role, you will be responsible for designing, developing, and supporting data pipelines using GCP Big Query, PySpark, Python, and Composer/Airflow. Key responsibilities:• Design and develop data pipelines using GCP Big...

  • Big Data Developer

    3 weeks ago


    Bengaluru, Karnataka, India Globalemployees Full time

    About the Role:We are seeking a highly skilled Data Scientist with expertise in Big Data to join our team at Globalemployers.Key Responsibilities:Develop, test, and deploy big data applications and pipelines.Work with Hadoop, Spark, Kafka, and related big data technologies.Optimize data workflows for performance, scalability, and efficiency.Collaborate with...

  • Senior Data Engineer

    1 month ago


    Bengaluru, Karnataka, India HS Solutions Inc Full time

    About the OpportunityWe are seeking a highly skilled Senior Data Engineer to join our team at HS Solutions Inc. in Bengaluru. The ideal candidate will have 6-8 years of experience in developing scalable Big Data applications and solutions on distributed platforms.Key Responsibilities:Design and develop scalable Big Data applications and solutions on...


  • Bengaluru, Karnataka, India Mercedes-Benz Full time

    About the RoleWe are seeking a highly skilled Senior Data Engineer to join our team at Mercedes-Benz Research and Development India Private Limited.ResponsibilitiesDesign and develop scalable big data platforms for fleet data analytics.Collaborate with cross-functional teams to integrate data engineering solutions.Analyze and resolve complex data engineering...


  • Bengaluru, Karnataka, India Mindteck Full time

    We are seeking a highly skilled Senior Data Architect to lead our big data solutions. As a key member of our team, you will be responsible for designing and implementing scalable and efficient data architectures.Based in Bangalore, this role offers an estimated annual salary of 12-15 lakhs (approximately $16,000-$20,000 USD), commensurate with...


  • Bengaluru, Karnataka, India Roku Full time

    About the TeamRoku runs one of the largest data lakes in the world, storing over 70 petabytes of data and running over 10 million queries per month. The Big Data team is responsible for building, running, and supporting the platform that makes this possible.About the RoleRoku is modernizing its Big Data Platform to improve user experience, minimize costs,...


  • Bengaluru, Karnataka, India TapTalent Full time

    Job Title: Senior Big Data Engineer - Cloud ExpertAt TapTalent, we are seeking a highly skilled Senior Big Data Engineer with expertise in Cloud Computing to join our team. As a Senior Big Data Engineer, you will be responsible for designing and implementing scalable data architectures using Cloud technologies. Your primary focus will be on building and...


  • Bengaluru, Karnataka, India F335 Deutsche India Private Limited, Bangalore Branch Full time

    Job DescriptionSenior Big Data Engineer is required to develop and deliver engineering solutions to accomplish business goals at F335 Deutsche India Private Limited, Bangalore Branch. The successful candidate will be responsible for designing and coding stable and scalable data ingestion workflows, integrating into existing workflows, and working with team...