Staff Software Engineer, Spark, Open Source, Distributed Systems

7 days ago

Bengaluru, Karnataka, India Google Full time ₹ 12,00,000 - ₹ 24,00,000 per year
Minimum qualifications:
  • Bachelor's degree or equivalent practical experience.
  • 8 years of experience with one or more general purpose programming languages, including Java, C/C++ or Python.
  • 3 years of experience with software design and architecture.
  • 3 years of experience with open source or developer technologies.
  • Experience in software development and engineering, incorporating design methodologies, leveraging open source technologies, and working with distributed computing systems, including Apache Spark, Apache Hadoop, and Apache Hive, with performance.
Preferred qualifications:
  • Master's degree or PhD in Engineering, Computer Science, or a related technical field.
  • 8 years of experience with data structures/algorithms.
  • 3 years of experience in a technical leadership role leading project teams and setting technical direction.
  • Experience in query processing and database internals.
  • Experience with big data technologies.
  • Ability to work effectively across teams in a distributed environment.


About the job

Google's software engineers develop the next-generation technologies that change how billions of users connect, explore, and interact with information and one another. Our products need to handle information at massive scale, and extend well beyond web search. We're looking for engineers who bring fresh ideas from all areas, including information retrieval, distributed computing, large-scale system design, networking and data storage, security, artificial intelligence, natural language processing, UI design and mobile; the list goes on and is growing every day. As a software engineer, you will work on a specific project critical to Google's needs with opportunities to switch teams and projects as you and our fast-paced business grow and evolve. We need our engineers to be versatile, display leadership qualities and be enthusiastic to take on new problems across the full-stack as we continue to push technology forward.

With your technical expertise you will manage project priorities, deadlines, and deliverables. You will design, develop, test, deploy, maintain, and enhance software solutions.

Google Cloud accelerates every organization's ability to digitally transform its business and industry. We deliver enterprise-grade solutions that leverage Google's cutting-edge technology, and tools that help developers build more sustainably. Customers in more than 200 countries and territories turn to Google Cloud as their trusted partner to enable growth and solve their most critical business problems.



Responsibilities
  • Lead development of next-generation features, positioning Cloud Dataproc as the preferred platform for Spark, Flink, Trino, and emerging cloud technologies. Define the roadmap for enhancing open-source technologies like Spark, Hive, Trino, Iceberg, Hudi, and Delta into Dataproc.
  • Establish benchmarks to identify and resolve performance bottlenecks, ensuring large-scale Spark job certification. Enhance the performance and efficiency of open-source technologies within the platform.
  • Drive the design and implementation of Data Lakes and Lake Houses, including Apache Iceberg and Hudi. Develop and implement software solutions leveraging Google technologies for accelerated cluster setup, streamlined operations, and comprehensive monitoring.
  • Collaborate with Google Cloud Platform (GCP) infrastructure teams to bring the latest GCP features to Dataproc users. Position Cloud Dataproc as the optimal choice for data science, ML and AI workloads.

  • Staff Software Engineer, Spark, Open Source, Distributed Systems

    1 week ago

    Bengaluru, Karnataka, India Google Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Minimum qualifications:Bachelor's degree or equivalent practical experience.8 years of experience with one or more general purpose programming languages, including Java, C/C++ or Python.3 years of experience with software design and architecture.3 years of experience with open source or developer technologies.Experience in software development and...

  • Senior Engineer, Spark, Open Source, Benchmarks

    1 day ago

    Bengaluru, Karnataka, India Google Full time ₹ 10,000 - ₹ 50,000 per year

    Minimum qualifications:Bachelor's degree or equivalent practical experience.5 years of experience with software development in one or more programming languages.3 years of experience testing, maintaining, or launching software products, and 1 year of experience with software design and architecture.Preferred qualifications:Experience working with data...

  • Staff Software Engineer, Distributed Data Systems

    7 days ago

    Bengaluru, Karnataka, India Onehouse Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    About Onehouse Onehouse is a mission-driven company dedicated to freeing data from data platform lock-in. We deliver the industry's most interoperable data lakehouse through a cloud-native managed service built on Apache Hudi. Onehouse enables organizations to ingest data at scale with minute-level freshness, centrally store it, and make available to any...

  • Staff Engineer

    7 days ago

    Bengaluru, Karnataka, India Acceldata Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    At Acceldata, We are revolutionizing data observability in how enterprises manage and observe data by offering comprehensive solutions tailored to each organization's unique needs. Our platform integrates various technologies, enabling seamless data observability for modern enterprises. About the Role: We are looking for a Staff Engineer (Apache Open Source...

  • Staff Engineer

    7 days ago

    Bengaluru, Karnataka, India Acceldata Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    At Acceldata, We are revolutionizing data observability in how enterprises manage and observe data by offering comprehensive solutions tailored to each organization's unique needs. Our platform integrates various technologies, enabling seamless data observability for modern enterprises.About the Role:We are looking for aStaff Engineer (Apache Open Source...

  • Software Engineer, Batch Compute

    5 days ago

    Bengaluru, Karnataka, India Stripe Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Who we areAbout StripeStripe is a financial infrastructure platform for businesses. Millions of companies—from the world's largest enterprises to the most ambitious startups—use Stripe to accept payments, grow their revenue, and accelerate new business opportunities. Our mission is to increase the GDP of the internet, and we have a staggering amount of...

  • Staff Software Development Engineer

    1 day ago

    Bengaluru, Karnataka, India Razorpay Software Private Limited Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Razorpay was founded by Shashank Kumar and Harshil Mathur in 2014. Razorpay is building a new-age digital banking hub (Neobank) for businesses in India with the mission is to enable frictionless banking and payments experiences for businesses of all shapes and sizes. What started as a B2B payments company is processing billions of dollars of payments for...

  • Staff Software Engineer

    7 days ago

    Bengaluru, Karnataka, India Hevo Data Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    About Hevo Data: Unleash the power of data at Hevo Join our team of brilliant minds as we revolutionize the world of data connectivity. Our mission is simple: to empower organisations with seamless access to their data universe, enabling smarter decisions and unlocking their full potential. Are you ready to be part of the data revolution that is shaping the...

  • Software Engineer Staff

    7 days ago

    Bengaluru, Karnataka, India Hewlett Packard Enterprise Full time ₹ 8,00,000 - ₹ 12,00,000 per year

    Software Engineer StaffThis role has been designed as 'Hybrid' with an expectation that you will work on average 2 days per week from an HPE office Who We Are: Hewlett Packard Enterprise is the global edge-to-cloud company advancing the way people live and work. We help companies connect, protect, analyze, and act on their data and applications wherever they...

  • Staff Software Engineering, Distributed Cloud Hosted

    1 day ago

    Bengaluru, Karnataka, India Google Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Minimum qualifications:Bachelor's degree or equivalent practical experience.8 years of experience in software development.5 years of experience building and developing large-scale infrastructure, distributed systems or networks, or experience with compute technologies, storage, or hardware architecture.Experience in architecting and designing a distributed...