Staff Apache Spark Engineer

1 week ago

Bengaluru India BairesDev Full time

Job Description At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide. When you apply for this position, you're taking the first step in a process that goes beyond the ordinary. We aim to align your passions and skills with our vacancies, setting you on a path to exceptional career development and success. Staff Apache Spark Engineer at BairesDev We are seeking a Staff Apache Spark Engineer to work on distributed data processing systems and real-time analytics. You'll contribute to Apache Spark and related ecosystem projects (Kafka, Hive, Iceberg), focusing on performance optimization and scalable architecture. What You'll Do: - Contribute actively to Apache Spark and related open source projects as a core contributor. - Design and optimize distributed data processing engines for batch and real-time analytical workloads. - Develop streaming analytics capabilities using Spark Structured Streaming and integration with Apache Kafka. - Enhance Spark SQL catalyst optimizer and distributed query execution systems. - Drive performance tuning initiatives for complex analytical operations. - Collaborate with engineering teams on technical solutions and system integration. What we are looking for: - 10+ years of experience in software development. - Strong expertise in distributed computing and large-scale data processing systems. - Deep understanding of Apache Spark architecture, including RDDs, DataFrames, and streaming. - Proficiency in Scala, Java, and Python (primary Spark development languages). - Proven experience in performance tuning of distributed systems at scale. - Core contributions to Apache open source projects (Spark, Kafka, Hive, or similar). - Advanced level of English. How we make your work (and your life) easier: - Excellent compensation in USD or your local currency if preferred. - Paid parental leave, vacation, & national holidays. - Innovative and multicultural work environment. - Collaborate and learn from the global Top 1% of talent in each area. - Supportive environment with mentorship, promotions, skill development, and diverse growth opportunities. Join a global team where your unique talents can truly thrive and make a significant impact Apply now

  • Staff Apache Kafka Engineer

    3 weeks ago

    Bengaluru, India BairesDev Full time

    Job Description At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide. When you apply for this...

  • Staff Apache Hive Engineer

    2 weeks ago

    Bengaluru, Karnataka, India BairesDev Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide.When you apply for this position, you're...

  • Senior Scala Developer

    7 days ago

    Bengaluru, India BairesDev Full time

    Job Description At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works remotely on roles that drive significant impact worldwide. When you apply...

  • Data Engineer

    4 weeks ago

    Bengaluru, India Wroots Global Private Limited Full time

    About the Role :We are looking for an experienced Data Engineer with strong hands-on expertise in building, optimizing, and maintaining large-scale data processing systems. The ideal candidate will have deep technical knowledge in Apache Spark, data lakehouse technologies, and streaming platforms, along with solid programming experience and exposure to...

  • Apache scala spark

    1 week ago

    Bengaluru, Chennai, India Athena Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    email-client -MNCApache ,spark ,SCALA requirementlocation -Chennai and Bangalorenotice period - immediate joining - joining 1 decexperience -6-9 years and 9+ years- Grade B2 and B3Mandatory skill Apache Spark and Scala (Python not reqd)

  • Apache Spark Administrator

    1 week ago

    India Marlabs Full time

    **Date**:Oct 11, 2024**Location**:Any Marlabs Office Location, IN**Company**:Marlabs Innovations Pvt Ltd**Description**: - Spark Administrator - 6 Weeks (Starting November 1st 2024)- Key Responsibilities- Install and configure the operating system (Linux distributions such as Ubuntu, CentOS, or RHEL) on cluster nodes, ensuring consistency across the...

  • Staff Apache Hadoop Engineer

    3 weeks ago

    Bengaluru, India BairesDev Full time

    Job Description At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide. When you apply for this...

  • Principal Data Engineer

    4 weeks ago

    Bengaluru, India Neemtree Full time

    Description :- Define and lead the data architecture vision and strategy, ensuring it supports analytics, ML, and business operations at scale.- Architect and manage cloud-native data platforms using Databricks and AWS, leveraging the lakehouse architecture to unify data engineering and ML workflows.- Build and optimize large-scale batch and streaming...

  • Staff Apache Ozone Engineer

    1 week ago

    Bengaluru, India BairesDev Full time

    Job Description At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide. When you apply for this...

  • Staff Apache Hadoop Engineer

    2 weeks ago

    Bengaluru, Karnataka, India BairesDev Full time ₹ 1,50,000 - ₹ 2,50,000 per year

    At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide.When you apply for this position, you're...