Apache Spark Associate Architect for Datasphere

1 week ago

Bengaluru, India SAP Full time

What you’ll do

-

As an associate architect in the SAP datasphere team, you will

Develop and execute high-level architectural designs with respect the Apache Spark components in Datasphere ensuring that solutions are scalable, secure, and optimized for performance, meeting both technical and business requirements as well as product standards. Provide technical leadership and guidance to development teams, ensuring adherence to architectural best practices and product standards. Author clear technical documentation and presentations in collaboration with product engineers and architects to communicate architecture and design options to educate engineering teams and product managers.

What you bring

8-10 years of overall software development experience 

At least 4 years of development experience in Apache Spark framework 

In-depth knowledge of Spark Core, Spark SQL and Spark streaming

Experience in working with Data Frames, Datasets and RDDs

Experience in designing and implementing scalable and performance Spark-based systems 

Ability to implement real-time data processing and batch processing workflows

Good understanding of Spark architecture including driver and executor interaction

Proficiency in writing clean and efficient code in one or more Spark programming languages such as Python/Scala/Java

Proficiency in Spark job optimization including partitioning, caching and resource allocation

Familiarity with tuning techniques like adjusting executor memory, shuffle partitions, and broadcast joins.

Experience building and managing ETL pipelines using Spark.

Ability to connect and work with relational databases using JDBC/ODBC

Experience deploying Spark jobs on cloud platforms such as AWS, Azure, Google Cloud

Proficiency in using monitoring and debugging tools such as Spark UI, Grafana or Promethus

Writing and optimizing complex SQL queries and understanding query execution plans and leveraging Spark SQL for distributed query processing.

#SAPInternalT3

​Bring out your best

SAP innovations help more than four hundred thousand customers worldwide work together more efficiently and use business insight more effectively. Originally known for leadership in enterprise resource planning (ERP) software, SAP has evolved to become a market leader in end-to-end business application software and related services for database, analytics, intelligent technologies, and experience management. As a cloud company with two hundred million users and more than one hundred thousand employees worldwide, we are purpose-driven and future-focused, with a highly collaborative team ethic and commitment to personal development. Whether connecting global industries, people, or platforms, we help ensure every challenge gets the solution it deserves. At SAP, you can bring out your best.

  • Apache Spark Associate Architect for Datasphere

    1 week ago

    Bengaluru, Karnataka, India SAP Full time

    **We help the world run better** At SAP, we enable you to bring out your best. Our company culture is focused on collaboration and a shared passion to help the world run better. How? We focus every day on building the foundation for tomorrow and creating a workplace that embraces differences, values flexibility, and is aligned to our purpose-driven and...

  • Apache Spark Associate Architect for Datasphere

    2 weeks ago

    Bengaluru, India SAP Full time

    What you’ll do - As an associate architect in the SAP datasphere team, you will Develop and execute high-level architectural designs with respect the Apache Spark components in Datasphere ensuring that solutions are scalable, secure, and optimized for performance, meeting both technical and business requirements as well as product standards. Provide...

  • Sr Backline Engineer

    1 week ago

    Bengaluru, India Databricks Full time

    Mission Databricks helps data teams solve the world’s toughest problems. As the leader in Unified Data Analytics, Databricks helps organizations make all their data ready for analytics, empower data-driven decisions across the organization, and rapidly adopt machine learning to outpace the competition Databricks is venture-backed and founded by the...

  • Sr Backline Engineer

    2 weeks ago

    Bengaluru, India Databricks Full time

    Mission Databricks helps data teams solve the world’s toughest problems. As the leader in Unified Data Analytics, Databricks helps organizations make all their data ready for analytics, empower data-driven decisions across the organization, and rapidly adopt machine learning to outpace the competition Databricks is venture-backed and founded by the...

  • Staff Technical Solutions Engineer

    1 week ago

    Bengaluru, India Databricks Full time

    P-1364 As a Spark Staff Technical Solutions Engineer, you will provide a deep dive technical and consulting related solutions for the challenging Apache Spark™/ML/AI/Delta/Streaming/Lakehouse reported issues by our customers and resolve any challenges involving the Databricks unified analytics platform with your highly comprehensive technical and customer...

  • Staff Technical Solutions Engineer

    2 weeks ago

    Bengaluru, India Databricks Full time

    P-1364 As a Spark Staff Technical Solutions Engineer, you will provide a deep dive technical and consulting related solutions for the challenging Apache Spark™/ML/AI/Delta/Streaming/Lakehouse reported issues by our customers and resolve any challenges involving the Databricks unified analytics platform with your highly comprehensive technical and customer...

  • Senior Technical Solutions Engineer

    1 week ago

    Bengaluru, India Databricks Full time

    P-1377 Databricks helps data teams solve the world’s toughest problems. As the leader in Unified Data Analytics, Databricks helps organizations make all their data ready for analytics, empower data-driven decisions across the organization, and rapidly adopt machine learning to outpace the competition Databricks is venture-backed and founded by the...

  • Senior Technical Solutions Engineer

    2 weeks ago

    Bengaluru, India Databricks Full time

    P-1377 Databricks helps data teams solve the world’s toughest problems. As the leader in Unified Data Analytics, Databricks helps organizations make all their data ready for analytics, empower data-driven decisions across the organization, and rapidly adopt machine learning to outpace the competition Databricks is venture-backed and founded by the original...

  • Associate architect – data

    4 weeks ago

    Bengaluru, India Quantiphi Full time

    Job Title : Associate Architect - Data EngineeringExperience : 6 to 9 YearsLocation : Bangalore / Mumbai (any) (Hybrid WFO)About the Role:We are looking for a skilled Data Engineers with hands-on experience in the Microsoft Azuredata ecosystem. The ideal candidate will play a critical role in designing, developing, andmanaging data pipelines and...

  • Associate architect – data

    6 days ago

    Bengaluru, India Quantiphi Full time

    Job Title : Associate Architect - Data EngineeringExperience : 6 to 9 YearsLocation : Bangalore / Mumbai (any) (Hybrid WFO)About the Role:We are looking for a skilled Data Engineers with hands-on experience in the Microsoft Azuredata ecosystem. The ideal candidate will play a critical role in designing, developing, andmanaging data pipelines and...