Senior Java Architect

2 weeks ago

New Delhi, India Hyperion AI Full time

Experience Level: Senior/Lead Java Architect (13+ years of experience)

Job Description:

We are seeking an experienced Senior Lead Java Architect with deep expertise in Apache Spark, Azure Data Services (ADLS, Azure Blob Storage, Azure Database), and Redpanda. The ideal candidate will play a pivotal role in designing, architecting, and leading the development of scalable data processing pipelines using Java and Spark. You will work closely with cross-functional teams to build data-intensive applications and drive best practices in cloud-based architecture. If you're passionate about data engineering, distributed systems, and cloud technologies, this role is for you.

Key Responsibilities:

  • Design and Architect Distributed Data Processing Systems:
  • Lead the design and implementation of large-scale Apache Spark jobs using Java for processing vast datasets from various sources, including blockchain data streams and financial transactions.
  • Architect robust, high-performance data pipelines integrated with Azure Data Lake Storage (ADLS), Azure Blob Storage, and Azure Databases for real-time and batch processing.
  • Development and Optimization of Spark Jobs:
  • Develop, optimize, and manage Spark jobs primarily in Java (JVM-based), ensuring high scalability, fault tolerance, and performance.
  • Implement data validation, schema management, and parquet file handling in Spark jobs for seamless integration with ADLS and Iceberg tables.
  • Integration with Redpanda:
  • Lead the integration of Redpanda (or Kafka-like message brokers) for real-time data ingestion, ensuring reliable, low-latency communication between the data ingestion services and downstream processing components.
  • Collaborate with the engineering team to build Redpanda consumers for real-time streaming data and ensuring that Spark jobs consume this data efficiently.
  • Azure Cloud Services:
  • Design cloud-native solutions leveraging Azure services such as Azure Data Lake Storage (ADLS), Azure Blob Storage, and Azure SQL/NoSQL Databases.
  • Collaborate with the DevOps team to ensure the seamless integration of Spark jobs with Azure services, including data ingestion and storage management.
  • Manage and optimize storage costs and performance for large datasets on Azure services.
  • Lead and Mentor Engineering Teams:
  • Provide technical leadership, mentoring, and guidance to a team of developers, ensuring best practices in Java development, Spark optimization, and cloud-native architecture.
  • Conduct regular code reviews, architecture assessments, and performance tuning exercises to ensure code quality and adherence to industry standards.
  • End-to-End Data Pipeline Management:
  • Manage the full lifecycle of data pipelines, from initial data ingestion in Redpanda to transforming, storing, and serving data using Azure Data Services and Spark.
  • Ensure that data pipelines are monitored, secure, and highly available by integrating with tools like Prometheus, Azure Monitor, and Grafana.
  • Performance Tuning and Resource Optimization:
  • Lead efforts in performance tuning and resource optimization for Spark jobs, ensuring minimal shuffling and efficient memory usage for large-scale data processing.
  • Develop caching mechanisms, partitioning, and task scheduling strategies to maximize throughput and reduce processing time.

Required Qualifications:

  • 13+ years of experience in software development with strong expertise in Java and JVM-based technologies.
  • 5+ years of experience working with Apache Spark for data processing in Java.
  • Extensive experience with Azure Data Services, including Azure Data Lake Storage (ADLS), Azure Blob Storage, and Azure Databases.
  • Strong expertise in distributed data processing and streaming frameworks, with hands-on experience integrating message brokers like Redpanda or Kafka.
  • Proven track record in cloud-native architecture design with a focus on Azure.
  • Hands-on experience with data storage optimization, partitioning, and Spark performance tuning.
  • Leadership and mentoring experience with the ability to guide and develop high-performing engineering teams.
  • Experience with CI/CD pipelines, containerization (Docker, Kubernetes), and DevOps practices.
  • Strong understanding of security best practices for cloud-based data pipelines.

Preferred Qualifications:

  • Experience with Iceberg tables and Parquet file formats in a distributed processing environment.
  • Knowledge of Prometheus, Grafana, or Azure Monitor for observability and monitoring.
  • Experience with streaming data processing using Redpanda and integration with Spark jobs for real-time data flows.
  • Familiarity with microservices architecture using Spring Boot and cloud-based deployment strategies.

  • Senior java architect

    2 weeks ago

    New Delhi, India Hyperion AI Full time

    Experience Level : Senior/Lead Java Architect (13+ years of experience) Job Description: We are seeking an experienced Senior Lead Java Architect with deep expertise in Apache Spark , Azure Data Services (ADLS, Azure Blob Storage, Azure Database), and Redpanda . The ideal candidate will play a pivotal role in designing, architecting, and...

  • Senior Java Architect

    2 weeks ago

    new delhi, India Hyperion AI Full time

    Experience Level : Senior/Lead Java Architect (13+ years of experience) Job Description: We are seeking an experienced Senior Lead Java Architect with deep expertise in Apache Spark , Azure Data Services (ADLS, Azure Blob Storage, Azure Database), and Redpanda . The ideal candidate will play a pivotal role in designing, architecting, and leading the...

  • Senior Java Architect

    7 days ago

    Delhi, Delhi, India SourceBae Full time

    Job Title: Senior Java ArchitectOverviewWe are seeking an experienced Senior Java Architect to lead the design and implementation of scalable, high-performing solutions. The ideal candidate will possess strong expertise in Java Spring, microservices architecture, and cloud technologies.Key Responsibilities:Solution Design & Architecture : Lead end-to-end...

  • Senior java architect

    2 weeks ago

    New Delhi, India Hyperion AI Full time

    Experience Level : Senior/Lead Java Architect (13+ years of experience)Job Description:We are seeking an experienced Senior Lead Java Architect with deep expertise in Apache Spark , Azure Data Services (ADLS, Azure Blob Storage, Azure Database), and Redpanda. The ideal candidate will play a pivotal role in designing, architecting, and leading the...

  • Senior Java Architect

    2 weeks ago

    New Delhi, India Hyperion AI Full time

    Experience Level: Senior/Lead Java Architect (13+ years of experience)Job Description:We are seeking an experienced Senior Lead Java Architect with deep expertise in Apache Spark, Azure Data Services (ADLS, Azure Blob Storage, Azure Database), and Redpanda. The ideal candidate will play a pivotal role in designing, architecting, and leading the development...

  • Senior Java Architect

    2 weeks ago

    New Delhi, India Hyperion AI Full time

    Experience Level : Senior/Lead Java Architect (13+ years of experience) Job Description: We are seeking an experienced Senior Lead Java Architect with deep expertise in Apache Spark , Azure Data Services (ADLS, Azure Blob Storage, Azure Database), and Redpanda . The ideal candidate will play a pivotal role in designing, architecting, and...

  • Senior Java Architect

    2 weeks ago

    New Delhi, India Hyperion AI Full time

    Experience Level : Senior/Lead Java Architect (13+ years of experience)Job Description:We are seeking anexperienced Senior Lead Java Architectwith deep expertise inApache Spark ,Azure Data Services(ADLS, Azure Blob Storage, Azure Database), andRedpanda . The ideal candidate will play a pivotal role in designing, architecting, and leading the development of...

  • Senior Java Enterprise Architect

    2 weeks ago

    Delhi, Delhi, India National Payments Corporation Of India (NPCI) Full time

    Job Title: Senior Java Enterprise ArchitectAbout the Role:We are seeking an experienced Senior Java Enterprise Architect to join our team at National Payments Corporation Of India (NPCI). As a key member of our architecture team, you will play a crucial role in designing and developing highly available and scalable platforms using Java and J2EE...

  • Java Architect

    1 month ago

    New Delhi, India Gemraj Technologies Ltd Full time

    Hands on Architecture experience for designing Web ApplicationsStrong knowledge of Design, Restful, Core Java, J2EE, Spring Boot, Elastic Search, SQL, ANGULAR, HTML, CSS, JSP, MYSQ, API DesigningKnowledge of Elastic search and JPA/Hibernet, Maven, HTML, CSS and JAVA script.Architectural & Designing & coding work for requirementFeasibility analysis of the...

  • Senior Java Software Architect

    1 month ago

    Noida/Greater Noida/Delhi NCR/Delhi/Gurgaon/Gurugram/Faridabad, India LEGPRO CONSULTANT PVT LTD Full time

    Job Title: Senior Java Software ArchitectAbout the Role:At LEGPRO CONSULTANT PVT LTD, we are seeking an experienced Senior Java Software Architect to lead our team in designing and developing high-quality computer software systems. As a Senior Java Software Architect, you will be responsible for researching, designing, and developing software systems that...

  • Senior Java Backend Developer

    7 days ago

    Delhi, Delhi, India Hrizen techno services Full time

    Job Title: Senior Java Backend Developer - Cloud ArchitectAbout the Role:Hrizen Techno Services is seeking an experienced Java Backend Developer - Cloud Architect to join our team. As a key member of our development team, you will be responsible for designing, developing, and maintaining high-quality Java-based applications.Key Responsibilities:

  • Senior Backend Developer

    3 weeks ago

    New Delhi, India h3 Technologies, LLC Full time

    Job Title: Senior Backend Developer - L2 (Java)Location: Delhi ExperienceRequired: 6+ years in Java backend developmentJob Type: Full-TimeJob Summary: We are seeking an experienced Senior Backend Developer specializing in Java to join our team. This role is crucial in implementing robust and scalable backend solutions by working closely with the Solution...

  • Senior Java Stack Architect

    7 days ago

    Delhi, Delhi, India Hirein5 Full time

    We are seeking a highly skilled Senior Java Stack Architect to join our dynamic team at Hirein5. With a strong background in designing, developing, and deploying web applications using Java technologies, you will be responsible for leading the development and enhancement of software solutions that meet the needs of our clients.The ideal candidate will have...

  • Senior java architect

    1 week ago

    Delhi, India Accolite Full time

    **Job Title: Java Architect (Front End & Cloud Expertise)**Experience : 10+ Years**Job Overview:**We are seeking a talented and experienced Java Architect with a strong background in front-end development and cloud technologies to join our dynamic team. In this role, you will be responsible for designing and implementing high-quality software solutions that...

  • Hyperion AI | Senior Java Architect

    1 week ago

    new delhi, India Hyperion AI Full time

    Experience Level : Senior/Lead Java Architect (10+ years of experience)Job Description:We are seeking anexperienced Senior Lead Java Architectwith deep expertise inApache Spark ,Azure Data Services(ADLS, Azure Blob Storage, Azure Database), andRedpanda . The ideal candidate will play a pivotal role in designing, architecting, and leading the development of...

  • Senior Java Architect

    1 week ago

    Delhi, India Accolite Full time

    **Job Title: Java Architect (Front End & Cloud Expertise)**Experience : 10+ Years**Job Overview:**We are seeking a talented and experienced Java Architect with a strong background in front-end development and cloud technologies to join our dynamic team. In this role, you will be responsible for designing and implementing high-quality software solutions that...

  • Senior Java Software Architect

    1 month ago

    Delhi, Delhi, India A Services company Full time

    Job Title: Senior Java Software ArchitectJob Summary:We are seeking a highly skilled Senior Java Software Architect to lead the design and development of robust web applications using Java, Spring boot, Microservices, and AWS. The ideal candidate will have 11+ years of experience in leading architectural design and implementation of web applications,...

  • Senior Java Cloud Architect

    2 days ago

    Delhi, Delhi, India HRemedies Full time

    Job DescriptionHRemedies is seeking an experienced Senior Java Cloud Architect to lead our modernization efforts to Spring Boot, Microservices, and cloud-native architecture.Key Responsibilities:Analyze existing Java codebase and design microservices architecture patterns for scalability, reliability, fault tolerance, and performance.Collaborate with clients...

  • Hyperion AI | Senior Java Architect

    2 weeks ago

    new delhi, India Hyperion AI Full time

    Experience Level : Senior/Lead Java Architect (13+ years of experience) Job Description: We are seeking an experienced Senior Lead Java Architect with deep expertise in Apache Spark , Azure Data Services (ADLS, Azure Blob Storage, Azure Database), and Redpanda . The ideal candidate will play a pivotal role in designing, architecting, and leading the...

  • Hyperion AI | Senior Java Architect

    2 weeks ago

    new delhi, India Hyperion AI Full time

    Experience Level : Senior/Lead Java Architect (13+ years of experience) Job Description: We are seeking an experienced Senior Lead Java Architect with deep expertise in Apache Spark , Azure Data Services (ADLS, Azure Blob Storage, Azure Database), and Redpanda . The ideal candidate will play a pivotal role in designing, architecting, and leading the...