Senior Spark Developer

The Senior Spark Tech Lead will be responsible for integrating and maintaining the Quantexa platform, a spark based software provided by a UK fintech, into our existing systems to enhance our anti-money laundering capabilities. This role requires a deep expertise in Spark development, as well as an ability to analyze and understand underlying data. Additionally, the candidate should have an interest in exploring open-source applications distributed by Apache, Kubernetes, OpenSearch and Oracle.

Should be able to work as a Scrum Master

Responsibilities

Direct Responsibilities

Integrate and upgrade the Quantexa tool with our existing systems for enhanced anti-money laundering measures.

Develop and maintain Spark-based applications deployed on Kubernetes clusters.

Conduct data analysis to understand and interpret underlying data structures.

Collaborate with cross-functional teams to ensure seamless integration and functionality.

Stay updated with the latest trends and best practices in Spark development and Kubernetes.

Contributing Responsibilities

Taking complete ownership of project activities and understand each tasks in details.

Ensure that the team delivers on time without any delays and deliveries are of high quality standards.

Estimation, Planning and scheduling of the project. Ensure all internal timelines are respected and project is on track.

Work with team to develop robust software adhering to the timelines & following all the standard guidelines.

Act proactively to ensure smooth team operations and effective collaboration

Make sure team adheres to all compliance processes and intervene if required

Task assignment to the team and tracking until task completion

Proactive Status reporting to the management.

Identify Risks in the project and highlight to Manager.

Create Contingency and Backup planning as necessary. Create Mitigation Plan.

Take decision by own based on situation.

Play the role of mentor and coach team members as and when required to meet the target goals

Gain functional knowledge on applications worked upon

Create knowledge repositories for future reference.

Arrange knowledge sharing sessions to enhance team's functional capability.

Evaluation of new tools and coming with POCs.

Provide feedback of team to upper management on timely basis

Technical & Behavioral Competencies

Key Responsibilities

Integrate and upgrade the Quantexa tool with our existing systems for enhanced anti-money laundering measures.

Develop and maintain Spark-based applications deployed on Kubernetes clusters.

Conduct data analysis to understand and interpret underlying data structures.

Collaborate with cross-functional teams to ensure seamless integration and functionality.

Stay updated with the latest trends and best practices in Spark development and Kubernetes.

Required Qualifications

7+ Years of experience in development

Extensive experience in Hadoop, Spark, Scala development (5 years min).

Strong analytical skills and experience in data analysis (SQL), data processing (such as ETL), parsing, data mapping and handling real-life data quality issues.

Excellent problem-solving abilities and attention to detail.

Strong communication and collaboration skills.

Experience in Agile development.

High quality coding skill, incl. code control, unit testing, design, and documentation (code, test). Experience with tools such as sonar. Experience with GIT, Jenkins.

Specific Qualifications (if required)

Experience with development and deployment of spark application and deployment on Kubernetes clusters

Hands-on development experience (Java, Scala, etc.) via system integration projects, Python, Elastic (optional).

Skills Referential

Behavioural Skills : (Please select up to 4 skills)

Ability to collaborate / Teamwork

Adaptability

Creativity & Innovation / Problem solving

Attention to detail / rigor

Transversal Skills: (Please select up to 5 skills)

Analytical Ability

Ability to develop and adapt a process

Ability to develop and leverage networks

Choose an item.

Choose an item.

Education Level:

Bachelor Degree or equivalent

Experience Level

At least 7 years

Fluent in English

Team player

Strong analytical skills

Quality oriented and well organized

Willing to work under pressure and mission oriented

Excellent Oral and Written Communication Skills, Motivational Skills, Results-Oriented