Pyspark Lead Developer Architect

3 days ago

Chennai, India Cloud Kinetics Technology Solutions Private Ltd Full time


Primary Skills:

Technical:

  • Must be a Subject Matter Expert in PySpark.
  • Hands on experience in creating PySpark jobs for data transformation and aggregation in Financial Market domain.
  • Experience in querying data from column oriented databases.
  • Strong understanding of Hadoop architecture and design best practices.
  • Thorough understanding of fundamental software engineering and computer science principles: objectoriented design structured design databases algorithms data structures usability refactoring debugging and configuration management
  • Ensure excellent quality of the code delivered such that stability and resiliency of the production system is not compromised.
  • Evaluate and implement innovative and continuous design improvements to PySpark framework.
  • Keeps current with new practice technical developments and innovation in market for potential use as solutions to requirements.
  • Adhere to best practices and coding standards document and publish the same.
  • Manage and support / maintain Cyber Risk Resilience Risk Stability Risks of application and drive towards low risk environment.

Functional:

  • Strong understanding of Financial Markets Asset Classes


Requirements

Responsibilities

  • Creating high performance pySpark applications which should be able to handle millions of rows from various file format to find Market Abusive Regulations (MAR).
  • Perform data analysis and apply ML/AI based libraries for innovative solutions to the requirements / problems in MAR solution.
  • Keep the technical specification documentation updated and version controlled.
  • This role will work closely with business analysts and Trade surveillance users.
  • Provide effort estimates of based on requirements.
  • Adhere to best practices coding standards.



PROGRAMMING SKILLS Must have experience and knowledge of current Java technologies and methodologies, and hands-on experience on the following: Web front-end experience React JS /Ember JS/HTML/CSS/JavaScript Core Java / JEE / JBoss EAP Object-relational Mappers JPA/Hibernate JMS XML / XSLT JUnit The application of design patterns Test-Driven Development (TDD) Relational database - Oracle An understanding of the following would also be advantageous and preferred: Workflow based processing REST API Development and Java Web Services Micro Services Architecture JBoss Rules Jasper

  • Pyspark Lead Developer

    2 months ago

    Chennai, India Cloud Kinetics Technology Solutions Private Ltd Full time

    Primary Skills:Technical:Must be a Subject Matter Expert in PySpark.Hands on experience in creating PySpark jobs for data transformation and aggregation in Financial Market domain.Experience in querying data from column oriented databases.Strong understanding of Hadoop architecture and design best practices.Thorough understanding of fundamental software...

  • Pyspark Lead Developer

    2 months ago

    Chennai, India Cloud Kinetics Technology Solutions Private Ltd Full time

    Job DescriptionPrimary Skills:Technical:Must be a Subject Matter Expert in PySpark.Hands on experience in creating PySpark jobs for data transformation and aggregation in Financial Market domain.Experience in querying data from column oriented databases.Strong understanding of Hadoop architecture and design best practices.Thorough understanding of...

  • Pyspark Lead Developer Architect

    1 month ago

    Chennai, India Cloud Kinetics Technology Solutions Private Ltd Full time

    PrimarySkills:Technical:Mustbe a Subject Matter Expert inPySpark.Handson experience in creating PySpark jobs for data transformation andaggregation in Financial Marketdomain.Experiencein querying data from column orienteddatabases.Strongunderstanding of Hadoop architecture and design bestpractices.Thoroughunderstanding of fundamental software engineering and...

  • Pyspark

    3 days ago

    Chennai, India Risk Resources LLP Full time

    Overview:The role of a Pyspark developer is crucial in our organization as they are responsible for developing and maintaining efficient scalable data processing applications using Pyspark.Key Responsibilities:Experience working in big data platforms for implementing projects with high SLA of data availability and data quality exposure to Azure cloud...

  • Python Pyspark Developer

    4 months ago

    Chennai, India RiskInsight Consulting Pvt Ltd Full time

    Responsibilities Design, develop, and deploy scalable data processing applications using Python and PySpark. Collaborate with data scientists and analysts to understand requirements and translate them into technical solutions. Write efficient and optimized code to process and analyze large volumes of data. Implement data ingestion processes from various data...

  • Pyspark Developer

    2 months ago

    Chennai, India Cloud Kinetics Technology Solutions Private Ltd Full time

    Primary Skills:Technical:Hands on experience in creating PySpark jobs for data transformation and aggregation in Financial Market domain.Experience in querying and processing from column oriented databases using python.Strong understanding of Hadoop architecture and design best practices.Thorough understanding of fundamental software engineering and computer...

  • Pyspark Developer

    2 months ago

    Chennai, India Cloud Kinetics Technology Solutions Private Ltd Full time

    PrimarySkills:Technical:Handson experience in creating PySpark jobs for data transformation andaggregation in Financial Marketdomain.Experiencein querying and processing from column oriented databases usingpython.Strongunderstanding of Hadoop architecture and design bestpractices.Thoroughunderstanding of fundamental software engineering and computerscience...

  • Pyspark Developer

    3 days ago

    Chennai, India Cloud Kinetics Technology Solutions Private Ltd Full time

    Primary Skills:Technical:Hands on experience in creating PySpark jobs for data transformation and aggregation in Financial Market domain.Experience in querying and processing from column oriented databases using python.Strong understanding of Hadoop architecture and design best practices.Thorough understanding of fundamental software engineering and computer...

  • Pyspark Developer

    2 months ago

    Chennai, India Cloud Kinetics Technology Solutions Private Ltd Full time

    Job DescriptionPrimary Skills:Technical:Hands on experience in creating PySpark jobs for data transformation and aggregation in Financial Market domain.Experience in querying and processing from column oriented databases using python.Strong understanding of Hadoop architecture and design best practices.Thorough understanding of fundamental software...

  • Data Engineer

    4 weeks ago

    Chennai, India SDNA Global Full time

    Job Description :Roles and Responsibilities :We are seeking an experienced Data Engineer to join our team. The ideal candidate will have 5 to 8 years of experience in data engineering and a strong background in building data pipelines, data warehousing, and data management solutions. In this role, you will work closely with data scientists, analysts, and...

  • Data Lead/ Data Architect

    1 month ago

    Chennai, India Optimum Solutions Pte Ltd Full time

    10+ Years experienceProven work experience as a Data ArchitectIn-depth understanding of database structure principlesExperience gathering and analysing system requirements.Knowledge of data mining and segmentation techniquesExpertise in SQL and OracleProficiency in MS ExcelFamiliarity with data visualization tools (e.g. Tableau, Power BI)Python...

  • Data Lead/ Data Architect

    1 month ago

    Chennai, India Optimum Solutions Pte Ltd Full time

    10+ Years experienceProven work experience as a Data ArchitectIn-depth understanding of database structure principlesExperience gathering and analysing system requirements.Knowledge of data mining and segmentation techniquesExpertise in SQL and OracleProficiency in MS ExcelFamiliarity with data visualization tools (e.g. Tableau, Power BI)Python...

  • Data Engineer Azure Pyspark

    3 days ago

    Chennai, India Overture Rede Private Limited Full time

    Job Title: Data Engineer Location: Specify Location or Remote Responsibilities: Design build and maintain scalable data pipelines using PySpark on Data bricks. Work with Delta Lake to ensure data reliability consistency and scalability. Utilize Azure Synapse for integrated analytics querying and big data processing. Develop and deploy data solutions on Azure...

  • Data Architect

    1 month ago

    Chennai, India Aspire Systems Full time

    Job Title: Data ArchitectLocation: Chennai, IndiaClient Base: KSA (Kingdom of Saudi Arabia)Experience Required: 10-12 yearsWork Location: Chennai, IndiaWorking Days: Sunday to ThursdayJob Description:We are seeking a highly skilled and experienced Data Architect to join our team, working on projects for KSA-based customers from our Chennai location. The...

  • Data Architect

    1 month ago

    Chennai, India Aspire Systems Full time

    Job Title: Data ArchitectLocation: Chennai, IndiaClient Base: KSA (Kingdom of Saudi Arabia)Experience Required: 10-12 yearsWork Location: Chennai, IndiaWorking Days: Sunday to ThursdayJob Description:We are seeking a highly skilled and experienced Data Architect to join our team, working on projects for KSA-based customers from our Chennai location. The...

  • Pyspark Dev

    4 months ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Pyspark Dev & QA Mandatory Skills BIG Data technology mentioned below Hadoop / Big Data (HDFS, PYTHON, SPARK-SQL, MapReduce) with PySpark Build CI/CD pipelines is required Outstanding coding, debugging and analytical skills Spark APIs to cleanse, explore, aggregate, transform, store & analyse available data Knowledge of installing, configuring, debugging...

  • Python Pyspark

    2 weeks ago

    Chennai, Tamil Nadu, India Virtusa Full time

    PySpark Developer Strong experience in technical leadership, mentoring MDM team of size approx. 10. Work experience 6+ years with skillset in Python (Pandas/ NumPy), PySpark Developer. Hands-on experience in generating/parsing XML, JSON documents, and REST API request/responses Hands-on experience writing complex SQL queries, exporting and importing large...

  • Principal Data Engineer

    1 month ago

    Chennai, India Petals Careers Full time

    Responsibilities :- Design, build, and operationalize large-scale enterprise data solutions and applications using one or more AWS data and analytics services - Glue, RedShift, Lambda, S3 Table Partition mechanism, Types of Partitions, column store index.- Maintenance and optimization of existing processes.- Requirement gathering and feasibility analysis,...

  • Python Pyspark

    3 months ago

    Chennai, India Virtusa Full time

    Overall Eight plus years of experience with min of five plus years experience in required technical skills Required technical skills: Python, PySpark, SQL AWS Cloud experience and Healthcare domain knowledge is desired **About Virtusa** Teamwork, quality of life, professional and personal development: values that Virtusa is proud to embody. When you join...

  • Python Pyspark

    3 months ago

    Chennai, India Virtusa Full time

    Overall Eight plus years of experience with min of five plus years experience in required technical skills Required technical skills: Python, PySpark, SQL AWS Cloud experience and Healthcare domain knowledge is desired Notice Period - Immediate to 30 days **About Virtusa** Teamwork, quality of life, professional and personal development: values that...