Pyspark Lead Developer

2 months ago

Chennai, India Cloud Kinetics Technology Solutions Private Ltd Full time
Job Description


Primary Skills:

Technical:

  • Must be a Subject Matter Expert in PySpark.
  • Hands on experience in creating PySpark jobs for data transformation and aggregation in Financial Market domain.
  • Experience in querying data from column oriented databases.
  • Strong understanding of Hadoop architecture and design best practices.
  • Thorough understanding of fundamental software engineering and computer science principles: object-oriented design, structured design, databases, algorithms, data structures, usability, refactoring, debugging, and configuration management
  • Ensure excellent quality of the code delivered such that stability and resiliency of the production system is not compromised.
  • Evaluate and implement innovative and continuous design improvements to PySpark framework.
  • Keeps current with new practice, technical developments, and innovation in market for potential use as solutions to requirements.
  • Adhere to best practices and coding standards, document and publish the same.
  • Manage and support / maintain Cyber Risk, Resilience Risk, Stability Risks of application and drive towards low risk environment.

Functional:

  • Strong understanding of Financial Markets Asset Classes


Requirements

Responsibilities

  • Creating high performance pySpark applications which should be able to handle millions of rows from various file format to find Market Abusive Regulations (MAR).
  • Perform data analysis and apply ML/AI based libraries for innovative solutions to the requirements / problems in MAR solution.
  • Keep the technical specification documentation updated and version controlled.
  • This role will work closely with business analysts and Trade surveillance users.
  • Provide effort estimates of based on requirements.
  • Adhere to best practices, coding standards.



Requirements
PROGRAMMING SKILLS Must have experience and knowledge of current Java technologies and methodologies, and hands-on experience on the following: · Web front-end experience – React JS /Ember JS/HTML/CSS/JavaScript · Core Java / JEE / JBoss EAP · Object-relational Mappers – JPA/Hibernate · JMS · XML / XSLT · JUnit · The application of design patterns · Test-Driven Development (TDD) · Relational database - Oracle An understanding of the following would also be advantageous and preferred: · Workflow based processing · REST API Development and Java Web Services · Micro Services Architecture · JBoss Rules · Jasper

  • Pyspark Lead Developer

    2 months ago

    Chennai, India Cloud Kinetics Technology Solutions Private Ltd Full time

    Primary Skills:Technical:Must be a Subject Matter Expert in PySpark.Hands on experience in creating PySpark jobs for data transformation and aggregation in Financial Market domain.Experience in querying data from column oriented databases.Strong understanding of Hadoop architecture and design best practices.Thorough understanding of fundamental software...

  • Pyspark Lead Developer Architect

    3 days ago

    Chennai, India Cloud Kinetics Technology Solutions Private Ltd Full time

    Primary Skills:Technical:Must be a Subject Matter Expert in PySpark.Hands on experience in creating PySpark jobs for data transformation and aggregation in Financial Market domain.Experience in querying data from column oriented databases.Strong understanding of Hadoop architecture and design best practices.Thorough understanding of fundamental software...

  • Pyspark

    3 days ago

    Chennai, India Risk Resources LLP Full time

    Overview:The role of a Pyspark developer is crucial in our organization as they are responsible for developing and maintaining efficient scalable data processing applications using Pyspark.Key Responsibilities:Experience working in big data platforms for implementing projects with high SLA of data availability and data quality exposure to Azure cloud...

  • Python Pyspark Developer

    4 months ago

    Chennai, India RiskInsight Consulting Pvt Ltd Full time

    Responsibilities Design, develop, and deploy scalable data processing applications using Python and PySpark. Collaborate with data scientists and analysts to understand requirements and translate them into technical solutions. Write efficient and optimized code to process and analyze large volumes of data. Implement data ingestion processes from various data...

  • Pyspark Lead Developer Architect

    1 month ago

    Chennai, India Cloud Kinetics Technology Solutions Private Ltd Full time

    PrimarySkills:Technical:Mustbe a Subject Matter Expert inPySpark.Handson experience in creating PySpark jobs for data transformation andaggregation in Financial Marketdomain.Experiencein querying data from column orienteddatabases.Strongunderstanding of Hadoop architecture and design bestpractices.Thoroughunderstanding of fundamental software engineering and...

  • Pyspark Developer

    2 months ago

    Chennai, India Cloud Kinetics Technology Solutions Private Ltd Full time

    Primary Skills:Technical:Hands on experience in creating PySpark jobs for data transformation and aggregation in Financial Market domain.Experience in querying and processing from column oriented databases using python.Strong understanding of Hadoop architecture and design best practices.Thorough understanding of fundamental software engineering and computer...

  • Pyspark Developer

    2 months ago

    Chennai, India Cloud Kinetics Technology Solutions Private Ltd Full time

    PrimarySkills:Technical:Handson experience in creating PySpark jobs for data transformation andaggregation in Financial Marketdomain.Experiencein querying and processing from column oriented databases usingpython.Strongunderstanding of Hadoop architecture and design bestpractices.Thoroughunderstanding of fundamental software engineering and computerscience...

  • Pyspark Developer

    3 days ago

    Chennai, India Cloud Kinetics Technology Solutions Private Ltd Full time

    Primary Skills:Technical:Hands on experience in creating PySpark jobs for data transformation and aggregation in Financial Market domain.Experience in querying and processing from column oriented databases using python.Strong understanding of Hadoop architecture and design best practices.Thorough understanding of fundamental software engineering and computer...

  • Pyspark Developer

    2 months ago

    Chennai, India Cloud Kinetics Technology Solutions Private Ltd Full time

    Job DescriptionPrimary Skills:Technical:Hands on experience in creating PySpark jobs for data transformation and aggregation in Financial Market domain.Experience in querying and processing from column oriented databases using python.Strong understanding of Hadoop architecture and design best practices.Thorough understanding of fundamental software...

  • Data Engineer Azure Pyspark

    3 days ago

    Chennai, India Overture Rede Private Limited Full time

    Job Title: Data Engineer Location: Specify Location or Remote Responsibilities: Design build and maintain scalable data pipelines using PySpark on Data bricks. Work with Delta Lake to ensure data reliability consistency and scalability. Utilize Azure Synapse for integrated analytics querying and big data processing. Develop and deploy data solutions on Azure...

  • Pyspark Dev

    4 months ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Pyspark Dev & QA Mandatory Skills BIG Data technology mentioned below Hadoop / Big Data (HDFS, PYTHON, SPARK-SQL, MapReduce) with PySpark Build CI/CD pipelines is required Outstanding coding, debugging and analytical skills Spark APIs to cleanse, explore, aggregate, transform, store & analyse available data Knowledge of installing, configuring, debugging...

  • Python Pyspark

    2 weeks ago

    Chennai, Tamil Nadu, India Virtusa Full time

    PySpark Developer Strong experience in technical leadership, mentoring MDM team of size approx. 10. Work experience 6+ years with skillset in Python (Pandas/ NumPy), PySpark Developer. Hands-on experience in generating/parsing XML, JSON documents, and REST API request/responses Hands-on experience writing complex SQL queries, exporting and importing large...

  • Python Pyspark

    3 months ago

    Chennai, India Virtusa Full time

    Overall Eight plus years of experience with min of five plus years experience in required technical skills Required technical skills: Python, PySpark, SQL AWS Cloud experience and Healthcare domain knowledge is desired **About Virtusa** Teamwork, quality of life, professional and personal development: values that Virtusa is proud to embody. When you join...

  • Data Engineer

    4 weeks ago

    Chennai, India SDNA Global Full time

    Job Description :Roles and Responsibilities :We are seeking an experienced Data Engineer to join our team. The ideal candidate will have 5 to 8 years of experience in data engineering and a strong background in building data pipelines, data warehousing, and data management solutions. In this role, you will work closely with data scientists, analysts, and...

  • Python Pyspark

    3 months ago

    Chennai, India Virtusa Full time

    Overall Eight plus years of experience with min of five plus years experience in required technical skills Required technical skills: Python, PySpark, SQL AWS Cloud experience and Healthcare domain knowledge is desired Notice Period - Immediate to 30 days **About Virtusa** Teamwork, quality of life, professional and personal development: values that...

  • Colan Infotech

    4 months ago

    Chennai, India Colan Infotech Pvt Ltd Full time

    Skill Set : Pyspark / Scala Spark, Python, Azure Databricks and Azure Data FactoryRoles And Responsibilities:- Should have programming skills with the ability to write optimized and reusable high-quality code.- Design, develop and maintain scalable data pipelines using Pyspark / Scala Spark, Databricks, Python, SQL.- Optimize SQL queries for efficient data...

  • Data Streaming

    4 months ago

    Chennai, India ACE Talent Consulting Full time

    ob Description: Spark/Scala/PySpark developer who knows how to fully exploit the potential of our Spark cluster. Should have ability to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data. **Responsibilities**: Create Scala/Spark/Pyspark jobs for data transformation and aggregation Produce...

  • Data Streaming

    4 months ago

    Chennai, India ACE Talent Consulting Full time

    Job Description: Spark/Scala/PySpark developer who knows how to fully exploit the potential of our Spark cluster. Should have ability to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data. **Responsibilities**: Create Scala/Spark/Pyspark jobs for data transformation and aggregation Produce...

  • Azure Data Engineer/Lead

    2 months ago

    Chennai, India Truelancer.com Full time

    Job Title : LEAD DATA ENGINEER - AzureYears of Experience : 6+ yearsLocation: Chennai, Coimbatore (Hybrid - WFO 3 days a week)Role Description :- As an Azure Lead Data Engineer, your primary role is to design, develop, and manage data systems on the Microsoft Azure cloud platform. You will be responsible for leading a team of data engineers and collaborating...

  • Python Pyspark

    4 months ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Data engineer 1) Overall 8+ years in IT Experience with relevant 5+ years of experience in Data Engineering. Required technical skills: Python, PySpark, SQL, AWS 2) Required Domain: Healthcare Insurance 4) Extensive data analysis skills and Good communication skills 5) Experience working in Agile **About Virtusa** Teamwork, quality of life,...