Azure Kafka_Parthiban_Capgemini

Skills  
• years of relevant IT experience in the BI/DW domain with minimum of hands-on experience on Azure modern data platform that includes Data Factory, Databricks, Synapse (Azure SQL DW) and Azure Data Lake

• Meaningful experience of data analysis and transformation using Python/R/Scala on Azure Databricks or Apache Spark

• Well versed NoSQL data store concepts

• Good knowledge in Distributed Processing using Databricks (preferred) or Apache Spark

• Ability to debug using tools like Ganglia UI, expertise in Optimizing Spark Jobs 

• The ability to work across structured, semi-structured, and unstructured data, extracting information and identifying linkages across disparate data sets

• Expert in creating data structures optimized for storage and various query patterns for e.g. Parquet and Delta Lake

• Meaningful experience in at least one database technology in each segment such as:
o Traditional RDBMS (MS SQL Server, Oracle)
o NoSQL (MongoDB, Cassandra, Neo4J, Cosmos DB, Gremlin)

• Understanding of Information Security principles to ensure compliant handling and management of data

• Effective in communication

• Proficient at working with large and complex code bases (Github, Gitflow, Fork/Pull Model) and Realtime data processing.

• Working experience in Agile methodologies (SCRUM, XP, Kanban)
Hands on development experience Data Factory, Databricks, Synapse (Azure SQL DW) and Azure Data Lake,Python/R/Scala . Expertise in traditional and NoSQL Database.

• Kafka Confluent admin and development

• ADB streaming knowledge.

---