Databricks Spark

3 days ago

Chennai Tamil Nadu, India Cognizant Full time

**Job Summary**

**Responsibilities**
- Develop and maintain scalable data processing solutions using Kafka Python and PySpark to enhance data flow and analytics capabilities.
- Collaborate with cross-functional teams to design and implement Databricks Workflows that streamline data operations and improve efficiency.
- Utilize Databricks SQL to perform complex data queries and generate actionable insights for business stakeholders.
- Ensure data integrity and quality by implementing robust data validation and error-handling mechanisms.
- Optimize existing data pipelines for performance and scalability ensuring they meet the evolving needs of the organization.
- Provide technical guidance and support to junior developers fostering a culture of continuous learning and improvement.
- Participate in code reviews and contribute to the development of best practices and coding standards.
- Work closely with product managers and business analysts to understand requirements and translate them into technical solutions.
- Monitor and troubleshoot data processing systems to ensure high availability and reliability.
- Stay updated with the latest industry trends and technologies to drive innovation and maintain a competitive edge.
- Document technical specifications and system designs to facilitate knowledge sharing and collaboration.
- Engage in regular team meetings to discuss project progress challenges and opportunities for improvement.
- Contribute to the companys purpose by developing solutions that enhance operational efficiency and deliver value to society.

**Qualifications**
- Possess strong expertise in Kafka Python Databricks SQL Databricks Workflows and PySpark with a proven track record of successful implementations.
- Demonstrate proficiency in designing and optimizing data pipelines for large-scale data processing.
- Exhibit excellent problem-solving skills and the ability to work effectively in a hybrid work model.
- Have experience in the Claims and Billing domains which is considered a plus.
- Show strong communication skills and the ability to collaborate with diverse teams.
- Display a commitment to continuous learning and staying abreast of emerging technologies.
- Hold a bachelors degree in Computer Science Information Technology or a related field.

**Certifications Required**

Certified Apache Kafka Developer Databricks Certified Data Engineer Associate

  • Databricks+Spark

    14 hours ago

    Chennai, Tamil Nadu, India Cognizant Technology Solutions Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job Title: - Databricks+SparkLocation: - KolkataMust Have SkillsMinimum 6 to 9 years of experienceHands on experience with Azure Databricks, Spark, Azure SQL DW/Synapse, and Azure functions.In-depth knowledge of relational database design, data warehousing and dimensional data modeling conceptsWorking experience with Python, Spark,.

  • Data Engineer

    4 weeks ago

    Chennai, India YO IT CONSULTING Full time

    Title : Data EngineerExperience : 4 to 6 yearsLocation : ChennaiMust Have :- Databricks- Spark- Snowflake Schema- AzureRole And Responsibilities :- Develop, maintain, and monitor data pipelines, ensuring reliability and prompt resolution of failures as they occur.- Collaborate with stakeholders to gather requirements, provide updates, and participate in...

  • Databricks

    1 week ago

    Chennai, Tamil Nadu, India Cognizant Full time

    **Job Summary** **Responsibilities** - Develop and optimize data solutions using Databricks SQL to ensure efficient data processing and analysis. - Implement and manage Databricks Workflows to automate data pipelines and improve operational efficiency. - Collaborate with cross-functional teams to understand business requirements and translate them into...

  • Databricks Platform Administrator

    3 days ago

    Chennai, Tamil Nadu, India Tiger Analytics Full time

    The Databricks Administrator will be responsible for providing technical expertise in deploying administration, configuration, and optimization of the Databricks platform to enable data analytics, machine learning, and data engineering activities within the organization. You will collaborate with data engineers, data scientists, and other stakeholders to...

  • Databricks spark

    1 week ago

    Chennai, Tamil Nadu, India Cognizant Technology Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job SummaryThe Sr. Developer role is crucial for driving innovation and efficiency in our hybrid work model. With a focus on Kafka Python Databricks SQL Databricks Workflows and PySpark the candidate will enhance our data processing capabilities. Experience in Claims and Billing domains is advantageous. This position requires 8 to 9 years of experience and...

  • Databrick

    1 week ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Key Responsibilities: Design, develop, and maintain scalable data pipelines using Apache Spark on Databricks. Write efficient and production-ready PySpark or Scala code for data transformation and ETL processes. Integrate data from various structured and unstructured sources into a unified platform. Implement Delta Lake and manage data versioning, updates,...

  • Data Architect- Databricks

    3 days ago

    Chennai, Tamil Nadu, India InfoCepts Full time

    **Position**: Data Architect (Databricks, Snowflake) **Purpose of the Position**: Design and optimize cloud-native data architectures on platforms like Databricks and Snowflake, enabling scalable data engineering, advanced analytics, and AI/ML solutions aligned with business needs. **Location**: Nagpur, Pune, Chennai, Bangalore **Type of Employment**:Full...

  • Databricks Bangalore

    1 week ago

    Chennai, Tamil Nadu Karnataka, India Diverse Lynx Full time

    Hiring for Databricks Bangalore Design develop and maintain ETL ELT pipelines using Azure Databricks PySpark and Delta Lake Build and optimize data pipelines for large-scale data ingestion transformation and processing Collaborate with data analysts data scientists and business teams to deliver reliable and high-performing datasets Develop data models manage...

  • Azure Databricks

    7 days ago

    Chennai, Tamil Nadu, India Cognizant Full time

    **Skill: Azure Databricks** **Experience**: 9 to 12 years **Location: AIA-Gurgaon** **Job Summary** As a Sr. Developer you will leverage your expertise in Databricks PySpark and AWS to drive innovative solutions in a hybrid work model. With 6 to 10 years of experience you will collaborate with cross-functional teams to enhance data processing...

  • Principal Consultant, Sr. Databricks Developer

    4 weeks ago

    Chennai, Tamil Nadu, India GENPACT Full time

    Ready to shape the future of work At Genpact we don t just adapt to change we drive it AI and digital innovation are redefining industries and we re leading the charge Genpact s AI Gigafactory our industry-first accelerator is an example of how we re scaling advanced technology solutions to help global enterprises work smarter grow faster and transform at...