Databricks spark

7 days ago

Chennai, Tamil Nadu, India Cognizant Technology Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Job Summary

The Sr. Developer role is crucial for driving innovation and efficiency in our hybrid work model. With a focus on Kafka Python Databricks SQL Databricks Workflows and PySpark the candidate will enhance our data processing capabilities. Experience in Claims and Billing domains is advantageous. This position requires 8 to 9 years of experience and offers a day shift schedule with no travel requirements.

Responsibilities

  • Develop and maintain scalable data processing solutions using Kafka Python and PySpark to enhance data flow and analytics capabilities.
  • Collaborate with cross-functional teams to design and implement Databricks Workflows that streamline data operations and improve efficiency.
  • Utilize Databricks SQL to perform complex data queries and generate actionable insights for business stakeholders.
  • Ensure data integrity and quality by implementing robust data validation and error-handling mechanisms.
  • Optimize existing data pipelines for performance and scalability ensuring they meet the evolving needs of the organization.
  • Provide technical guidance and support to junior developers fostering a culture of continuous learning and improvement.
  • Participate in code reviews and contribute to the development of best practices and coding standards.
  • Work closely with product managers and business analysts to understand requirements and translate them into technical solutions.
  • Monitor and troubleshoot data processing systems to ensure high availability and reliability.
  • Stay updated with the latest industry trends and technologies to drive innovation and maintain a competitive edge.
  • Document technical specifications and system designs to facilitate knowledge sharing and collaboration.
  • Engage in regular team meetings to discuss project progress challenges and opportunities for improvement.
  • Contribute to the companys purpose by developing solutions that enhance operational efficiency and deliver value to society.

Qualifications

  • Possess strong expertise in Kafka Python Databricks SQL Databricks Workflows and PySpark with a proven track record of successful implementations.
  • Demonstrate proficiency in designing and optimizing data pipelines for large-scale data processing.
  • Exhibit excellent problem-solving skills and the ability to work effectively in a hybrid work model.
  • Have experience in the Claims and Billing domains which is considered a plus.
  • Show strong communication skills and the ability to collaborate with diverse teams.
  • Display a commitment to continuous learning and staying abreast of emerging technologies.
  • Hold a bachelors degree in Computer Science Information Technology or a related field.

Certifications Required

Certified Apache Kafka Developer Databricks Certified Data Engineer Associate

  • Databricks+Spark

    18 hours ago

    Chennai, Tamil Nadu, India Cognizant Technology Solutions Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job Title: - Databricks+SparkLocation: - KolkataMust Have SkillsMinimum 6 to 9 years of experienceHands on experience with Azure Databricks, Spark, Azure SQL DW/Synapse, and Azure functions.In-depth knowledge of relational database design, data warehousing and dimensional data modeling conceptsWorking experience with Python, Spark,.

  • Databricks Architect

    7 days ago

    Chennai, Tamil Nadu, India DataTerrain, Inc. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job Title: Databricks ArchitectExperience: 12+ yearsJob Summary:We are seeking an experienced Databricks Architect to design, implement, and optimize data solutions on the Databricks platform. The ideal candidate will have deep expertise in big data technologies, cloud platforms, and analytics architectures, with strong communication skills to interact...

  • Databrick

    17 hours ago

    Chennai, Tamil Nadu, India Virtusa Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    We are seeking an experienced Databricks Developer with 5 to 7 years of expertise in big data engineering and analytics platforms. The ideal candidate will have deep experience in Apache Spark Databricks PySpark and cloud platforms such as Azure or AWS and a strong background in data pipeline development performance optimization and data governance.Key...

  • Databricks Architect

    2 days ago

    Chennai, Tamil Nadu, India NotionMindz Technology LLP Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Role:Databricks ArchitectDesignation:Data ArchitectExperience:7–10 yearsLocation:ChennaiAbout the role:We're looking for aDatabricks Architectto design, build, and optimize data pipelines and large-scale integrations for enterprise-grade data environments. The ideal candidate will bring strong experience inAzure Databricks, Data Factory, and SQL-based data...

  • Azure Databricks, Data Factory and SQL

    21 hours ago

    Chennai, Tamil Nadu, India Cognizant Technology Solutions Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Skill: DatabricksExperience: 6 to 13 yearsLocation: BhubaneswarJob descriptionResponsibilities:Designing, building, and deploying data pipelines and workflows using Databricks.Developing and optimizing Apache Spark applications for data processing and analytics.Integrating Databricks with various data sources and platforms.Implementing data quality checks...

  • AWS Glue Databricks Python

    23 hours ago

    Chennai, Tamil Nadu, India Changeleaders Consulting Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    The Candidate must have 4 Year of relevant of experience in Databricks AWS Python Hands on experience on AWS Cloud platform especially S3 Glue Lamda. Experience on spark scripting.

  • Databricks AI Engineer

    2 days ago

    Chennai, Tamil Nadu, India Enoah Isolution Full time ₹ 1,50,00,000 - ₹ 3,00,00,000 per year

    Role & responsibilitiesProgramming & Query Languages: Proficient in Python and SQL for data analysis, modeling, and database management.Machine Learning & AI Models: Experience with regression, decision trees, SVM, deep learning (CNNs, RNNs, Transformers), and advanced AI (GNNs, RL, GANs, LLMs such as GPT-4, Claude, LLaMA).Natural Language & Vision AI:...

  • Senior Data Engineer – Databricks

    19 hours ago

    Chennai, Tamil Nadu, India Zillion Technologies Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Senior Data Engineer – DatabricksLocation: Hyderabad/ChennaiWork Schedule: Timings 2:30 PM to 11:30 PMRole Overview:We are seeking a highly skilled Senior Data Engineer with deep expertise in Databricks to design, build, and optimize scalable data pipelines and architectures. You will collaborate with cross-functional teams including data scientists,...

  • DataBricks / AWS Redshift

    1 week ago

    Chennai, Tamil Nadu, India Virtusa Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Key responsibilities include designing developing and maintaining scalable data pipelines using Apache Spark on Databricks Writing efficient and production ready PySpark or Scala code for data transformation and ETL processes Integrating data from various structured and unstructured sources into a unified platform Implementing Delta Lake and managing data...

  • Senior Data Engineer- Azure Databricks

    23 hours ago

    Chennai, Tamil Nadu, India InfoCepts Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Position: Data Engineer- Azure DatabricksPurpose of the Position:To design, build, and optimize scalable data pipelines and solutions using Azure Databricks and related technologies, enabling Zodiac Maritime to make faster, data-driven decisions as part of its data transformation journey.Proficiency in data integration techniques, ETL processes and data...