Databricks spark

6 days ago

Chennai, Tamil Nadu, India Cognizant Technology Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Job Summary

The Sr. Developer role is crucial for driving innovation and efficiency in our hybrid work model. With a focus on Kafka Python Databricks SQL Databricks Workflows and PySpark the candidate will enhance our data processing capabilities. Experience in Claims and Billing domains is advantageous. This position requires 8 to 9 years of experience and offers a day shift schedule with no travel requirements.

Responsibilities

  • Develop and maintain scalable data processing solutions using Kafka Python and PySpark to enhance data flow and analytics capabilities.
  • Collaborate with cross-functional teams to design and implement Databricks Workflows that streamline data operations and improve efficiency.
  • Utilize Databricks SQL to perform complex data queries and generate actionable insights for business stakeholders.
  • Ensure data integrity and quality by implementing robust data validation and error-handling mechanisms.
  • Optimize existing data pipelines for performance and scalability ensuring they meet the evolving needs of the organization.
  • Provide technical guidance and support to junior developers fostering a culture of continuous learning and improvement.
  • Participate in code reviews and contribute to the development of best practices and coding standards.
  • Work closely with product managers and business analysts to understand requirements and translate them into technical solutions.
  • Monitor and troubleshoot data processing systems to ensure high availability and reliability.
  • Stay updated with the latest industry trends and technologies to drive innovation and maintain a competitive edge.
  • Document technical specifications and system designs to facilitate knowledge sharing and collaboration.
  • Engage in regular team meetings to discuss project progress challenges and opportunities for improvement.
  • Contribute to the companys purpose by developing solutions that enhance operational efficiency and deliver value to society.

Qualifications

  • Possess strong expertise in Kafka Python Databricks SQL Databricks Workflows and PySpark with a proven track record of successful implementations.
  • Demonstrate proficiency in designing and optimizing data pipelines for large-scale data processing.
  • Exhibit excellent problem-solving skills and the ability to work effectively in a hybrid work model.
  • Have experience in the Claims and Billing domains which is considered a plus.
  • Show strong communication skills and the ability to collaborate with diverse teams.
  • Display a commitment to continuous learning and staying abreast of emerging technologies.
  • Hold a bachelors degree in Computer Science Information Technology or a related field.

Certifications Required

Certified Apache Kafka Developer Databricks Certified Data Engineer Associate

  • Databricks

    2 weeks ago

    Chennai, Tamil Nadu, India Cognizant Technology Solutions Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job SummaryJoin our dynamic team as a Developer where you will leverage your expertise in Spark in Scala Databricks and Amazon technologies to drive innovative solutions. With a hybrid work model and day shifts you will collaborate with cross-functional teams to optimize data processes and enhance our data infrastructure. Your contributions will directly...

  • Databricks Developer

    2 weeks ago

    Chennai, Tamil Nadu, India Tekskills Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Job Description:• Data Pipeline Design: Architect and implement scalable, reliable data pipelines using Databricks, Spark, and Delta Lake• Medallion Architecture Implementation: Ingest raw data into the Bronze layer, process and clean it in the Silver layer, and aggregate/transform it for analytics in the Gold layer• Data Modeling: Design and optimize...

  • Databricks Architect

    6 days ago

    Chennai, Tamil Nadu, India DataTerrain, Inc. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job Title: Databricks ArchitectExperience: 12+ yearsJob Summary:We are seeking an experienced Databricks Architect to design, implement, and optimize data solutions on the Databricks platform. The ideal candidate will have deep expertise in big data technologies, cloud platforms, and analytics architectures, with strong communication skills to interact...

  • Databricks Developers

    2 weeks ago

    Chennai, Tamil Nadu, India Cynosure Corporate Solutions Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Key Responsibilities:Develop and optimize data pipelines in Databricks for transforming and processing data from various sources.Integrate data using Unity Catalog and external data sources (data lakes, APIs, etc.).Write Spark SQL and PySpark scripts for data transformations, optimizations, and creating views/procedures.Perform data analysis to identify...

  • Databricks Data Engineer 6+ years

    2 weeks ago

    Chennai, Tamil Nadu, India Raah Techservices Full time ₹ 4,69,000 - ₹ 18,17,548 per year

    Position OverviewWe are looking for an experienced Databricks Data Engineer with 6+ years of expertise in building and managing large-scale data platforms. The ideal candidate will have strong hands-on experience with Databricks, Spark, and Cloud Data Engineering practices. This role involves designing, developing, and optimizing data pipelines, enabling...

  • DataBricks / AWS Redshift

    4 days ago

    Chennai, Tamil Nadu, India Virtusa Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Key responsibilities include designing developing and maintaining scalable data pipelines using Apache Spark on Databricks Writing efficient and production ready PySpark or Scala code for data transformation and ETL processes Integrating data from various structured and unstructured sources into a unified platform Implementing Delta Lake and managing data...

  • Data Architect- Databricks

    1 week ago

    Chennai, Tamil Nadu, India InfoCepts Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Position: Data Architect (Databricks, Snowflake)Location: Nagpur, Pune, Chennai, BangaloreType of Employment: Full time EmploymentPurpose of the Position:Design and optimize cloud-native data architectures on platforms like Databricks and Snowflake, enabling scalable data engineering, advanced analytics, and AI/ML solutions aligned with business needs.Key...

  • Data Engineer- Azure Databricks

    2 weeks ago

    Chennai, Tamil Nadu, India InfoCepts Full time ₹ 6,00,000 - ₹ 12,00,000 per year

    Position: Data Engineer ? Azure DatabricksPurpose of the Position:To design, build, and optimize scalable data pipelines and solutions using Azure Databricks and related technologies, enabling Zodiac Maritime to make faster, data-driven decisions as part of its data transformation journey.Proficiency in data integration techniques, ETL processes and data...

  • Senior Databricks Engineer

    6 days ago

    Chennai, Tamil Nadu, India Zorba Consulting Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Experience : 8-15 years. Primary Roles And Responsibilities : Developing Modern Data Warehouse solutions using Databricks and AWS/ Azure Stack. Ability to provide solutions that are forward-thinking in data engineering and analytics space. Collaborate with DW/BI leads to understand new ETL pipeline development requirements. Triage issues to find gaps in...

  • Azure Data Factory,Azure Python, Pyspark, Databricks, SQL Professional

    6 days ago

    Chennai, Tamil Nadu, India Suzva Software Technologies Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Experience with Azure datafactory,Azure ,Python, Pyspark, Databricks, SQL Key ResponsibilitiesDesign and build ETL/ELT pipelines to ingest, transform, and load data from clinical, omics, research, and operational sources.Optimize performance and scalability of data flows using tools like Apache Spark, Databricks, or AWS Glue.Collaborate with domain experts...