Databricks

1 day ago

Chennai Tamil Nadu, India Cognizant Full time

**Job Summary**

**Responsibilities**
- Lead the design and implementation of data processing pipelines using Spark in Scala and PySpark
- Oversee the administration of Databricks Unity Catalog and manage data governance
- Provide expertise in Delta Sharing and ensure secure data sharing practices
- Develop and maintain Databricks CLI scripts for automation and operational efficiency
- Implement Delta Live Pipelines for real-time data processing and analytics
- Manage risk by identifying and mitigating potential data-related issues
- Integrate Apache Airflow for orchestrating complex data workflows
- Optimize data storage and retrieval using Amazon S3 and Amazon Redshift
- Write efficient and maintainable code in Python for various data processing tasks
- Create and manage Databricks SQL queries for data analysis and reporting
- Ensure data integrity and performance with Databricks Delta Lake
- Develop and monitor Databricks Workflows to automate data tasks
- Collaborate with cross-functional teams to understand business requirements and deliver solutions
- Communicate effectively in English both written and spoken to document processes and interact with stakeholders
- Stay updated with the latest industry trends and best practices in data engineering and Property & Casualty Insurance

**Qualifications**
- Possess strong technical skills in Spark in Scala PySpark and Databricks ecosystem
- Have experience with Delta Sharing Databricks Unity Catalog Admin and Databricks CLI
- Demonstrate expertise in Delta Live Pipelines and Structured Streaming
- Show proficiency in Risk Management and Apache Airflow
- Be skilled in using Amazon S3 and Amazon Redshift for data storage solutions
- Have advanced knowledge of Python for data processing
- Be experienced in Databricks SQL Databricks Delta Lake and Databricks Workflows
- Have a background in Property & Casualty Insurance
- Be proficient in English for effective communication
- Be able to work in a hybrid model with day shifts
- Have a minimum of 8 years and a maximum of 12 years of relevant experience
- Be detail-oriented and capable of managing multiple tasks simultaneously
- Be committed to continuous learning and professional development.

  • Databricks

    1 day ago

    Chennai, Tamil Nadu, India Cognizant Full time

    **Job Summary** Join our dynamic team as a Developer where you will leverage your expertise in Spark in Scala Databricks and Amazon technologies to drive innovative solutions. With a hybrid work model and day shifts you will collaborate with cross-functional teams to optimize data processes and enhance our data infrastructure. Your contributions will...

  • Databricks

    1 week ago

    Chennai, Tamil Nadu, India Cognizant Full time

    **Job Summary** **Responsibilities** - Develop and optimize data solutions using Databricks SQL to ensure efficient data processing and analysis. - Implement and manage Databricks Workflows to automate data pipelines and improve operational efficiency. - Collaborate with cross-functional teams to understand business requirements and translate them into...

  • Databrick

    2 weeks ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Key Responsibilities: Design, develop, and maintain scalable data pipelines using Apache Spark on Databricks. Write efficient and production-ready PySpark or Scala code for data transformation and ETL processes. Integrate data from various structured and unstructured sources into a unified platform. Implement Delta Lake and manage data versioning, updates,...

  • Pyspark+databricks

    7 days ago

    Chennai, Tamil Nadu, India Cognizant Full time

    **Exp: 4 to 13 years** **Skill: Data Bricks+Pyspark** **Location : Bangalore/Hyderabad/Kolkota/Pune/Chennai** **Technical Skills**: Python,PySpark,Azure Data Lake Store,Databricks Workflows,Databricks SQL **Responsibilities**: - Develop and optimize data solutions using Azure Data Lake Store to support business needs. - Utilize Python to create...

  • Databricks Platform Administrator

    4 days ago

    Chennai, Tamil Nadu, India Tiger Analytics Full time

    The Databricks Administrator will be responsible for providing technical expertise in deploying administration, configuration, and optimization of the Databricks platform to enable data analytics, machine learning, and data engineering activities within the organization. You will collaborate with data engineers, data scientists, and other stakeholders to...

  • Azure Databricks Trainer

    1 day ago

    Chennai, Tamil Nadu, India Ednue Technologies Private Limited Full time

    We are seeking a knowledgeable and experienced **Azure Databricks Trainer** to design and deliver comprehensive training sessions for professionals and aspiring data engineers. The trainer will be responsible for conducting hands-on workshops, developing course content, and mentoring participants on real-world Azure Databricks projects. **Key...

  • Databricks Setup with Aws

    1 hour ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Job Description: We are looking for a Databricks Setup Specialist to lead the deployment and configuration of Databricks on AWS, focusing on PrivateLink integration and Unity Catalog administration. This role involves collaboration with AWS DevOps and Security teams to create a secure and compliant data platform. Key Responsibilities: Deploy Databricks...

  • Data Architect- Databricks

    2 weeks ago

    Chennai, Tamil Nadu, India InfoCepts Full time

    **Position**: Data Architect (Databricks, Snowflake) **Purpose of the Position**: Design and optimize cloud-native data architectures on platforms like Databricks and Snowflake, enabling scalable data engineering, advanced analytics, and AI/ML solutions aligned with business needs. **Location**: Nagpur, Pune, Chennai, Bangalore **Type of Employment**:Full...

  • Data Architect- Databricks

    4 days ago

    Chennai, Tamil Nadu, India InfoCepts Full time

    **Position**: Data Architect (Databricks, Snowflake) **Purpose of the Position**: Design and optimize cloud-native data architectures on platforms like Databricks and Snowflake, enabling scalable data engineering, advanced analytics, and AI/ML solutions aligned with business needs. **Location**: Nagpur, Pune, Chennai, Bangalore **Type of Employment**:Full...

  • Databricks

    2 days ago

    Chennai, Tamil Nadu, India Cognizant Technology Solutions Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    Job SummaryJoin our dynamic team as a Developer where you will leverage your expertise in Spark in Scala Databricks and Amazon technologies to drive innovative solutions. With a hybrid work model and day shifts you will collaborate with cross-functional teams to optimize data processes and enhance our data infrastructure. Your contributions will directly...