Databricks

2 weeks ago

Hyderabad, Telangana, India Cognizant Technology Solutions Full time ₹ 20,00,000 - ₹ 25,00,000 per year

Job Summary

We are seeking a highly skilled Sr. Developer with 8 to 12 years of experience to join our team. The ideal candidate will have extensive experience in Spark in Scala Delta Sharing Databricks Unity Catalog Admin Databricks CLI Delta Live Pipelines Structured Streaming Risk Management Apache Airflow Amazon S3 Amazon Redshift Python Databricks SQL Databricks Delta Lake Databricks Workflows and PySpark. This is a hybrid role with day shifts and no travel required.

Responsibilities

  • Develop and maintain scalable data pipelines using Spark in Scala to ensure efficient data processing and transformation.
  • Implement and manage Delta Sharing to facilitate secure and efficient data sharing across various platforms.
  • Administer Databricks Unity Catalog to ensure proper data governance and access control.
  • Utilize Databricks CLI for seamless integration and automation of Databricks workflows.
  • Design and implement Delta Live Pipelines to automate data processing and ensure data quality.
  • Leverage Structured Streaming to handle real-time data processing and analytics.
  • Apply risk management techniques to identify and mitigate potential data-related risks.
  • Integrate and manage Apache Airflow for orchestrating complex data workflows.
  • Utilize Amazon S3 for scalable and secure data storage solutions.
  • Implement data warehousing solutions using Amazon Redshift for efficient data querying and reporting.
  • Develop and maintain Python scripts for data processing and automation tasks.
  • Utilize Databricks SQL for querying and analyzing large datasets.
  • Implement Databricks Delta Lake to ensure data reliability and performance.
  • Manage Databricks Workflows to streamline data processing and analytics tasks.
  • Develop and maintain PySpark applications for large-scale data processing.
  • Collaborate with cross-functional teams to ensure seamless data integration and delivery.
  • Provide technical expertise and support to team members and stakeholders.
  • Ensure all data solutions align with company goals and industry best practices.
  • Continuously monitor and optimize data workflows for performance and efficiency.
  • Stay updated with the latest industry trends and technologies to ensure the company remains competitive.
  • Contribute to the companys mission by delivering high-quality data solutions that drive business success.

Qualifications

  • Must have extensive experience with Spark in Scala Delta Sharing Databricks Unity Catalog Admin Databricks CLI Delta Live Pipelines Structured Streaming Risk Management Apache Airflow Amazon S3 Amazon Redshift Python Databricks SQL Databricks Delta Lake Databricks Workflows and PySpark.
  • Strong problem-solving skills and the ability to work independently.
  • Excellent communication and collaboration skills.
  • Proven track record of delivering high-quality data solutions.
  • Ability to work in a hybrid work model.
  • Strong understanding of data governance and security best practices.
  • Experience with real-time data processing and analytics.
  • Knowledge of data warehousing and ETL processes.
  • Familiarity with cloud-based data storage and processing solutions.
  • Ability to manage and optimize complex data workflows.
  • Strong attention to detail and commitment to data quality.
  • Ability to stay updated with the latest industry trends and technologies.

  • Databricks+spark

    5 days ago

    Hyderabad, Telangana, India Cognizant Technology Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job SummaryWe are seeking a highly skilled Sr. Developer with 8 to 12 years of experience to join our team in a hybrid work model. The ideal candidate will have expertise in Databricks SQL Databricks Delta Lake Databricks Workflows and PySpark. Experience in Sales & Marketing domains is a plus. The role involves developing and optimizing data solutions to...

  • Databricks Architect

    2 weeks ago

    Hyderabad, Telangana, India Paltech Consulting Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Key Responsibilities:Design and develop the migration strategies and processesCollaborate with stakeholders to understand business requirements and technical challenges.Analyze current data and scope for optimization during the migration process.Define the architecture and roadmap for cloud-based data and analytics transformation on Databricks.Design,...

  • Databricks Engineer

    4 days ago

    Hyderabad, Telangana, India NTT DATA Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Design, build, and maintain robust ETL/ELT data pipelines using Apache Spark on Databricks. Implement data Lakehouse architecture using Delta Lake for cost-effective data storage and analytics. Use Databricks Workflows for orchestrating batch and streaming pipelines. Develop and maintain CI/CD pipelines for data applications using tools such as Azure DevOps,...

  • Databricks Developer

    2 weeks ago

    Hyderabad, Telangana, India Zetamicron Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Job Title:Databricks DeveloperLocation:Bangalore, Chennai, Hyderabad, PuneExperience:5–7 YearsAbout the RoleWe are seeking an experiencedDatabricks Developerwith 5–7 years of expertise in Big Data engineering and analytics platforms. The ideal candidate will have strong hands-on experience inApache Spark, Databricks, PySpark, and cloud platforms such...

  • Databricks + Pyspark

    5 days ago

    Hyderabad, Telangana, India Cognizant Full time ₹ 1,00,00,000 - ₹ 3,00,00,000 per year

    Skills- Databricks+ PysparkExperience: 4 to 13 yearsLocation: AIA-PuneWe are looking for a highly skilled Data Engineer with expertise in PySpark and Databricks to design, build, and optimize scalable data pipelines for processing massive datasets.Key Responsibilities:Build & Optimize Pipelines: Develop high-throughput ETL workflows using PySpark on...

  • Pyspark + Databricks

    6 days ago

    Hyderabad, Telangana, India Cognizant Technology Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job SummaryWe are seeking a highly skilled Sr. Developer with 6 to 10 years of experience to join our team. The ideal candidate will have expertise in Databricks SQL Databricks Workflows and PySpark. Experience in the Cards & Payments domain is a plus. This is a hybrid work model with day shifts and no travel required.ResponsibilitiesDevelop and maintain...

  • Databricks Engineer

    2 weeks ago

    Hyderabad, Telangana, India ValueMomentum Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Company DescriptionValueMomentum is a leading solutions provider for the global property and casualty insurance industry, backed by deep domain and technology capabilities. Our mission is to help insurers achieve sustained growth and high performance by enhancing stakeholder value and building resilient societies. Trusted by over 100 insurers globally, we...

  • Databricks with Python

    5 days ago

    Hyderabad, Telangana, India Tata Consultancy Services Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Role - Databricks with PythonExp - 5 to 9 YrsLocation - HyderabadNote - This is a scheduled drive. Only eligible candidates who receive invite from us will be allowed inside Campus.Job DescriptionExtensive expertise in designing and implementing data load processes using Azure Data Factory, Azure Databricks, Delta Lake, Azure Delta Lake Storage and...

  • Azure Databricks Engineer

    2 weeks ago

    Hyderabad, Telangana, India Tekskills Inc. Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Role: Azure Databricks EngineerJob Location: Pune, MH / Chennai, TN / Hyderabad. TG / Indore, MP / Kolkata, WB / Bangalore, TNWork Mode: (WFO)Skills Required: Digital: DatabricksExperience: 6-8 Years(5+ yrs)Job Description:Should build Databricks Notebook developed in spark sql/pysparkShould be able to create unity catalog tablesShould have good hands on...

  • Databricks Engineer

    2 weeks ago

    Hyderabad, Telangana, India Oblytech Full time US$ 90,000 - US$ 1,20,000 per year

    Job Title:Databricks Engineer (PySpark Developer)Employment Type:Full-Time | Permanent | Remote (Work From Home)Industry:IT Services & Consulting | Software Development | Data Engineering | Cloud ServicesFunctional Area:Data Engineering | Big Data | Cloud Platforms | AnalyticsAbout Oblytech:Oblytech is a fast-growing IT consulting and software services firm,...