Databricks Solutions Architect

3 weeks ago

Hyderabad, Telangana, India Tiger Analytics Full time

Job Title: Databricks Solutions Architect

We're seeking an experienced Databricks Expert/Architect to join Tiger Analytics's data services team. This role will play a key part in designing, implementing, and optimizing Databricks solutions across the organization.

Key Responsibilities:

  1. Analyze business requirements and translate them into technical specifications for data pipelines, data lakes, and analytical processes on the Databricks platform.
  2. Design and architect end-to-end data solutions, including data ingestion, storage, transformation, and presentation layers, to meet business needs and performance requirements.
  3. Lead the setup, configuration, and optimization of Databricks clusters, workspaces, and jobs to ensure the platform operates efficiently and meets performance benchmarks.
  4. Manage access controls and security configurations to ensure data privacy and compliance.
  5. Design and implement data integration processes, ETL workflows, and data pipelines to extract, transform, and load data from various sources into the Databricks platform.
  6. Optimize ETL processes to achieve high data quality and reduce latency.
  7. Monitor and optimize query performance and overall platform performance to ensure efficient execution of analytical queries and data processing jobs.
  8. Identify and resolve performance bottlenecks in the Databricks environment.
  9. Establish and enforce best practices, standards, and guidelines for Databricks development, ensuring data quality, consistency, and maintainability.
  10. Implement data governance and data lineage processes to ensure data accuracy and traceability.
  11. Mentor and train team members on Databricks best practices, features, and capabilities.
  12. Conduct knowledge-sharing sessions and workshops to foster a data-driven culture within the organization.
  13. Will be responsible for Databricks Practice Technical/Partnership initiatives.
  14. Build skills in technical areas which support the deployment and integration of Databricks-based solutions to complete customer projects.

Requirements:

  • Strong in Unity Catalog, Delta Lake, dbConnect, db API 2.0
  • Databricks workflow orchestration, Security management, Platform governance, and Data Security
  • Must know new features available in Databricks and their implications along with various possible use cases.
  • Must have followed various architectural principles to design best suited per problem.
  • Must be well-versed with Databricks Lakehouse concept and its implementation in enterprise environments.
  • Must have a strong understanding of Data warehousing and various governance and security standards around Databricks.
  • Must know about cluster optimization and its integration with various cloud services.
  • Must have a good understanding of creating complex data pipelines.
  • Must be strong in SQL and Spark-SQL.
  • Must have strong performance optimization skills to improve efficiency and reduce cost.
  • Must have worked on designing both Batch and streaming data pipelines.
  • Must have extensive knowledge of Spark and Hive data processing frameworks.
  • Must have worked on any cloud (Azure, AWS, GCP) and most common services like ADLS/S3, ADF/Lambda, CosmosDB/DynamoDB, ASB/SQS, and Cloud databases.
  • Must be strong in writing unit test cases and integration tests.
  • Responsible for setting best practices around Databricks CI/CD.
  • Must understand composable architecture to take fullest advantage of Databricks capabilities.
  • Good to have Rest API knowledge.
  • Good to have an understanding of cost distribution.
  • Good to have if worked on a migration project to build a Unified data platform.
  • SQL Endpoint – Photon engine
  • Hands-on Experience to design and build Databricks-based solutions on any cloud platform.
  • Must be very good at designing End-to-End solutions on cloud platforms.
  • Must have good knowledge of Data Engineering concepts and related services of the cloud.
  • In-depth hands-on implementation knowledge of Databricks. Delta Lake, Delta table - Managing Delta Tables, Delta live tables, Databricks Cluster Configuration, Cluster policies.
  • Experience handling structured and unstructured datasets
  • Strong proficiency in programming languages like Python, Pyspark, Scala, or SQL.
  • Experience with Cloud platforms like AWS understanding of cloud-based data storage and computing services.
  • Familiarity with big data technologies like Apache Spark, Hadoop, and data lake architectures.
  • Develop and maintain data pipelines, ETL workflows, and analytical processes on the Databricks platform.
  • Should have good experience in Data Engineering in Databricks Batch process and Streaming
  • Should have good experience in creating Workflows & Scheduling pipelines.
  • Should have good exposure to how to make packages or libraries available in DB.
  • Familiarity with Databricks default runtimes
  • Databricks Certified Data Engineer Associate/Professional Certification (Desirable).
  • Should have experience working in Agile methodology
  • Strong verbal and written communication skills.
  • Strong analytical and problem-solving skills with a high attention to detail.

  • Enterprise Databricks Solutions Architect

    2 days ago

    Hyderabad, Telangana, India Tiger Analytics Full time

    We are seeking a highly skilled and experienced Enterprise Databricks Solutions Architect to join our global data services team at Tiger Analytics. This individual will play a key role in designing, implementing, and optimizing Databricks solutions across our organization.About the RoleThis is a full-time opportunity with an estimated salary of $140,000 -...

  • Azure Databricks Architect

    1 month ago

    Hyderabad, Telangana, India ValueMomentum Full time

    Job Summary:We are seeking an experienced Databricks Architect to join our team at ValueMomentum. As a key member of our Data and Analytics team, you will be responsible for designing and implementing scalable and efficient data solutions using the Databricks platform.Key Responsibilities:Develop and implement enterprise data architecture strategies aligned...

  • Senior Data Architect

    4 weeks ago

    Hyderabad, Telangana, India TechVedika Full time

    About TechVedikaWe are a product engineering company focused on Artificial Intelligence (AI) solutions, providing cutting-edge products and services to various industries.Our mission is to deliver simple, elegant tech solutions that perform complex tasks, enabling organizations to improve operational efficiency and unleash new business potential.Job...

  • Senior Data Architect

    2 weeks ago

    Hyderabad, Telangana, India TechVedika Full time

    At Tech Vedika, we are seeking a highly skilled Senior Data Architect to lead our Databricks initiatives. This role requires expertise in designing and implementing scalable data architectures, including Data Lakes, Lakehouses, and Data Warehouses.Key Responsibilities:· Serve as the subject matter expert for all Databricks-related activities, including...

  • Data Architect

    3 weeks ago

    Hyderabad, Telangana, India ValueMomentum Full time

    Job Title: Data Architect - Enterprise SolutionsJob Summary:ValueMomentum is seeking an experienced Data Architect to lead the design and implementation of enterprise data solutions using Databricks. The ideal candidate will have a strong background in data engineering, architecture, and governance, with expertise in cloud-based data...

  • AWS Solutions Architect

    1 week ago

    Hyderabad, Telangana, India Visionyle Solutions Full time

    **Job Summary**We are seeking an experienced AWS Solutions Architect to join our team at Visionyle Solutions. The ideal candidate will have a strong background in designing and implementing cloud-based data management frameworks, with expertise in AWS tools and technologies. **Responsibilities**Main Responsibilities:Develop migration strategies using AWS...

  • Salesforce Solutions Architect

    2 weeks ago

    Hyderabad, Telangana, India PRUDENT GLOBALTECH SOLUTIONS PRIVATE LIMITED Full time

    Job Title: Salesforce Solutions ArchitectAt PRUDENT GLOBALTECH SOLUTIONS PRIVATE LIMITED, we are seeking an experienced Salesforce Solutions Architect to lead our Salesforce development initiatives. This role will be responsible for designing and implementing scalable Salesforce solutions that meet the evolving needs of our business.About Us:With over 25...

  • Azure Databricks Data Architect

    3 weeks ago

    Hyderabad, Telangana, India Nuviso Networks Full time

    Data Engineering Role OverviewAt Nuviso Networks, we are seeking a skilled Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines using Azure Databricks. Your expertise in data engineering and cloud architecture will enable us to efficiently ingest and transform data from...

  • Cloud and Big Data Solutions Architect

    4 weeks ago

    Hyderabad, Telangana, India Tiger Analytics Full time

    Job Description Position Overview:Tiger Analytics is seeking a skilled Cloud and Big Data Solutions Architect to lead the design and implementation of data and analytics solutions with a focus on infrastructure and platform strategies. The successful candidate will be responsible for developing and driving comprehensive infrastructure and platform strategies...

  • Enterprise Data Architect for Cloud-Native Solutions

    3 days ago

    Hyderabad, Telangana, India ValueMomentum Full time

    About the RoleWe are seeking an experienced Enterprise Data Architect to lead our cloud-native data strategy. As a key member of our team, you will be responsible for designing and implementing scalable and efficient data solutions using the Databricks platform.

  • Senior Data Architect

    1 week ago

    Hyderabad, Telangana, India UnitedHealth Group Full time

    About UnitedHealth Group UnitedHealth Group is a global organization that delivers care, aided by technology to help millions of people live healthier lives. Our mission is to help people live healthier lives and make the health system work better for everyone. Job Summary We are seeking an experienced Senior Data Architect to lead our team in designing and...

  • Azure Data Solutions Architect

    3 weeks ago

    Hyderabad, Telangana, India Innover Global Inc Full time

    Job Title: Azure Data Solutions ArchitectAbout the Role:We are seeking an experienced Azure Data Solutions Architect to join our team at Innover Global Inc. As a key member of our data engineering team, you will be responsible for designing and implementing scalable data solutions using Azure technologies.Key Responsibilities:Design and implement data...

  • Azure Databricks Data Engineer Lead

    3 weeks ago

    Hyderabad, Telangana, India Alignity Solutions Full time

    About the RoleWe are seeking a skilled Azure Databricks Data Engineer to join our team at Alignity Solutions. The ideal candidate will have experience in designing, developing, and maintaining data pipelines and data streams using Databricks, SQL, PySpark, Spark, Python, and Azure Data Factory (ADF).The successful candidate will be responsible for designing...

  • Data Solutions Architect

    2 weeks ago

    Hyderabad, Telangana, India 860 GapTech India Full time

    About the RoleWe are seeking an experienced Data Solutions Architect to join our team at 860 GapTech India. In this role, you will design and implement scalable data solutions, leveraging technologies such as Databricks, Delta Lake, PySpark (Apache Spark), and Vector DB.Key Responsibilities:Design and optimize data pipelines using PySpark (Apache Spark),...

  • Azure Data Solutions Architect

    4 days ago

    Hyderabad, Telangana, India Proclink Full time

    **Job Description for Proclink**We are seeking an experienced Azure Data Solutions Architect to join our team at Proclink. As a key member of our data engineering team, you will design and implement scalable data solutions on Microsoft Azure.**About the Role**The successful candidate will have a strong background in designing and developing data pipelines...

  • Data Lake Schema Architect

    3 days ago

    Hyderabad, Telangana, India AUGUS IT SOLUTIONS Full time

    About AUGUS IT SOLUTIONSAUGUS IT SOLUTIONS is a forward-thinking technology company that delivers innovative solutions to businesses.Job OverviewWe are seeking an experienced Data Lake Schema Architect to join our team. This role will be responsible for designing, building, and maintaining large-scale data lakes on Azure Databricks.Key ResponsibilitiesDesign...

  • Cloud Solutions Architect

    4 weeks ago

    Hyderabad, Telangana, India TalentSmart Solutions Full time

    Job Title: Cloud Solutions ArchitectJob Summary:We are seeking an experienced Cloud Solutions Architect to join our team at TalentSmart Solutions. As a Cloud Solutions Architect, you will be responsible for designing and implementing cloud-based solutions that meet the needs of our clients.Key Responsibilities:Design and implement cloud-based solutions using...

  • Data Solution Architect

    2 months ago

    Hyderabad, Telangana, India Techstar Software Development India Pvt. Ltd. Full time

    Position Summary : As a Data Solutions Architect, you will be responsible for architecting, designing, implementing, and maintaining our organization's data solutions. You will work closely with cross-functional teams to understand business requirements and translate them into robust, scalable, and efficient data solutions. The ideal candidate will have...

  • Senior Databricks Developer

    4 weeks ago

    Hyderabad, Telangana, India Innover Global Inc Full time

    **Job Title:** Senior Databricks DeveloperAt Innover Global Inc, we are seeking a highly skilled and experienced Senior Databricks Developer to join our team. As a key member of our data engineering team, you will be responsible for designing, developing, and implementing large-scale data processing systems using Azure Databricks.Key Responsibilities:Design...

  • Cloud Data Solutions Architect

    3 days ago

    Hyderabad, Telangana, India Anblicks Full time

    We are seeking a highly skilled Cloud Data Solutions Architect to join our team at Anblicks.The ideal candidate will have experience in designing and implementing robust data pipelines using the Microsoft Stack, with a focus on Azure-based solutions.About the Role:This position involves developing and implementing ETL processes using Azure Data Factory,...