Databricks Setup with Aws

9 hours ago

Chennai Tamil Nadu, India Virtusa Full time

Job Description:
We are looking for a Databricks Setup Specialist to lead the deployment and configuration of Databricks on AWS, focusing on PrivateLink integration and Unity Catalog administration. This role involves collaboration with AWS DevOps and Security teams to create a secure and compliant data platform.
Key Responsibilities:
Deploy Databricks workspaces on AWS with secure access via PrivateLink.
Integrate Databricks with VPCs, IAM roles, and security groups alongside AWS DevOps engineers.
Administer Unity Catalog, managing workspace binding, permissions, and audit logging.
Implement best practices for identity federation and role-based access controls (RBAC).
Troubleshoot configuration and connectivity issues to minimize downtime.
Document setup procedures and provide training for users and administrators.
Required Qualifications:
3+ years of experience with Databricks on AWS.
Expertise in AWS services like VPC, PrivateLink, IAM, and S3.
Experience with Unity Catalog in enterprise settings.
Familiarity with DevOps tools for infrastructure automation.
Understanding of data governance and compliance requirements.
Strong problem-solving and communication skills.
Preferred Qualifications:
Experience with multi-workspace Databricks environments.
Knowledge of Lakehouse architecture and data lineage tools.
Relevant Databricks and AWS certifications.

**About Virtusa**

Teamwork, quality of life, professional and personal development: values that Virtusa is proud to embody. When you join us, you join a team of 27,000 people globally that cares about your growth — one that seeks to provide you with exciting projects, opportunities and work with state of the art technologies throughout your career with us.

Great minds, great potential: it all comes together at Virtusa. We value collaboration and the team environment of our company, and seek to provide great minds with a dynamic place to nurture new ideas and foster excellence.

Virtusa was founded on principles of equal opportunity for all, and so does not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, non-disqualifying physical or mental disability, national origin, veteran status or any other basis covered by appropriate law. All employment is decided on the basis of qualifications, merit, and business need.

  • Databrick

    2 weeks ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Key Responsibilities: Design, develop, and maintain scalable data pipelines using Apache Spark on Databricks. Write efficient and production-ready PySpark or Scala code for data transformation and ETL processes. Integrate data from various structured and unstructured sources into a unified platform. Implement Delta Lake and manage data versioning, updates,...

  • Aws Services

    1 week ago

    Chennai, Tamil Nadu, India Cognizant Full time

    **Job Summary** **Responsibilities** - Develop and maintain data pipelines using Redshift Databricks and Kafka to ensure seamless data flow. - Implement and manage AWS Lambda functions to automate data processing tasks. - Utilize AWS Services to design and deploy scalable and secure data solutions. - Oversee the integration of AWS Glue for efficient data...

  • Data Architect- Databricks

    2 weeks ago

    Chennai, Tamil Nadu, India InfoCepts Full time

    **Position**: Data Architect (Databricks, Snowflake) **Purpose of the Position**: Design and optimize cloud-native data architectures on platforms like Databricks and Snowflake, enabling scalable data engineering, advanced analytics, and AI/ML solutions aligned with business needs. **Location**: Nagpur, Pune, Chennai, Bangalore **Type of Employment**:Full...

  • Data Architect- Databricks

    4 days ago

    Chennai, Tamil Nadu, India InfoCepts Full time

    **Position**: Data Architect (Databricks, Snowflake) **Purpose of the Position**: Design and optimize cloud-native data architectures on platforms like Databricks and Snowflake, enabling scalable data engineering, advanced analytics, and AI/ML solutions aligned with business needs. **Location**: Nagpur, Pune, Chennai, Bangalore **Type of Employment**:Full...

  • Databricks Platform Administrator

    4 days ago

    Chennai, Tamil Nadu, India Tiger Analytics Full time

    The Databricks Administrator will be responsible for providing technical expertise in deploying administration, configuration, and optimization of the Databricks platform to enable data analytics, machine learning, and data engineering activities within the organization. You will collaborate with data engineers, data scientists, and other stakeholders to...

  • Data Architect- Databricks

    2 days ago

    Chennai, Tamil Nadu, India InfoCepts Full time

    **Position**: Data Architect (Databricks, Snowflake) **Location**: Nagpur, Pune, Chennai, Bangalore **Type of Employment**:Full time Employment **Purpose of the Position**: Design and optimize cloud-native data architectures on platforms like Databricks and Snowflake, enabling scalable data engineering, advanced analytics, and AI/ML solutions aligned with...

  • Aws Cdk

    4 days ago

    Chennai, Tamil Nadu, India Virtusa Full time

    P1-C3- TSTS Design, implement, and maintain cloud infrastructure using AWS Cloud Development Kit (CDK) Develop and evolve Infrastructure as Code (IaC) to ensure efficient provisioning and management of AWS resources Configure and manage various AWS services, including but not limited to EC2, VPC, Security Group, NACL, S3, CloudFormation, CloudWatch, AWS...

  • Etl Databricks

    2 days ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Develop and maintain a metadata driven generic ETL framework for automating ETL code Design, build, and optimize ETL/ELT pipelines using Databricks (PySpark/SQL) on AWS. InsureMO rating engine experience required. Ingest data from a variety of structured and unstructured sources (APIs, RDBMS, flat files, streaming). Develop and maintain robust data pipelines...

  • Databricks Lead Engineer

    16 hours ago

    Bengaluru, Chennai, Hyderabad, India Rhohyve Ai Labs Full time US$ 1,25,000 - US$ 1,75,000 per year

    Required Skills & Experience6-9 years of experience in Consulting, Data, and Analytics, with a strong background in Databricks solutions.Hands-on experience in designing and implementing big data solutions, including data pipelines for large and complex datasets.Expertise in Python, PySpark, and orchestration frameworks such as Airflow, Oozie, Luigi,...

  • Databrick Pyspark

    5 days ago

    Chennai, Tamil Nadu, India Virtusa Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Key Responsibilities:Design, develop, and maintain scalable data pipelines using Apache Spark on Databricks. Write efficient and production-ready PySpark or Scala code for data transformation and ETL processes. Integrate data from various structured and unstructured sources into a unified platform. Implement Delta Lake and manage data versioning, updates,...