Databricks Setup with Aws

3 days ago

Chennai Tamil Nadu, India Virtusa Full time

Job Description:
We are looking for a Databricks Setup Specialist to lead the deployment and configuration of Databricks on AWS, focusing on PrivateLink integration and Unity Catalog administration. This role involves collaboration with AWS DevOps and Security teams to create a secure and compliant data platform.
Key Responsibilities:
Deploy Databricks workspaces on AWS with secure access via PrivateLink.
Integrate Databricks with VPCs, IAM roles, and security groups alongside AWS DevOps engineers.
Administer Unity Catalog, managing workspace binding, permissions, and audit logging.
Implement best practices for identity federation and role-based access controls (RBAC).
Troubleshoot configuration and connectivity issues to minimize downtime.
Document setup procedures and provide training for users and administrators.
Required Qualifications:
3+ years of experience with Databricks on AWS.
Expertise in AWS services like VPC, PrivateLink, IAM, and S3.
Experience with Unity Catalog in enterprise settings.
Familiarity with DevOps tools for infrastructure automation.
Understanding of data governance and compliance requirements.
Strong problem-solving and communication skills.
Preferred Qualifications:
Experience with multi-workspace Databricks environments.
Knowledge of Lakehouse architecture and data lineage tools.
Relevant Databricks and AWS certifications.

**About Virtusa**

Teamwork, quality of life, professional and personal development: values that Virtusa is proud to embody. When you join us, you join a team of 27,000 people globally that cares about your growth — one that seeks to provide you with exciting projects, opportunities and work with state of the art technologies throughout your career with us.

Great minds, great potential: it all comes together at Virtusa. We value collaboration and the team environment of our company, and seek to provide great minds with a dynamic place to nurture new ideas and foster excellence.

Virtusa was founded on principles of equal opportunity for all, and so does not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, non-disqualifying physical or mental disability, national origin, veteran status or any other basis covered by appropriate law. All employment is decided on the basis of qualifications, merit, and business need.

  • Databricks / Aws Redshift

    5 days ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Key responsibilities include designing developing and maintaining scalable data pipelines using Apache Spark on Databricks Writing efficient and production ready PySpark or Scala code for data transformation and ETL processes Integrating data from various structured and unstructured sources into a unified platform Implementing Delta Lake and managing data...

  • Databrick

    2 weeks ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Key Responsibilities: Design, develop, and maintain scalable data pipelines using Apache Spark on Databricks. Write efficient and production-ready PySpark or Scala code for data transformation and ETL processes. Integrate data from various structured and unstructured sources into a unified platform. Implement Delta Lake and manage data versioning, updates,...

  • DataBricks / AWS Redshift

    4 days ago

    Chennai, Tamil Nadu, India Virtusa Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Key responsibilities include designing developing and maintaining scalable data pipelines using Apache Spark on Databricks Writing efficient and production ready PySpark or Scala code for data transformation and ETL processes Integrating data from various structured and unstructured sources into a unified platform Implementing Delta Lake and managing data...

  • 15h Left! Aws Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India virtusa Full time

    AWS Data Engineer Design and build scalable data pipelines using AWS services like AWS Glue Amazon Redshift and S3 Develop efficient ETL processes for data extraction transformation and loading into data warehouses and lakes Create and manage applications using Python SQL Databricks and various AWS technologies Automate repetitive tasks and build...

  • Informatica Cloud Mdm Administrator

    6 days ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Job Title Informatica Cloud MDM Administrator AWS CI CD Multi Environment Setup Experience 12 Years Domain Expertise: Insurance, Master Data Management, Cloud Infrastructure Professional Summary Experienced Informatica Cloud MDM Administrator with a strong background in setting up and managing enterprise-grade MDM platforms from the ground up. Skilled in...

  • Databricks Lead Engineer

    1 week ago

    Bengaluru, Chennai, Hyderabad, India Rhohyve Ai Labs Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Required Skills & Experience6-9 years of experience in Consulting, Data, and Analytics, with a strong background in Databricks solutions.Hands-on experience in designing and implementing big data solutions, including data pipelines for large and complex datasets.Expertise in Python, PySpark, and orchestration frameworks such as Airflow, Oozie, Luigi,...

  • Principal Consultant, Sr. Databricks Developer

    1 day ago

    Chennai, Tamil Nadu, India GENPACT Full time

    Ready to shape the future of work At Genpact we don t just adapt to change we drive it AI and digital innovation are redefining industries and we re leading the charge Genpact s AI Gigafactory our industry-first accelerator is an example of how we re scaling advanced technology solutions to help global enterprises work smarter grow faster and transform at...

  • Databrick Pyspark

    3 weeks ago

    Chennai, India Virtusa Full time

    Key Responsibilities: Design, develop, and maintain scalable data pipelines using Apache Spark on Databricks. Write efficient and production-ready PySpark or Scala code for data transformation and ETL processes. Integrate data from various structured and unstructured sources into a unified platform. Implement Delta Lake and manage data versioning, updates,...

  • Databricks Developer

    3 weeks ago

    Chennai, India Tekskills Full time

    Job Description:• Data Pipeline Design: Architect and implement scalable, reliable data pipelines using Databricks, Spark, and Delta Lake• Medallion Architecture Implementation: Ingest raw data into the Bronze layer, process and clean it in the Silver layer, and aggregate/transform it for analytics in the Gold layer• Data Modeling: Design and optimize...

  • Databricks Developers

    3 weeks ago

    Chennai, India Cynosure Corporate Solutions Full time

    Key Responsibilities: Develop and optimize data pipelines in Databricks for transforming and processing data from various sources. Integrate data using Unity Catalog and external data sources (data lakes, APIs, etc.). Write Spark SQL and PySpark scripts for data transformations, optimizations, and creating views/procedures. Perform data analysis to identify...