Databricks / Aws Redshift

4 days ago

Chennai Tamil Nadu, India Virtusa Full time

Key responsibilities include designing developing and maintaining scalable data pipelines using Apache Spark on Databricks Writing efficient and production ready PySpark or Scala code for data transformation and ETL processes Integrating data from various structured and unstructured sources into a unified platform Implementing Delta Lake and managing data versioning updates and schema evolution Optimizing data processing workflows for performance scalability and cost efficiency Collaborating with data scientists analysts and business stakeholders to deliver high quality datasets Implementing data quality checks validation routines and logging mechanisms Monitoring and debugging production jobs using Databricks jobs notebooks and clusters Ensuring security privacy and compliance standards are met throughout the data lifecycle Providing guidance and mentorship to junior team members Required skills and qualifications include four to six years of experience in Big Data development Hands on experience with Databricks including Workflows Notebooks Delta Live Tables and Unity Catalog Strong programming skills in PySpark and or Scala Solid understanding of Delta Lake architecture Proficiency in SQL for data analysis and transformation Experience with cloud platforms such as Azure including Azure Data Lake Data Factory and Synapse or AWS including S3 Glue and Redshift Familiarity with CI and CD for Databricks deployments such as using GitHub Actions or Azure DevOps Knowledge of data governance cataloguing and security best practices Experience working in an Agile or Scrum environment Required skills include Databricks Redshift and SQL Preferred skills include Python and Snowflake Additional preferred experience includes working with Databricks Unity Catalog and Delta Live Tables exposure to machine learning workflows in Databricks and experience with orchestration or messaging tools such as Apache Airflow or Kafka Certifications such as Databricks Certified Data Engineer Associate or Professional Azure or AWS certification are also desirable

**About Virtusa**

Teamwork, quality of life, professional and personal development: values that Virtusa is proud to embody. When you join us, you join a team of 27,000 people globally that cares about your growth — one that seeks to provide you with exciting projects, opportunities and work with state of the art technologies throughout your career with us.

Great minds, great potential: it all comes together at Virtusa. We value collaboration and the team environment of our company, and seek to provide great minds with a dynamic place to nurture new ideas and foster excellence.

Virtusa was founded on principles of equal opportunity for all, and so does not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, non-disqualifying physical or mental disability, national origin, veteran status or any other basis covered by appropriate law. All employment is decided on the basis of qualifications, merit, and business need.

  • DataBricks / AWS Redshift

    1 week ago

    Chennai, Tamil Nadu, India Virtusa Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Key responsibilities include designing developing and maintaining scalable data pipelines using Apache Spark on Databricks Writing efficient and production ready PySpark or Scala code for data transformation and ETL processes Integrating data from various structured and unstructured sources into a unified platform Implementing Delta Lake and managing data...

  • Aws Redshift

    6 days ago

    Chennai, Tamil Nadu, India Virtusa Full time

    **Job Description**: Design and implement efficient data models and structures to support high-performance queries and analytics. Experience Database design and architecting data solutions with AWS Redshift. Analyse and optimize complex SQL queries to ensure optimal performance in Redshift. Identify and resolve performance bottlenecks. Utilize Redshift...

  • Databricks Data Engineer

    5 days ago

    tamil nadu, India Tata Consultancy Services Full time

    About the RoleThe Databricks Data Engineer will play a crucial role in migrating and optimizing data workflows, ensuring high performance and reliability in our data operations.Experience- 3 to 6 yearsLocation - Kolkata /Chennai/TrivandrumRequired Technical Skill Set ------ Databricks, AWS, Python & PySparkGood to have- Experience in AWS platform (S3, EC2,...

  • Databricks Data Engineer

    1 week ago

    tamil nadu, India Tata Consultancy Services Full time

    About the Role The Databricks Data Engineer will play a crucial role in migrating and optimizing data workflows, ensuring high performance and reliability in our data operations. Experience- 3 to 6 years Location - Kolkata /Chennai/Trivandrum Required Technical Skill Set ------ Databricks, AWS, Python & PySpark Good to have- Experience in AWS platform (S3,...

  • Aws Services

    1 week ago

    Chennai, Tamil Nadu, India Cognizant Full time

    **Job Summary** **Responsibilities** - Develop and maintain data pipelines using Redshift Databricks and Kafka to ensure seamless data flow. - Implement and manage AWS Lambda functions to automate data processing tasks. - Utilize AWS Services to design and deploy scalable and secure data solutions. - Oversee the integration of AWS Glue for efficient data...

  • Databricks Engineer

    2 weeks ago

    tamil nadu, India Tata Consultancy Services Full time

    TCS present an excellent opportunity for Databricks Engineer Job Location: Chennai (ONLY) Experience required : 6-10 yrs Virtual Interview Date: 25-Oct-25 (Saturday) Must-Have Data Engineer having , Databricks & AWS development experience Development experience in Python, Pyspark Hands-on experience in writing complex SQLs and performance optimization of...

  • Aws Data

    2 weeks ago

    Chennai, Tamil Nadu, India virtusa Full time

    P3-C3-TSTSAWS Data EngineerDesign and build scalable data pipelines using AWS services like AWS Glue Amazon Redshift and QS SNS Cloudwatch Step function CDK or Terrafoam Develop efficient ETL processes for data extraction transformation and loading into data warehouses and lakes Create and manage applications using Python Pyspark SQL Databricks and various...

  • Databricks Data Engineer

    4 weeks ago

    Chennai, Tamil Nadu, India, Tamil Nadu Tata Consultancy Services Full time

    About the RoleThe Databricks Data Engineer will play a crucial role in migrating and optimizing data workflows, ensuring high performance and reliability in our data operations.Experience- 3 to 6 yearsLocation - Kolkata /Chennai/TrivandrumRequired Technical Skill Set ------ Databricks, AWS, Python & PySparkGood to have- Experience in AWS platform (S3, EC2,...

  • Databricks Setup with Aws

    2 days ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Job Description: We are looking for a Databricks Setup Specialist to lead the deployment and configuration of Databricks on AWS, focusing on PrivateLink integration and Unity Catalog administration. This role involves collaboration with AWS DevOps and Security teams to create a secure and compliant data platform. Key Responsibilities: Deploy Databricks...

  • Databrick

    2 weeks ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Key Responsibilities: Design, develop, and maintain scalable data pipelines using Apache Spark on Databricks. Write efficient and production-ready PySpark or Scala code for data transformation and ETL processes. Integrate data from various structured and unstructured sources into a unified platform. Implement Delta Lake and manage data versioning, updates,...