Etl Databricks

3 days ago

Chennai Tamil Nadu, India Virtusa Full time

Develop and maintain a metadata driven generic ETL framework for automating ETL code
Design, build, and optimize ETL/ELT pipelines using Databricks (PySpark/SQL) on AWS.
InsureMO rating engine experience required.
Ingest data from a variety of structured and unstructured sources (APIs, RDBMS, flat files, streaming).
Develop and maintain robust data pipelines for batch and streaming data using Delta Lake and Spark Structured Streaming.
Implement data quality checks, validations, and logging mechanisms.
Optimize pipeline performance, cost, and reliability.
Collaborate with data analysts, BI, and business teams to deliver fit for purpose datasets.
Support data modeling efforts (star, snowflake schemas) de norm tables approach and assist with data warehousing initiatives.
Work with orchestration tools Databricks Workflows to schedule and monitor pipelines.
Follow best practices for version control, CI/CD, and collaborative development

**Skills**:Hands-on experience in ETL/Data Engineering roles.
Strong expertise in Databricks (PySpark, SQL, Delta Lake), Databricks Data Engineer Certification preferred
Experience with Spark optimization, partitioning, caching, and handling large-scale datasets.
Proficiency in SQL and scripting in Python or Scala.
Solid understanding of data lakehouse/medallion architectures and modern data platforms.
Experience working with cloud storage systems like AWS S3
Familiarity with DevOps practices Git, CI/CD, Terraform, etc.
Strong debugging, troubleshooting, and performance-tuning skills.

**About Virtusa**

Teamwork, quality of life, professional and personal development: values that Virtusa is proud to embody. When you join us, you join a team of 27,000 people globally that cares about your growth — one that seeks to provide you with exciting projects, opportunities and work with state of the art technologies throughout your career with us.

Great minds, great potential: it all comes together at Virtusa. We value collaboration and the team environment of our company, and seek to provide great minds with a dynamic place to nurture new ideas and foster excellence.

Virtusa was founded on principles of equal opportunity for all, and so does not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, non-disqualifying physical or mental disability, national origin, veteran status or any other basis covered by appropriate law. All employment is decided on the basis of qualifications, merit, and business need.

  • Databrick

    2 weeks ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Key Responsibilities: Design, develop, and maintain scalable data pipelines using Apache Spark on Databricks. Write efficient and production-ready PySpark or Scala code for data transformation and ETL processes. Integrate data from various structured and unstructured sources into a unified platform. Implement Delta Lake and manage data versioning, updates,...

  • Data Architect- Databricks

    2 weeks ago

    Chennai, Tamil Nadu, India InfoCepts Full time

    **Position**: Data Architect (Databricks, Snowflake) **Purpose of the Position**: Design and optimize cloud-native data architectures on platforms like Databricks and Snowflake, enabling scalable data engineering, advanced analytics, and AI/ML solutions aligned with business needs. **Location**: Nagpur, Pune, Chennai, Bangalore **Type of Employment**:Full...

  • Data Architect- Databricks

    5 days ago

    Chennai, Tamil Nadu, India InfoCepts Full time

    **Position**: Data Architect (Databricks, Snowflake) **Purpose of the Position**: Design and optimize cloud-native data architectures on platforms like Databricks and Snowflake, enabling scalable data engineering, advanced analytics, and AI/ML solutions aligned with business needs. **Location**: Nagpur, Pune, Chennai, Bangalore **Type of Employment**:Full...

  • Azure Databricks Trainer

    3 days ago

    Chennai, Tamil Nadu, India Ednue Technologies Private Limited Full time

    We are seeking a knowledgeable and experienced **Azure Databricks Trainer** to design and deliver comprehensive training sessions for professionals and aspiring data engineers. The trainer will be responsible for conducting hands-on workshops, developing course content, and mentoring participants on real-world Azure Databricks projects. **Key...

  • Data Architect- Databricks

    3 days ago

    Chennai, Tamil Nadu, India InfoCepts Full time

    **Position**: Data Architect (Databricks, Snowflake) **Location**: Nagpur, Pune, Chennai, Bangalore **Type of Employment**:Full time Employment **Purpose of the Position**: Design and optimize cloud-native data architectures on platforms like Databricks and Snowflake, enabling scalable data engineering, advanced analytics, and AI/ML solutions aligned with...

  • Azure Databricks/datafactory

    3 days ago

    Chennai, Tamil Nadu, India Cognizant Full time

    Working with event based / streaming technologies to ingest and process data Working with other members of the project team to support delivery of additional project components (API interfaces, Search). - Evaluating the performance and applicability of multiple tools against customer requirements Working within an Agile delivery / DevOps methodology to...

  • Databricks Developers

    6 days ago

    Chennai, Tamil Nadu, India Cynosure Corporate Solutions Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Key Responsibilities:Develop and optimize data pipelines in Databricks for transforming and processing data from various sources.Integrate data using Unity Catalog and external data sources (data lakes, APIs, etc.).Write Spark SQL and PySpark scripts for data transformations, optimizations, and creating views/procedures.Perform data analysis to identify...

  • ETL Support Engineer

    6 days ago

    Chennai, Tamil Nadu, India Star Health Insurance Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    ETL Support Engineer:Maintain ETL pipelines using Databricks and PySpark to extract, transform, and load data from various sources.Troubleshoot and resolve issues related to data transformations or loading processes in Databricks.Develop and maintain documentation for ETL processes, configurations, and data mappings.Strong proficiency in Databricks with...

  • Azure DataBricks

    6 days ago

    Chennai, Tamil Nadu, India Hexaware Technologies Full time ₹ 15,00,000 - ₹ 28,00,000 per year

    We're on the lookout for passionate Azure Data Engineers who want to redefine the future of data with Databricks & Azure.Role: ADB – Data Engineer (Azure, Databricks, ADF, PySpark/Python)Date: Saturday, 13/09/2025Mode: F2F Discussion at Hexaware Technologies , Siruseri ,Chennai ,Bring your expertise in:Datawarehousing & ETL (10+ yrs)Azure, ADF, Synapse,...

  • ETL Data Architect

    3 days ago

    Chennai, Tamil Nadu, India beBeeData Full time ₹ 18,00,000 - ₹ 24,00,000

    ETL Data ArchitectJob Summary:We are seeking an experienced ETL Data Architect to design and develop scalable data pipelines and architectures using Databricks.Key Responsibilities:Collaborate with stakeholders to understand data requirements and translate them into technical solutions.Design, develop, and deploy scalable data pipelines and ETL processes...