Etl Databricks

2 weeks ago

Pune Maharashtra, India Virtusa Full time

Looking for a offshore Techlead with databricks engineer experience and lead the team from offshore.
Develop and maintain a metadata driven generic ETL framework for automating ETL code
Design, build, and optimize ETL/ELT pipelines using Databricks (PySpark/SQL) on AWS.
Ingest data from a variety of structured and unstructured sources (APIs, RDBMS, flat files, streaming).
Develop and maintain robust data pipelines for batch and streaming data using Delta Lake and Spark Structured Streaming.
Implement data quality checks, validations, and logging mechanisms.
Optimize pipeline performance, cost, and reliability.
Collaborate with data analysts, BI, and business teams to deliver fit for purpose datasets.
Support data modeling efforts (star, snowflake schemas) de norm tables approach and assist with data warehousing initiatives.
Work with orchestration tools Databricks Workflows to schedule and monitor pipelines.
Follow best practices for version control, CI/CD, and collaborative development

**Skills**:Hands-on experience in ETL/Data Engineering roles.
Strong expertise in Databricks (PySpark, SQL, Delta Lake), Databricks Data Engineer Certification preferred
Experience with Spark optimization, partitioning, caching, and handling large-scale datasets.
Proficiency in SQL and scripting in Python or Scala.
Solid understanding of data lakehouse/medallion architectures and modern data platforms.
Experience working with cloud storage systems like AWS S3
Familiarity with DevOps practices Git, CI/CD, Terraform, etc.
Strong debugging, troubleshooting, and performance-tuning skills.

**About Virtusa**

Teamwork, quality of life, professional and personal development: values that Virtusa is proud to embody. When you join us, you join a team of 27,000 people globally that cares about your growth — one that seeks to provide you with exciting projects, opportunities and work with state of the art technologies throughout your career with us.

Great minds, great potential: it all comes together at Virtusa. We value collaboration and the team environment of our company, and seek to provide great minds with a dynamic place to nurture new ideas and foster excellence.

Virtusa was founded on principles of equal opportunity for all, and so does not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, non-disqualifying physical or mental disability, national origin, veteran status or any other basis covered by appropriate law. All employment is decided on the basis of qualifications, merit, and business need.

  • ETL Databricks

    3 weeks ago

    Pune, India Virtusa Full time

    Looking for a offshore Techlead with databricks engineer experience and lead the team from offshore. Develop and maintain a metadata driven generic ETL framework for automating ETL code Design, build, and optimize ETL/ELT pipelines using Databricks (PySpark/SQL) on AWS . Ingest data from a variety of structured and unstructured sources (APIs, RDBMS, flat...

  • ETL Databricks

    2 weeks ago

    Pune, Maharashtra, India Virtusa Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Looking for a offshore Techlead with databricks engineer experience and lead the team from offshore.Develop and maintain a metadata driven generic ETL framework for automating ETL codeDesign, build, and optimize ETL/ELT pipelines using Databricks (PySpark/SQL) on AWS .Ingest data from a variety of structured and unstructured sources (APIs, RDBMS, flat files,...

  • Databricks Engineer

    3 weeks ago

    Delhi, Bengaluru, Pune, India Impetus Technologies Full time

    Job Description - Databricks Engineers with 2-6 years of hand-on experience in Pyspark, Cloud Platform and SQL. - Proven experience in implementing data solutions on the Databricks platform withhands on experience in setting up Databricks cluster, working in Databricks modules,data pipelines for ingestingand transforming data from various sources into...

  • Databricks Engineer

    4 weeks ago

    Delhi, Bengaluru, Pune, India Impetus Technologies Full time

    Job Description- Databricks Engineers with 2-6 years of hand-on experience in Pyspark, Cloud Platform and SQL.- Proven experience in implementing data solutions on the Databricks platform withhands on experience in setting up Databricks cluster, working in Databricks modules,data pipelines for ingestingand transforming data from various sources into...

  • Databricks Engineer

    3 weeks ago

    Pune, India Bitwise Solutions Full time

    Role: DataBricks Engineer Notice Period: Immediate joiners only Location: Pune Work Mode: Hybrid Position Overview: We need a skilled Databricks Data Engineer with strong analytical and troubleshooting abilities. The ideal candidate should have hands-on experience in building scalable data pipelines using PySpark on cloud platforms like Azure/AWS along with...

  • Databricks Engineer

    2 weeks ago

    Pune, Maharashtra, India Bitwise Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Role: DataBricks EngineerNotice Period: Immediate joiners onlyLocation: PuneWork Mode: HybridPosition Overview:We need a skilled Databricks Data Engineer with strong analytical and troubleshooting abilities. The ideal candidate should have hands-on experience in building scalable data pipelines using PySpark on cloud platforms like Azure/AWS along with a...

  • Databricks Architect

    3 weeks ago

    Delhi, Bengaluru, Pune, India Impetus Technologies Full time

    Job Description - Overall 10-18 yrs. of Data Engineering experience with Minimum 4+ years of hands on experience in Databricks. Ready to travel Onsite and work at client location. - Proven hands-on experience as a Databricks Architect or similar role with a deep understanding of the Databricks platform and its capabilities. - Analyze business requirements...

  • Capco - Data Engineer - ETL/Azure Databricks

    3 weeks ago

    Pune, India Capco Technologies Pvt Ltd Full time

    Senior Consultant - Data EngineerExperience : 8+ YearsEmployment Type : Full-timeJob Overview : We are seeking a highly skilled Senior Consultant - Data Engineer with 8+ years of experience, particularly within the Financial Services sector. You will be instrumental in designing, building, and optimizing data pipelines and ETL processes in Azure...

  • Databricks Lead

    1 week ago

    Pune, Maharashtra, India Bitwise Solutions Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Bitwise id hiring for Expert Databricks Engineer for Pune Location.PFB JD for your reference. Relevant candidate please share resume on Email ID- Noted - We are looking for candidate who can join in 30 Days Only.Mandatory skills needed:1) Strong hands-on experience with Databricks - PySpark notebooks and Datbricks pipeline development2) Experience using...

  • Informatica Developer with Databricks

    5 days ago

    Navi Mumbai, Maharashtra, India, Maharashtra Celebal Technologies Full time

    Job Description – ETL Migration Specialist (Informatica to Databricks)Location: Navi MumbaiExperience: 5-7+ YearsDuration: FulltimePosition OverviewWe are looking for an experienced ETL Migration Specialist with strong expertise in Informatica PowerCenter and Databricks (PySpark). The ideal candidate will be responsible for analysing and reverse...