Databricks SQL Engineer

1 week ago

Chennai, India Mastek Ltd Full time

Job Title : Databricks SQL Engineer (with Pharma/Life Sciences background)

About the Role :

We are seeking a highly skilled Databricks SQL Engineer with Pharma background to join our Data Engineering team. The ideal candidate will have strong expertise in Databricks, Pyspark, SQL, Spark SQL, Delta Lake, and data lakehouse architectures, and will be responsible for building scalable and efficient data models, queries, and pipelines that power analytical insights and business decisions.

Key Responsibilities :

- Design, develop, and optimize SQL-based data models and queries in Databricks for large-scale datasets.

- Work with Delta Lake, Unity Catalog, and Databricks Notebooks to manage and secure data assets.

- Implement and maintain ETL/ELT pipelines using Databricks workflows and SQL.

- Develop documentation and data dictionaries to promote self-service analytics.

- Ensure data quality and consistency through testing, validation, and monitoring.

- Optimize data lakehouse architecture for cost and performance, leveraging Spark SQL best practices.

- Support CI/CD practices for deploying notebooks and SQL jobs to production.

Required Qualifications:

- 5+ years of experience working with Databricks SQL, Pyspark, Spark SQL, or similar distributed SQL engines.

- Experience in the Pharma Domain

- Strong experience with Delta Lake, structured streaming, and data lakehouse paradigms.

- Proficiency in SQL development, query tuning, and working with large datasets.

- Experience with version control tools (Git) and collaborative development practices.

- Experience working in cloud environments (AWS, Azure, or GCP), preferably with Azure Databricks or Databricks on AWS.

- Understanding of data governance, access control, and tools like Unity Catalog or Purview.

- Familiarity with CI/CD for data pipelines and automation tools like Databricks Repos, Terraform, or dbx.

- Exposure to Python or PySpark for scripting and data manipulation.

(ref:hirist.tech)

  • Etl Databricks

    2 weeks ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Develop and maintain a metadata driven generic ETL framework for automating ETL code Design, build, and optimize ETL/ELT pipelines using Databricks (PySpark/SQL) on AWS. InsureMO rating engine experience required. Ingest data from a variety of structured and unstructured sources (APIs, RDBMS, flat files, streaming). Develop and maintain robust data pipelines...

  • Databricks

    6 days ago

    Chennai, India Cognizant Technology Solutions Full time

    Job Summary Join our dynamic team as a Developer where you will leverage your expertise in Spark in Scala Databricks and Amazon technologies to drive innovative solutions. With a hybrid work model and day shifts you will collaborate with cross-functional teams to optimize data processes and enhance our data infrastructure. Your contributions will directly...

  • Eucloid - Senior Data Engineer - SQL/Python/Databricks

    1 week ago

    Chennai, India Eucloid Data Solutions Pvt Ltd Full time

    Job Description : Eucloid is looking for a senior Data Engineer with hands-on expertise in Databricks to join our Data Platform team supporting various business applications. The ideal candidate will support the development of data infrastructure on Databricks for our clients by participating in activities which may include starting from upstream and...

  • Databricks

    2 weeks ago

    Chennai, Tamil Nadu, India Cognizant Full time

    **Job Summary** Join our dynamic team as a Developer where you will leverage your expertise in Spark in Scala Databricks and Amazon technologies to drive innovative solutions. With a hybrid work model and day shifts you will collaborate with cross-functional teams to optimize data processes and enhance our data infrastructure. Your contributions will...

  • Databricks

    2 weeks ago

    Chennai, Tamil Nadu, India Cognizant Full time

    **Job Summary** **Responsibilities** - Lead the design and implementation of data processing pipelines using Spark in Scala and PySpark - Oversee the administration of Databricks Unity Catalog and manage data governance - Provide expertise in Delta Sharing and ensure secure data sharing practices - Develop and maintain Databricks CLI scripts for automation...

  • Databricks Developers

    6 days ago

    Chennai, India Cynosure Corporate Solutions Full time

    Key Responsibilities: Develop and optimize data pipelines in Databricks for transforming and processing data from various sources. Integrate data using Unity Catalog and external data sources (data lakes, APIs, etc.). Write Spark SQL and PySpark scripts for data transformations, optimizations, and creating views/procedures. Perform data analysis to identify...

  • Senior data engineer

    4 days ago

    Chennai, India Tredence Inc. Full time

    Role: Senior Databricks Engineer / Databricks Technical Lead/ Data Architect Experience: 6-12 years Location: Bangalore, Chennai, Delhi, Pune, Kolkata About Tredence: Tredence focuses on last-mile delivery of powerful insights into profitable actions by uniting its strengths in business analytics, data science and software engineering. The largest...

  • Databricks Architect

    3 weeks ago

    Chennai, Tamil Nadu, India Datacrew Full time

    Job Title: Senior Databricks Data Engineer/Databricks ArchitectExperience: 5-8 yearsLocation: Chennai / Oman (Onsite)Note: Initial onsite for 3 months at Oman and back to Chennai to work from OffshoreEmployment Type: ContractRole Summary:The Databricks Architect will be responsible for designing, implementing, and optimizing scalable data architectures...

  • Senior Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India beBeeDataengineer Full time ₹ 15,00,000 - ₹ 28,00,000

    Job Title: Databricks LeadWe are seeking a seasoned Data Engineer with expertise in developing and implementing ETL pipelines using Databricks on cloud AWS.Key Responsibilities:Design, implement, and manage data analytics using Databricks.Develop data pipelines using Python, PySpark, and SQL.Collaborate with cross-functional teams to deliver projects on...

  • Senior Data Engineer

    1 week ago

    Chennai, India Tredence Inc. Full time

    Role: Senior Databricks Engineer / Databricks Technical Lead/ Data ArchitectExperience: 6-12 yearsLocation: Bangalore, Chennai, Delhi, Pune, KolkataAbout Tredence:Tredence focuses on last-mile delivery of powerful insights into profitable actions by uniting its strengths in business analytics, data science and software engineering. The largest companies...