Senior Data Engineer

4 weeks ago

Bangalore Urban, India USEReady Full time

Job Title: Senior Databricks Engineer

Experience Level: 5-8 Years

Job Summary

As a Senior Databricks Engineer, you will be responsible for designing, developing, and optimizing our data architecture and pipelines on the Databricks Lakehouse Platform. You will leverage your deep expertise in Spark, Delta Lake, and cloud technologies to build scalable and reliable solutions that process large-scale data efficiently. The ideal candidate is a proactive problem-solver with a strong background in data engineering and a passion for mentoring and driving technical excellence.

Key Responsibilities

  • Architect and Develop: Design, build, and maintain robust and scalable data pipelines and ETL/ELT processes using Databricks, Apache Spark, and Python/Scala.
  • Lakehouse Implementation: Architect and implement solutions on the Databricks Lakehouse Platform, including data modeling, ingestion, storage, and processing using Delta Lake.
  • Performance Optimization: Analyze and tune the performance of Spark jobs and Databricks clusters to ensure high efficiency, reliability, and cost-effectiveness.
  • Data Governance: Implement and manage data governance, data quality, and security best practices within Databricks, utilizing features like Unity Catalog.
  • Collaboration: Work closely with data scientists, analysts, and business stakeholders to understand their data requirements and deliver high-quality, end-to-end data solutions.
  • Automation & CI/CD: Develop and maintain CI/CD pipelines for automated testing and deployment of data workflows (e.g., using Azure DevOps, Jenkins, GitHub Actions).
  • Mentorship: Mentor junior engineers, conduct code reviews, and champion best practices in data engineering and software development.
  • Innovation: Stay current with the latest advancements in the Databricks platform and the broader big data ecosystem, and drive the adoption of new tools and technologies.


Required Qualifications and Skills

  • Experience: 5-8 years of professional experience in a data engineering role.
  • Databricks Expertise: Proven hands-on experience architecting and building solutions on the Databricks platform.
  • Core Technical Skills:
  • Strong proficiency in Python or Scala .
  • Expert-level knowledge of Apache Spark , including Spark SQL and DataFrame APIs.
  • In-depth understanding and practical experience with Delta Lake .
  • Advanced SQL skills and experience with data warehousing concepts.
  • Cloud Proficiency: Solid experience with at least one major cloud platform (Azure , AWS , or GCP ), including their data storage and processing services (e.g., Azure Data Lake Storage, AWS S3, Google Cloud Storage).
  • ETL/ELT: Demonstrable experience in designing and implementing complex ETL/ELT pipelines.
  • Problem-Solving: Excellent analytical and problem-solving abilities with a keen attention to detail.
  • Communication: Strong verbal and written communication skills, with the ability to explain complex technical concepts to non-technical stakeholders.
  • Data Visualization: Experience supporting data visualization tools like Power BI or Tableau

  • Senior Data Engineer

    2 weeks ago

    Bangalore Urban, India Sonata Software Full time

    The Senior Software Engineer supports functions which require automation/systems development, including initial development and ongoing support of the suite of applications that would help manage loan, pricing, enterprise data pipeline, or other functions depending on the department. As the Application Developer, you must be proficient in Python, Data...

  • Senior Data Engineer

    2 weeks ago

    Bangalore Urban, India Sonata Software Full time

    The Senior Software Engineer supports functions which require automation/systems development, including initial development and ongoing support of the suite of applications that would help manage loan, pricing, enterprise data pipeline, or other functions depending on the department. As the Application Developer, you must be proficient in Python, Data...

  • Senior Data Engineer

    5 days ago

    Bangalore Urban, India Tata Consultancy Services Full time

    Greetings from TCS!!!TCS has been a great pioneer in feeding the fire of Techies like you. We are a global leader in the technology arena and there’s nothing that can stop us from growing together.Your role is of key importance, as it lays down the foundation for the entire project.Make sure you have a valid EP number before the interview. To create an EP...

  • Senior Data Engineer

    1 week ago

    Bangalore Urban, India Tata Consultancy Services Full time

    Greetings from TCS!!! TCS has been a great pioneer in feeding the fire of Techies like you. We are a global leader in the technology arena and there’s nothing that can stop us from growing together. Your role is of key importance, as it lays down the foundation for the entire project. Make sure you have a valid EP number before the interview. To create an...

  • Senior Data Engineer

    4 weeks ago

    Bangalore Urban, India USEReady Full time

    Job Title: Senior Databricks EngineerExperience Level: 5-8 YearsJob SummaryAs a Senior Databricks Engineer, you will be responsible for designing, developing, and optimizing our data architecture and pipelines on the Databricks Lakehouse Platform. You will leverage your deep expertise in Spark, Delta Lake, and cloud technologies to build scalable and...

  • Sr. Data Engineer

    2 weeks ago

    Bangalore Urban, India Tata Consultancy Services Full time

    Greetings from TCS!We are Looking for Senior Data EngineerExperience : 8 + YearsLocation : BangaloreMust-HaveThe Senior Data Engineer is the one who designs and builds data foundations and end to end solutions for the Shell Business to maximize value from data. The role helps create a data-driven thinking within the organization, not just within IT teams,...

  • Senior Data Scientist

    2 weeks ago

    Bangalore Urban, India Analyttica Datalab Full time

    Senior Data Scientist Location: Bangalore About the Role We are seeking a Senior Data Scientist with deep expertise in data science methodologies and exceptional proficiency in Python. You will lead end-to-end analytical initiatives — from problem framing and data preparation to model development, validation, deployment, and monitoring. This role requires...

  • Senior Data Scientist

    2 weeks ago

    Bangalore Urban, India Analyttica Datalab Full time

    Senior Data ScientistLocation: BangaloreAbout the RoleWe are seeking a Senior Data Scientist with deep expertise in data science methodologies and exceptional proficiency in Python. You will lead end-to-end analytical initiatives — from problem framing and data preparation to model development, validation, deployment, and monitoring. This role requires...

  • Senior Data Engineer

    4 weeks ago

    Bangalore Urban, India Emids Full time

    Hi All,Greetings for the day!!We are currently hiring for Senior Data Engineer (Snowflake) for Emids(MNC) at Bangalore location.Job Title: Senior Data Engineer (Snowflake, Postgres)Location: Bangalore,Hyderabad,NoidaJob Type : FulltimeExperience : 6- 12 YrsNotice Period: Immediate to 15 days Only Job Role:We are looking for a highly skilled and experienced...

  • Senior Engineer

    2 weeks ago

    Bangalore Urban, India Tata Consultancy Services Full time

    TCS has been a great pioneer in feeding the fire of Young techies like you. We are global leaders in the technology arena and there's nothing that can stop us from growing together. TCS Hiring for skill " Snowflake + DBT developer". Role: Senior Engineer Required Technical Skill Set: Snowflake, DBT, Python Data Engineering Experience: 6-8 Years Work...