Senior Data Engineer

4 days ago

bangalore district, India USEReady Full time

Job Title: Senior Databricks Engineer

Experience Level: 5-8 Years

Job Summary

As a Senior Databricks Engineer, you will be responsible for designing, developing, and optimizing our data architecture and pipelines on the Databricks Lakehouse Platform. You will leverage your deep expertise in Spark, Delta Lake, and cloud technologies to build scalable and reliable solutions that process large-scale data efficiently. The ideal candidate is a proactive problem-solver with a strong background in data engineering and a passion for mentoring and driving technical excellence.

Key Responsibilities

  • Architect and Develop: Design, build, and maintain robust and scalable data pipelines and ETL/ELT processes using Databricks, Apache Spark, and Python/Scala.
  • Lakehouse Implementation: Architect and implement solutions on the Databricks Lakehouse Platform, including data modeling, ingestion, storage, and processing using Delta Lake.
  • Performance Optimization: Analyze and tune the performance of Spark jobs and Databricks clusters to ensure high efficiency, reliability, and cost-effectiveness.
  • Data Governance: Implement and manage data governance, data quality, and security best practices within Databricks, utilizing features like Unity Catalog.
  • Collaboration: Work closely with data scientists, analysts, and business stakeholders to understand their data requirements and deliver high-quality, end-to-end data solutions.
  • Automation & CI/CD: Develop and maintain CI/CD pipelines for automated testing and deployment of data workflows (e.g., using Azure DevOps, Jenkins, GitHub Actions).
  • Mentorship: Mentor junior engineers, conduct code reviews, and champion best practices in data engineering and software development.
  • Innovation: Stay current with the latest advancements in the Databricks platform and the broader big data ecosystem, and drive the adoption of new tools and technologies.

Required Qualifications and Skills

  • Experience: 5-8 years of professional experience in a data engineering role.
  • Databricks Expertise: Proven hands-on experience architecting and building solutions on the Databricks platform.
  • Core Technical Skills:
  • Strong proficiency in Python or Scala .
  • Expert-level knowledge of Apache Spark , including Spark SQL and DataFrame APIs.
  • In-depth understanding and practical experience with Delta Lake .
  • Advanced SQL skills and experience with data warehousing concepts.
  • Cloud Proficiency: Solid experience with at least one major cloud platform ( Azure , AWS , or GCP ), including their data storage and processing services (e.g., Azure Data Lake Storage, AWS S3, Google Cloud Storage).
  • ETL/ELT: Demonstrable experience in designing and implementing complex ETL/ELT pipelines.
  • Problem-Solving: Excellent analytical and problem-solving abilities with a keen attention to detail.
  • Communication: Strong verbal and written communication skills, with the ability to explain complex technical concepts to non-technical stakeholders.
  • Data Visualization: Experience supporting data visualization tools like Power BI or Tableau

  • Senior Data Engineer

    38 minutes ago

    bangalore district, India 6thStreet.com Full time

    Job Title: Senior Data Engineer Location: Bangalore Experience: 3-5 Years Department: Data & Analytics Employment Type: Full-time About the Role We’re looking for a Senior Data Engineer to lead the design, development, and scaling of our data infrastructure. You’ll shape best practices, mentor junior engineers, and ensure our data ecosystem empowers...

  • Senior Data Engineer

    2 weeks ago

    bangalore district, India ITC Infotech Full time

    Senior Data Engineer - Azure Stack Experience - 3-12 years Location : Bangalore, Kolkata, Gurgaon, Pune We are looking for a highly skilled Senior Data Engineer with deep expertise in Azure Stack to join our growing team. As a Senior Data Engineer, you will be instrumental in designing and developing robust data pipelines, data lakes, and cloud-based...

  • Senior/Lead Data Engineer

    4 days ago

    bangalore, India Eucloid Data Solutions Full time

    About The Role:Eucloid is looking for a senior/ lead Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure on Databricks for our clients by participating in activities which may includestarting from up- stream and down-stream technology selection to designing...

  • Senior Data Engineer

    2 weeks ago

    bangalore district, India Persistent Systems Full time

    About Position: We are seeking a highly skilled and experienced Senior Data Engineer to join our growing data team. The ideal candidate will have deep expertise in SQL/PLSQL development , AWS cloud services , DBT (Data Build Tool) , Python , and Snowflake . You will be responsible for designing, building, and maintaining scalable data pipelines...

  • Senior Data Engineer

    4 days ago

    bangalore district, India Syncarp Full time

    Looking for Lead Engineer / Technical Lead for a Global IT Service Provider based out of Chennai/Bangalore/Hyderabad/Pune Locations. Job Description: Lead Engineer: Exp: 5-8 Years Design, develop, and maintain simulation services and tools for ML feature, model, and rule evaluation. Build and optimize data pipelines for point-in-time historical data...

  • Senior Data Engineer

    1 week ago

    bangalore district, India Grid Dynamics Full time

    About Company Grid Dynamics (Nasdaq:GDYN) is a digital-native technology services provider that accelerates growth and bolsters competitive advantage for Fortune 1000 companies. Grid Dynamics provides digital transformation consulting and implementation services in omnichannel customer experience, big data analytics, search, artificial intelligence, cloud...

  • Senior Data Engineer

    2 weeks ago

    bangalore district, India JRD Systems Full time

    We are seeking a Senior Data Engineer with deep expertise in AWS Cloud Development . This role is critical to both the development of new data products and the modernization of existing platforms. The ideal candidate is a seasoned data engineer with hands-on experience designing, building, and optimizing large-scale data pipelines and architectures ...

  • Data Engineer

    4 days ago

    bangalore, India Data-Hat AI Full time

    Department: Data Engineering & AI Solutions Reports To: Lead Data Solutions Architect Travel: International travel required (up to 30–40%)  Position Summary: We are hiring a senior-level Data Engineer to lead the design, development, and optimization of high-performance data infrastructure that underpins mission-critical AI systems. With 12+ years of...

  • Senior Data Engineer

    6 days ago

    bangalore district, India Josys Full time

    Location: Bengaluru, Karnataka, India About The Role We're looking for an experienced Senior Data Engineer (6-8 years) to join our data team. You'll be key in building and maintaining our data systems on AWS. You'll use your strong skills in big data tools and cloud technology to help our analytics team get valuable insights from our data. You'll be in...

  • Senior Data Engineer

    20 minutes ago

    bangalore district, India Quantiphi Full time

    Role: Senior Data Engineer Experience Level: 4 to 7 Years Work location: Mumbai/Bangalore (Hybrid) Required Skills: ● Bachelor’s degree in Computer Science or similar field or equivalent work experience. ● 3+ years of experience on Data Warehousing, Data Engineering or Data Integration projects. ● Expert with data warehousing concepts, strategies,...