Senior Big Data Architect

4 days ago

Meerut, Uttar Pradesh, India beBeeDatabricks Full time ₹ 15,00,000 - ₹ 30,00,000
Lead Databricks Engineer

About the Role:

We are seeking a highly skilled Lead Databricks Engineer to join our team. As a key member of our data engineering group, you will be responsible for designing, developing, and maintaining large-scale data pipelines and ETL processes on Databricks.

The successful candidate will have a proven track record of leading architecture and implementation of scalable, secure, and high-performance data solutions. You will collaborate with business stakeholders, data scientists, and analysts to gather requirements and deliver actionable data solutions.

You will optimize data workflows for performance, cost-efficiency, and reliability. This includes integrating data from multiple structured and unstructured sources into Delta Lake and cloud data warehouses.

As a technical leader, you will mentor junior engineers and provide guidance on data engineering best practices. Additionally, you will troubleshoot and resolve issues in data pipelines, ensuring high data availability and integrity.

Key Responsibilities:

  • Design, develop, and maintain data pipelines and ETL processes on Databricks.
  • Lead the architecture and implementation of scalable, secure, and high-performance data solutions.
  • Collaborate with business stakeholders, data scientists, and analysts to gather requirements and deliver actionable data solutions.
  • Optimize data workflows for performance, cost-efficiency, and reliability.
  • Integrate data from multiple structured and unstructured sources into Delta Lake and cloud data warehouses.
  • Mentor junior engineers and provide technical leadership in data engineering best practices.
  • Troubleshoot and resolve issues in data pipelines, ensuring high data availability and integrity.

Required Skills & Experience:

  • 7+ years of experience in Data Engineering.
  • Proven Subject Matter Expertise in Databricks (including Databricks SQL, Delta Lake, and MLflow).
  • Strong experience with PySpark, Spark SQL, and Scala.
  • Hands-on experience with cloud platforms (Azure preferred; AWS/GCP a plus).
  • Proficiency in data modeling, ETL development, and data warehousing concepts.
  • Solid understanding of big data ecosystems (Hadoop, Kafka, etc.).
  • Strong SQL development skills with a focus on performance tuning.
  • Experience in implementing CI/CD pipelines for data solutions.

Benefits:

Our company offers a dynamic and supportive work environment that fosters growth and innovation. We provide competitive compensation and benefits packages, as well as opportunities for professional development and advancement.

Others:

This is an exciting opportunity to join our team and contribute to the development of cutting-edge data engineering solutions. If you are a motivated and experienced data engineer looking for a new challenge, we encourage you to apply.

  • Big Data Platform Engineer

    2 days ago

    Meerut, Uttar Pradesh, India beBeetechnicallead Full time ₹ 18,50,000 - ₹ 23,50,000

    We are seeking a highly skilled Technical Lead to join our team. The successful candidate will be responsible for building a scalable and extensible big data platform that provides the foundation for collecting, storing, modeling, and analyzing massive data sets from multiple channels.Responsibilities:Align with key client initiativesInterface daily with...

  • Big Data Specialist

    2 days ago

    Meerut, Uttar Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 21,00,000

    Unlock New Career Opportunities in Data AnalyticsJob Description:As a seasoned data professional, you will be responsible for developing and implementing large-scale big data solutions using cutting-edge technologies.Must-Have Skills:Expertise in PySpark with hands-on experience in building scalable data pipelinesStrong understanding of Hive and its...

  • Big Data Specialist

    2 days ago

    Meerut, Uttar Pradesh, India beBeeBigData Full time ₹ 18,05,999 - ₹ 25,10,999

    Job Title: Big Data SpecialistWe are seeking a skilled professional to fill the role of a big data specialist.About the Role:The successful candidate will be responsible for designing, implementing, and maintaining large-scale data clusters using Cloudera technology.Key Responsibilities:Data Cluster Design: Develop high-performance data clusters for big data...

  • Big Data Developer

    3 weeks ago

    Meerut, Uttar Pradesh, India Tata Consultancy Services Full time

    Role : Data Engineering/Big Data Development Location: HyderabadExperience: 4 to 10 YearsMust Have skills : Bigdata Hadoop Platform -Cloudera

  • Data Solutions Architect

    2 days ago

    Meerut, Uttar Pradesh, India beBeeInformatica Full time ₹ 18,00,000 - ₹ 25,00,000

    ETL Developer PositionWe are seeking an experienced ETL developer to join our team and work on delivering cutting-edge data solutions.This role involves collaborating with data architects and analysts to design, develop, and test ETL architecture, transformation jobs, and workflow schedules using Informatica BDM/DEI.The ideal candidate will have expertise in...

  • Chief Data Architect

    3 days ago

    Meerut, Uttar Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Azure Data EngineerWe are seeking a skilled data professional to join our organization and spearhead large-scale data processing initiatives on the Microsoft Azure platform.The ideal candidate will have hands-on experience with Azure services, specifically ADF (Azure Data Factory), ADB (Azure Database), PySpark, and SQL. You should be able to architect and...

  • Cloud Data Solutions Architect

    1 day ago

    Meerut, Uttar Pradesh, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Cloud Data Solutions ArchitectWe're seeking a highly skilled Cloud Data Solutions Architect to lead the design, development, and management of enterprise data solutions on Google Cloud Platform (GCP).Design and architect end-to-end data solutions on GCP, aligning with business and technical requirements.Define data models, storage strategies, data ingestion,...

  • Strategic Data Architect

    4 days ago

    Meerut, Uttar Pradesh, India beBeeDataEngineering Full time ₹ 16,62,413 - ₹ 30,19,569

    Job Title: Senior Data Engineer  Our team is seeking a skilled and experienced Senior Data Engineer to lead our big data engineering efforts. This individual will work closely with clients and team members to understand their requirements and develop architectures that meet their needs.The successful candidate will provide technical leadership and guidance...

  • Senior Data Systems Engineer

    2 days ago

    Meerut, Uttar Pradesh, India beBeeData Full time ₹ 12,00,000 - ₹ 25,00,000

    Job Title: Data ArchitectAbout the Role:We are looking for a highly skilled and experienced Data Architect to join our team. As a Data Architect, you will be responsible for designing and developing scalable data pipelines and infrastructure that support real-time and batch data processing.Your key responsibilities will include:Designing robust data...

  • Big Data Processing Expert

    2 days ago

    Meerut, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data Engineer- Apache DruidKey Responsibilities:Develop and maintain large-scale data processing systems using Apache Druid.Design and implement data models for efficient querying and analytics.Collaborate with cross-functional teams to integrate Druid with various data sources and platforms.Optimize system performance, scalability, and...