Senior Big Data Specialist

1 day ago

Kanpur, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000
Job Title: Senior Data Engineer

Duties and Responsibilities

  • We are seeking a highly skilled Data Engineer to design, develop, and maintain scalable data pipelines and ETL processes on Databricks.
  • The ideal candidate will lead the architecture and implementation of high-performance data solutions, collaborating with business stakeholders, data scientists, and analysts to gather requirements and deliver actionable data insights.
  • This role requires expertise in optimizing data workflows for performance, cost-efficiency, and reliability, as well as integrating data from multiple structured and unstructured sources into Delta Lake and cloud data warehouses.
  • The selected candidate will implement best practices in data governance, security, and quality management, mentoring junior engineers and providing technical leadership in data engineering best practices.
  • In addition, this position involves troubleshooting and resolving issues in data pipelines, ensuring high data availability and integrity.

Required Skills and Qualifications

  • A minimum of 7 years of experience in Data Engineering is required.
  • Proven Subject Matter Expertise in Databricks (including Databricks SQL, Delta Lake, and MLflow) is essential.
  • Strong experience with PySpark, Spark SQL, and Scala is necessary.
  • Hands-on experience with cloud platforms (Azure preferred; AWS/GCP a plus) is required.
  • Proficiency in data modeling, ETL development, and data warehousing concepts is crucial.
  • Solid understanding of big data ecosystems (Hadoop, Kafka, etc.) is necessary.
  • Strong SQL development skills with a focus on performance tuning are required.
  • Experience in implementing CI/CD pipelines for data solutions is desirable.

Benefits of Working with Us

This role offers opportunities for professional growth, collaboration with a talented team, and the chance to work on challenging projects that drive business success.

Other Opportunities

We encourage continuous learning and offer resources for skill development and career advancement.

  • Senior Big Data Specialist

    1 day ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 13,50,524 - ₹ 25,10,060

    Job Title: Data EngineerAt our organization, we're looking for a skilled Data Engineer to join our team.We require:Hands on experience in Python and PySpark.Able to build pySpark applications using Spark Dataframes in Python.Experience in optimizing spark jobs that processes huge volumes of data.Proficient in version control tools like Git.Familiarity with...

  • Data Architect and Engineer

    1 day ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Senior Data Engineer NeededWe're looking for a highly skilled data engineer to join our team.Hands-on expertise in Python and PySpark is essential.5+ years of experience in data engineering with a focus on big data, telemetry, and streaming analytics.The ideal candidate will have strong business acumen, excellent communication skills, and the ability to work...

  • Senior Data Systems Specialist

    3 days ago

    Kanpur, Uttar Pradesh, India beBeeDatabase Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Description:We are seeking a highly skilled Database Architect to join our team. This role goes beyond traditional database management—you will design scalable, high-performance database systems that seamlessly integrate with AI and ML components.The ideal candidate will have deep expertise in database architecture while also contributing to broader...

  • Senior Informatica Developer

    4 days ago

    Kanpur, Uttar Pradesh, India Pacific Data Integrators Full time

    Job Title: Senior Data Engineer (IDMC-CDI)Primary Skills: SQL, IDMC (CDI), ETL Concepts BaseLocation: RemoteShift: US EST (5 PM to 2 AM IST)Experience: 5-12 YearsAbout UsWe are PDI (Pacific Data Integrators), a leading provider of Data Management, Analytics, and Cloud solutions. As an Informatica Platinum Partner and a reseller of various market-leading...

  • Senior Process Mining Specialist

    2 days ago

    Kanpur, Uttar Pradesh, India beBeeProcessMining Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Senior Process Mining Specialist - BFSI SectorAbout the Role:In a rapidly evolving business landscape, organizations in the BFSI sector are increasingly seeking innovative solutions to optimize operations and drive growth. As a Senior Process Mining Specialist, you will play a key role in helping our clients unlock valuable insights and improve...

  • Quality Assurance

    2 days ago

    Kanpur, Uttar Pradesh, India BIG Language Solutions Full time

    Quality Assurance (QA) SpecialistPosition: Quality Assurance (QA) SpecialistExperience Required: 3+ Years Start Date: ImmediateLocation: NoidaEmployment Type: Full-TimeTo support the quality and accuracy of multilingual projects by applying linguistic expertise and meticulous review processes, ensuring all deliverables meet client and company standards while...

  • Data Strategist

    5 days ago

    Kanpur, Uttar Pradesh, India beBeeDataStrategist Full time ₹ 1,50,00,000 - ₹ 2,01,00,000

    About This RoleWe are seeking a Data Architect to join our team and contribute to the design and implementation of data strategies for corporate finance functions.Key Responsibilities:Develop scalable analytical solutions leveraging cloud and big data technologies.Collaborate with senior stakeholders and business consultants to design and implement technical...

  • Senior Data Pipeline Architect

    1 day ago

    Kanpur, Uttar Pradesh, India beBeeEngineeringManager Full time ₹ 20,00,000 - ₹ 25,00,000

    About the RoleWe are seeking an experienced Engineering Manager to lead a team of data engineers in designing and delivering scalable data pipelines and analytics platforms using Big Data platforms, Microsoft Azure, and Databricks.Main ResponsibilitiesLead, mentor, and grow a team of data engineers working on large-scale distributed data systems.Architect...

  • Senior Data Engineer

    2 weeks ago

    Kanpur, Uttar Pradesh, India Betterbeans Consultants Pvt Ltd Full time

    Role : Senior Data EngineerExperience : 8-10 years' experienceSkill Set Required : Azure Data FactoryAzure DatabricksPythonSQL, PL/SQLSSISJob Role/Job Description : Senior Data Engineer will be responsible for designing, implementing, and maintaining data solutions on the Microsoft Azure Data platform and SQL Server (SSIS, SSAS, UC4 Atomic)Collaborate with...

  • Advanced Data Insights Professional

    52 minutes ago

    Kanpur, Uttar Pradesh, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Business Insight SpecialistA Senior Business Insight Specialist plays a vital role in driving business growth through data-driven insights and strategic decision-making.Key Responsibilities:Design and develop robust data pipelines using Python to extract, transform, and load data from various sources.Conduct in-depth analysis of large datasets using...