Technical Specialist – Big Data Engineer

3 days ago

Kolkata, West Bengal, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000
Job Title: Software Engineering Professional

We are looking for an experienced Software Engineering Professional to join our team.

This is a highly demanding role that requires the ability to design and implement scalable data architectures, develop and optimize ETL/ELT pipelines, and perform data ingestion, transformation, and orchestration across structured and unstructured datasets.

The ideal candidate will have strong experience in Azure Data Engineering with Data Factory, Synapse, Fabric, Databricks, and expertise in SQL, T-SQL, and Python for data processing and transformation.

In addition, the candidate should have hands-on experience with PySpark for big data processing, experience in designing ETL/ELT pipelines and working with large datasets, and proficiency with Azure Data Lake (ADLS) Gen2 for data storage and processing.

  • Key Responsibilities:
  • Develop and optimize ETL/ELT pipelines using Azure Data Factory (ADF), Databricks, and Synapse Analytics.
  • Design and implement scalable data architectures following Medallion Architecture and best practices.
  • Perform data ingestion, transformation, and orchestration across structured and unstructured datasets.
  • Write and optimize SQL queries for data extraction, transformation, and analysis.
  • Implement CI/CD pipelines for data workflows using Azure DevOps.
  • Collaborate with stakeholders to understand data requirements and ensure data availability.
  • Ensure data governance, security, and compliance in Azure-based data solutions.
  • Monitor and troubleshoot data pipelines to maintain reliability and performance.
  • Stay updated with the latest Azure and big data technologies and implement best practices.
  • Resolving issues that are raised during code/review, through exhaustive systematic analysis of the root cause, and being able to justify the decision taken.
  • Carrying out POCs to make sure that suggested design/technologies meet the requirements.

About Us:

We are a global technology company with over two decades of core IT experience. Our differentiators are technological prowess with unwavering back-end support on a wide range of technologies and industry-leading platforms.

We take pride in building innovative, efficient, and robust IT solutions for our clients. We match the client's business goals with our technology expertise and immaculate execution capabilities to solve issues that matter to the end-user.

We believe in empowering our people to perform and grow by offering opportunities, learning, and inspiration—to 'act and accomplish'. If you are a self-starter looking for an open and collaborative work culture to excel in career, we are the place for you.

  • Mastering Big Data Solutions

    2 days ago

    Kolkata, West Bengal, India beBeeData Full time ₹ 0 - ₹ 1

    Big Data Specialist\We are seeking a skilled Big Data Specialist to drive data innovation and growth across our global operations.\The ideal candidate will have at least 5-8 years of experience in big data engineering with strong expertise in:\Key Technologies:\Palantir Foundry / WorkshopPySparkTypescript\About the Role:\You will leverage your expertise in...

  • Big Data Platform Engineer

    3 weeks ago

    Kolkata, West Bengal, India Peoplemint Consultant LLP Full time

    Job Summary :We are seeking an experienced Big Data Platform Engineer to join our banking technology team. The ideal candidate will be responsible for designing, building, and maintaining scalable big data platforms to support advanced analytics, reporting, and regulatory compliance. This role demands strong technical acumen in distributed systems, data...

  • Big Data Systems Developer

    3 days ago

    Kolkata, West Bengal, India beBeeData Full time ₹ 18,00,000 - ₹ 24,00,000

    Job DescriptionWe are seeking a skilled Data Engineer to join our team. As a Data Engineer, you will play a critical role in building and maintaining complex data systems.Your primary responsibility will be to work closely with our data science team to design, develop, and deploy data pipelines that collect, transform, and analyze large datasets. You will...

  • Big Data Platform Leader

    3 days ago

    Kolkata, West Bengal, India beBeeTechnical Full time US$ 1,80,000 - US$ 2,50,000

    Job DescriptionWe are seeking a seasoned professional to lead our team of engineers in building a scalable and extensible big data platform.The ideal candidate will have a strong track record of relevant work experience and a computer science or related technical discipline.Responsibilities include:Aligning the organization with key client...

  • Senior Data Engineer

    3 days ago

    Kolkata, West Bengal, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 20,00,000

    As a data engineering specialist, you will be responsible for designing and developing scalable ETL workflows using Informatica BDM/DEI. This role requires strong expertise in data warehousing concepts, including efficient data modeling and transformation.The ideal candidate will have 5+ years of experience in Informatica BDM/DEI and a proven track record of...

  • Big Data Specialist

    2 days ago

    Kolkata, West Bengal, India beBeeData Full time ₹ 10,00,000 - ₹ 15,00,000

    Data Scientist PositionWe are seeking a highly skilled data scientist to join our team and contribute to the development of cutting-edge solutions.Key Responsibilities:Develop and implement advanced data processing and analytics capabilities using PySpark and Hive.Design and implement scalable data ingestion pipelines from disparate sources onto Big Data...

  • Data Engineering Specialist

    2 days ago

    Kolkata, West Bengal, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title:Data Engineering SpecialistAs a senior data engineer, you will be responsible for designing and implementing scalable, high-performance data pipelines using AWS cloud services and Databricks.Key Responsibilities:Develop and maintain data pipelines and workflows using AWS (S3, Redshift, Glue, EMR, Lambda, Kinesis, Athena) and Databricks.Build and...

  • Big Data Cluster Manager

    2 days ago

    Kolkata, West Bengal, India beBeeDataClusterManager Full time ₹ 90,00,000 - ₹ 1,50,00,000

    Job Title: Big Data Cluster ManagerThis role is responsible for overseeing the management and maintenance of complex data clusters, ensuring optimal performance and efficiency. The ideal candidate will have experience in managing Linux systems and be proficient in building and supporting IT infrastructure.Key Responsibilities:Design and implement scalable...

  • Data Engineer with Expertise in Big Data

    3 days ago

    Kolkata, West Bengal, India beBeeDataEngineering Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Data Engineer with Expertise in PySparkAbout the RoleThis position involves working as a Data Engineer to design, develop and maintain scalable data pipelines using PySpark.Key ResponsibilitiesDesign and implement data pipelines for batch and streaming data processingDevelop robust code in Python with expertise in data structures, algorithms, and...

  • Senior Technical Support Specialist

    7 days ago

    Kolkata, West Bengal, India beBeeExpert Full time ₹ 96,00,000 - ₹ 1,68,00,000

    Job SummaryWe are seeking a skilled technical professional to deliver exceptional support for our data security and governance platform.This role involves resolving complex product issues, mentoring junior engineers, and collaborating cross-functionally to ensure customer satisfaction and retention.Key ResponsibilitiesAct as the primary technical contact for...