Big Data Platform Engineer

3 weeks ago

Kolkata, West Bengal, India Peoplemint Consultant LLP Full time

Job Summary :

We are seeking an experienced Big Data Platform Engineer to join our banking technology team. The ideal candidate will be responsible for designing, building, and maintaining scalable big data platforms to support advanced analytics, reporting, and regulatory compliance. This role demands strong technical acumen in distributed systems, data integration, and platform engineering, along with experience in BFSI environments.

Key Responsibilities :

- Design, develop, and maintain big data infrastructure and platforms (Hadoop, Spark, Kafka, etc.) to support large-scale data processing.

- Work closely with data scientists, analysts, and business stakeholders to support real-time and batch data pipelines.

- Ensure high availability, scalability, and security of big data environments.

- Automate workflows for data ingestion, transformation, and job scheduling using tools like Apache Airflow or Oozie.

- Monitor and optimize system performance and troubleshoot platform-related issues.

- Manage platform upgrades, patching, and support with strict adherence to SLAs and compliance.

- Implement governance, access control, and data privacy policies as per banking regulations (e.g., RBI, GDPR, etc.).

- Collaborate with DevOps and cloud teams for containerization, CI/CD pipelines, and cloud-based big data solutions (AWS/GCP/Azure).

- Document architecture, processes, and best practices.

Required Skills & Qualifications :

- Bachelor's/Master's degree in Computer Science, Engineering, or a related field.

- Minimum 5 years of hands-on experience in big data technologies.

- Expertise in Hadoop ecosystem (HDFS, Hive, HBase, Impala, etc.).

- Strong experience in Apache Spark (PySpark/Scala) and Kafka.

- Proficient in Linux system administration and scripting (Shell, Python).

- Experience with data pipeline orchestration tools (Airflow, Oozie).

- Familiarity with cloud platforms (AWS EMR, GCP DataProc, Azure HDInsight) is a plus.

- Knowledge of data security and compliance in financial systems.

- Solid understanding of relational and NoSQL databases.

- Strong problem-solving, analytical, and communication skills.

Preferred Qualifications :

- Certifications in Hadoop, Spark, or Cloud Platforms (AWS Big Data Specialty, etc.).

- Experience in banking/financial domain with understanding of regulatory requirements and risk/data governance frameworks.

- Exposure to MDM, metadata management, and data lineage tools.

(ref:hirist.tech)

  • High-Performance Data Engineer Position

    1 day ago

    Kolkata, West Bengal, India beBeeData Full time ₹ 1,62,87,736 - ₹ 2,03,09,871

    Big Data Engineer Role OverviewWe're seeking a talented Big Data Engineer to join our team to help us scale and optimize our data processing capabilities. The ideal candidate will have strong analytical thinking and problem-solving skills, excellent attention to detail, and the ability to work effectively in a collaborative environment.Key...

  • Backend Platform Developer

    4 days ago

    Kolkata, West Bengal, India beBeeBackend Full time US$ 2,00,000 - US$ 2,50,000

    Job OpportunityWe are seeking a talented individual to join our team as a Staff Software Engineer, Backend. This is an exceptional opportunity for someone who is passionate about building innovative software solutions and working in a collaborative environment.Key Responsibilities:Architect and develop platforms for financial dataProvide technical leadership...

  • Senior Big Data Engineer

    1 day ago

    Kolkata, West Bengal, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Opportunity: Senior Data EngineerWe are seeking a highly skilled Data Engineer to join our team and contribute to the development of a cutting-edge application for a leading retailer. As a key member of our engineering team, you will be responsible for designing, implementing, and maintaining large-scale data systems.Key Responsibilities:Design and...

  • Data Platform Architect

    24 minutes ago

    Kolkata, West Bengal, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000

    Senior Data EngineerWe are seeking a skilled Senior Data Engineer to design, build, and maintain our modern data platform. This role is perfect for someone with deep technical expertise in ETL pipeline design, data modeling, and data infrastructure who thrives in a fast-paced environment.You will work across the full data stack from ingestion and...

  • Senior Data Engineering Specialist

    1 day ago

    Kolkata, West Bengal, India beBeeData Full time ₹ 18,00,000 - ₹ 25,00,000

    High-Performance Data Solutions ArchitectKey Responsibilities:Design, develop, and maintain data pipelines and ETL processes on Databricks to ensure high-performance, scalability, and reliability.Lead the architecture and implementation of secure, scalable, and high-performance data solutions for business stakeholders and data scientists.Collaborate with...

  • Senior Cloud Data Architect

    2 days ago

    Kolkata, West Bengal, India beBeeDataEngineering Full time ₹ 20,00,000 - ₹ 25,00,000

    Job TitleWe are seeking an experienced Data Engineering professional with expertise in designing and building large-scale data platforms on the Google Cloud Platform (GCP).Key ResponsibilitiesDesign and develop robust data pipelines for ingestion, transformation, and analytics.Utilize Big Data tools such as Hadoop, Spark, and Kafka to process and analyze...

  • Big Data Solutions Architect

    1 day ago

    Kolkata, West Bengal, India beBeeDataAnalyst Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Data AnalystExperience: 4–7 Years Location: Remote Employment Type: Full-TimeWe are seeking a highly skilled professional to play a critical role in data extraction, integration, and advanced analysis by leveraging both traditional programming and emerging technologies.The ideal candidate will have expertise in SQL, Python, and Java, along with...

  • Cloud-Native Data Engineering Manager

    1 day ago

    Kolkata, West Bengal, India beBeeEngineering Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Key Role Summary:We are seeking a highly experienced and strategic leader to manage a team of talented data engineers. The ideal candidate will design and deliver scalable data pipelines and analytics platforms, ensuring high performance, reliability, and security in a cloud-native environment.Leadership Responsibilities:Lead, mentor, and grow a team of data...

  • Databricks Cloud Engineer

    7 days ago

    Kolkata, West Bengal, India beBeeDataEngineer Full time

    Unlock Your Career Potential in Data EngineeringJob DescriptionWe are seeking a highly skilled and experienced Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, implementing, and maintaining large-scale data processing systems on the Databricks platform.Key Responsibilities:Design and implement efficient data...

  • Data Architect

    4 weeks ago

    Kolkata, West Bengal, India Axim Digitech Full time

    Job Profile : - Design and implement scalable and efficient data architectures to support business needs. - Develop data models (conceptual, logical, and physical) that align with organizational goals. - Lead the database design and optimization efforts for structured and unstructured data. - Establish ETL pipelines and data integration strategies for...