Data Engineer

5 days ago

Bengaluru, Karnataka, India OA Compserve Pvt. Ltd Full time ₹ 20,00,000 - ₹ 25,00,000 per year

We are seeking a highly skilled and results-driven Data Engineer with proven expertise in data engineering, reporting, and analytics, specializing in Palantir Foundry. The ideal candidate will have experience designing scalable data architectures, building robust pipelines, and enabling actionable insights through optimized data solutions.

This role requires hands-on technical excellence, a strong understanding of big data ecosystems, and the ability to work closely with cross-functional teams to translate business requirements into efficient technical implementations.

Key Responsibilities

  • Design, develop, and maintain end-to-end data pipelines using Palantir Foundry, including the code repository, Contour for data analysis, health checks, and data expectations.
  • Build and optimize PySpark-based ETL pipelines for large-scale structured and unstructured data processing.
  • Manage data ingestion, transformation, and quality monitoring for high volume, high-velocity datasets.
  • Design scalable and efficient data models aligning with business requirements and analytics needs.
  • Implement data warehousing solutions and manage data lake / lakehouse architecture for analytics.
  • Optimize Spark and PySpark pipelines for cost efficiency, scalability, and low latency.
  • Leverage Hadoop/Spark ecosystems to handle distributed data processing effectively.
  • Collaborate with analytics and BI teams to enable self-service analytics and robust reporting capabilities.
  • Support financial reporting and valuations (e.g., IFRS17) where applicable.
  • Define and implement data validation, profiling, and lineage tracking within Palantir Foundry.
  • Establish best practices for data security, privacy, and compliance.
  • Work closely with data scientists, analysts, and business teams to deliver actionable insights.
  • Translate complex business problems into scalable, technical data solutions.
  • 3+ years of hands-on experience in data engineering and big data ecosystems.
  • Expertise in Palantir Foundry (pipelines, Contour, health checks, expectations).
  • Strong programming skills in PySpark, Python, and SQL for data manipulation and analysis.
  • Solid understanding of Hadoop, Spark, Kafka, and distributed data systems.
  • Experience in data warehousing concepts (Snowflake, Redshift, BigQuery, or similar) is a plus.
  • Exposure to cloud platforms (AWS, GCP, or Azure) and their big data services (Glue, EMR, Dataproc, Databricks).
  • Experience in CI/CD for data pipelines (Git, Jenkins, or equivalent).
  • Understanding of containerization & orchestration (Docker, Kubernetes, Airflow, or Dagster).
  • Knowledge of real-time data streaming frameworks (Kafka, Flink, or Kinesis).
  • Familiarity with data cataloging & governance tools (Alation, Collibra).
  • Familiarity with TypeScript, HTML, CSS, JavaScript, and frameworks like React, Angular, , or
  • Experience with modern visualization tools (Tableau, Power BI, Looker, or Superset).
  • Exposure to financial services, risk modeling, or IFRS17 reporting is a strong plus.

Academic Background

  • Bachelors or Masters degree in Mathematics, Statistics, Operations Research, Industrial Engineering, Computer Science, Econometrics, or Information Technology.

Soft Skills

  • Strong analytical and problem-solving skills.
  • Excellent communication and stakeholder management abilities.
  • Ability to work in agile, cross-functional teams with a high degree of ownership.

)

  • Data Engineer

    7 days ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    As a Data Engineer, your main responsibilities will involve: The project is a migration of a SAS code to DBT code, so it's important that the candidate has experience in this type of projects. Analyze data models and derive logical conclusions. Processes modeling. Hands on development and monitoring of the Azure cloud Platform and various associated...

  • Data Engineer

    1 day ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Design and implement tailored data solutions to meet customer needs and use cases, spanning from streaming to data lakes, analytics, and beyond within a dynamically evolving technical stack. Provide thought leadership by recommending the most appropriate technologies and solutions for a given use case, covering the entire spectrum from the application layer...

  • Data Engineer

    7 days ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 8,00,000 - ₹ 16,00,000 per year

    Build scalable data pipelines from structured and unstructured sources Design ontologies that model business entities, relationships, and process flows Collaborate with domain SMEs and ML engineers to structure agent inputs and outputs Implement transformation, validation, and enrichment logic across diverse datasets Maintain and optimize cloud data...

  • Data Engineer

    7 days ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Migrate ETL workflows from SAP BODS to AWS Glue/dbt/Talend. Develop and maintain scalable ETL pipelines in AWS. Write PySpark scripts for large-scale data processing. Optimize SQL queries and transformations for AWS PostgreSQL. Work with Cloud Engineers to ensure smooth deployment and performance tuning. Integrate data pipelines with existing Unix systems....

  • Sr. Data Engineer Data Science

    1 day ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Req ID: 342869NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Sr. Data Engineer Data Science to join our team in Bangalore, Karnātaka (IN-KA), India (IN). ...

  • Data Engineer

    1 day ago

    Bengaluru, Karnataka, India NTT DATA, Inc. Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Key Responsibilities:Design and implement tailored data solutions to meet customer needs and use cases, spanning from streaming to data lakes, analytics, and beyond within a dynamically evolving technical stack.Provide thought leadership by recommending the most appropriate technologies and solutions for a given use case, covering the entire spectrum from...

  • Data Engineer

    5 days ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    Req ID: 343998NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer - Sr ETL Developer (Architect) to join our team in Bangalore, Karnātaka (IN-KA), India (IN). "Job...

  • Data Engineer

    4 days ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 4,00,000 - ₹ 8,00,000 per year

    Req ID: 344005NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer - ETL Developer to join our team in Bangalore, Karnātaka (IN-KA), India (IN). "Job Duties: Design,...

  • Data Engineer

    7 days ago

    Bengaluru, Karnataka, India NTT DATA North America Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer - DBT to join our team in Bangalore, Karnātaka (IN-KA), India (IN)."Job Duties: We are seeking an Engineer to...

  • Sr Data Engineer

    1 week ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Req ID: 340286NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Sr Data Engineer (Life and Annuity) to join our team in Bangalore, Karnātaka (IN-KA), India (IN). Job Duties: At...