Data Engineer for Scalable Big Data Solutions

2 weeks ago

Bengaluru, Karnataka, India Talent21 Management and Shared Service Pvt Ltd Full time

About the Role:

Talent21 Management and Shared Service Pvt Ltd is seeking a highly experienced Data Engineer to join our Epsilon Attribution/Forecasting Product Development team.

Key Responsibilities:

  • Design, implement, and optimize data processing solutions using Scala, Spark, and Hadoop to ensure high performance and scalability.
  • Collaborate with cross-functional teams to deploy big data solutions on our on-premises and cloud infrastructure.
  • Build, schedule, and maintain workflows to streamline data integration and transformation processes.
  • Perform data troubleshooting, document processes, and communicate technical concepts clearly to both technical and non-technical stakeholders.
  • Continuously enhance our attribution engine/forecasting engine to improve accuracy and efficiency.

Requirements:

  • A minimum of 12 years of experience in Scala programming language.
  • Extensive experience with Apache Spark for Big Data processing, including performance tuning and resource optimization.
  • In-depth understanding of the Hadoop ecosystem, including HDFS, YARN, and MapReduce.
  • Expertise in designing and implementing scalable, fault-tolerant data pipelines with end-to-end monitoring and alerting.
  • Proficiency in Python for developing infrastructure modules.
  • Solid grasp of database systems and SQLs for writing efficient queries (RDBMS/Warehouse) to handle large datasets.
  • Familiarity with design patterns and best practices for efficient data modelling, partitioning strategies, and sharding for distributed systems.
  • End-to-end ownership with definition, development, and documentation of software objectives, business requirements, deliverables, and specifications in collaboration with stakeholders.
  • Experience in working on GIT (or equivalent source control) and solid understanding of Unit and integration test frameworks.
  • Able to collaborate with stakeholders/teams to understand requirements and develop a working solution.
  • Able to work within tight deadlines and effectively prioritize and execute tasks in a high-pressure environment.
  • Able to mentor junior staff.

Preferred Qualifications:

  • Hands-on experience with Databricks for unified data analytics, including Databricks Notebooks, Delta Lake, and Catalogues.
  • Proficiency in using the ELK (Elasticsearch, Logstash, Kibana) stack for real-time search, log analysis, and visualization.
  • Strong background in analytics, including the ability to derive actionable insights from large datasets and support data-driven decision-making.
  • Experience with data visualization tools like Tableau, Power BI, or Grafana.
  • Familiarity with Docker for containerization and Kubernetes for orchestration.

Estimated Salary: $180,000 - $250,000 per year, depending on location and experience.

  • Cloud Data Engineer for Scalable Big Data Solutions

    3 days ago

    Bengaluru, Karnataka, India AppSierra Solutions Pvt Ltd Full time

    We are seeking an experienced Cloud Data Bricks Engineer to spearhead our team's big data initiatives. The ideal candidate will have a strong background in software development, with 5-7 years of experience on AWS DataBricks.Key Skills and QualificationsAWS Data Bricks: Strong knowledge and hands-on experience required.DevOps Kubernetes: Experience in...

  • Big Data Engineering Expert for Scalable Solutions

    2 weeks ago

    Bengaluru, Karnataka, India Madhees Techno Consulting Pvt. Ltd Full time

    At Madhees Techno Consulting Pvt. Ltd, we are seeking an experienced Big Data Engineer to join our team and play a crucial role in designing, developing, and implementing scalable and efficient big data solutions.About the RoleThis is a hybrid position that requires collaboration with cross-functional teams to understand business requirements and translate...

  • Big Data Expert

    4 weeks ago

    Bengaluru, Karnataka, India KrtrimaIQ Cognitive Solution Full time

    We are seeking an experienced Big Data Expert to join KrtrimaIQ Cognitive Solution. This role requires hands-on expertise in big data technologies and the ability to design and implement robust data processing solutions.The ideal candidate should have over 5 years of experience with Kafka, PySpark, Python/Scala, Spark, SQL, and the Hadoop ecosystem. Key...

  • Big Data Engineer

    1 day ago

    Bengaluru, Karnataka, India Coffeee Full time

    Coffeee is seeking an experienced Big Data Engineer to join our innovative team. With a focus on designing scalable solutions, this role requires a strong background in Hadoop development and data engineering.About the Role:We are looking for a skilled Big Data Engineer who can develop, design, and implement Hadoop-based solutions, including data ingestion...

  • Data Architect for Scalable Big Data Solutions

    3 weeks ago

    Bengaluru, Karnataka, India myGwork Full time

    About the RoleWe are seeking a skilled Data Architect to join our team at WWGS Data & Analytics. As a Data Architect, you will play a crucial role in designing and implementing next-generation data pipelines and BI solutions that support business analyst, data scientist, and customer reporting needs.Key Responsibilities:Design and implement scalable big data...

  • Big Data Solutions Developer

    2 weeks ago

    Bengaluru, Karnataka, India LinkedIn Full time

    Job SummaryWe are seeking a highly skilled Big Data Solutions Developer to join our team. As a Big Data Solutions Developer, you will be responsible for designing, building, and deploying large-scale big data solutions that power our platform. You will work closely with our engineering teams to ensure the scalability, reliability, and performance of our big...

  • Cloud Data Engineer

    1 day ago

    Bengaluru, Karnataka, India Juniper Consultancy Services Full time

    Juniper Consultancy Services is seeking a highly skilled Cloud Data Engineer to design, implement, and maintain scalable data pipelines and infrastructure leveraging AWS services and big data technologies.Key ResponsibilitiesDesign and develop ETL processes using PySpark, Python, and AWS Glue to extract, transform, and load large datasets from S3, Redshift,...

  • Data Engineer for Scalable Enterprise Systems

    3 days ago

    Bengaluru, Karnataka, India HEVO DATA Full time

    About Hevo DataWe empower businesses to drive growth by consolidating data from multiple sources, enabling faster analytics and informed decision making.Mission and VisionOur mission is to build technology that's simple to adopt and easy to access. We aim to unlock the potential of data for everyone.A Day in the Life of a Data Engineer at HevoYou'll work on...

  • Senior Big Data Professional

    2 weeks ago

    Bengaluru, Karnataka, India Visionyle Solutions Full time

    Visionyle Solutions is looking for an exceptional Senior Big Data Professional to join our team. As a key member of our data engineering team, you will be responsible for designing and developing scalable Big Data applications or solutions on distributed platforms.The estimated salary for this role is ₹1,800,000 - ₹3,000,000 per annum, depending on...

  • Big Data Applications Specialist

    2 weeks ago

    Bengaluru, Karnataka, India Visionyle Solutions Full time

    Visionyle Solutions is hiring a talented Big Data Applications Specialist to join our team. As a key member of our data engineering team, you will be responsible for designing and developing scalable Big Data applications or solutions on distributed platforms.The estimated salary for this role is ₹1,200,000 - ₹2,000,000 per annum, depending on...

  • Data Engineering Lead with Scalable Platforms

    2 weeks ago

    Bengaluru, Karnataka, India HEVO DATA Full time

    Unlock Career Growth as a Data Engineering LeadWe're seeking an exceptional Data Engineering Lead to join our team at Hevo Data. In this role, you'll have the opportunity to own and build scalable platform services that support thousands of customers' data pipelines and ensure load data reliability to cloud warehouses.The Hevo platform core services will...

  • Data Architect for Big Data Engineering

    3 weeks ago

    Bengaluru, Karnataka, India NeerInfo Solutions Full time

    Hiring for Fortune IT Company at NeerInfo SolutionsEstimated Salary: ₹80,00,000 - ₹1,20,00,000 per annumAbout the RoleWe are seeking an experienced Data Architect for Big Data Engineering to join our team at NeerInfo Solutions. As a key member of our organization, you will play a vital role in designing, implementing, and optimizing large-scale data...

  • Senior Big Data Engineer for Scalable Data Pipelines

    1 month ago

    Bengaluru, Karnataka, India WEN Full time

    Big Data Developer Opportunity at WENWe are seeking a highly skilled Big Data Developer to join our dynamic team at WEN. The ideal candidate will have hands-on experience in developing and optimizing data processing pipelines and possess strong expertise in big data technologies.Job Description OverviewIn this role, you will be responsible for designing,...

  • Big Data Solutions Architect

    4 weeks ago

    Bengaluru, Karnataka, India Madhees Techno Consulting Pvt. Ltd Full time

    At Madhees Techno Consulting Pvt. Ltd, we are seeking a highly skilled Senior Data Engineer to join our team. With a strong background in big data engineering and a passion for innovation, you will design, develop, and implement scalable and efficient big data solutions using Spark, Scala, and other relevant frameworks.

  • Big Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Idyllic Services Pvt Ltd Full time

    Job Title: Big Data EngineerWe are looking for a highly skilled Big Data Engineer to join our team at Idyllic Services Pvt Ltd. In this role, you will design, build, and maintain scalable data pipelines, optimize data processes, and ensure seamless integration of data from various sources.About UsWe are a leading provider of innovative data solutions,...

  • Big Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Coders Brain Technology Private Limited Full time

    Job DescriptionWe are seeking a highly skilled Big Data Engineer to join our team at Coders Brain Technology Private Limited. As a key member of our data engineering team, you will be responsible for designing, developing, and deploying scalable big data pipelines using AWS Databricks.About the RoleDesign, develop, and deploy complex big data pipelines using...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Recro Full time

    Job Title: Data Engineer – Scalable Cloud SolutionsLocation: OnsiteExperience: 5+ yearsCompensation Package:We offer a highly competitive salary of $180,000 per annum.Job Description:Data Engineers play a pivotal role in designing and building scalable data solutions. As a Senior Data Engineer at Recro, you will be responsible for developing and optimizing...

  • Data Engineer Position with Scalable Data Solutions

    4 weeks ago

    Bengaluru, Karnataka, India Talent500 Full time

    OverviewTalent500 is a leading technology firm seeking an experienced Data Engineer to join our team. In this role, you will design and implement scalable data pipelines, working closely with cross-functional teams to drive business growth.SalaryThe estimated salary for this position is $120,000 per annum, commensurate with experience.Job DescriptionWe are...

  • Big Data Solutions Architect

    2 weeks ago

    Bengaluru, Karnataka, India XHire Full time

    **About the Role:**XHire is seeking a Big Data Solutions Architect to lead the design, development, and implementation of our data platforms. As a key member of our data organization, you will collaborate with cross-functional teams to deliver high-quality results and drive business growth.**Compensation Package:** $120,000 - $150,000 per year, depending on...

  • Snowflake Engineer for Big Data Solutions

    1 month ago

    Bengaluru, Karnataka, India S P A Enterprise Full time

    S P A Enterprise is a leading provider of big data engineering services, and we're looking for talented individuals to join our team.We're offering a highly competitive salary of $180,000 - $250,000 per year, depending on experience, plus benefits such as medical, dental, and vision insurance, 401(k) matching, and generous paid time off.Job DescriptionWe're...