Big Data Engineer

2 weeks ago

Hyderabad, India Teamware Solutions Full time

Job Description Key Responsibilities: - Design and implement scalable, fault-tolerant data pipelines using Scala, Kafka, and other Big Data technologies. - Build real-time data streaming applications to process high-volume datasets from various sources. - Integrate and manage containerized applications using Docker and orchestrate deployments with Kubernetes. - Work with distributed data processing frameworks such as Apache Spark, Flink, or Kafka Streams. - Optimize data ingestion, transformation, and storage processes for performance and reliability. - Collaborate with data scientists, analysts, and backend engineers to meet analytical and operational needs. - Monitor and troubleshoot production issues in real time, ensuring high availability and low latency. - Implement data quality, observability, and monitoring best practices across the pipeline. Qualifications and Requirements: - Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field. - 4+ years of experience in Big Data and streaming platform development. - Proficient in Scala (required); knowledge of Java or Python is a plus. - Hands-on experience with Apache Kafka (Kafka Connect, Kafka Streams, or Confluent Platform). - Strong experience in Docker containerization and Kubernetes orchestration. - Familiarity with Big Data frameworks: Apache Spark, Hadoop, or Flink. - Working knowledge of CI/CD pipelines, Git, and monitoring tools (e.g., Prometheus, Grafana). - Experience with both batch and stream processing architectures. Desirable Skills and Certifications: - Experience with cloud data platforms (e.g., AWS EMR, GCP Dataflow, Azure Synapse). - Understanding of data lake, data warehouse, and data mesh principles. - Familiarity with NoSQL databases (e.g., Cassandra, HBase, or MongoDB). - Certifications like Confluent Certified Developer, CKA/CKAD, or Databricks Spark Developer. - Knowledge of security and compliance in Big Data environments (encryption, RBAC, auditing).

  • Big Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Sky Systems, Inc. (SkySys) Full time

    Role:Big Data EngineerPosition Type:Full-Time Contract (40hrs/week)Contract Duration:Long TermWork Schedule: 8 hours/day (Mon-Fri)Location: Hybrid - Hyderabad, India (3x days onsite/week)We are looking for a skilledBig Data Engineer (5+ years' experience)to join Experian's dynamic data engineering team. You will design and optimize scalable data pipelines,...

  • Big Data engineer

    2 weeks ago

    Hyderabad, Telangana, India Cozzera Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job Title:Data Engineer Location:Hyderabad | Type:Full-time Description:We're hiring aBig Data Engineerto design and implement scalable data pipelines and streaming applications. You'll work with modern big data technologies and cloud platforms to power real-time data solutions. Key Skills:Strong software engineering with Java, Scala, or PythonExpertise in...

  • Big Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India IDESLABS PRIVATE LIMITED Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    We are looking for a skilled Big Data Engineer with 6 to 22 years of experience. The ideal candidate will have expertise in Pyspark, Azure Data Bricks, and workflows. This position is available as a contract role across Pan India.Roles and ResponsibilityDesign and develop scalable big data systems using Pyspark and Azure Data Bricks.Implement and manage...

  • Big Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Incedo Inc Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Data Engineers – Big Data Platform At Incedo, we're looking for experienced Big Data Engineers (5–8 years) to join our AI & Data team in Hyderabad — and be part of building cutting-edge data ecosystems for the Banking domain. What You'll Do • Design, develop & optimize large-scale data pipelines using Hadoop, Hive, HDFS, and Kafka •...

  • Big Data Engineer

    3 days ago

    Hyderabad, Telangana, India Sheryl strategic solutions Pvt. LTD . Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Description : Position : Big Data Engineer Location : Hyderabad, India (Hybrid, 2-3 times per week on any weekdays) Duration : 1 year Job Summary : Experian FSD is seeking a skilled and motivated Big Data Engineer to join our dynamic data engineering team. This hybrid role based in Hyderabad offers the opportunity to work on cutting-edge data...

  • Big Data Engineer

    2 weeks ago

    Hyderabad, Bengaluru, India Fusion Plus Solutions Full time

    Job Description We are looking for a skilled Big Data Engineer with 5 to 7 years of hands-on experience in Big Data technologies. The ideal candidate should have strong expertise in Hadoop, HDFS, Hive, SQL, and Unix, and be able to work on large-scale data processing systems. Roles and Responsibilities: - Work on design, development, and optimization of Big...

  • Big Data Engineer

    4 weeks ago

    Hyderabad, India Rapinno Tech Solutions Pvt.Ltd. Full time

    Big Data Engineer Locations: Hyderabad | Chennai | Pune | Bangalore

  • Big Data Engineer

    4 weeks ago

    Hyderabad, India Rapinno Tech Solutions Pvt.Ltd. Full time

    Big Data Engineer Locations: Hyderabad | Chennai | Pune | Bangalore

  • Big Data Engineer

    4 weeks ago

    Hyderabad, India Rapinno Tech Solutions Pvt.Ltd. Full time

    Big Data Engineer Locations: Hyderabad | Chennai | Pune | Bangalore

  • Big Data Engineer

    3 weeks ago

    Hyderabad, India Tata Consultancy Services Full time

    Dear Candidate Greetings from TATA Consultancy Services Job Openings at TCS Skill : Data Engineer (Spark scala)Exp range: 5yrs to 10 yrs Location: Chennai/Pune/Bangalore/Hyd/DelhiNotice period –30 days Pls find the Job Description below. 1)should be able to design strategies and programs to collect, store, analyse and visualize data from various sources....