Big Data Engineer

4 weeks ago

Hyderabad, India People Prime Worldwide Full time

About the CompanyOur client is a trusted global innovator of IT and business services, present in 50+ countries. They specialize in digital & IT modernization, consulting, managed services, and industry-specific solutions. With a commitment to long-term success, they empower clients and society to move confidently into the digital future.Position: Data EngineerExperience: 6+ YearsLocation: HyderabadEmployment Type: Full-timeAbout the RoleWe are looking for a highly skilled Data Engineer with 6+ years of experience in designing and building scalable data pipelines, big data processing solutions, and cloud-based data platforms. The ideal candidate will have strong expertise in Big Data technologies, PySpark, Python, AWS, and Snowflake, and will work closely with data architects, analysts, and business stakeholders to deliver efficient, reliable, and high-performance data solutions.Key Responsibilities- Design, develop, and maintain scalable and high-performance ETL/ELT pipelines using PySpark, Python, and SQL.- Build and optimize Big Data processing workflows using Spark, Hive, and other distributed data frameworks.- Develop ETL/ELT frameworks and workflows on AWS cloud using services such as S3, Glue, EMR, Lambda, Step Functions, and Redshift.- Implement and maintain enterprise-grade Snowflake data models, warehouses, data sharing, and performance tuning.- Collaborate with data architects to design end-to-end data architecture, ensuring scalability, security, and reliability.- Perform data ingestion, transformation, quality validation, and metadata management across various structured and unstructured data sources.- Build CI/CD pipelines for data applications using tools like Git, Jenkins, or AWS CodePipeline.- Monitor and optimize data pipelines for cost efficiency, performance, and reliability.- Troubleshoot production issues and implement solutions for long-term stability and scalability.- Ensure compliance with data governance, security best practices, and industry standards.Required Skills & Qualifications- 6+ years of hands-on experience as a Data Engineer or similar role.- Strong proficiency in PySpark, Spark SQL, and Big Data processing frameworks.- Advanced programming skills in Python (data processing, automation, modular coding).- Deep experience with AWS cloud services (S3, EMR, Glue, Lambda, IAM, Redshift, Athena).- Hands-on experience working with Snowflake—data modeling, pipelines, SnowSQL, performance tuning.- Strong SQL skills with experience in optimizing complex queries.- Experience with workflow orchestration tools (Airflow / AWS Step Functions / Apache Oozie).- Knowledge of DevOps practices, CI/CD, and version control (Git).- Experience handling large-scale, distributed data systems in production environments.- Strong understanding of data warehousing, ETL/ELT patterns, and best practices.Nice-to-Have Skills- Experience with streaming technologies (Kafka, Kinesis).- Knowledge of infrastructure-as-code (Terraform / CloudFormation).- Familiarity with data catalog and governance tools (AWS Glue Data Catalog, Collibra, Alation).- Exposure to ML data pipelines or MLOps frameworks.

  • Big Data Engineer

    5 days ago

    hyderabad, India beBeeData Full time

    Job Description:We are seeking a skilled Big Data Engineer to join our team. The ideal candidate will have expertise in designing, developing, and maintaining large-scale data processing systems.Required Skills and Qualifications:4-7 years of experience in big data engineeringStrong background in programming languages such as Java, Python, or ScalaExperience...

  • Big Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Sky Systems, Inc. (SkySys) Full time

    Role:Big Data EngineerPosition Type:Full-Time Contract (40hrs/week)Contract Duration:Long TermWork Schedule: 8 hours/day (Mon-Fri)Location: Hybrid - Hyderabad, India (3x days onsite/week)We are looking for a skilledBig Data Engineer (5+ years' experience)to join Experian's dynamic data engineering team. You will design and optimize scalable data pipelines,...

  • Big Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Harp Technologies Services Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Job Title: Bigdata + PysparkLocation: HyderabadExperience Level: 5-8 YearsNotice Period: Immediate to 30 DaysInterview Mode: L1 Virtual / L2 F2F (Mandatory)Work Mode: HybridJob SummaryWe are seeking a Specialist with 5 to 8 years of experience in the Big Data Hadoop Ecosystem to design and implement scalable data solutions.Detailed Job DescriptionWork...

  • Big Data Engineer

    4 weeks ago

    Hyderabad, India People Prime Worldwide Full time

    About the Company Our client is a trusted global innovator of IT and business services, present in 50+ countries. They specialize in digital & IT modernization, consulting, managed services, and industry-specific solutions. With a commitment to long-term success, they empower clients and society to move confidently into the digital future. Position: Data...

  • Big Data Engineer

    17 hours ago

    Bengaluru, Chennai, Hyderabad, India Net2source Llp Full time

    Hiring: Big Data/Hadoop Engineer (PySpark, Hive) | 4 to 6 YearsWe are looking for talented Big Data/Hadoop Engineer for one of our premium clientsWork Locations: Bangalore (KA) | Chennai (TN) | Hyderabad (TG)Experience: 4 to 6 YearsRequired Skills:Strong hands-on experience in PySpark and HiveExpertise in Big Data & Hadoop Ecosystem toolsAbility to design...

  • Big Data Engineer

    4 weeks ago

    Hyderabad, India People Prime Worldwide Full time

    About the CompanyOur client is a trusted global innovator of IT and business services, present in 50+ countries. They specialize in digital & IT modernization, consulting, managed services, and industry-specific solutions. With a commitment to long-term success, they empower clients and society to move confidently into the digital future.Position: Data...

  • Big Data

    1 day ago

    Hyderabad, India Greenwich Associates Full time

    Please enter job description - 5+ years software development experience in an enterprise level engineering environment demonstrating increasing levels of technical leadership. 3+ years of work experience with ETL, Data Modeling, and Data Architecture. Experience with Big Data technologies such as Hadoop/Hive/Spark with Spark as primary. Fluent in one or more...

  • Big Data Engineer

    4 days ago

    Hyderabad, India Anicalls (Pty) Ltd Full time

    Big Data Engineer  Specification- • 2 – 7 years of recent experience in data engineering. • Bachelor's Degree or more in Computer Science or a related field. • A solid track record of data management showing your flawless execution and attention to detail. • Strong knowledge of and experience with statistics. • Programming experience, ideally in...

  • Big Data Developer

    3 weeks ago

    Hyderabad, India Tata Consultancy Services Full time

    Job Title :- Data Engineer - BigdataExperience: 4 to 8 YearsLocation: Hyderabad,Chennai, BangaloreJob DescriptionRequired Skills:4+ years of experience in Big data and pysparkMust-HaveGood work experience on Big Data Platforms like Hadoop, Spark, Scala, Hive, Impala, SQLGood-to-HaveGood Spark, Pyspark,Big Data experienceSpark UI/Optimization/debugging...

  • Big Data Developer

    2 weeks ago

    Hyderabad, India Tata Consultancy Services Full time

    Job Title :- Data Engineer - BigdataExperience: 4 to 8 YearsLocation: Hyderabad,Chennai, BangaloreJob DescriptionRequired Skills:4+ years of experience in Big data and pysparkMust-HaveGood work experience on Big Data Platforms like Hadoop, Spark, Scala, Hive, Impala, SQLGood-to-HaveGood Spark, Pyspark,Big Data experienceSpark UI/Optimization/debugging...