Big Data Engineer

3 days ago

hyderabad, India People Prime Worldwide Full time

About the CompanyOur client is a trusted global innovator of IT and business services, present in 50+ countries. They specialize in digital & IT modernization, consulting, managed services, and industry-specific solutions. With a commitment to long-term success, they empower clients and society to move confidently into the digital future.Position: Data EngineerExperience: 6+ YearsLocation: HyderabadEmployment Type: Full-timeAbout the RoleWe are looking for a highly skilled Data Engineer with 6+ years of experience in designing and building scalable data pipelines, big data processing solutions, and cloud-based data platforms. The ideal candidate will have strong expertise in Big Data technologies, PySpark, Python, AWS, and Snowflake, and will work closely with data architects, analysts, and business stakeholders to deliver efficient, reliable, and high-performance data solutions.Key ResponsibilitiesDesign, develop, and maintain scalable and high-performance ETL/ELT pipelines using PySpark, Python, and SQL.Build and optimize Big Data processing workflows using Spark, Hive, and other distributed data frameworks.Develop ETL/ELT frameworks and workflows on AWS cloud using services such as S3, Glue, EMR, Lambda, Step Functions, and Redshift.Implement and maintain enterprise-grade Snowflake data models, warehouses, data sharing, and performance tuning.Collaborate with data architects to design end-to-end data architecture, ensuring scalability, security, and reliability.Perform data ingestion, transformation, quality validation, and metadata management across various structured and unstructured data sources.Build CI/CD pipelines for data applications using tools like Git, Jenkins, or AWS CodePipeline.Monitor and optimize data pipelines for cost efficiency, performance, and reliability.Troubleshoot production issues and implement solutions for long-term stability and scalability.Ensure compliance with data governance, security best practices, and industry standards.Required Skills & Qualifications6+ years of hands-on experience as a Data Engineer or similar role.Strong proficiency in PySpark, Spark SQL, and Big Data processing frameworks.Advanced programming skills in Python (data processing, automation, modular coding).Deep experience with AWS cloud services (S3, EMR, Glue, Lambda, IAM, Redshift, Athena).Hands-on experience working with Snowflake—data modeling, pipelines, SnowSQL, performance tuning.Strong SQL skills with experience in optimizing complex queries.Experience with workflow orchestration tools (Airflow / AWS Step Functions / Apache Oozie).Knowledge of DevOps practices, CI/CD, and version control (Git).Experience handling large-scale, distributed data systems in production environments.Strong understanding of data warehousing, ETL/ELT patterns, and best practices.Nice-to-Have SkillsExperience with streaming technologies (Kafka, Kinesis).Knowledge of infrastructure-as-code (Terraform / CloudFormation).Familiarity with data catalog and governance tools (AWS Glue Data Catalog, Collibra, Alation).Exposure to ML data pipelines or MLOps frameworks.

  • Big Data Engineer

    1 week ago

    Hyderabad, Telangana, India Sky Systems, Inc. (SkySys) Full time

    Role:Big Data EngineerPosition Type:Full-Time Contract (40hrs/week)Contract Duration:Long TermWork Schedule: 8 hours/day (Mon-Fri)Location: Hybrid - Hyderabad, India (3x days onsite/week)We are looking for a skilledBig Data Engineer (5+ years' experience)to join Experian's dynamic data engineering team. You will design and optimize scalable data pipelines,...

  • Big data engineer

    7 days ago

    Hyderabad, India Virtusa Full time

    Big data engineer - CREQ Description Position: Big data engineer Primary: big data concepts, aws, pyspark Location: HYD Create Trigger based automation framework for Data Migration Identify roles/access needed for data migration from federated bucket to managed bucket and Build APIs for the same Integrate CDMS framework with Lake and Data bridge API Data...

  • Big Data Engineer

    1 week ago

    Hyderabad, Telangana, India IDESLABS PRIVATE LIMITED Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    We are looking for a skilled Big Data Engineer with 6 to 22 years of experience. The ideal candidate will have expertise in Pyspark, Azure Data Bricks, and workflows. This position is available as a contract role across Pan India.Roles and ResponsibilityDesign and develop scalable big data systems using Pyspark and Azure Data Bricks.Implement and manage...

  • Big Data Engineer

    1 week ago

    Hyderabad, Telangana, India Harp Technologies Services Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Job Title: Bigdata + PysparkLocation: HyderabadExperience Level: 5-8 YearsNotice Period: Immediate to 30 DaysInterview Mode: L1 Virtual / L2 F2F (Mandatory)Work Mode: HybridJob SummaryWe are seeking a Specialist with 5 to 8 years of experience in the Big Data Hadoop Ecosystem to design and implement scalable data solutions.Detailed Job DescriptionWork...

  • Big Data Engineer

    4 weeks ago

    Hyderabad, Bengaluru, India Fusion Plus Solutions Full time

    Job Description We are looking for a skilled Big Data Engineer with 5 to 7 years of hands-on experience in Big Data technologies. The ideal candidate should have strong expertise in Hadoop, HDFS, Hive, SQL, and Unix, and be able to work on large-scale data processing systems. Roles and Responsibilities: - Work on design, development, and optimization of Big...

  • Big Data Engineer

    4 days ago

    Hyderabad, India People Prime Worldwide Full time

    About the Company Our client is a trusted global innovator of IT and business services, present in 50+ countries. They specialize in digital & IT modernization, consulting, managed services, and industry-specific solutions. With a commitment to long-term success, they empower clients and society to move confidently into the digital future. Position: Data...

  • Big Data Engineer

    3 days ago

    Hyderabad, India People Prime Worldwide Full time

    About the Company Our client is a trusted global innovator of IT and business services, present in 50+ countries. They specialize in digital & IT modernization, consulting, managed services, and industry-specific solutions. With a commitment to long-term success, they empower clients and society to move confidently into the digital future. Position: Data...

  • Big Data Engineer

    3 days ago

    Hyderabad, India People Prime Worldwide Full time

    About the Company Our client is a trusted global innovator of IT and business services, present in 50+ countries. They specialize in digital & IT modernization, consulting, managed services, and industry-specific solutions. With a commitment to long-term success, they empower clients and society to move confidently into the digital future. Position: Data...

  • Big Data Engineer

    2 days ago

    Hyderabad, India People Prime Worldwide Full time

    About the Company Our client is a trusted global innovator of IT and business services, present in 50+ countries. They specialize in digital & IT modernization, consulting, managed services, and industry-specific solutions. With a commitment to long-term success, they empower clients and society to move confidently into the digital future. Position: Data...

  • Big Data QA/Test Engineer

    2 weeks ago

    Hyderabad, Telangana, India deltaclass technology solutions Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Role : Bigdata QA Test Engineer Exp : 6+Yrs Notice :Immediate to 15 Days Location:Hyderabad (Hybrid) Big Data QA/Test engineer – 3 Openings Hyderabad or Chennai (Hybrid) Big Data QA/Test Engineer AWS + Data-bricks + Big Data + Python + DevOps Big Data QA/Test Engineer AWS + Data-bricks + Big Data + Python + DevOps Big Data QA/Test Engineer AWS +...