Pyspark, AWS Engineer Lead

1 week ago


Hyderabad, Telangana, India Virtusa Full time
Pyspark, AWS Engineer Lead
  • CREQ187058 Description 6 8 years of relevant work experience showing growth as a Data Engineer.
Hands On programming experienceImplementation Experience on Kafka, Kinesis, Spark, AWS Glue, AWS LakeFormation.

Experience of performance optimization in Batch and Real time processing applicationsExpertise in Data Governance and Data Security ImplementationWorked on Scheduling tools like AirflowGood hands-on design and programming skills building reusable tools and productsExperience developing in AWS or similar cloud platforms.


Preferred:
, ECS, EKS, S3, EMR, DynamoDB, Aurora, Redshift, QuickSight or similar.
Good knowledge of Python. Good to have Java/Scala.

Familiarity with systems with very high volume of transactions, micro service design, or data processing pipelines (Spark).Experience in efficient data processing using open table format Delta, Iceberg.

Knowledge and hands-on experience with server-less technologies such as Lambda, MSK, MWAA, Kinesis Analytics a plus.
Expertise in practices like Agile, Peer reviews, Continuous IntegrationPrimary Location Hyderabad, Andhra Pradesh, India Job Type Experienced Primary Skills AWS Lake Formation, AWS Glue, Java, Spark, Kafka, Python, Amazon Redshift Years of Experience 8 Qualification

6-8years of relevant work experience showing growth as a Data Engineer.
Hands On programming experienceImplementation Experience on Kafka, Kinesis, Spark, AWS Glue, AWS LakeFormation.

Experience of performance optimization in Batch and Real time processing applicationsExpertise in Data Governance and Data Security ImplementationWorked on Scheduling tools like AirflowGood hands-on design and programming skills building reusable tools and productsExperience developing in AWS or similar cloud platforms.


Preferred:
, ECS, EKS, S3, EMR, DynamoDB, Aurora, Redshift, QuickSight or similar.
Good knowledge of Python. Good to have Java/Scala.

Familiarity with systems with very high volume of transactions, micro service design, or data processing pipelines (Spark).Experience in efficient data processing using open table format Delta, Iceberg.

Knowledge and hands-on experience with server-less technologies such as Lambda, MSK, MWAA, Kinesis Analytics a plus.
Expertise in practices like Agile, Peer reviews, Continuous IntegrationTravel No

  • Hyderabad, Telangana, India Virtusa Full time

    Pyspark, AWS Engineer Lead - CREQ187058 Description 6-8 years of relevant work experience showing growth as a Data Engineer.Hands On programming experienceImplementation Experience on Kafka, Kinesis, Spark, AWS Glue, AWS LakeFormation.Experience of performance optimization in Batch and Real time processing applicationsExpertise in Data Governance and Data...

  • Pyspark Lead

    7 days ago


    Hyderabad, Telangana, India CIEL HR Services Full time

    We are hiring for our client-An International IT service provider Role- Pyspark Lead Mandatory skill set- Python+Spark+Scala(experience in all 3 skill sets should be strong & mandatory) ,Data Engineering+ AWS Exp. Level- 5-12 yrs Location- Bangalore & Hyderabad Notice Period-Immediate to 15days Job Description Design and implement scalable and...


  • Hyderabad, Telangana, India Virtusa Full time

    Python Pyspark with AWS - CREQ188741 Description Responsibilities:1.Analyze various data sources present with the client, including structured and unstructured data formatsentities across different data sources to identify relationships and data inconsistencies3.Analyze existing reports created with Tableau and other tools to understand data usage and...


  • Hyderabad, Telangana, India Virtusa Full time

    Python Pyspark with AWS - CREQ188741 DescriptionResponsibilities:Analyze various data sources present with the client, including structured and unstructured data formats entities across different data sources to identify relationships and data inconsistenciesAnalyze existing reports created with Tableau and other tools to understand data usage and identify...

  • AWS Data Engineer

    1 week ago


    Hyderabad, Telangana, India ProArch Full time

    ProArch is seeking an experienced AWS Data Engineer to join our team. As an AWS Data Engineer, you will be responsible for designing, building, and maintaining data solutions on the AWS platform.Job Description:Must to Have Skills : AWS - Data Engineer - PySpark, Glue, S3, Athena To work in the capacity of AWS Cloud developer. Scripting/programming in...

  • AWS Data Engineer

    7 days ago


    Hyderabad, Telangana, India ProArch Full time

    ProArch is seeking an experienced AWS Data Engineer to join our team. As an AWS Data Engineer, you will be responsible for designing, building, and maintaining data solutions on the AWS platform.Job Description:Must to Have Skills : AWS Data EngineerPySpark, Glue, S3, AthenaTo work in the capacity of AWS Cloud developer.Scripting/programming in...

  • Pyspark

    7 days ago


    Hyderabad, Telangana, India Nityo Infotech Full time

    Lead and guide development activities of data ingestions to AWS cloud and Databricks data harmonization meeting the functional and non-functional requirements.Collaborate with SME and business SME to develop architectural requirements to ensure client satisfaction with productDefine the standards processes and the best practices to be followed for...


  • Hyderabad, Telangana, India Virtusa Full time

    Pyspark Data Engineer - CREQ190586 Description 4 to 6 years of relevant work experience showing growth as a Data Engineer Hands On programming experience Implementation Experience on Kafka, Kinesis, Spark, AWS Glue, AWS LakeFormation. Experience of performance optimization in Batch and Real time processing applications Expertise in Data Governance and Data...

  • Pyspark

    7 days ago


    Hyderabad, Telangana, India Cognizant Technology Solutions Full time

    Job descriptionPySpark Developer / PySpark Data EngineerShould have at least three years of strong experience in PySparkStrong SQL ExpertiseStrong communication and client-facing skillsEmployee Status :Full Time EmployeeShift :Day JobTravel :NoJob Posting :Jun **About Cognizant


  • Hyderabad, Telangana, India Virtusa Full time

    Pyspark Data Engineer CREQ187054 Description 6 8 years of relevant work experience showing growth as a Data Engineer.Hands On programming experienceImplementation Experience on Kafka, Kinesis, Spark, AWS Glue, AWS LakeFormation.Experience of performance optimization in Batch and Real time processing applicationsExpertise in Data Governance and Data Security...


  • Hyderabad, Telangana, India Virtusa Full time

    Pyspark Data Engineer CREQ190586 Description 4 to 6 years of relevant work experience showing growth as a Data EngineerHands On programming experienceImplementation Experience on Kafka, Kinesis, Spark, AWS Glue, AWS LakeFormation.Experience of performance optimization in Batch and Real time processing applicationsExpertise in Data Governance and Data...

  • AWS Data Engineer

    1 week ago


    Hyderabad, Telangana, India Corporate Resources Full time

    Job DescriptionDataEngineer on AWS who is adept in cloud friendly tech such as Python and PySpark.Goodknowledge of AWS services such as S3, Lambda, Glue, AthenaIndustryIT-Software / Software ServicesFunctional AreaIT Software - Application Programming, MaintenanceJob RoleSoftware DeveloperJob TypePermanentQualificationUG QualificationB.Tech/B.E - CivilPG...


  • Hyderabad, Telangana, India Genpact Full time

    Genpact (NYSE: G) is a global professional services and solutions firm delivering outcomes that shape the future. Our 125,000+ people across 30+ countries are driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients. Powered by our purpose – the relentless pursuit of a world that works better for people –...

  • Data Engineer

    7 days ago


    Hyderabad, Telangana, India WALLERO TECHNOLOGIES PRIVATE LIMITED Full time

    Roles & Responsibilities :- Tech Stack Expertise : Proficiency in Python, Pyspark, and Databricks, utilizing these tools to develop robust data pipelines, perform data transformations, and optimize data workflows within the Supply Chain realm.- Innovative Problem Solving : Demonstrate technical expertise by thinking creatively and providing innovative...

  • AWS Data Architect

    1 week ago


    Hyderabad, Telangana, India HuQuo Full time

    Role Description: Data Engineer - Big Data The ideal candidate is a hands-on technology developer with experience in developing scalable applications and platforms. They must be at ease working in an agile environment with little supervision. The person should be a self-motivated person with a passion for problem solving and continuous learning. Role and...

  • AWS Data Engineer

    3 weeks ago


    Hyderabad, Telangana, India DATABUZZPRIVATE LIMITED Full time

    Job Description : As a Data Engineer, you will play a crucial role in designing, developing, and maintaining data architecture and infrastructure. The successful candidate should possess a strong foundation in Python, pyspark, SQL, and ETL processes, with a demonstrated ability to implement solutions in a cloud environment.Mandatory Skills : - Design, build,...

  • AWS Data Engineer

    1 week ago


    Hyderabad, Telangana, India Corporate Resources Full time

    Job Description DataEngineer on AWS who is adept in cloud friendly tech such as Python and PySpark. Goodknowledge of AWS services such as S3, Lambda, Glue, Athena Industry IT-Software / Software Services Functional Area IT Software - Application Programming, Maintenance Job Role Software Developer Job Type Permanent Qualification UG...


  • Hyderabad, Telangana, India Xpetize Full time

    Years of Experience : 3+ Years**Joining Time frame :immediate to 1 Week**Positions: 4Mandatory skills:Mandatory skills: Experience in Python/Pyspark/Scala SQL GCP/AWS/Azure Hadoop / HDFS/Other Databases Airflow/Other Automation tools Linux/ Bash scripting Kafka/Flume/Kinesis/Data StreamingJob Type : Full Time**About Us


  • Hyderabad, Telangana, India KPI Partners Full time

    We are looking for a PySpark solutions developer and data engineer that is able to design and build solutions for one of our Fortune 500 Client programs, which aims to build a data standardized and curation-based Hadoop cluster. This high visibility, fast-paced key initiative will integrate data across internal and external sources, provide analytical...

  • Data Engineer

    7 days ago


    Hyderabad, Telangana, India Risk Resources LLP Full time

    Job Description :The Cloud Engineer will focus on Capture data and transforming the data as per business needs. We receive data from multiple internal and external sources which is maintained in Data marts within Data lakeMandatory Skills :- 5 + years of experience on Apachespark, Databricks, Elastic Search, PySpark, Delta Lake, Data Factory, Data Bricks,...