Aws + Pyspark

2 days ago

Chennai Tamil Nadu, India Cognizant Full time

**Skills
- AWS +Pyspark**

**Experience: 4 to 9 years**

**Location: AIA-Pune**

**As an AWS Data Engineer, you will design, build, and manage robust data pipelines on the AWS cloud platform. You will leverage your expertise in AWS data services and programming languages to ingest, process, transform, and store data efficiently, enabling data-driven insights and decision-making.**

**Key Responsibilities**:

- **Data Pipeline Development: Design, implement, and maintain scalable data pipelines using AWS services such as S3, Glue, Lambda, EMR, and Kinesis.**
- **Data Processing and Transformation: Utilize PySpark, Spark, and SQL to perform complex data transformations and aggregations on large datasets within the AWS ecosystem.**
- **Data Storage and Management: Design and implement data storage solutions using Amazon S3, RDS, DynamoDB, and Redshift, ensuring data quality, integrity, and accessibility.**
- **Data Modeling and Warehousing: Develop and maintain data models to support analytics and reporting, leveraging Redshift as the data warehousing solution.**
- **Infrastructure and Cloud Technologies: Provision and manage scalable data infrastructure using EC2, VPC, IAM, CloudFormation, and other relevant AWS services.**
- **Performance Optimization and Monitoring: Continuously monitor and optimize data pipelines and systems using CloudWatch, identifying and resolving performance bottlenecks.**
- **Collaboration and Knowledge Sharing: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and provide technical guidance.**

**Qualifications**:

- **Proficiency in Python, SQL, PySpark, and Spark.**
- **Strong expertise in AWS data services (S3, Glue, Lambda, EMR, Redshift, DynamoDB, RDS).**
- **Experience with data warehousing, ETL processes, and data modeling.**
- **Excellent problem-solving, analytical, and communication skills.

  • AWS + Pyspark

    2 days ago

    Chennai, Tamil Nadu, India Cognizant Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Skills- AWS +PysparkExperience: 4 to 9 yearsLocation: AIA-PuneAs an AWS Data Engineer, you will design, build, and manage robust data pipelines on the AWS cloud platform. You will leverage your expertise in AWS data services and programming languages to ingest, process, transform, and store data efficiently, enabling data-driven insights and...

  • TCS Hiring for AWS Senior Data Engineer with Pyspark, AWS, Glue_Chennai

    3 weeks ago

    Chennai, Tamil Nadu, India, Tamil Nadu Tata Consultancy Services Full time

    Job Title: AWS Senior Data Engineer with Pyspark, AWS, GlueLocation: ChennaiExperience: 6 to 10 YearsNotice Period: 30-45 daysJob Description:Must: PySpark, AWS[ETL Concepts, S3, Glue, EMR, Redshift, DMS, AppFlow] ,Qlik Replicate, Data TestingNice To Have: Hadoop, Teradata Background, IaC[Cloud Formation / Terraform], GitKind Regards,Priyankha M

  • Etl Databricks with Aws

    6 days ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Develop and maintain a metadata driven generic ETL framework for automating ETL code Design, build, and optimize ETL/ELT pipelines using Databricks (PySpark/SQL) on AWS. Ingest data from a variety of structured and unstructured sources (APIs, RDBMS, flat files, streaming). Develop and maintain robust data pipelines for batch and streaming data using Delta...

  • Aws Data

    4 days ago

    Chennai, Tamil Nadu, India virtusa Full time

    P3-C3-TSTSAWS Data EngineerDesign and build scalable data pipelines using AWS services like AWS Glue Amazon Redshift and QS SNS Cloudwatch Step function CDK or Terrafoam Develop efficient ETL processes for data extraction transformation and loading into data warehouses and lakes Create and manage applications using Python Pyspark SQL Databricks and various...

  • Pyspark / Reltio

    1 week ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Key Responsibilities: Develop and maintain data pipelines using PySpark in distributed computing environments (e.g., AWS EMR, Databricks). Integrate and synchronize data between enterprise systems and the Reltio MDM platform. Design and implement data transformation, cleansing, and enrichment processes. Collaborate with data architects, business analysts,...

  • Aws data engineer

    1 day ago

    Chennai, India Tata Consultancy Services Full time

    Experience: 5-10 YrsLocation - Bangalore, Chennai, Hyderabad, Pune, Kochi, Bhubaneshawar, KolkataKey SkillsAWS Lambda, Python, Boto3 , Pyspark, GlueMust have SkillsStrong experience in Python to package, deploy and monitor data science appsKnowledge in Python based automationKnowledge of Boto3 and related Python packagesWorking experience in AWS and AWS...

  • AWS Data Engineer

    3 weeks ago

    Chennai, India Tata Consultancy Services Full time

    Experience: 5-10 Yrs Location - Bangalore,Chennai,Hyderabad,Pune,Kochi,Bhubaneshawar,Kolkata Key Skills AWS Lambda, Python, Boto3 ,Pyspark, Glue Must have Skills Strong experience in Python to package, deploy and monitor data science apps Knowledge in Python based automation Knowledge of Boto3 and related Python packages Working experience in AWS and AWS...

  • Pyspark - Machine Learning

    2 weeks ago

    Chennai, Tamil Nadu, India Virtusa Full time

    7+ years of experience in Big Data with strong expertise in Spark and Scala Mandatory Skills: Big Data Primarily Spark and Scala Strong Knowledge in HDFS, Hive, Impala with knowledge on Unix, Oracle, Autosys, Good to Have : Agile Methodology and Banking Expertise Strong Communication Skills Not limited to Spark batch, need Spark streaming experience No SQL...

  • TCS Hiring for AWS Senior Data Engineer with Pyspark, AWS, Glue_Chennai

    1 week ago

    Chennai, India Tata Consultancy Services Full time

    Job Title: AWS Senior Data Engineer with Pyspark, AWS, GlueLocation: ChennaiExperience: 6 to 10 YearsNotice Period: 30-45 daysJob Description:Must: PySpark, AWS(ETL Concepts, S3, Glue, EMR, Redshift, DMS, AppFlow) ,Qlik Replicate, Data TestingNice To Have: Hadoop, Teradata Background, IaC(Cloud Formation / Terraform), GitKind Regards,Priyankha M

  • TCS Hiring for AWS Senior Data Engineer with Pyspark, AWS, Glue_Chennai

    1 week ago

    chennai, India Tata Consultancy Services Full time

    Job Title: AWS Senior Data Engineer with Pyspark, AWS, GlueLocation: ChennaiExperience: 6 to 10 YearsNotice Period: 30-45 daysJob Description:Must: PySpark, AWS[ETL Concepts, S3, Glue, EMR, Redshift, DMS, AppFlow] ,Qlik Replicate, Data TestingNice To Have: Hadoop, Teradata Background, IaC[Cloud Formation / Terraform], GitKind Regards,Priyankha M