AWS Data Engineer

1 day ago

Bengaluru, India Neerinfo Solutions Full time

Key Responsibilities :

AWS Data Infrastructure Development :

- Design, develop, and maintain AWS-based data pipelines using services such as AWS Glue, AWS Redshift, Amazon S3, and AWS Lambda.

- Build ETL (Extract, Transform, Load) processes, integrating batch and near real-time data from various sources to Amazon Redshift, S3, or other cloud-based storage solutions.

- Develop data transformation scripts using Python and PySpark to process large datasets in the cloud.

Big Data Technologies :

- Work with Apache Spark and the Hadoop ecosystem to manage large-scale data processing workloads.

- Optimize Apache Spark and Hadoop jobs for performance and scalability, ensuring data pipelines run efficiently at scale.

SQL and Database Optimization :

- Write and optimize complex SQL queries for data manipulation and aggregation in cloud data warehouses.

- Experience in AWS Redshift for OLAP workloads, Hive for big data processing, or similar data warehouse solutions.

Cloud and Data Security :

- Implement security measures to ensure the protection and privacy of sensitive data within the AWS ecosystem, following industry best practices.

- Work closely with data security teams to ensure compliance with data governance and regulatory requirements.

Scheduling and Automation :

- Experience with scheduling tools like Apache Airflow for workflow automation and pipeline orchestration.

- Set up and maintain automated pipelines, monitor job performance, and manage job failures to ensure the continuity of data workflows.

Documentation and Best Practices :

- Ensure readable and maintainable documentation of data engineering components being developed for transparency, knowledge sharing, and onboarding.

- Follow coding best practices and standards for Python and other technologies used in the development of data pipelines.

Collaboration and Cross-Functional Teams :

- Collaborate with data scientists, analysts, and other engineers to understand data requirements and deliver solutions that enhance business decision-making.

- Participate in agile development processes, contributing to sprint planning and progress tracking.

Required Qualifications and Skills :

Experience :

- 4 to 8 years of experience in data engineering, with significant hands-on experience in AWS services and data pipeline development.

- Strong experience with AWS services including Redshift, Glue, EMR, Lambda, and S3.

- In-depth experience with Apache Spark and Hadoop ecosystem for distributed data processing and analysis.

- Proficiency in Python and PySpark for data engineering tasks, data transformation, and automation.

Technical Expertise :

- Strong understanding of SQL for data manipulation, performance tuning, and data retrieval from cloud-based data warehouses (AWS Redshift, Hive).

- Expertise in designing and developing ETL pipelines (batch and near real-time) for integrating data across different systems and platforms.

- Exposure to data scheduling tools like Apache Airflow for orchestrating and automating workflows.

- Hands-on experience with data security practices and the ability to implement security measures to protect data in AWS.

Cloud & Big Data Technologies :

- Advanced experience with AWS data services (Redshift, Glue, S3, Lambda, EMR) for big data analytics and storage solutions.

- Familiarity with Hadoop, Spark, and other big data processing frameworks.

Tools & Frameworks :

- Knowledge of version control systems like Git for managing codebases and collaborating with teams.

- Experience working in agile development environments, contributing to sprint planning and continuous & Troubleshooting :

- Excellent problem-solving and debugging skills, with the ability to resolve data pipeline issues and improve system performance.

- Ability to troubleshoot and resolve issues related to data pipelines, databases, and AWS infrastructure.

Soft Skills :

- Strong communication and collaboration skills, with the ability to work effectively in cross-functional teams.

- Ability to document and communicate technical requirements and solutions clearly and concisely.

Preferred Skills :

- Experience with data lake architecture and integration of data across multiple sources.

- Certifications in AWS (e.g, AWS Certified Big Data - Specialty, AWS Certified Solutions Architect).

- Experience with containerization technologies such as Docker and orchestration tools like Kubernetes.

- Familiarity with Data Governance and Data Quality practices.

Education :

- Bachelor's or Master's degree in Computer Science, Information Technology, Data Engineering, or a related field

(ref:hirist.tech)

  • Sr Data Scientist, AWS Support Strategy Analytics and Data Science, Support Engineering

    6 months ago

    Bengaluru, India AWS India - Karnataka - D11 Full time

    AWS Sales, Marketing, and Global Services (SMGS) is responsible for driving revenue, adoption, and growth from the largest and fastest growing small- and mid-market accounts to enterprise-level customers including public sector. The AWS Global Support team interacts with leading companies and believes that world-class support is critical to customer success....

  • AWS Data Engineer

    6 months ago

    Bengaluru, India Torry Harris Full time

    Are you an AWS Data Engineer looking for an exciting new opportunity? Look no further than Torry Harris! We are a leading digital transformation company, and we are currently seeking a talented and experienced AWS Data Engineer to join our team.As an AWS Data Engineer at Torry Harris, you will be responsible for designing, implementing, and maintaining our...

  • Cloud Engineer

    2 weeks ago

    Bengaluru, Karnataka, India AWS India - Karnataka Full time

    About the RoleWe are seeking a highly skilled Cloud Support Engineer to join our team at Amazon Web Services (AWS) India - Karnataka. As a key member of our support organization, you will be responsible for providing exceptional customer service and technical expertise to our global customers.Key ResponsibilitiesLeverage your extensive experience in network,...

  • AWS Data Engineer

    2 weeks ago

    Bengaluru, India Spruce IT Pvt. Ltd. Full time

    Job Title : AWS Data Engineer. Company : Spruce InfoTech, Inc. Job Type : Contract-to-Hire (C2H) with MNC. Location : Bangalore. Experience : 5+ years. Job Description : Spruce InfoTech, Inc. is hiring a skilled AWS Data Engineer to join a leading MNC on a Contract-to-Hire basis. The ideal candidate will bring robust experience in cloud data engineering,...

  • AWS Data Engineer

    5 hours ago

    Bengaluru, India Bhavitha Tech, CMMi Level 3 Company Full time

    Company Description Bhavitha Tech, a CMMi Level 3 Company located in Bengaluru, specializes in digital transformation services including Application Modernization, Custom Product Development, AWS Cloud Consulting, Industrial 4.0 IoT, Consumer IoT, and ERP software implementation & support. With a diverse client portfolio ranging from enterprise companies...

  • Data Engineer, Infra-Finance Business intelligence

    5 months ago

    Bengaluru, India AWS India - Karnataka Full time

    Do you have a desire to make a major contribution to the future in the rapid growth environment of Cloud Computing? Are you passionate about data and code? Does the prospect of dealing with massive volumes of data excite you? Do you want to build data engineering solutions that process millions of records? Amazon Web Services is looking for a Data Engineer...

  • AWS Support Manager

    3 weeks ago

    Bengaluru, Karnataka, India AWS India - Karnataka Full time

    AWS Support Manager Job DescriptionDrive technical excellenceAWS India Support is seeking an experienced AWS Support Manager to lead a team of 15-20 Support Engineers in India. As a Support Manager, you will be responsible for managing day-to-day operations, ensuring SLA commitments are met, and driving personnel management to ensure engineers have what they...

  • AWS Data Engineer

    2 weeks ago

    Bengaluru, India Tata Consultancy Services Full time

    Experience: 4+ YearsLocation: bangalore/hyderabadRole: AWS Data EngineerJob Description:Good hands on experience in Python programming and Pyspark and snowflake· Data Engineering experience using AWS core services (Lambda, Glue, EMR and RedShift) · Required skill set- SQL, Airflow· Have Experience with snowflake or Hive or AWS S3· Responsibility – Will...

  • Aws data engineer

    2 weeks ago

    Bengaluru, India Tata Consultancy Services Full time

    Experience: 4+ YearsLocation: bangalore/hyderabadRole: AWS Data EngineerJob Description:Good hands on experience in Python programming and Pyspark and snowflake· Data Engineering experience using AWS core services (Lambda, Glue, EMR and Red Shift) · Required skill set- SQL, Airflow· Have Experience with snowflake or Hive or AWS S3· Responsibility –...

  • Aws data engineer

    2 weeks ago

    Bengaluru, India Deloitte Full time

    We are looking for Candidates who are having experience in AWS Data Engineer.Experience Required: 5+ years.Location: Bengaluru, Karnataka, IndiaKey Responsibilities:Build & manage ETL pipelines using Python , Py Spark & AWS Glue.Process and store data on AWS S3 & optimize queries in Redshift.Develop serverless solutions with AWS Lambda & design...

  • AWS Data Engineer

    2 weeks ago

    Bengaluru, India Deloitte Full time

    We are looking for Candidates who are having experience in AWS Data Engineer .Experience Required: 5+ years.Location: Bengaluru, Karnataka, IndiaKey Responsibilities:Build & manage ETL pipelines using Python , PySpark & AWS Glue .Process and store data on AWS S3 & optimize queries in Redshift .Develop serverless solutions with AWS Lambda & design...

  • AWS Data Engineer

    5 hours ago

    Bengaluru, India Tata Consultancy Services Full time

    Dear Candidate, Greetings from TATA CONSULTANCY SERVICES LIMITED!!! Skill: AWS Data Engineer + Python & Scala Location: PAN INDIA Experience: 4 to 8 years Industry experience in Data Engineering on AWS cloud with glue, redshift , Athena experience. Ability to write high quality, maintainable, and robust code, often in SQL, Scala and Python. 3+ Years of...

  • AWS Data Engineer

    2 weeks ago

    Bengaluru, India Tata Consultancy Services Full time

    We are looking forAWS Data EngineerAWS Data EngineerExperience Range- 5 to 10years.Work Location -Kolkata/Bangalore/Pune/Bhubaneswar /DelhiMode of Interview: Online InterviewInterview Date: 27th Nov 24 (Wednesday)Interview Time: 10 am to 5pmMandatory Skills: AWS Data engineering, Python, Concepts and implementation experience of API, Problem-Solving and...

  • AWS Data Engineer

    4 weeks ago

    Bengaluru, India NEC Software Solutions (India) Full time

    Job DescriptionWhat you’ll be doing: Building Scalable Data Pipelines: Designing and developing high-quality, scalable ETL pipelines for processing big data using AWS analytical services, leveraging no-code tools, and reusable Python libraries to ensure efficiency and reusability.Collaborating & Aligning with Project Goals: Working closely with...

  • AWS Data Engineer

    2 weeks ago

    Bengaluru, India Tata Consultancy Services Full time

    Experience: 4+ YearsLocation: bangalore/hyderabadRole: AWS Data EngineerJob Description:Good hands on experience in Python programming and Pyspark and snowflake· Data Engineering experience using AWS core services (Lambda, Glue, EMR and RedShift) · Required skill set- SQL, Airflow· Have Experience with snowflake or Hive or AWS S3· Responsibility – Will...

  • AWS Data Engineer

    2 weeks ago

    Bengaluru, India Tata Consultancy Services Full time

    Experience: 4+ Years Location: bangalore/hyderabad Role: AWS Data Engineer Job Description: Good hands on experience in Python programming and Pyspark and snowflake · Data Engineering experience using AWS core services (Lambda, Glue, EMR and RedShift) · Required skill set- SQL, Airflow · Have Experience with snowflake or Hive or AWS S3 · Responsibility...

  • AWS Data Engineer

    2 weeks ago

    Bengaluru, India Tata Consultancy Services Full time

    We are looking for AWS Data EngineerAWS Data EngineerExperience Range- 5 to 10years.Work Location - Kolkata/Bangalore/Pune/Bhubaneswar /DelhiMode of Interview: Online InterviewInterview Date: 27th Nov 24 (Wednesday)Interview Time: 10 am to 5pmMandatory Skills: AWS Data engineering, Python, Concepts and implementation experience of API, Problem-Solving and...

  • Aws data engineer

    2 weeks ago

    Bengaluru, India Tata Consultancy Services Full time

    We are looking for AWS Data EngineerAWS Data EngineerExperience Range- 5 to 10years.Work Location - Kolkata/Bangalore/Pune/Bhubaneswar /DelhiMode of Interview: Online InterviewInterview Date: 27th Nov 24 (Wednesday)Interview Time: 10 am to 5pmMandatory Skills: AWS Data engineering, Python, Concepts and implementation experience of API, Problem-Solving and...

  • AWS Data Engineer

    2 weeks ago

    Bengaluru, India Deloitte Full time

    We are looking for Candidates who are having experience in AWS Data Engineer .Experience Required: 5+ years.Location: Bengaluru, Karnataka, IndiaKey Responsibilities:Build & manage ETL pipelines usingPython,PySpark&AWS Glue .Process and store data onAWS S3& optimize queries inRedshift .Develop serverless solutions withAWS Lambda& design workflows withStep...

  • Aws Solutions Architect

    3 months ago

    Bengaluru, Karnataka, India AWS India - Karnataka Full time

    8+ years of specific technology domain areas (e.g. software development, cloud computing, systems engineering, infrastructure, security, networking, data & analytics) experience - 10+ years of IT development or implementation/consulting in the software or Internet industries experience Imagine being a part of a team where your ideas can help reshape the...