AWS Data Engineer

3 weeks ago

Bengaluru, India Synechron Full time

Job Title: Senior Data Engineer - AWS, Python, PySpark, Hive, SQL

Location: Pune/Bangalore

Experience Required: 7+ years

Role Summary: As a Senior Data Engineer, you will lead the design and implementation of scalable, high-performance data solutions on AWS. You will leverage your deep expertise in Python, PySpark, Hive, and SQL to build and optimize data pipelines, support data modeling, and contribute to the architecture that supports big data processing and analytics.

Key Responsibilities:

  • Design and develop robust, scalable data pipelines using AWS services, Python, PySpark, Hive, and SQL.
  • Optimize data processing and storage for performance and cost.
  • Implement data security and compliance best practices in collaboration with the IT security team.
  • Lead the migration of legacy data warehouses and data marts to AWS cloud-based data lake and data warehouse solutions.
  • Work closely with data scientists and analysts to support their data needs and assist in building complex queries and data analysis pipelines.
  • Create and maintain comprehensive documentation of data pipelines, architectures, and workflows.
  • Continuously evaluate new technologies and AWS services to enhance data capabilities and performance.
  • Collaborate with cross-functional teams to understand their data needs and translate them into technical requirements.
  • Participate in code reviews and ensure that all solutions are aligned to pre-defined architectural specifications.
  • Provide mentoring and guidance to junior data engineers and team members.

Day-to-Day Activities:

  • Engage in daily stand-up meetings to discuss progress, obstacles, and timelines.
  • Design, code, test, and deploy new data pipelines and enhancements to existing pipelines.
  • Perform data modeling and ETL development as required for ongoing projects.
  • Monitor AWS resources and optimize the cost and performance of data applications.
  • Conduct data analysis and troubleshooting on complex data processing issues.
  • Collaborate with BI and analytics teams to assist in data-related technical issues and support their data infrastructure needs.
  • Lead performance tuning activities on SQL queries and PySpark jobs.
  • Ensure the integrity and availability of data by implementing proper backup and disaster recovery strategies.
  • Remain up-to-date with industry trends and advancements in big data and cloud computing.

Qualifications:

  • Bachelor's or Master’s degree in Computer Science, Engineering, or a related technical field.
  • 7+ years of experience in a data engineering role with a strong focus on AWS, Python, PySpark, Hive, and SQL.
  • Proven experience in designing and delivering large-scale data warehousing and data processing solutions.
  • Strong understanding of AWS services such as S3, EC2, EMR, RDS, Redshift, Glue, Lambda, and Athena.
  • Deep knowledge of big data technologies and ETL tools.
  • Excellent problem-solving, analytical, and communication skills.
  • Ability to work independently as well as part of a team in a fast-paced environment.

Preferred Qualifications:

  • AWS certifications related to data engineering or architecture.
  • Experience with additional big data technologies like Kafka, Spark Streaming, or Hadoop.
  • Familiarity with machine learning concepts and tools.

  • AWS Data Engineer

    3 months ago

    Bengaluru, India Torry Harris Full time

    Are you an AWS Data Engineer looking for an exciting new opportunity? Look no further than Torry Harris! We are a leading digital transformation company, and we are currently seeking a talented and experienced AWS Data Engineer to join our team.As an AWS Data Engineer at Torry Harris, you will be responsible for designing, implementing, and maintaining our...

  • Sr Data Scientist, AWS Support Strategy Analytics and Data Science, Support Engineering

    3 months ago

    Bengaluru, India AWS India - Karnataka - D11 Full time

    AWS Sales, Marketing, and Global Services (SMGS) is responsible for driving revenue, adoption, and growth from the largest and fastest growing small- and mid-market accounts to enterprise-level customers including public sector. The AWS Global Support team interacts with leading companies and believes that world-class support is critical to customer success....

  • AWS Data Engineer

    3 weeks ago

    Bengaluru, India Synechron Full time

    Job Title: Senior Data Engineer - AWS, Python, PySpark, Hive, SQLLocation: Pune/BangaloreExperience Required: 7+ yearsRole Summary: As a Senior Data Engineer, you will lead the design and implementation of scalable, high-performance data solutions on AWS. You will leverage your deep expertise in Python, PySpark, Hive, and SQL to build and optimize data...

  • AWS Data Engineer

    6 days ago

    Bengaluru, India Synechron Full time

    Job Title: Senior Data Engineer - AWS, Python, PySpark, Hive, SQL Location: Pune/Bangalore Experience Required: 7+ years Role Summary: As a Senior Data Engineer, you will lead the design and implementation of scalable, high-performance data solutions on AWS. You will leverage your deep expertise in Python, PySpark, Hive, and SQL to build and optimize...

  • Data Engineer

    3 weeks ago

    Bengaluru, India Acumant Full time

    Company DescriptionAcumant is a IT Services and Solutions company specializing in innovative and scalable data solutions and business applications. The company is dedicated to integrating sustainability, data, and AI solutions to empower organizations with cutting-edge technology. Acumant's focus is on driving businesses forward with tailored and...

  • Data Engineer, Infra-Finance Business intelligence

    2 months ago

    Bengaluru, India AWS India - Karnataka Full time

    Do you have a desire to make a major contribution to the future in the rapid growth environment of Cloud Computing? Are you passionate about data and code? Does the prospect of dealing with massive volumes of data excite you? Do you want to build data engineering solutions that process millions of records? Amazon Web Services is looking for a Data Engineer...

  • AWS Data Engineer

    5 days ago

    Bengaluru, India Genpact Full time

    Genpact (NYSE: G) is a global professional services and solutions firm delivering outcomes that shape the future. Our 125,000+ people across 30+ countries are driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients. Powered by our purpose – the relentless pursuit of a world that works better for people – we...

  • Data Engineer

    3 weeks ago

    Bengaluru, India Acumant Full time

    Company Description Acumant is a IT Services and Solutions company specializing in innovative and scalable data solutions and business applications. The company is dedicated to integrating sustainability, data, and AI solutions to empower organizations with cutting-edge technology. Acumant's focus is on driving businesses forward with tailored and...

  • Data Engineer

    3 weeks ago

    Bengaluru, India Acumant Full time

    Company DescriptionAcumant is a IT Services and Solutions company specializing in innovative and scalable data solutions and business applications. The company is dedicated to integrating sustainability, data, and AI solutions to empower organizations with cutting-edge technology. Acumant's focus is on driving businesses forward with tailored and...

  • Data Engineer

    3 weeks ago

    Bengaluru, India Acumant Full time

    Company DescriptionAcumant is a IT Services and Solutions company specializing in innovative and scalable data solutions and business applications. The company is dedicated to integrating sustainability, data, and AI solutions to empower organizations with cutting-edge technology. Acumant's focus is on driving businesses forward with tailored and...

  • AWS Data Engineer

    3 days ago

    Bengaluru, India NAM Info Inc Full time

    Job Title: AWS Data Engineer with PySparkLocation: Bangalore, ChennaiFulltimeJob Description:Preferred Skills and Qualifications:Proficient in AWS Cloud services and infrastructure.Strong experience with Databricks and PySpark.Extensive knowledge of data gathering ingestion, manipulation, orchestration, and optimization.Proven track record of leading...

  • AWS Data Engineer

    3 months ago

    Bengaluru, India EDGESOFT Full time

    Job Description :- AWS Data Engineer with min of 5 to 7 years of experience.- Collaborate with business analysts to understand and gather requirements for existing or new ETL pipelines.- Connect with stakeholders daily to discuss project progress and updates.- Work within an Agile process to deliver projects in a timely and efficient manner.- Design and...

  • Data Engineer

    2 months ago

    Bengaluru, India Cayian India Pvt Limited Full time

    Cayian is a global IT and business service provider that delivers consultancy, capability, and system integration services to our partners. Our unique partner model is optimised to accelerate delivery, continuously improve quality, and drive business growth. Find out more about us here.  The role As a Data Engineer, you will be a part of...

  • AWS Data Engineer

    1 week ago

    Bengaluru, India Genpact Full time

    Genpact (NYSE: G) is a global professional services and solutions firm delivering outcomes that shape the future. Our 125,000+ people across 30+ countries are driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients. Powered by our purpose – the relentless pursuit of a world that works better for people – we...

  • AWS Data Engineer

    6 days ago

    Bengaluru, India Genpact Full time

    Genpact (NYSE: G) is a global professional services and solutions firm delivering outcomes that shape the future. Our 125,000+ people across 30+ countries are driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients. Powered by our purpose – the relentless pursuit of a world that works better for people – we...

  • Data Engineer

    3 months ago

    Bengaluru, India Spectrum Consulting Full time

    Position : Data Infrastructure EngineerCore work that needs accomplished :Architect, design and build Data access solution using various tool sets on Client's AWS and other platforms like DBT, Airflow, Atlan to enable clients data governance goalsThe solution will enable managing access to enterprise data at various layers of clients data solutions and...

  • Aws Cloud Architect

    2 months ago

    Bengaluru, India NTT DATA Full time

    **Req ID**: 248981 We are currently seeking a AWS Cloud Architect to join our team in Bangalore, Karnātaka (IN-KA), India (IN). Background As an Amazon Web Services (AWS) cloud architect, you will be responsible to architect and manage the cloud environment for a Digital Banking Asset. Duties include architecting and implementing cloud solutions,...

  • AWS Data Engineer

    3 weeks ago

    Bengaluru, India Birlasoft Full time

    About Birlasoft:Birlasoft, a powerhouse where domain expertise, enterprise solutions, and digital technologies converge to redefine business processes. We take pride in our consultative and design thinking approach, driving societal progress by enabling our customers to run businesses with unmatched efficiency and innovation. As part of the CK Birla Group, a...

  • AWS Data Engineer

    3 weeks ago

    Bengaluru, India Birlasoft Full time

    About Birlasoft: Birlasoft, a powerhouse where domain expertise, enterprise solutions, and digital technologies converge to redefine business processes. We take pride in our consultative and design thinking approach, driving societal progress by enabling our customers to run businesses with unmatched efficiency and innovation. As part of the CK Birla...

  • AWS Data Engineer

    3 weeks ago

    Bengaluru, India Birlasoft Full time

    About Birlasoft:Birlasoft, a powerhouse where domain expertise, enterprise solutions, and digital technologies converge to redefine business processes. We take pride in our consultative and design thinking approach, driving societal progress by enabling our customers to run businesses with unmatched efficiency and innovation. As part of the CK Birla Group, a...