Thoucentric - AWS Data Engineer - PySpark/SQL/Apache Airflow

3 weeks ago

Bengaluru, Karnataka, India Thoucentric Technology Pvt. Ltd. Full time

About Us :

Thoucentric is the Consulting arm of Xoriant, a prominent digital engineering services company with 5000+ employees. We are headquartered in Bangalore with presence across multiple locations in India, US, UK, Singapore & Australia Globally.

As the Consulting business of Xoriant, We help clients with Business Consulting, Program & Project Management, Digital Transformation, Product Management, Process & Technology Solutioning and Execution including Analytics & Emerging Tech areas cutting across functional areas such as Supply Chain, Finance & HR, Sales & Distribution across US, UK, Singapore and Australia. Our unique consulting framework allows us to focus on execution rather than pure advisory. We are working closely with marquee names in the global consumer & packaged goods (CPG) industry, new age tech and start-up ecosystem. Xoriant (Parent entity) started in 1990 and is a Sunnyvale, CA headquartered digital engineering firm with offices in the USA, Europe, and Asia. Xoriant is backed by ChrysCapital, a leading private equity firm. Our strengths are now combined with Xoriants capabilities in AI & Data, cloud, security and operations services proven for 30 years.

We have been certified as "Great Place to Work" by AIM and have been ranked as "50 Best Firms for Data Scientists to Work For".

We have an experienced consulting team of over 450+ world-class business and technology consultants based across six global locations, supporting clients through their expert insights, entrepreneurial approach and focus on delivery excellence. We have also built point solutions and products through Thoucentric labs using AI/ML in the supply chain space.

Job Description :

Position Overview :

We are looking for an experienced AWS Data Engineer to design, develop, and maintain data solutions using core AWS services. The ideal candidate will have hands-on experience with tools like Amazon S3, Redshift, AWS Glue, and DynamoDB, and will build scalable, efficient, and secure data pipelines and architectures. The role also requires strong expertise in PySpark, SQL, and workflow orchestration tools like Airflow.

Key Responsibilities :

1. Data Pipeline Development :

- Develop and manage ETL/ELT workflows using AWS Glue and PySpark to process large datasets.

- Automate data workflows using Apache Airflow and other orchestration tools.

2. Data Storage and Management :

- Architect and manage data storage in Amazon S3 , ensuring performance, cost-efficiency, and security.

- Create and optimize Amazon Redshift clusters for data warehousing and analytics workloads.

- Design scalable NoSQL solutions using Amazon DynamoDB for real-time data needs.

3. Compute and Serverless :

- Build and deploy serverless solutions using AWS Lambda for event-driven data processing.

- Configure and manage virtual machine instances using Amazon EC2 for custom data processing tasks.

4. Security and Monitoring :

- Implement fine-grained access controls with AWS IAM to ensure data security.

- Set up monitoring, logging, and alerts using AWS CloudWatch for proactive system health management.

5. Data Integration and Transformation :

- Create efficient SQL queries to handle data transformations and analytics.

- Integrate and process structured and unstructured data from multiple sources.

- Design data models and implement them in Redshift or DynamoDB for optimized query performance.

6. Collaboration and Optimization :

- Collaborate with data scientists, analysts, and stakeholders to gather requirements and deliver solutions.

- Continuously optimize data processing workflows to improve performance and reduce costs.

Required Skills and Qualifications :

- Core AWS Expertise

- Strong knowledge of Amazon S3 , Redshift , AWS Glue , Lambda , DynamoDB , EC2 , IAM , and CloudWatch .

Technical Proficiency :

- Hands-on experience with PySpark for big data processing.

- Proficient in writing complex SQL queries for data manipulation and analysis.

- Expertise in workflow orchestration using Apache Airflow or similar tools.

Experience :

- 4+ years in data engineering with a focus on AWS technologies.

- Experience in designing scalable, fault-tolerant data pipelines.

- Familiarity with data modeling, ETL/ELT, and data warehousing principles.

Soft Skills :

- Strong problem-solving and analytical skills.

- Excellent communication and collaboration abilities.

- Ability to manage priorities in a fast-paced environment

Note : For your candidature to be considered on this job, you need to apply necessarily on the company's redirected page of this job. Please make sure you apply on the redirected page as well.

(ref:hirist.tech)

  • AWS Big data pyspark SQL Spark Airflow

    6 days ago

    Bengaluru, Karnataka, India ScaleneWorks Full time

    Job Title: AWS Big Data Engineer About the Role: We are looking for a highly skilled AWS Big Data Engineer with extensive experience in PySpark, SQL, Spark, and Airflow. The ideal candidate will have a strong background in coding and a deep understanding of big data technologies. You will be responsible for designing, developing, and maintaining big data...

  • Expert in Apache Airflow and AWS

    2 days ago

    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    About the Role:We are looking for an Expert in Apache Airflow and AWS to lead our team in delivering innovative solutions using these technologies.The successful candidate will have extensive experience in designing and implementing data processing pipelines using Apache Beam and Apache Airflow.Apart from technical expertise, the candidate should possess...

  • Thoucentric - Data Engineer - SQL/Python

    3 weeks ago

    Bengaluru, Karnataka, India Thoucentric Technology Pvt. Ltd. Full time

    About Us : Thoucentric is the Consulting arm of Xoriant, a prominent digital engineering services company with 5000+ employees. We are headquartered in Bangalore with presence across multiple locations in India, US, UK, Singapore & Australia Globally.As the Consulting business of Xoriant, We help clients with Business Consulting, Program & Project...

  • Thoucentric - Data Engineer - SQL/Python

    2 weeks ago

    Bengaluru, Karnataka, India Thoucentric Technology Pvt. Ltd. Full time

    About Us : Thoucentric is the Consulting arm of Xoriant, a prominent digital engineering services company with 5000+ employees. We are headquartered in Bangalore with presence across multiple locations in India, US, UK, Singapore & Australia Globally.As the Consulting business of Xoriant, We help clients with Business Consulting, Program & Project...

  • Thoucentric - Data Scientist - NLP

    4 weeks ago

    Bengaluru, Karnataka, India Thoucentric Technology Pvt. Ltd. Full time

    Job Description :At Thoucentric, we work on various problem statements.The most popular ones are :- Building capabilities that address a market need, basis our ongoing research efforts- Solving a specific use case for a current or potential client based on challenges on-ground- Developing new systems that help be a better employer and a better partner to...

  • AWS Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Role- AWS Data EngineerExperience- 4 to 8 YrsRequired Technical Skill Set - Python,Pyspark, Hive, Snowflake,AWS (S3,Lambda, Glue, EMR and RedShift)Must-Have· Good hands on experience in Python programming and Pyspark· Data Engineering experience using AWS core services (Lambda, Glue, EMR and RedShift)· Required skill set- SQL, Airflow ·Must – Have...

  • Senior Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Infometry Full time

    Job Summary : We are seeking an experienced Senior Data Engineer with strong expertise in Databricks, Apache Airflow, SQL, and Python to design and optimize data pipelines, ETL workflows, and cloud-based data solutions. The ideal candidate will have deep experience in big data processing, cloud-based analytics, and automation to drive efficient and scalable...

  • Senior Data Engineer

    1 week ago

    Bengaluru, Karnataka, India Infometry Full time

    Job Summary : We are seeking an experienced Senior Data Engineer with strong expertise in Databricks, Apache Airflow, SQL, and Python to design and optimize data pipelines, ETL workflows, and cloud-based data solutions. The ideal candidate will have deep experience in big data processing, cloud-based analytics, and automation to drive efficient and scalable...

  • AWS Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India Neerinfo Solutions Full time

    Key Responsibilities :AWS Data Infrastructure Development :- Design, develop, and maintain AWS-based data pipelines using services such as AWS Glue, AWS Redshift, Amazon S3, and AWS Lambda.- Build ETL (Extract, Transform, Load) processes, integrating batch and near real-time data from various sources to Amazon Redshift, S3, or other cloud-based storage...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India O2F Info Solutions Pvt. Ltd. Full time

    Job Summary :We are seeking a highly skilled Senior Data Engineer with 4 to 8 years of experience in building robust data pipelines and working extensively with PySpark to join our data engineering team. Key Responsibilities :Data Pipeline Development : - Design, build, and maintain scalable data pipelines using PySpark to process large datasets and support...

  • Data Engineer

    1 week ago

    Bengaluru, Karnataka, India O2F Info Solutions Pvt. Ltd. Full time

    Job Summary :We are seeking a highly skilled Senior Data Engineer with 4 to 8 years of experience in building robust data pipelines and working extensively with PySpark to join our data engineering team. Key Responsibilities :Data Pipeline Development : - Design, build, and maintain scalable data pipelines using PySpark to process large datasets and support...

  • Data Engineer

    14 hours ago

    Bengaluru, Karnataka, India Tekgence Inc Full time

    Job DescriptionRole: Data EngineerLocation: BangaloreJob Summary – We are looking for Data Engineers with PySpark and AWS Glue experiences.Years of experience needed – · 5+ years of experience Technical Skills: · Data Engineers with PySpark and AWS Glue experiences.· AWS mandatory. GCP and Azure add-on· Proven experience as a Data Engineer or similar...

  • Walkin interview for Java Microservices, Apache Airflow+AWS

    3 weeks ago

    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Walkin interview for Java Microservices, Apache Airflow +AWSExperience -6to9yrsBangalore( Bhuwalka ) Location onlyJDApache Beam:-Must have- Java Microservices, Apache Airflow +AWSStrong in Core Java, Advance java (Collection API)Experience on apache beam java development at least 1-2 yearsBasic understanding in cloud dataflowGood knowledge in database...

  • Walkin interview for Java Microservices, Apache Airflow+AWS

    2 weeks ago

    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Walkin interview for Java Microservices, Apache Airflow +AWSExperience -6to9yrsBangalore( Bhuwalka ) Location onlyJDApache Beam:-Must have- Java Microservices, Apache Airflow +AWSStrong in Core Java, Advance java (Collection API)Experience on apache beam java development at least 1-2 yearsBasic understanding in cloud dataflowGood knowledge in database...

  • Walkin interview for Java Microservices, Apache Airflow+AWS

    2 days ago

    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Walkin interview for Java Microservices, Apache Airflow +AWSExperience -6to9yrsBangalore( Bhuwalka ) Location onlyJDApache Beam:-Must have- Java Microservices, Apache Airflow +AWS- Strong in Core Java, Advance java (Collection API)- Experience on apache beam java development at least 1-2 years- Basic understanding in cloud dataflow- Good knowledge in...

  • Data Engineer

    1 week ago

    Bengaluru, Karnataka, India Xcel Hire Management Solutions Full time

    Position : Data Engineer (Py-Spark)Experience : 5+ yearsLocation : Skills :- MUST HAVE: AWS, iceberg, glue, Athena, lambda, emr, SQL, Py-Spark- Python(Advance Should know about inheritance and Class)- Athena, Redshift, IAM iceberg, Apache Airflow, GitImportant :Data Engineer skills :- Data warehousing ( e.g redshift)- Fetch data from database ( PostgreSQL,...

  • Data Engineer

    7 hours ago

    Bengaluru, Karnataka, India Tekgence Inc Full time

    Job DescriptionJob DescriptionRole: Data EngineerLocation: BangaloreJob SummaryWe are looking for Data Engineers with PySpark and AWS Glue experiences.Years of experience needed5+ years of experienceTechnical Skills:Data Engineers with PySpark and AWS Glue experiences.AWS mandatory. GCP and Azure add-onProven experience as a Data Engineer or similar role in...

  • Thoucentric - Data Scientist - Python

    2 weeks ago

    Bengaluru, Karnataka, India Thoucentric Technology Pvt. Ltd. Full time

    About Us :Thoucentric is the Consulting arm of Xoriant, a prominent digital engineering services company with 5000+ employees. We are headquartered in Bangalore with presence across multiple locations in India, US, UK, Singapore & Australia Globally.As the Consulting business of Xoriant, We help clients with Business Consulting, Program & Project Management,...

  • Thoucentric - Data Scientist - Python

    1 week ago

    Bengaluru, Karnataka, India Thoucentric Technology Pvt. Ltd. Full time

    About Us :Thoucentric is the Consulting arm of Xoriant, a prominent digital engineering services company with 5000+ employees. We are headquartered in Bangalore with presence across multiple locations in India, US, UK, Singapore & Australia Globally.As the Consulting business of Xoriant, We help clients with Business Consulting, Program & Project Management,...

  • Thoucentric - Data Scientist - Python

    3 weeks ago

    Bengaluru, Karnataka, India Thoucentric Technology Pvt. Ltd. Full time

    About Us :Thoucentric is the Consulting arm of Xoriant, a prominent digital engineering services company with 5000+ employees. We are headquartered in Bangalore with presence across multiple locations in India, US, UK, Singapore & Australia Globally.As the Consulting business of Xoriant, We help clients with Business Consulting, Program & Project Management,...