Thoucentric - AWS Data Engineer - PySpark/SQL/Apache Airflow

2 weeks ago

Bengaluru, India Thoucentric Technology Pvt. Ltd. Full time

About Us :


Thoucentric is the Consulting arm of Xoriant, a prominent digital engineering services company with 5000+ employees. We are headquartered in Bangalore with presence across multiple locations in India, US, UK, Singapore & Australia Globally.


As the Consulting business of Xoriant, We help clients with Business Consulting, Program & Project Management, Digital Transformation, Product Management, Process & Technology Solutioning and Execution including Analytics & Emerging Tech areas cutting across functional areas such as Supply Chain, Finance & HR, Sales & Distribution across US, UK, Singapore and Australia. Our unique consulting framework allows us to focus on execution rather than pure advisory. We are working closely with marquee names in the global consumer & packaged goods (CPG) industry, new age tech and start-up ecosystem. Xoriant (Parent entity) started in 1990 and is a Sunnyvale, CA headquartered digital engineering firm with offices in the USA, Europe, and Asia. Xoriant is backed by ChrysCapital, a leading private equity firm. Our strengths are now combined with Xoriants capabilities in AI & Data, cloud, security and operations services proven for 30 years.


We have been certified as "Great Place to Work" by AIM and have been ranked as "50 Best Firms for Data Scientists to Work For".


We have an experienced consulting team of over 450+ world-class business and technology consultants based across six global locations, supporting clients through their expert insights, entrepreneurial approach and focus on delivery excellence. We have also built point solutions and products through Thoucentric labs using AI/ML in the supply chain space.


Job Description :


Position Overview :


We are looking for an experienced AWS Data Engineer to design, develop, and maintain data solutions using core AWS services. The ideal candidate will have hands-on experience with tools like Amazon S3, Redshift, AWS Glue, and DynamoDB, and will build scalable, efficient, and secure data pipelines and architectures. The role also requires strong expertise in PySpark, SQL, and workflow orchestration tools like Airflow.


Key Responsibilities :


1. Data Pipeline Development :


- Develop and manage ETL/ELT workflows using AWS Glue and PySpark to process large datasets.


- Automate data workflows using Apache Airflow and other orchestration tools.


2. Data Storage and Management :


- Architect and manage data storage in Amazon S3 , ensuring performance, cost-efficiency, and security.


- Create and optimize Amazon Redshift clusters for data warehousing and analytics workloads.


- Design scalable NoSQL solutions using Amazon DynamoDB for real-time data needs.


3. Compute and Serverless :


- Build and deploy serverless solutions using AWS Lambda for event-driven data processing.


- Configure and manage virtual machine instances using Amazon EC2 for custom data processing tasks.


4. Security and Monitoring :


- Implement fine-grained access controls with AWS IAM to ensure data security.


- Set up monitoring, logging, and alerts using AWS CloudWatch for proactive system health management.


5. Data Integration and Transformation :


- Create efficient SQL queries to handle data transformations and analytics.


- Integrate and process structured and unstructured data from multiple sources.


- Design data models and implement them in Redshift or DynamoDB for optimized query performance.


6. Collaboration and Optimization :


- Collaborate with data scientists, analysts, and stakeholders to gather requirements and deliver solutions.


- Continuously optimize data processing workflows to improve performance and reduce costs.


Required Skills and Qualifications :


- Core AWS Expertise


- Strong knowledge of Amazon S3 , Redshift , AWS Glue , Lambda , DynamoDB , EC2 , IAM , and CloudWatch .


Technical Proficiency :


- Hands-on experience with PySpark for big data processing.


- Proficient in writing complex SQL queries for data manipulation and analysis.


- Expertise in workflow orchestration using Apache Airflow or similar tools.


Experience :


- 4+ years in data engineering with a focus on AWS technologies.


- Experience in designing scalable, fault-tolerant data pipelines.


- Familiarity with data modeling, ETL/ELT, and data warehousing principles.


Soft Skills :


- Strong problem-solving and analytical skills.


- Excellent communication and collaboration abilities.


- Ability to manage priorities in a fast-paced environment


Note : For your candidature to be considered on this job, you need to apply necessarily on the company's redirected page of this job. Please make sure you apply on the redirected page as well.

(ref:hirist.tech)

  • Thoucentric - AWS Data Engineer - ETL/PySpark

    6 days ago

    Bengaluru, India Thoucentric Technology Pvt. Ltd. Full time

    We are looking for an experienced AWS Data Engineer to design, develop, and maintain data solutions using core AWS services. The ideal candidate will have hands-on experience with tools like Amazon S3, Redshift, AWS Glue, and DynamoDB, and will build scalable, efficient, and secure data pipelines and architectures. The role also requires strong expertise in...

  • Data Engineer Python PySpark Apache Airflow NoSQL

    3 months ago

    Bengaluru, India ConsultBae India Private limited Full time

    Data Engineer(Python PySpark Apache Airflow NoSQL)Location: Bengaluru Karnataka India (Onsite)Experience: 35yearsResponsibilities:Build optimize and maintain scalable ETL pipelines for dataingestion and processing. Develop and manageworkflows using Apache Airflow for scheduling and orchestratingtasks. Work with distributed computingtechnologies (PySpark) to...

  • AWS Data Engineer

    3 weeks ago

    Bengaluru, India Thoucentric Full time

    Job DescriptionPosition Overview     We are looking for an experienced   AWS Data Engineer   to design, develop, and maintain data solutions using core AWS services. The ideal candidate will have hands-on experience with tools like Amazon S3, Redshift, AWS Glue, and DynamoDB, and will build scalable, efficient, and secure data pipelines and...

  • AWS Data Engineer

    3 weeks ago

    Bengaluru, India Thoucentric Full time

    Position Overview     We are looking for an experienced   AWS Data Engineer   to design, develop, and maintain data solutions using core AWS services. The ideal candidate will have hands-on experience with tools like Amazon S3, Redshift, AWS Glue, and DynamoDB, and will build scalable, efficient, and secure data pipelines and architectures. The role...

  • Highly Skilled AWS Data Architect

    3 weeks ago

    Bengaluru, Karnataka, India Thoucentric Full time

    Job Title: Highly Skilled AWS Data Architect">About Thoucentric: We are a forward-thinking company dedicated to leveraging cutting-edge technologies to drive innovation and growth.">Salary: $150,000 - $200,000 per year, depending on experience.">Job Description:">Position Overview: We are seeking an experienced AWS Data Architect to design, develop, and...

  • AWS Data Solutions Architect

    3 weeks ago

    Bengaluru, Karnataka, India Thoucentric Full time

    Job Title: AWS Data Solutions ArchitectWe are seeking an experienced AWS Data Solutions Architect to join our team at Thoucentric.About the Role:The ideal candidate will have hands-on experience with designing, developing, and maintaining data solutions using core AWS services. They will work closely with data scientists, analysts, and stakeholders to gather...

  • Data Engineer

    2 weeks ago

    Bengaluru, India Coders Brain Technology Private Limited Full time

    Job Description : Data Engineer- Role : Data Engineer- Work Location : Bangalore (Work From Office)- Experience : 3+ yearsJob Overview :We are looking for a proficient Data Engineer with expertise in Amazon Redshift, Python, Apache Airflow, dbt (Data Build Tool), API integration, and AWS. This role will be responsible for developing and maintaining scalable...

  • Thoucentric - Data Engineer - SQL/Python

    4 weeks ago

    Bengaluru, India Thoucentric Technology Pvt. Ltd. Full time

    About Us :Thoucentric is the Consulting arm of Xoriant, a prominent digital engineering services company with 5000+ employees. We are headquartered in Bangalore with presence across multiple locations in India, US, UK, Singapore & Australia Globally.As the Consulting business of Xoriant, We help clients with Business Consulting, Program & Project Management,...

  • Delta Lake Architect

    6 months ago

    Bengaluru, Karnataka, India Thoucentric Full time

    **About us**: **Thoucentric is the Consulting arm of Xoriant**, a prominent digital engineering services company with 5000+ employees. We are headquartered in Bangalore with presence across multiple locations in India, US, UK, Singapore & Australia Globally. As the Consulting business of **Xoriant**, We help clients with Business Consulting, Program &...

  • ▷ 15h Left: AWS Data Engineer

    1 month ago

    Bengaluru, India Tata Consultancy Services Full time

    AWS Data Engineer:Required Technical Skill Set: AWS Redshift, Glue, PySparkLocation of Requirement: Chennai/Mumbai/Bangalore/Pune/Indore/HyderabadDesired Experience Range: 4-10 YearsMust-Have:- Strong hands-on experience in Python programming and PySpark- Experience using AWS services (RedShift, Glue, EMR, S3 & Lambda)- Experience working with Apache Spark...

  • AWS Data Engineer

    1 month ago

    Bengaluru, India Tata Consultancy Services Full time

    AWS Data Engineer:Required Technical Skill Set: AWS Redshift, Glue, PySparkLocation of Requirement: Chennai/Mumbai/Bangalore/Pune/Indore/HyderabadDesired Experience Range: 4-10 YearsMust-Have:Strong hands-on experience in Python programming and PySparkExperience using AWS services (RedShift, Glue, EMR, S3 & Lambda)Experience working with Apache Spark and...

  • Senior Data Engineer

    4 weeks ago

    Bengaluru, India Cyber Sphere LLC Full time

    Job Description :Required Minimum Qualifications :- Bachelor's degree in computer science, CIS, or related field- 5-7 years of IT experience in software engineering or related field- Experience on project(s) involving the implementation of software development life cycles (SDLC)Primary Skills : PySpark, SQL, GCP EcoSystem(Biq Query, Cloud Composer,...

  • Thoucentric - Data Scientist - NLP

    6 days ago

    Bengaluru, India Thoucentric Technology Pvt. Ltd. Full time

    Job Description :At Thoucentric, we work on various problem statements.The most popular ones are :- Building capabilities that address a market need, basis our ongoing research efforts- Solving a specific use case for a current or potential client based on challenges on-ground- Developing new systems that help be a better employer and a better partner to...

  • Thoucentric - Data Scientist - NLP

    6 days ago

    Bengaluru, India Thoucentric Technology Private Limited Full time

    Thoucentric, we work on various problem statements.The most popular ones are :- Building capabilities that address a market need, basis our ongoing research efforts- Solving a specific use case for a current or potential client based on challenges on-ground- Developing new systems that help be a better employer and a better partner to clients- All of these...

  • [15h Left] AWS Data Engineer

    7 days ago

    Bengaluru, India NeerInfo Solutions Full time

    Hiring for Fortune IT CompanyRole: AWS Data EngineerExperience: 4 to 8 YearsLocation: Pune / Bengaluru / ChennaiNotice Period: Immediate or currently servingJD:- Strong hands-on experience in Python programming and PySpark- Experience using AWS services (RedShift, Glue, EMR, S3 & Lambda)- Experience working with Apache Spark and Hadoop ecosystem.- Experience...

  • AWS Data Engineer

    1 week ago

    Bengaluru, India NeerInfo Solutions Full time

    Hiring for Fortune IT CompanyRole: AWS Data EngineerExperience: 4 to 8 YearsLocation: Pune / Bengaluru / ChennaiNotice Period: Immediate or currently servingJD:Strong hands-on experience in Python programming and PySparkExperience using AWS services (RedShift, Glue, EMR, S3 & Lambda)Experience working with Apache Spark and Hadoop ecosystem.Experience in...

  • AWS Data Engineer

    1 week ago

    Bengaluru, India NeerInfo Solutions Full time

    Hiring for Fortune IT Company Role: AWS Data Engineer Experience: 4 to 8 Years Location: Pune / Bengaluru / Chennai Notice Period: Immediate or currently serving JD: Strong hands-on experience in Python programming and PySpark Experience using AWS services (RedShift, Glue, EMR, S3 & Lambda) Experience working with Apache Spark and Hadoop ecosystem....

  • Principal Data Engineer

    4 months ago

    Bengaluru, India gethyr Full time

    Responsibilities : Data Pipeline Development : Design, implement, and maintain scalable data pipelines using PySpark to process and transform large datasets efficiently.Workflow Orchestration : Develop, schedule, and monitor complex data workflows and ETL processes using Apache Airflow.Data Management : Manage and optimize data storage solutions, ensuring...

  • Data Engineer

    1 month ago

    Bengaluru, India Thoucentric Full time

    Job DescriptionJob Overview: We are looking for an experienced Advanced Analytics Consultant to join our analytics team. In this role, you willanalyse and interpret complex datasets to support digital support platforms, payments, and risk platformperformance. The ideal candidate will possess a strong foundation in SQL, Python, and Tableau, with expertise...

  • AWS Data Engineer

    3 days ago

    Bengaluru, India Neerinfo Solutions Full time

    Key Responsibilities :AWS Data Infrastructure Development :- Design, develop, and maintain AWS-based data pipelines using services such as AWS Glue, AWS Redshift, Amazon S3, and AWS Lambda.- Build ETL (Extract, Transform, Load) processes, integrating batch and near real-time data from various sources to Amazon Redshift, S3, or other cloud-based storage...