Python Pyspark Aws Dbt
6 days ago
Job Summary:
Design and implement ETL workflows using AWS Glue, Python, and PySpark.
Develop and optimize queries using Amazon Athena and Redshift.
Build scalable data pipelines to ingest, transform, and load data from various sources.
Ensure data quality, integrity, and security across AWS services.
Collaborate with data analysts, data scientists, and business stakeholders to deliver data solutions.
Monitor and troubleshoot ETL jobs and cloud infrastructure performance.
Automate data workflows and integrate with CI/CD pipelines.
Required Skills & Qualifications:
Hands-on experience with AWS Glue, Athena, and Redshift.
Strong programming skills in Python and PySpark.
Experience with ETL design, implementation, and optimization.
Familiarity with S3, Lambda, CloudWatch, and other AWS services.
Understanding of data warehousing concepts and performance tuning in Redshift.
Experience with schema design, partitioning, and query optimization in Athena.
Proficiency in version control (Git) and agile development practices.
** About Virtusa**
Teamwork, quality of life, professional and personal development: values that Virtusa is proud to embody. When you join us, you join a team of 27,000 people globally that cares about your growth — one that seeks to provide you with exciting projects, opportunities and work with state of the art technologies throughout your career with us.
Great minds, great potential: it all comes together at Virtusa. We value collaboration and the team environment of our company, and seek to provide great minds with a dynamic place to nurture new ideas and foster excellence.
Virtusa was founded on principles of equal opportunity for all, and so does not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, non-disqualifying physical or mental disability, national origin, veteran status or any other basis covered by appropriate law. All employment is decided on the basis of qualifications, merit, and business need.
-
Bigdata Pyspark
1 week ago
Andhra Pradesh, India Virtusa Full timeOverall 10+ years of experience in Datawarehouse and Hadoop platform. MUST have experience with Python/PySpark and Hive in Big Data environments Should have strong skills in writing complex SQL Queries and good understanding of Data warehouse concepts. exposure to migration of legacy Data warehouse platform to Hadoop platform experience will be a big...
-
Andhra Pradesh, India Growel Softech Pvt. Ltd. Full time ₹ 9,00,000 - ₹ 12,00,000 per year- 1.Hands on industry experience in design and coding from scratch in AWS Glue-Pyspark with services like S3 DynamoDB StepFunctions etc. 2.Hands on industry experience in design and coding from scratch in Snowflake 3.Experience in Pyspark/Snowflake 1 to 3 years with overall around 5 years of experience in building data/analytics solutions Level Senior...
-
AWS Data Engineer
1 week ago
Andhra Pradesh, India Inityinfotech Full time ₹ 15,00,000 - ₹ 25,00,000 per yearAWS Data EngineerExperience :- 5+ yearsLocation : RemoteJob DescriptionDesign, development, and implementation of performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR.Writing reusable, testable, and efficient codeIntegration of data storage solutions in spark – especially with AWS S3 object storage. Performance tuning of pySpark...
-
Python Pyspark with Aws
1 week ago
Andhra Pradesh, India Virtusa Full time**Responsibilities**: 1.Analyze various data sources present with the client, including structured and unstructured data formats 2.Map entities across different data sources to identify relationships and data inconsistencies 3.Analyze existing reports created with Tableau and other tools to understand data usage and identify gaps 4.Colloborate with data...
-
Aws Python
6 days ago
Andhra Pradesh, India Virtusa Full timeJD for Data Engineer Python At least 5 to 8 years of experience in AWS Python programming and who can design, build, test & deploy the code. Should have experience in using following AWS services: AWS SQS, AWS MSK, AWS RDS Aurora DB, BOTO 3. Very strong SQL knowledge is a must, should be able to understand build complex queries. He/she should be closely...
-
Python Aws
4 days ago
Andhra Pradesh, India Virtusa Full timePython AWS Job Summary We are looking for a highly motivated and skilled Back End Engineer with expertise in Python FastAPI and AWS cloud services to join our engineering team This role focuses on building and deploying robust APIs and MCP Microservice Control Plane server components leveraging modern containerization and cloud native technologies Key...
-
Aws De
6 days ago
Andhra Pradesh, India Virtusa Full timeMust Have Strong hands-on Experience in AWS Native Services Python, Pyspark Data Engineering(S3, EMR, Glue, Sqoop) Serverless Experience(Lambda, Step Functions) Strong SQL AWS Athena, Cloud Formation Templates, Crawler Strong Data analysis and troubleshooting skill Datamart / Data Warehousing Multi-Dimensional OLAP Star/Snowflake Schema Design **About...
-
Net with Aws
4 days ago
Andhra Pradesh, India Virtusa Full timeRole:.Net Developer **Skills**:Minimum 8 years of development and design experience in C# and AWS Lambda Comprehensive knowledge in C# programming language Strong knowledge of RESTful APIs, AWS Lambda, Terraform and ReactJS Hands on experience in using MongoDB Familiarity with other cloud services (e.g., S3, Athena, Cloudwatch, Cloudformation, DynamoDB),...
-
Delhi, Andhra Pradesh, India Legal operations Full time ₹ 12,00,000 - ₹ 36,00,000 per yearAt EY, we're all in to shape your future with confidence. We'll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. Job Description – Data Engineer (Snowflake, dbt, Airflow, Insurance Domain)Position: Data EngineerRank-...
-
Python, Pyspark, SQL, GCP
1 week ago
uttar pradesh, India Tata Consultancy Services Full timeTCS present an excellent opportunity for Python, Pyspark, SQL, GCPJob Location: TCS Noida YamunaExperience required : 7-12 yrsWalk in Interview Date: 08-Nov-25 (SaturdayMust-Have. ReactJS and Python f/w for Backend – preferably FAST API. Kubernetes/Docker (preferably AKS). Strong Hands-on experience . Handling large volumes of data on web pages (preferably...