AWS Big data pyspark SQL Spark Airflow
1 week ago
Job Title: AWS Big Data Engineer
About the Role:
We are looking for a highly skilled AWS Big Data Engineer with extensive experience in PySpark, SQL, Spark, and Airflow. The ideal candidate will have a strong background in coding and a deep understanding of big data technologies. You will be responsible for designing, developing, and maintaining big data solutions on AWS, ensuring data is processed efficiently and effectively.
Key Responsibilities:
Design, develop, and maintain big data solutions using AWS services.
Write and optimize complex SQL queries.
Develop and manage data pipelines using PySpark and Spark.
Implement and manage workflows using Apache Airflow.
Collaborate with data scientists and analysts to understand data requirements.
Ensure data quality and integrity throughout the data lifecycle.
Troubleshoot and resolve issues related to big data processing.
Stay updated with the latest trends and best practices in big data technologies.
Key Skills and Qualifications:
Proven experience with AWS big data services.
Strong coding experience in PySpark and SQL.
Expertise in Spark and Airflow.
Excellent problem-solving and analytical skills.
Ability to work independently and as part of a team.
Strong communication and documentation skills.
Relevant certifications (e.g., AWS Certified Big Data - Specialty) are a plus.
-
Senior Big Data Specialist
3 days ago
Bengaluru, Karnataka, India UST Full timeWe're looking for a Senior Big Data Specialist to join our team at UST. As a key member of our data engineering team, you will be responsible for designing, implementing, and optimizing big data solutions using PySpark, GCP, Spark, Hadoop, and Hive.The ideal candidate will have strong hands-on experience in PySpark & GCP and a solid understanding of big data...
-
Bengaluru, Karnataka, India Thoucentric Technology Pvt. Ltd. Full timeAbout Us : Thoucentric is the Consulting arm of Xoriant, a prominent digital engineering services company with 5000+ employees. We are headquartered in Bangalore with presence across multiple locations in India, US, UK, Singapore & Australia Globally. As the Consulting business of Xoriant, We help clients with Business Consulting, Program & Project...
-
AWS Data Engineer
4 weeks ago
Bengaluru, Karnataka, India Neerinfo Solutions Full timeKey Responsibilities :AWS Data Infrastructure Development :- Design, develop, and maintain AWS-based data pipelines using services such as AWS Glue, AWS Redshift, Amazon S3, and AWS Lambda.- Build ETL (Extract, Transform, Load) processes, integrating batch and near real-time data from various sources to Amazon Redshift, S3, or other cloud-based storage...
-
Big Data Engineer
4 hours ago
Bengaluru, Karnataka, India Scoop Technologies Pvt Ltd Full timeJob Title : Big Data Engineer Location : Bangalore Employment Type : Full-time Work Mode : 4 Days Work from Office Notice Period : Immediate to 30 days Job Description : We are looking for a Big Data Engineer with expertise in Hadoop, Hive, SQL, Python, PySpark, and AWS to join our team in Bangalore. The ideal candidate will have hands-on experience in...
-
Big Data Architect Lead
3 days ago
Bengaluru, Karnataka, India Recro Full timeJob OverviewAs a Senior Data Engineer at Recro, you will be responsible for designing and implementing large-scale data processing systems. Your expertise in distributed systems, big data technologies, and cloud platforms will enable our company to extract insights from vast amounts of data.We are looking for a highly skilled professional with a deep...
-
Big Data Engineer
4 hours ago
Bengaluru, Karnataka, India Scoop Technologies Full timeJob Role : Big Data Engineer Work Location : Bangalore (CV Ramen Nagar location) Experience : 7 Years Notice Period : Immediate - 30 days Mandatory Skills : Big Data, Python, SQL, Spark/Pyspark, AWS Cloud JD and required Skills & Responsibilities : - Actively participate in all phases of the software development lifecycle, including requirements gathering,...
-
Big Data Engineer
4 hours ago
Bengaluru, Karnataka, India Scoop Technologies Pvt Ltd Full timeJob Role : Big Data Engineer Work Location : Bangalore (CV Ramen Nagar location) Experience : 7 Years Notice Period : Immediate - 30 days Mandatory Skills : Big Data, Python, SQL, Spark/Pyspark, AWS (EMR, S3), Hive, Hadoop JD and required Skills & Responsibilities : - Actively participate in all phases of the software development lifecycle, including...
-
Aws Data Engineer
3 weeks ago
Bengaluru, Karnataka, India InfoWryt Solutions LLP Full timeJob DescriptionOverviewThe AWS Data Engineer plays a critical role in helping organizations harness the power of data through the AWS cloud platform. This position involves designing, building, and maintaining data architectures and pipelines that support analytical and operational needs. Working closely with data scientists, analysts, and stakeholders, the...
-
Data Science Specialist
3 days ago
Bengaluru, Karnataka, India Tata Consultancy Services Full timeRequired Skills and QualificationsTo be successful in this role, you will need:4-8 years of experience in software development with a focus on big data and cloud computing.Strong knowledge of SQL, Python, Pyspark/Spark/Scala, and AWS Services such as Glue and Redshift.Familiarity with Airflow DAGs and good understanding of ETL data warehousing concepts.
-
Big Data Developer
1 day ago
Bengaluru, Karnataka, India ARA Resources Full timeRequired Skills and QualificationsExperience: 4 to 6 years of experience as a Databricks Developer. Education: Bachelor's or Master's degree in Computer Science, Information Systems, Engineering, or equivalent. Skills: Strong experience in Databricks and AWS, hands-on experience with Python, PySpark, Scala, SQL, experience with Spark performance tuning and...
-
Sr. Data Engineer
3 weeks ago
Bengaluru, Karnataka, India Elev8 HR Consultancy Full timeMandatory Skills:• Big Data• Python• SQL• Spark / PySpark• AWS Cloud Job Description: We are seeking an experienced Senior Data Engineer who will be responsible for designing, developing, and maintaining scalable data solutions. The ideal candidate should have expertise in Big Data, Python, SQL, Spark/PySpark, and AWS Cloud and will work closely...
-
AWS Big Data Architect
6 days ago
Bengaluru, Karnataka, India Talent Worx Full timeAbout UsTalent Worx is a top-tier company that provides cutting-edge data solutions, and we're searching for a seasoned AWS Data Engineer to create and maintain advanced data pipelines and analytics platforms using AWS services.Job OverviewThe ideal candidate will have extensive experience in designing, implementing, and maintaining scalable,...
-
AWS Data Engineer
4 days ago
Bengaluru, Karnataka, India Tata Consultancy Services Full timeGreetings from TCSTCS is hiring for AWS Data EngineerDesired experience range: 4 to 8 YearsJob location: Bangalore / Chennai / Pune (5 days work from office)Must have skills1. SQL2. Python3. Pyspark/ spark / Scala4. AWS Services – Glue5. AWS Redshift6. Airflow DAGsGood to have1. Good knowledge in ETL Data WarehousingRegardsAankhi DebTalent Acquisition -...
-
Big Data Engineer
4 days ago
Bengaluru, Karnataka, India Scoop Technologies Pvt Ltd Full timeJob Title : Big Data EngineerLocation : BangaloreEmployment Type : Full-timeWork Mode : 4 Days Work from OfficeNotice Period : Immediate to 30 daysJob Description :We are looking for a Big Data Engineer with expertise in Hadoop, Hive, SQL, Python, PySpark, and AWS to join our team in Bangalore. The ideal candidate will have hands-on experience in designing,...
-
AWS Data Engineer
4 days ago
Bengaluru, Karnataka, India Tata Consultancy Services Full timeGreetings from TCSTCS is hiring for AWS Data EngineerDesired experience range: 4 to 8 YearsJob location: Bangalore / Chennai / Pune (5 days work from office)Must have skillsSQLPythonPyspark/ spark / ScalaAWS Services – GlueAWS RedshiftAirflow DAGsGood to haveGood knowledge in ETL Data WarehousingRegardsAankhi DebTalent Acquisition - TCS
-
AWS Data Engineer
3 weeks ago
Bengaluru, Karnataka, India Tata Consultancy Services Full timeRole- AWS Data EngineerExperience- 4 to 8 YrsRequired Technical Skill Set - Python,Pyspark, Hive, Snowflake,AWS (S3,Lambda, Glue, EMR and RedShift)Must-Have· Good hands on experience in Python programming and Pyspark· Data Engineering experience using AWS core services (Lambda, Glue, EMR and RedShift)· Required skill set- SQL, Airflow ·Must – Have...
-
Data Engineer
2 weeks ago
Bengaluru, Karnataka, India Xcel Hire Management Solutions Full timePosition : Data Engineer (Py-Spark)Experience : 5+ yearsLocation : Skills :- MUST HAVE: AWS, iceberg, glue, Athena, lambda, emr, SQL, Py-Spark- Python(Advance Should know about inheritance and Class)- Athena, Redshift, IAM iceberg, Apache Airflow, GitImportant :Data Engineer skills :- Data warehousing ( e.g redshift)- Fetch data from database ( PostgreSQL,...
-
Lead Big Data Engineer
4 hours ago
Bengaluru, Karnataka, India TalentXO Full timeWhat You'll Be Doing : - Build highly scalable, available, fault-tolerant distributed data processing systems (batch and streaming systems) processing over 100s of terabytes of data ingested every day and petabyte-sized data warehouse and elasticsearch cluster. - Build quality data solutions and refine existing diverse datasets to simplified models...
-
Big Data Engineer
3 weeks ago
Bengaluru, Karnataka, India Scoop Technologies Full timeJob Role : Big Data Engineer Work Location : Bangalore (CV Ramen Nagar location)Experience : 7+ Years Notice Period : Immediate - 30 daysMandatory Skills : Big Data, Python, SQL, Spark/Pyspark, AWS Cloud JD and required Skills & Responsibilities : - Actively participate in all phases of the software development lifecycle, including requirements gathering,...
-
Big Data Engineer
2 weeks ago
Bengaluru, Karnataka, India Scoop Technologies Full timeJob Role : Big Data Engineer Work Location : Bangalore (CV Ramen Nagar location)Experience : 7+ Years Notice Period : Immediate - 30 daysMandatory Skills : Big Data, Python, SQL, Spark/Pyspark, AWS Cloud JD and required Skills & Responsibilities : - Actively participate in all phases of the software development lifecycle, including requirements gathering,...