AWS Big data pyspark SQL Spark Airflow
9 hours ago
Job Title: AWS Big Data Engineer
About the Role:
We are looking for a highly skilled AWS Big Data Engineer with extensive experience in PySpark, SQL, Spark, and Airflow. The ideal candidate will have a strong background in coding and a deep understanding of big data technologies. You will be responsible for designing, developing, and maintaining big data solutions on AWS, ensuring data is processed efficiently and effectively.
Key Responsibilities:
Design, develop, and maintain big data solutions using AWS services.
Write and optimize complex SQL queries.
Develop and manage data pipelines using PySpark and Spark.
Implement and manage workflows using Apache Airflow.
Collaborate with data scientists and analysts to understand data requirements.
Ensure data quality and integrity throughout the data lifecycle.
Troubleshoot and resolve issues related to big data processing.
Stay updated with the latest trends and best practices in big data technologies.
Key Skills and Qualifications:
Proven experience with AWS big data services.
Strong coding experience in PySpark and SQL.
Expertise in Spark and Airflow.
Excellent problem-solving and analytical skills.
Ability to work independently and as part of a team.
Strong communication and documentation skills.
Relevant certifications (e.g., AWS Certified Big Data - Specialty) are a plus.
-
Bengaluru, India UST Full timeCandidates ready to join immediately can share their details via email for quick processing.CCTC | ECTC | Notice Period | Location fast for immediate attention! ⏳5+ years if Experience.Roles and Responsibiliti- esDesign, develop, and maintain scalable data pipelines using Spark (PySpark or Spark with Scala- ).Build data ingestion and transformation...
-
Bengaluru, India UST Full timeCandidates ready to join immediately can share their details via email for quick processing.CCTC | ECTC | Notice Period | Location fast for immediate attention! ⏳5+ years if Experience.Roles and Responsibiliti- esDesign, develop, and maintain scalable data pipelines using Spark (PySpark or Spark with Scala- ).Build data ingestion and transformation...
-
Bengaluru, India UST Full timeCandidates ready to join immediately can share their details via email for quick processing. 📌 CCTC | ECTC | Notice Period | Location Preference nitin.patil@ust.com Act fast for immediate attention! ⏳📩 5+ years if Experienc e.Roles and Responsibiliti - esDesign, develop, and maintain scalable data pipelines using Spark (PySpark or Spark with...
-
Big Data Engineer
2 weeks ago
Bengaluru, India ANB IT Solutions Pvt Ltd Full timeJob Description :- Minimum of 7+ years of progressive experience in Data Engineering, with a proven track record in pipeline creation and implementation.- Strong awareness and hands-on experience with various data tools and platforms, including Hive, Hadoop, Starburst, and other Big Data technologies.- Proficiency in programming languages such as SQL,...
-
Bengaluru, India UST Full timeCandidates ready to join immediately can share their details via email for quick processing.CCTC | ECTC | Notice Period | Location PreferenceAct fast for immediate attention! ⏳5+ years if Experience.Roles and Responsibiliti- esDesign, develop, and maintain scalable data pipelines using Spark (PySpark or Spark with Scala- ).Build data ingestion and...
-
Data Engineer
14 hours ago
Bengaluru, India The Nielsen Company Full timeAt Nielsen, we believe that career growth is a partnership. You ultimately own, fuel and set the journey. By joining our team of nearly 14,000 associates, you will become part of a community that will help you to succeed. We champion you because when you succeed, we do too. Embark on a new initiative, explore a fresh approach, and take license to think big,...
-
▷ 15h Left: Senior Data Engineer
2 weeks ago
Bengaluru, India Axheleon IT Services Private Limited Full timeJob Description About the Role We are looking for a highly skilled Senior Data Engineer with a strong background in Python, PySpark, Hive, and distributed data processing frameworks. The ideal candidate will have 5+ years of hands-on experience in building and optimizing big data pipelines, with expertise in both batch and streaming data processing using...
-
Big Data/Pyspark Developer
2 weeks ago
Bengaluru, India Contactx Resource Management Pvt. Ltd. Full timeSenior Big Data Engineer / PySpark Developer with deep expertise in distributed computing, PySpark, and Spark performance tuning for both batch and streaming data processing. Hands-on experience with Hive, open table formats such as Iceberg, and workflow orchestration tools like Airflow or Oozie. Strong coding skills adhering to OOP principles, design...
-
Big Data/Pyspark Developer
5 days ago
Bengaluru, Karnataka, India Contactx Resource Management Pvt. Ltd. Full time ₹ 12,00,000 - ₹ 36,00,000 per yearSenior Big Data Engineer / PySpark Developer with deep expertise in distributed computing, PySpark, and Spark performance tuning for both batch and streaming data processing.Hands-on experience with Hive, open table formats such as Iceberg, and workflow orchestration tools like Airflow or Oozie.Strong coding skills adhering to OOP principles, design...
-
Data Streaming
2 days ago
Bengaluru, India ACE Talent Consulting Full timeJob Description: Spark/Scala/PySpark developer who knows how to fully exploit the potential of our Spark cluster. Should have ability to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data. **Responsibilities**: Create Scala/Spark/Pyspark jobs for data transformation and aggregation Produce...
