AWS Big data pyspark SQL Spark Airflow

1 week ago


Bengaluru, Karnataka, India ScaleneWorks Full time

Job Title: AWS Big Data Engineer

About the Role:
We are looking for a highly skilled AWS Big Data Engineer with extensive experience in PySpark, SQL, Spark, and Airflow. The ideal candidate will have a strong background in coding and a deep understanding of big data technologies. You will be responsible for designing, developing, and maintaining big data solutions on AWS, ensuring data is processed efficiently and effectively.

Key Responsibilities:
Design, develop, and maintain big data solutions using AWS services.
Write and optimize complex SQL queries.
Develop and manage data pipelines using PySpark and Spark.
Implement and manage workflows using Apache Airflow.
Collaborate with data scientists and analysts to understand data requirements.
Ensure data quality and integrity throughout the data lifecycle.
Troubleshoot and resolve issues related to big data processing.
Stay updated with the latest trends and best practices in big data technologies.

Key Skills and Qualifications:
Proven experience with AWS big data services.
Strong coding experience in PySpark and SQL.
Expertise in Spark and Airflow.
Excellent problem-solving and analytical skills.
Ability to work independently and as part of a team.
Strong communication and documentation skills.
Relevant certifications (e.g., AWS Certified Big Data - Specialty) are a plus.



  • Bengaluru, Karnataka, India UST Full time

    We're looking for a Senior Big Data Specialist to join our team at UST. As a key member of our data engineering team, you will be responsible for designing, implementing, and optimizing big data solutions using PySpark, GCP, Spark, Hadoop, and Hive.The ideal candidate will have strong hands-on experience in PySpark & GCP and a solid understanding of big data...


  • Bengaluru, Karnataka, India Thoucentric Technology Pvt. Ltd. Full time

    About Us : Thoucentric is the Consulting arm of Xoriant, a prominent digital engineering services company with 5000+ employees. We are headquartered in Bangalore with presence across multiple locations in India, US, UK, Singapore & Australia Globally. As the Consulting business of Xoriant, We help clients with Business Consulting, Program & Project...

  • AWS Data Engineer

    4 weeks ago


    Bengaluru, Karnataka, India Neerinfo Solutions Full time

    Key Responsibilities :AWS Data Infrastructure Development :- Design, develop, and maintain AWS-based data pipelines using services such as AWS Glue, AWS Redshift, Amazon S3, and AWS Lambda.- Build ETL (Extract, Transform, Load) processes, integrating batch and near real-time data from various sources to Amazon Redshift, S3, or other cloud-based storage...

  • Big Data Engineer

    4 hours ago


    Bengaluru, Karnataka, India Scoop Technologies Pvt Ltd Full time

    Job Title : Big Data Engineer Location : Bangalore Employment Type : Full-time Work Mode : 4 Days Work from Office Notice Period : Immediate to 30 days Job Description : We are looking for a Big Data Engineer with expertise in Hadoop, Hive, SQL, Python, PySpark, and AWS to join our team in Bangalore. The ideal candidate will have hands-on experience in...


  • Bengaluru, Karnataka, India Recro Full time

    Job OverviewAs a Senior Data Engineer at Recro, you will be responsible for designing and implementing large-scale data processing systems. Your expertise in distributed systems, big data technologies, and cloud platforms will enable our company to extract insights from vast amounts of data.We are looking for a highly skilled professional with a deep...

  • Big Data Engineer

    4 hours ago


    Bengaluru, Karnataka, India Scoop Technologies Full time

    Job Role : Big Data Engineer Work Location : Bangalore (CV Ramen Nagar location) Experience : 7 Years Notice Period : Immediate - 30 days Mandatory Skills : Big Data, Python, SQL, Spark/Pyspark, AWS Cloud JD and required Skills & Responsibilities : - Actively participate in all phases of the software development lifecycle, including requirements gathering,...

  • Big Data Engineer

    4 hours ago


    Bengaluru, Karnataka, India Scoop Technologies Pvt Ltd Full time

    Job Role : Big Data Engineer Work Location : Bangalore (CV Ramen Nagar location) Experience : 7 Years Notice Period : Immediate - 30 days Mandatory Skills : Big Data, Python, SQL, Spark/Pyspark, AWS (EMR, S3), Hive, Hadoop JD and required Skills & Responsibilities : - Actively participate in all phases of the software development lifecycle, including...

  • Aws Data Engineer

    3 weeks ago


    Bengaluru, Karnataka, India InfoWryt Solutions LLP Full time

    Job DescriptionOverviewThe AWS Data Engineer plays a critical role in helping organizations harness the power of data through the AWS cloud platform. This position involves designing, building, and maintaining data architectures and pipelines that support analytical and operational needs. Working closely with data scientists, analysts, and stakeholders, the...


  • Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Required Skills and QualificationsTo be successful in this role, you will need:4-8 years of experience in software development with a focus on big data and cloud computing.Strong knowledge of SQL, Python, Pyspark/Spark/Scala, and AWS Services such as Glue and Redshift.Familiarity with Airflow DAGs and good understanding of ETL data warehousing concepts.

  • Big Data Developer

    1 day ago


    Bengaluru, Karnataka, India ARA Resources Full time

    Required Skills and QualificationsExperience: 4 to 6 years of experience as a Databricks Developer. Education: Bachelor's or Master's degree in Computer Science, Information Systems, Engineering, or equivalent. Skills: Strong experience in Databricks and AWS, hands-on experience with Python, PySpark, Scala, SQL, experience with Spark performance tuning and...

  • Sr. Data Engineer

    3 weeks ago


    Bengaluru, Karnataka, India Elev8 HR Consultancy Full time

    Mandatory Skills:• Big Data• Python• SQL• Spark / PySpark• AWS Cloud Job Description: We are seeking an experienced Senior Data Engineer who will be responsible for designing, developing, and maintaining scalable data solutions. The ideal candidate should have expertise in Big Data, Python, SQL, Spark/PySpark, and AWS Cloud and will work closely...


  • Bengaluru, Karnataka, India Talent Worx Full time

    About UsTalent Worx is a top-tier company that provides cutting-edge data solutions, and we're searching for a seasoned AWS Data Engineer to create and maintain advanced data pipelines and analytics platforms using AWS services.Job OverviewThe ideal candidate will have extensive experience in designing, implementing, and maintaining scalable,...

  • AWS Data Engineer

    4 days ago


    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Greetings from TCSTCS is hiring for AWS Data EngineerDesired experience range: 4 to 8 YearsJob location: Bangalore / Chennai / Pune (5 days work from office)Must have skills1. SQL2. Python3. Pyspark/ spark / Scala4. AWS Services – Glue5. AWS Redshift6. Airflow DAGsGood to have1. Good knowledge in ETL Data WarehousingRegardsAankhi DebTalent Acquisition -...

  • Big Data Engineer

    4 days ago


    Bengaluru, Karnataka, India Scoop Technologies Pvt Ltd Full time

    Job Title : Big Data EngineerLocation : BangaloreEmployment Type : Full-timeWork Mode : 4 Days Work from OfficeNotice Period : Immediate to 30 daysJob Description :We are looking for a Big Data Engineer with expertise in Hadoop, Hive, SQL, Python, PySpark, and AWS to join our team in Bangalore. The ideal candidate will have hands-on experience in designing,...

  • AWS Data Engineer

    4 days ago


    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Greetings from TCSTCS is hiring for AWS Data EngineerDesired experience range: 4 to 8 YearsJob location: Bangalore / Chennai / Pune (5 days work from office)Must have skillsSQLPythonPyspark/ spark / ScalaAWS Services – GlueAWS RedshiftAirflow DAGsGood to haveGood knowledge in ETL Data WarehousingRegardsAankhi DebTalent Acquisition - TCS

  • AWS Data Engineer

    3 weeks ago


    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Role- AWS Data EngineerExperience- 4 to 8 YrsRequired Technical Skill Set - Python,Pyspark, Hive, Snowflake,AWS (S3,Lambda, Glue, EMR and RedShift)Must-Have· Good hands on experience in Python programming and Pyspark· Data Engineering experience using AWS core services (Lambda, Glue, EMR and RedShift)· Required skill set- SQL, Airflow ·Must – Have...

  • Data Engineer

    2 weeks ago


    Bengaluru, Karnataka, India Xcel Hire Management Solutions Full time

    Position : Data Engineer (Py-Spark)Experience : 5+ yearsLocation : Skills :- MUST HAVE: AWS, iceberg, glue, Athena, lambda, emr, SQL, Py-Spark- Python(Advance Should know about inheritance and Class)- Athena, Redshift, IAM iceberg, Apache Airflow, GitImportant :Data Engineer skills :- Data warehousing ( e.g redshift)- Fetch data from database ( PostgreSQL,...


  • Bengaluru, Karnataka, India TalentXO Full time

    What You'll Be Doing : - Build highly scalable, available, fault-tolerant distributed data processing systems (batch and streaming systems) processing over 100s of terabytes of data ingested every day and petabyte-sized data warehouse and elasticsearch cluster. - Build quality data solutions and refine existing diverse datasets to simplified models...

  • Big Data Engineer

    3 weeks ago


    Bengaluru, Karnataka, India Scoop Technologies Full time

    Job Role : Big Data Engineer Work Location : Bangalore (CV Ramen Nagar location)Experience : 7+ Years Notice Period : Immediate - 30 daysMandatory Skills : Big Data, Python, SQL, Spark/Pyspark, AWS Cloud JD and required Skills & Responsibilities : - Actively participate in all phases of the software development lifecycle, including requirements gathering,...

  • Big Data Engineer

    2 weeks ago


    Bengaluru, Karnataka, India Scoop Technologies Full time

    Job Role : Big Data Engineer Work Location : Bangalore (CV Ramen Nagar location)Experience : 7+ Years Notice Period : Immediate - 30 daysMandatory Skills : Big Data, Python, SQL, Spark/Pyspark, AWS Cloud JD and required Skills & Responsibilities : - Actively participate in all phases of the software development lifecycle, including requirements gathering,...