PySpark Data Engineer

6 days ago

Thrissur, Kerala, India BLS360 Full time

Role Overview

This is a contract remote role. We are seeking a highly motivated PySpark Data Engineer with 3–4 years of experience in building, optimizing, and maintaining large-scale data processing solutions. The ideal candidate will bring hands-on expertise in PySpark, Python, SQL, and real-time data streaming and will contribute to the design and implementation of scalable data pipelines in cloud environments such as Azure.

·         Location: India. Remote.

·         Job type: Contract

Key Responsibilities

  • Design, develop, and optimize data pipelines using PySpark, Spark, and Python.
  • Implement ETL/ELT solutions to integrate structured and unstructured data from multiple sources.
  • Develop and maintain Hive queries and ensure seamless data transformations.
  • Write and optimize SQL queries for large datasets.
  • Build, monitor, and support real-time streaming solutions using Apache Kafka.
  • Work on Databricks to create, manage, and optimize data workflows.
  • Deploy and manage solutions in Azure cloud services (e.g., Data Lake, Synapse, Azure Databricks).
  • Debug, troubleshoot, and optimize code for performance and scalability.
  • Collaborate with cross-functional teams to deliver data-driven solutions.
  • Adhere to best practices for coding, testing, and deployment.

Required Skills & Qualifications

  • 3–4 years of experience in data engineering with a focus on PySpark and Python.
  • Strong knowledge of Spark, Hive, SQL, and Unix shell scripting.
  • Experience with Databricks in a production environment.
  • Hands-on experience with Azure cloud services (preferred: Data Lake, Synapse, Databricks).
  • Experience working with Apache Kafka or other real-time streaming technologies.
  • Strong problem-solving and debugging skills.
  • Excellent written and verbal communication skills.
  • Ability to work independently in a fast-paced environment and adapt to new technologies quickly.

Nice-to-Have Skills

  • Exposure to CI/CD pipelines for data solutions.
  • Knowledge of data governance and security best practices.
  • Experience with Agile/Scrum methodology.

  • Senior Data Engineer

    7 days ago

    Thrissur, Kerala, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Title: Senior Data EngineerWe are seeking a highly motivated and experienced data engineer to join our team.The ideal candidate will have hands-on expertise in PySpark, Python, SQL, and real-time data streaming and contribute to the design and implementation of scalable data pipelines in cloud environments.Key Responsibilities:Design, develop, and...

  • AWS Data Engineer Position

    6 days ago

    Thrissur, Kerala, India beBeeDataEngineer Full time ₹ 6,00,000 - ₹ 14,00,000

    Job Title: Data Processing Specialist">About the Role:The ideal candidate will have experience with AWS, Python, and PySpark.They will be responsible for designing, developing, and maintaining large-scale data processing systems using Databricks.Key Responsibilities:Develop scalable and efficient data pipelines using AWS services.Collaborate with...

  • Big Data Engineer Position

    4 days ago

    Thrissur, Kerala, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Data Engineering SpecialistWe are seeking an experienced data engineering professional with expertise in big data pipelines and distributed data systems.The ideal candidate will have a proven track record of designing, developing, and optimizing large-scale data processing systems, leveraging skills in Scala, Spark (PySpark), and Python.The role...

  • Data Engineering Specialist

    1 week ago

    Thrissur, Kerala, India beBeeAzure Full time ₹ 20,00,000 - ₹ 30,00,000

    Azure Data Engineer Role OverviewThis is a key position in our organization's efforts to successfully migrate large-scale SQL workloads from on-premise environments to Azure. As a skilled Azure Data Engineer, you will lead the migration process, ensuring high data integrity, minimal downtime, and performance optimization. Key Responsibilities Data Pipeline...

  • Azure Cloud Data Engineering Specialist

    6 days ago

    Thrissur, Kerala, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job ProfileAzure data engineer with expertise in designing and implementing scalable data processing pipelines leveraging Data Bricks, Azure Data Factory, and PySpark.

  • Data Engineering Specialist

    6 days ago

    Thrissur, Kerala, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Data Engineering ExpertAbout the Role:We are seeking a skilled data engineer to play a critical role in designing, developing, and managing data pipelines and transformation frameworks that enable analytics, reporting, and business intelligence capabilities across the enterprise.The ideal candidate will work closely with data architects, platform...

  • Advanced Data Pipeline Specialist

    1 week ago

    Thrissur, Kerala, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Data Engineer PositionOur organization is seeking an experienced and skilled Data Engineer to play a pivotal role in our team.The ideal candidate will have a solid background in data engineering, focusing on PySpark, Python, and SQL. Familiarity with Azure Databricks is beneficial.Main Responsibilities:Create, develop, and maintain scalable data pipelines...

  • Expert Data Pipeline Architect

    2 weeks ago

    Thrissur, Kerala, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Freelance Data EngineerWe are seeking a Freelance Data Engineer with expertise in designing, building, and optimizing large-scale data pipelines using Python, SQL, Linux/Bash, PySpark, AWS, and Databricks.Job Description:Data Engineering is a key component of our organization's ability to drive business outcomes through data-driven decision making. We are...

  • Chief Data Pipeline Architect

    1 week ago

    Thrissur, Kerala, India beBeeData Full time ₹ 8,00,000 - ₹ 12,00,000

    Job Description:We are seeking an experienced Data Engineer to design, build and maintain scalable data pipelines in a cloud environment. The ideal candidate will have hands-on experience with AWS services, including EMR, Glue, PySpark, Airflow and Python.Requirements:A minimum of 5 years of experience in data engineering or related roles.Strong proficiency...

  • Mastering Data Pipelines

    4 days ago

    Thrissur, Kerala, India beBeeDataEngineer Full time ₹ 16,00,000 - ₹ 20,10,000

    Job OverviewWe are seeking an experienced Data Engineer to design, build, and optimize scalable data pipelines and ETL workflows.The ideal candidate will have strong expertise in Python, Spark, PySpark, and experience working with streaming data and collaborating with data scientists and business stakeholders to deliver high-quality solutions.Required Skills...