Big Data Engineer

7 days ago

MumbaiPuneBangalore, India Techno Wise Full time

Job Description :
We are seeking a talented Big Data Engineer proficient in PySpark to join our dynamic team.

The ideal candidate will play a pivotal role in designing, implementing, and maintaining scalable data solutions leveraging the Big Data technologies like PySpark.

This role requires a strong understanding of big data technologies, data engineering best practices, and the ability to collaborate effectively with cross-functional teams.


Responsibilities :

  • Design, develop, and implement end-to-end data pipelines using PySpark to process, cleanse, and transform large-scale datasets.
  • Collaborate with data scientists, analysts, and stakeholders to understand data requirements and translate them into technical solutions.
  • Optimize and tune Spark jobs for performance and scalability, ensuring efficient resource utilization and processing speed.
  • Develop and maintain data ingestion processes from various sources such as databases, streams, and file systems into the Databricks environment.
  • Implement data quality checks and validation processes to ensure the accuracy, completeness, and reliability of the data.
  • Have worked on Data Orchestration Tools like Airflow.
  • Working on story\task using Agile Methodology.
  • Must have hand-on on GitHub and GitLab
  • Work closely with DevOps teams to automate deployment processes and ensure the reliability and availability of data pipelines.
  • Monitor, troubleshoot, and resolve issues related to data pipeline failures, performance degradation, and system errors.
  • Stay updated on emerging technologies and industry trends in big data, distributed computing, and cloud platforms to drive continuous improvement and innovation.
)

  • Big Data Engineer

    1 month ago

    bangalore, India Virtusa Full time

    Big Data Engineer - CREQ188642 Description Looking for a Senior Software Engineer with strong analytical skills to help build and enhance our Big data application. This individual's day-to-day work will involve using Java, Scala, SQL, Big Data, Linux, git, and maven/gradle to design and build software solutions that are fast, scalable, and...

  • Big Data Engineer

    2 weeks ago

    Mumbai/Bangalore/Pune, India Techno Wise Full time

    Job Role : Big Data EngineerRelevant Experience : 3+ yearsLocation : Navi Mumbai /Bengaluru/PuneNotice Period : Immediate or serving Notice PeriodPrimary Skills : Big Data, PySpark, Cloud- Azure or AWS, Databricks, SQL, PythonOverview : We are seeking a talented Big Data Engineer proficient in PySpark to join our dynamic team. The ideal candidate will play...

  • Big Data Engineer

    7 days ago

    Mumbai/Bangalore/Pune, IN Techno Wise Full time

    Job Role : Big Data EngineerRelevant Experience : 3+ yearsLocation : Navi Mumbai /Bengaluru/PuneNotice Period : Immediate or serving Notice PeriodPrimary Skills : Big Data, PySpark, Cloud- Azure or AWS, Databricks, SQL, PythonOverview : We are seeking a talented Big Data Engineer proficient in PySpark to join our dynamic team. The ideal candidate will play a...

  • Big Data Engineer

    3 weeks ago

    bangalore, India Awign Expert Full time

    This is a remote position. Job Title: Big Data Solutions Engineer Reporting to: Director, Product Management Location: India (Remote) Our Ideal Candidate Our company is seeking a talented Big Data Solutions Engineer to join our team. The ideal candidate will be responsible for designing, developing, and implementing our proprietary...

  • Big Data Engineer

    3 weeks ago

    bangalore, India Awign Expert Full time

    This is a remote position. Job Title: Big Data Solutions Engineer Reporting to: Director, Product Management Location: India (Remote) Our Ideal Candidate Our company is seeking a talented Big Data Solutions Engineer to join our team. The ideal candidate will be responsible for designing, developing, and implementing our proprietary...

  • Big Data Engineer

    7 days ago

    Pune, Maharashtra, India Stratacent Full time

    Position Title:Big Data EngineerLocation:Pune/ BangaloreExperience : 5+ YearsShift Timings:UK ShiftPosition Summary:We seek an experienced Senior Big Data Engineer to lead and enhance our data engineering initiatives. The ideal candidate will possess advanced expertise in Impala, Spark, Scala, Hive, Hadoop, and other Big Data technologies. You will be...

  • Big Data Engineer

    2 weeks ago

    pune, India Stratacent Full time

    Position Title:  Big Data Engineer Location:  Pune/ Bangalore Experience : 5+ Years Shift Timings:  UK Shift Position Summary: We seek an experienced Senior Big Data Engineer to lead and enhance our data engineering initiatives. The ideal candidate will possess advanced expertise in Impala, Spark, Scala, Hive, Hadoop, and other Big Data technologies. You...

  • Big Data Engineer

    7 days ago

    Pune, Maharashtra, India Stratacent Full time

    Position Title: Big Data EngineerLocation: Pune/ BangaloreExperience: 5+ YearsShift Timings: UK ShiftPosition Summary:We seek an experienced Senior Big Data Engineer to lead and enhance our data engineering initiatives. The ideal candidate will possess advanced expertise in Impala, Spark, Scala, Hive, Hadoop, and other Big Data technologies. You will be...

  • Big Data Engineer

    7 days ago

    Pune, Maharashtra, India Stratacent Full time

    Position Title: Big Data Engineer Location: Pune/ Bangalore Experience : 5+ Years Shift Timings: UK Shift Position Summary: We seek an experienced Senior Big Data Engineer to lead and enhance our data engineering initiatives. The ideal candidate will possess advanced expertise in Impala, Spark, Scala, Hive, Hadoop, and other Big Data technologies. You...

  • Big Data Engineer

    4 weeks ago

    Pune, India Qualys Full time

    We are seeking a talented Big Data Engineer to deliver roadmap features of Enterprise TruRisk Platform which would help customers to Measure, Communicate and Eliminate Cyber Risks. Working with a team of engineers and architects, you will be responsible for prototyping, designing, developing and supporting a highly scalable, distributed SaaS based Security...

  • Big Data Engineer

    2 months ago

    pune, India Qualys Full time

    We are seeking a talented Big Data Engineer to deliver roadmap features of Enterprise TruRisk Platform which would help customers to Measure, Communicate and Eliminate Cyber Risks.Working with a team of engineers and architects, you will be responsible for prototyping, designing, developing and supporting a highly scalable, distributed SaaS based Security...

  • Big Data Engineer

    3 weeks ago

    pune, India Qualys Full time

    We are seeking a talented Big Data Engineer to deliver roadmap features of Enterprise TruRisk Platform which would help customers to Measure, Communicate and Eliminate Cyber Risks.Working with a team of engineers and architects, you will be responsible for prototyping, designing, developing and supporting a highly scalable, distributed SaaS based Security...

  • Big Data Engineer

    7 days ago

    Pune, Maharashtra, India Techno Wise Full time

    Job Role : Big Data EngineerRelevant Experience : 3+ yearsLocation : Navi Mumbai /Bengaluru/PuneNotice Period : Immediate or serving Notice PeriodPrimary Skills : Big Data, PySpark, Cloud- Azure or AWS, Databricks, SQL, PythonOverview :We are seeking a talented Big Data Engineer proficient in PySpark to join our dynamic team. The ideal candidate will play a...

  • Big Data Engineer

    7 days ago

    Pune, Maharashtra, India Qualys Full time

    We are seeking a talented Big Data Engineer to deliver roadmap features of Enterprise TruRisk Platform which would help customers to Measure, Communicate and Eliminate Cyber Risks. Working with a team of engineers and architects, you will be responsible for prototyping, designing, developing and supporting a highly scalable, distributed SaaS based Security...

  • Big Data Engineer

    7 days ago

    Pune, Maharashtra, India Draftmason Enterprises Full time

    Designation : Big Data EngineerExperience : - 8 YearsSalary Range : upto 30 LPAJob Location : Kharadi, PuneType Of Employment : FT, PermanentWorking Mode : Hybrid (3 days WFO)Working Days : Mon - FriReports To : Director - Data EngineeringThe Opportunity :We are looking for talented, motivated, and self-driven Big Data Engineers who specialize in data-driven...

  • Big Data Engineer

    2 months ago

    Pune, India Qualys Full time

    We are seeking a talented Big Data Engineer to deliver roadmap features of Enterprise TruRisk Platform which would help customers to Measure, Communicate and Eliminate Cyber Risks. Working with a team of engineers and architects, you will be responsible for prototyping, designing, developing and supporting a highly scalable, distributed SaaS based Security...

  • Big Data Engineer

    2 months ago

    Pune, India Qualys Full time

    We are seeking a talented Big Data Engineer to deliver roadmap features of Enterprise TruRisk Platform which would help customers to Measure, Communicate and Eliminate Cyber Risks. Working with a team of engineers and architects, you will be responsible for prototyping, designing, developing and supporting a highly scalable, distributed SaaS based Security...

  • Big Data Engineer

    2 months ago

    Pune, India Qualys Full time

    We are seeking a talented Big Data Engineer to deliver roadmap features of Enterprise TruRisk Platform which would help customers to Measure, Communicate and Eliminate Cyber Risks. Working with a team of engineers and architects, you will be responsible for prototyping, designing, developing and supporting a highly scalable, distributed SaaS based Security...

  • Big Data Engineer

    3 weeks ago

    Pune, India Qualys Full time

    We are seeking a talented Big Data Engineer to deliver roadmap features of Enterprise TruRisk Platform which would help customers to Measure, Communicate and Eliminate Cyber Risks. Working with a team of engineers and architects, you will be responsible for prototyping, designing, developing and supporting a highly scalable, distributed SaaS based Security...

  • Big Data Engineer

    3 weeks ago

    Pune, India Qualys Full time

    We are seeking a talented Big Data Engineer to deliver roadmap features of Enterprise TruRisk Platform which would help customers to Measure, Communicate and Eliminate Cyber Risks. Working with a team of engineers and architects, you will be responsible for prototyping, designing, developing and supporting a highly scalable, distributed SaaS based Security...