Data Engineer- PySpark

3 days ago

Chennai, India Cynosure Corporate Solutions Full time
  • Build up data pipelines for consumption by the data science team.
  • Clear understanding and experience with Python and PySpark.
  • Experience in writing Python programs and SQL queries.
  • Experience in SQL Query tuning.
  • Build and maintain data pipelines in Pyspark with SQL and Python.
  • Knowledge of Cloud (Azure/AWS) technologies is additional. Suggest and implement best practices in data integration.
  • Split the planned deliverables into tasks and assign them to the team.
  • Good oral, written and presentation skills.

Requirements

  • Degree in Computer Science, IT, or a similar field; a master's is a plus. Hands-on experience with Python and Pyspark
  • 3-5 years of hands-on experience in Python, PySpark, and SQL.
  • Great numerical and analytical skills.
  • Proven expertise in building and optimizing data pipelines using Spark and Hadoop ecosystem tools.
  • Experience working on cloud platforms like Azure or AWS.
  • Should be able to collaborate and coordinate in a remote environment. Be a problem solver and be proactive to solve the challenges that come his way.

  • Pyspark Developer

    7 days ago

    Chennai, Tamil Nadu, India MP DOMINIC AND CO Full time

    Job Summary - Design develop and implement scalable data pipelines and streaming use cases using PySpark and Spark on a distributed computing platform - Possess strong programming skills in Spark streaming - Have familiarity with cloud platforms like GCP - Gain experience in big data technologies such as Hadoop Hive and HDFS - Perform ETL operations...

  • Senior Data Engineering

    1 week ago

    Bengaluru / Bangalore, Delhi NCR, Chennai, India beBeePySparkDeveloper Full time ₹ 15,00,000 - ₹ 28,00,000

    Data Engineer & PySpark Developer JobWe are seeking a highly skilled data engineer and PySpark developer to join our team. The ideal candidate will have experience with designing, building, and optimizing large-scale data pipelines on Databricks Unified Analytics Platform.Design, build, and optimize scalable ETL/ELT pipelines using PySpark on...

  • Senior Data Engineer

    6 days ago

    Chennai, India Pan Asia HR Solutions Full time

    Job Title : Senior Data EngineerExperience : 6+ YearsLocation : Chennai, IndiaWork Mode : Hybrid (2-3 days/week onsite)About the Role :We are looking for a highly skilled Senior Data Engineer with a strong background in Python, Java, PySpark, and Google Cloud Platform (GCP) to join our growing data team in Chennai. The ideal candidate should have a deep...

  • Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India CX Data Labs Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    HiRole - Azure Data engineerLocation - Chennai, Gurugram (5days Onsite)Full timeImmediate joiner Only (15days Notice period only)Technical Skills:o Proficiency in SQL for complex queries, database design, and optimization.o Strong experience with PySpark for data transformation and processing.o Hands-on experience with Databricks for building and managing...

  • Data Engineer

    3 days ago

    Chennai, India CX Data Labs Full time

    Hi Role - Azure Data engineer Location - Chennai, Gurugram (5days Onsite) Full time Immediate joiner Only (15days Notice period only) Technical Skills: o Proficiency in SQL for complex queries, database design, and optimization. o Strong experience with PySpark for data transformation and processing. o Hands-on experience with Databricks for building and...

  • AWS Data Engineer

    12 hours ago

    Chennai, India Tata Consultancy Services Full time

    Dear CandidateGreetings from TATA Consultancy ServicesJob Openings at TCSSkill - AWS Data Engineer - Redshift, Pyspark, GlueExp range - 5 yrs - 8yrsLocation- ChennaiNotice period 30 daysPls find the Job Description below.Good hands-on experience in Python programming and PysparkData Engineering experience using AWS core services (Lambda, Glue, EMR and...

  • PySpark Developer

    6 days ago

    Hyderabad, Bengaluru, Chennai, India Coders Brain Technology Private Limited Full time

    Job Description ROLE RESPONSIBILITIES Data Engineering and Processing: Develop and manage data pipelines using PySpark on Databricks. Implement ETL/ELT processes to process structured and unstructured data at scale. Optimize data pipelines for performance, scalability, and cost-efficiency in Databricks. Databricks Platform Expertise: Experience in...

  • AWS Data Engineer

    2 days ago

    Chennai, India Tata Consultancy Services Full time

    Dear CandidateGreetings from TATA Consultancy ServicesJob Openings at TCSSkill - AWS Data Engineer - Redshift, Pyspark, GlueExp range - 5 yrs - 8yrsLocation- ChennaiNotice period 30 daysPls find the Job Description below.Good hands-on experience in Python programming and PysparkData Engineering experience using AWS core services (Lambda, Glue, EMR and...

  • AWS Data Engineer

    2 days ago

    Chennai, India Tata Consultancy Services Full time

    Dear CandidateGreetings from TATA Consultancy ServicesJob Openings at TCSSkill - AWS Data Engineer - Redshift, Pyspark, GlueExp range - 5 yrs - 8yrsLocation- ChennaiNotice period 30 daysPls find the Job Description below.Good hands-on experience in Python programming and PysparkData Engineering experience using AWS core services (Lambda, Glue, EMR and...

  • AWS Data Engineer

    1 day ago

    Chennai, India Tata Consultancy Services Full time

    Dear Candidate Greetings from TATA Consultancy Services Job Openings at TCS Skill - AWS Data Engineer - Redshift, Pyspark, Glue Exp range - 5 yrs - 8yrs Location- Chennai Notice period 30 days Pls find the Job Description below. Good hands-on experience in Python programming and Pyspark Data Engineering experience using AWS core services (Lambda, Glue,...