Python Pyspark Developer

1 day ago


Noida, Uttar Pradesh, India Zigsaw Full time ₹ 1,00,00,000 - ₹ 2,00,00,000 per year

Roles & Responsibilities:
Design, development and implementation of performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR
Writing reusable, testable, and efficient code
Integration of data storage solutions in spark – especially with AWS S3 object storage.
Performance tuning of pySpark scripts.
Need to ensure overall build delivery quality is good and on time delivery is done at all times.
Should be able to handle meetings with customers with ease.
Need to have excellent communication skills to interact with customer.Education/Experience
Technical/Functional Skills (Mandatory skills):
5+ years of experience in programming with python. Strong proficiency in python
Familiarity with functional programming concepts
3+ years of Hands on experience in developing ETL data pipelines using pyspark on AWS EMR
Hands-on experience of JSON processing using python
Good understanding of Spark's RDD API
Good understanding of Spark's Dataframe and API
Experience in configuring EMR clusters on AWS
Experience and good understanding of Apache Spark Data sources API.
Experience of dealing with AWS S3 object storage from Spark.
Experience in troubleshooting spark jobs. Knowledge of monitoring Spark jobs using Spark UI
Performance tuning of Spark jobs.
Understanding fundamental design principles behind a business processes
 
Nice to have skills:
Knowledge of AWS SDK CLI
Experience of setting up continuous integration/deployment of spark jobs to EMR clusters
Knowledge of scheduling spark applications in AWS EMR cluster.
Understanding the differences between Hadoop MapReduce and Apache Spark
Proficient understanding of code versioning tools as Git, SVN
Experience minimum 4-7years


  • Pyspark Developer

    1 day ago


    Noida, Uttar Pradesh, India Zigsaw Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Roles & Responsibilities: Design development and implementation of performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR Writing reusable testable and efficient code Integration of data storage solutions in spark & ndash especially with AWS S3 object storage. Performance tuning of pySpark scripts. Need to ensure overall build...

  • Developer - PySpark

    3 days ago


    Noida, Uttar Pradesh, India Compunnel Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    We are looking for skilled PySparkDevelopers to design and develop scalable data processing solutions. Therole involves working with big data platforms, building ETL pipelines,and collaborating with cross-functional teams to ensure data availability,performance, and quality.Qualification:BE / B. TechLocation:Pune,Key ResponsibilitiesDesign, develop, and...

  • Python SDET

    2 weeks ago


    Noida, Uttar Pradesh, India Iris Software Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Must have:Minimum 5+ years of experience in a QE/ Agile environment with a focus on technical, automated validations leveraging variety of tools/ technologies/ frameworks.Undergraduate degree in TechnologyVery good Python-coding skills, with focus on Spark/PySparkStrong QA or Development background with focus on delivering excellent, well-tested, quality...

  • Data Engineer

    2 weeks ago


    Noida, Uttar Pradesh, India Source-Right Full time ₹ 5,00,000 - ₹ 12,00,000 per year

    Position: Data Engineer (AWS QuickSight, Glue, PySpark) (Noida) (CE46SF RM 3386)Education Required: Bachelor's / Masters / PhD:Bachelor's or master's in computer science, Statistics, Mathematics, Data Science, EngineeringAWS certification (e.g., AWS Certified Data Analytics – Specialty, AWS Certified Developer)Must Have SkillsProficiency in AWS cloud...


  • Noida, Uttar Pradesh, India NTT DATA North America Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Business Consulting-Technical analyst with ETL,GCP using Pyspark to join our team in Noida, Uttar Pradesh (IN-UP), India...


  • Noida, Uttar Pradesh, India Algoworks Technologies Pvt. Ltd Full time ₹ 27,00,000 - ₹ 32,40,000 per year

    Job SummaryWe are seeking a highly skilled Data Engineer with a strong background in Databricks and a passion for data engineering. The ideal candidate will have extensive experience in designing and implementing data solutions using various technologies, including PySpark, Spark SQL, and Power BI. This role requires a proactive individual who can work...


  • Noida, Uttar Pradesh, India EXL Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    DescriptionJob DescriptionJob Title: Data Engineer - PySpark, Python, SQL, Git, AWS Services – Glue, Lambda, Step Functions, S3, Athena. Job Description:We are seeking a talented Data Engineer with expertise in PySpark, Python, SQL, Git, and AWS to join our dynamic team. The ideal candidate will have a strong background in data engineering, data...

  • Python Developer

    4 days ago


    Noida, Uttar Pradesh, India Talentoj Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    JOB DESCRIPTIONPosition-DeveloperFunction-DeliveryTechnology-PythonExperience-3-5 yearsROLE & RESPONSIBILITIES:A Python Developer is responsible for writing Python code using pre-defined packages for data and machine learning purposes. They work with frameworks like Pandas, TensorFlow, PyTorch, and Scikit-Learn, automate workflows, and collaborate with data...

  • Python Developer

    1 day ago


    Noida, Uttar Pradesh, India Big Oh Tech Full time ₹ 8,00,000 - ₹ 18,00,000 per year

    Looking for a Python Developer with strong hands-on experience in Python, NumPy, REST API development, MongoDB, Redis, SQL, PostgreSQL, and GitHub. A candidate should be proficient in building scalable backend services, integrating multiple databases, and working in an agile environment.ResponsibilitiesDevelop and maintain Python-based backend...


  • Noida, Uttar Pradesh, India Topcoder Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Primary skill: Azure Databricks, SQL, PySparkAdditional skills: Power BIDetailed JD: -Extensive experience in designing and building data pipelines within an Azure Databricks Lakehouse environment.Strong understanding and practical experience with Spark and other big data technologies.Integrated spark jobs in Databricks to read and write structured, batch...