Python Pyspark Developer

3 days ago


Noida, Uttar Pradesh, India Zigsaw Full time ₹ 20,00,000 - ₹ 25,00,000 per year

Roles & Responsibilities:
Design, development and implementation of performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR
Writing reusable, testable, and efficient code
Integration of data storage solutions in spark – especially with AWS S3 object storage.
Performance tuning of pySpark scripts.
Need to ensure overall build delivery quality is good and on time delivery is done at all times.
Should be able to handle meetings with customers with ease.
Need to have excellent communication skills to interact with customer.Education/Experience
Technical/Functional Skills (Mandatory skills):
5+ years of experience in programming with python. Strong proficiency in python
Familiarity with functional programming concepts
3+ years of Hands on experience in developing ETL data pipelines using pyspark on AWS EMR
Hands-on experience of JSON processing using python
Good understanding of Spark's RDD API
Good understanding of Spark's Dataframe and API
Experience in configuring EMR clusters on AWS
Experience and good understanding of Apache Spark Data sources API.
Experience of dealing with AWS S3 object storage from Spark.
Experience in troubleshooting spark jobs. Knowledge of monitoring Spark jobs using Spark UI
Performance tuning of Spark jobs.
Understanding fundamental design principles behind a business processes
 
Nice to have skills:
Knowledge of AWS SDK CLI
Experience of setting up continuous integration/deployment of spark jobs to EMR clusters
Knowledge of scheduling spark applications in AWS EMR cluster.
Understanding the differences between Hadoop MapReduce and Apache Spark
Proficient understanding of code versioning tools as Git, SVN
Experience minimum 4-7years


  • Pyspark Developer

    3 days ago


    Noida, Uttar Pradesh, India Zigsaw Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Roles & Responsibilities: Design development and implementation of performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR Writing reusable testable and efficient code Integration of data storage solutions in spark & ndash especially with AWS S3 object storage. Performance tuning of pySpark scripts. Need to ensure overall build...

  • Developer - PySpark

    4 days ago


    Noida, Uttar Pradesh, India Compunnel Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    We are looking for skilled PySparkDevelopers to design and develop scalable data processing solutions. Therole involves working with big data platforms, building ETL pipelines,and collaborating with cross-functional teams to ensure data availability,performance, and quality.Qualification:BE / B. TechLocation:Pune,Key ResponsibilitiesDesign, develop, and...


  • Noida, Uttar Pradesh, India Tata Consultancy Services Full time ₹ 18,00,000 - ₹ 25,00,000 per year

    Greetings from TCS Walk-in Interview for Python Full Stack DeveloperExperience Range : 4- 6 yearsJob Location: BangaloreJob Description:Must-Have Skills· Strong hands-on experience with Databricks and Apache Airflow for data orchestration.· Expertise in PySpark, SQL, and Python for large-scale data processing.· Proven experience in building and...


  • Noida, Uttar Pradesh, India NTT DATA Global Delivery Services Ltd Full time ₹ 1,00,00,000 - ₹ 2,00,00,000 per year

    Business Consulting-Technical analyst with ETL,GCP using Pyspark Req ID: 335235 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Business Consulting-Technical analyst with...

  • Python SDET

    2 weeks ago


    Noida, Uttar Pradesh, India Iris Software Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Must have:Minimum 5+ years of experience in a QE/ Agile environment with a focus on technical, automated validations leveraging variety of tools/ technologies/ frameworks.Undergraduate degree in TechnologyVery good Python-coding skills, with focus on Spark/PySparkStrong QA or Development background with focus on delivering excellent, well-tested, quality...


  • Noida, Uttar Pradesh, India NTT DATA North America Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Business Consulting-Technical analyst with ETL,GCP using Pyspark to join our team in Noida, Uttar Pradesh (IN-UP), India...


  • Noida, Uttar Pradesh, India Topcoder Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Primary skill: Azure Databricks, SQL, PySparkAdditional skills: Power BIDetailed JD: -Extensive experience in designing and building data pipelines within an Azure Databricks Lakehouse environment.Strong understanding and practical experience with Spark and other big data technologies.Integrated spark jobs in Databricks to read and write structured, batch...

  • Python Developer

    2 weeks ago


    Noida, Uttar Pradesh, India Zeply Full time ₹ 40,00,000 - ₹ 1,20,00,000 per year

    Company DescriptionWe suggest you enter details here.Role DescriptionThis is a full-time on-site role for a Python Developer located in Noida. The Python Developer will be responsible for developing server-side logic, ensuring high performance and responsiveness to requests from the front-end, and integrating elements developed by front-end developers. The...

  • Python Developer

    2 weeks ago


    Noida, Uttar Pradesh, India EV Software Solutions Private Limited Full time ₹ 2,86,000 - ₹ 13,88,691 per year

    3-6 years of experience as a Python developer.Knowledge of fast APIUnderstanding of frameworks connected to Python, such as flask, Django, etc.Chat GPT API knowledge communicate with Python.Reprogramming existing databases to improve functionality.A deep understanding and multi-process architecture and the threading limitations of Python.Familiarity with...

  • Python Developer

    3 days ago


    Noida, Uttar Pradesh, India Topcoder Full time ₹ 40,00,000 - ₹ 1,20,00,000 per year

    Python Handson with backend development.Skills - Python, AWS Serverless. AWS services (Lambda, S3, API Gateway)Drill down - sql-alchemy, requests, lambdas, API Integration.Strong Python experienceProficient in API writingSolid SQL experience. Knowledge on SQL programmingGood understanding of system design (Low Level & High Level)