Pyspark Developer

2 days ago


Noida, Uttar Pradesh, India Zigsaw Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Roles & Responsibilities:
Design development and implementation of performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR
Writing reusable testable and efficient code
Integration of data storage solutions in spark & ndash especially with AWS S3 object storage.
Performance tuning of pySpark scripts.
Need to ensure overall build delivery quality is good and on time delivery is done at all times.
Should be able to handle meetings with customers with ease.
Need to have excellent communication skills to interact with customer.Education/Experience
Technical/Functional Skills (Mandatory skills):
5+ years of experience in programming with python. Strong proficiency in python
Familiarity with functional programming concepts
3+ years of Hands on experience in developing ETL data pipelines using pyspark on AWS EMR
Hands-on experience of JSON processing using python
Good understanding of Spark& rsquo s RDD API
Good understanding of Spark& rsquo s Dataframe and API
Experience in configuring EMR clusters on AWS
Experience and good understanding of Apache Spark Data sources API.
Experience of dealing with AWS S3 object storage from Spark.
Experience in troubleshooting spark jobs. Knowledge of monitoring Spark jobs using Spark UI
Performance tuning of Spark jobs.
Understanding fundamental design principles behind a business processes
 
Nice to have skills:
Knowledge of AWS SDK CLI
Experience of setting up continuous integration/deployment of spark jobs to EMR clusters
Knowledge of scheduling spark applications in AWS EMR cluster.
Understanding the differences between Hadoop MapReduce and Apache Spark
Proficient understanding of code versioning tools as Git SVN
Experience minimum 4-7years


  • Developer - PySpark

    4 days ago


    Noida, Uttar Pradesh, India Compunnel Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    We are looking for skilled PySparkDevelopers to design and develop scalable data processing solutions. Therole involves working with big data platforms, building ETL pipelines,and collaborating with cross-functional teams to ensure data availability,performance, and quality.Qualification:BE / B. TechLocation:Pune,Key ResponsibilitiesDesign, develop, and...


  • Noida, Uttar Pradesh, India Zigsaw Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Roles & Responsibilities: Design, development and implementation of performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR Writing reusable, testable, and efficient code Integration of data storage solutions in spark – especially with AWS S3 object storage. Performance tuning of pySpark scripts. Need to ensure overall build delivery...


  • Noida, Uttar Pradesh, India NTT DATA Global Delivery Services Ltd Full time ₹ 1,00,00,000 - ₹ 2,00,00,000 per year

    Business Consulting-Technical analyst with ETL,GCP using Pyspark Req ID: 335235 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Business Consulting-Technical analyst with...


  • Noida, Uttar Pradesh, India NTT DATA North America Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Business Consulting-Technical analyst with ETL,GCP using Pyspark to join our team in Noida, Uttar Pradesh (IN-UP), India...


  • Noida, Uttar Pradesh, India Topcoder Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Primary skill: Azure Databricks, SQL, PySparkAdditional skills: Power BIDetailed JD: -Extensive experience in designing and building data pipelines within an Azure Databricks Lakehouse environment.Strong understanding and practical experience with Spark and other big data technologies.Integrated spark jobs in Databricks to read and write structured, batch...


  • Noida, Uttar Pradesh, India Tata Consultancy Services Full time ₹ 18,00,000 - ₹ 25,00,000 per year

    Greetings from TCS Walk-in Interview for Python Full Stack DeveloperExperience Range : 4- 6 yearsJob Location: BangaloreJob Description:Must-Have Skills· Strong hands-on experience with Databricks and Apache Airflow for data orchestration.· Expertise in PySpark, SQL, and Python for large-scale data processing.· Proven experience in building and...

  • Data Scientist

    2 days ago


    Noida, Uttar Pradesh, India Growel Softech Pvt. Ltd. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    About the Role : We are looking for an experienced Data Scientist to lead the design and development of AI/ML solutions leveraging Azure Machine Learning and Generative AI frameworks. The ideal candidate will have hands-on expertise in Python, PySpark, Azure ML, and OpenAI, with a proven track record of building, deploying, and optimizing data-driven...

  • Group Manager

    2 weeks ago


    Noida, Uttar Pradesh, India WNS Global Services Pvt Ltd Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Azure Databricks (Python/PySpark, SQL), and GCP Group Manager-Analytics Company Description WNS (Holdings) Limited (NYSE: WNS), is a leading Business Process Management (BPM) company. We combine our deep industry knowledge with technology and analytics expertise to co-create innovative, digital-led transformational solutions with clients across 10...

  • Associate- BIM

    2 weeks ago


    Noida, Uttar Pradesh, India Axtria - Ingenious Insights Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Position SummaryTo be a driven business analyst who can work on complex Analytical problems and help the customer in better business decision making especially in the area of pharma/life sciences (domain).Job ResponsibilitiesEngage with Client to participate in requirement gathering, Status update on work, UAT and be the key partner in the overall...


  • Noida, Uttar Pradesh, India Eice Technology Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Job Opening: Azure Data Engineer Microsoft Fabric SpecialistLocation: Noida, Sector 62 (Hybrid)Experience: 5+ YearsJoin: ImmediateAbout the Role:We are looking for an experienced Azure Data Engineer with strong expertise in Microsoft Fabric and modern data engineering practices. The ideal candidate will be skilled in developing scalable data pipelines,...