Pyspark/Hadoop Developer

2 weeks ago

Bengaluru, India 12542 Citicorp Services India Private Limited Full time
Data Engineer:The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working with tech leads, product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment.Responsibilities:Responsible for design and development of big data solutions. Partner with domain experts, product managers, analyst, and data scientists to develop Big Data pipelines in HadoopResponsible for moving all legacy workloads to cloud platformWork with data scientist to build Client pipelines using heterogeneous sources and provide engineering services for data science applicationsEnsure automation through CI/CD across platforms both in cloud and on-premisesDefine needs around maintainability, testability, performance, security, quality and usability for data platformDrive implementation, consistent patterns, reusable components, and coding standards for data engineering processesConvert SAS based pipelines into languages like PySpark, Scala to execute on Hadoop and non-Hadoop ecosystemsTune Big data applications on Hadoop and non-Hadoop platforms for optimal performanceEvaluate new IT developments and evolving business requirements and recommend appropriate systems alternatives and/or enhancements to current systems by analyzing business processes, systems and industry standards.Applies in-depth understanding of how data analytics collectively integrate within the sub-function as well as coordinates and contributes to the objectives of the entire function.Produces detailed analysis of issues where the best course of action is not evident from the information available, but actions must be recommended/taken.Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.Qualifications:6-9 years of total IT experience3+ years of experience with Hadoop (Cloudera)/big data technologiesKnowledge of the Hadoop ecosystem and Big Data technologies Hands-on experience with the Hadoop eco-system (HDFS, MapReduce, Hive, Pig, Impala, Spark, Kafka, Kudu, Solr)Experience on designing and developing Data Pipelines for Data Ingestion or Transformation using Java or Scala or Python.Experience with Spark programming (pyspark or scala or java)Hands-on experience with Python/Pyspark/Scala and basic libraries for machine learning is required;Exposure to containerization and related technologies ( Docker, Kubernetes)Exposure to aspects of DevOps (source control, continuous integration, deployments, etc.)Proficient in programming in Java or Python with prior Apache Beam/Spark experience a plus.System level understanding - Data structures, algorithms, distributed storage & computeCan-do attitude on solving complex business problems, good interpersonal and teamwork skillsPossess team management experience and have led a team of data engineers and analysts.Experience in Snowflake is a plus.Education:Bachelor’s degree/University degree or equivalent experience------------------------------------------------------Job Family Group:Technology

------------------------------------------------------Job Family:Data Analytics

------------------------------------------------------Time Type:Full time

------------------------------------------------------Citi is an equal opportunity and affirmative action employer.Qualified applicants will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.Citigroup Inc. and its subsidiaries ("Citi”) invite all qualified interested applicants to apply for career opportunities. If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review .

  • Hadoop Developer

    3 weeks ago

    Bengaluru, India Tech Mahindra Full time

    Job Description: Good hands on experience in Hadoop Strong technical knowledge in pyspark Good communication to handle client requirement. Must Have Skills: Hadoop + Pyspark Experience: 4-6 years Notice Period: 0-15 Days Work Timing: Regular Shift

  • Pyspark Developer

    4 weeks ago

    Bengaluru, India Manvision Technologies Full time

    SNRequired InformationDetails1RolePySpark Developer2Required Technical Skill SetPySpark, Python, HDFS, Hadoop, SQL3No of Requirements024Desired Experience Range5+ yearsLocation of RequirementIndiaDesired Competencies (Technical/Behavioral Competency)Must-HaveSound programming knowledge on PySpark & SQL in terms of processing large amount of semi structured &...

  • Pyspark Developer

    3 days ago

    Bengaluru, Karnataka, India Manvision Technologies Full time

    SNRequired InformationDetails1RolePySpark Developer2Required Technical Skill SetPySpark, Python, HDFS, Hadoop, SQL3No of Requirements024Desired Experience Range5+ yearsLocation of RequirementIndiaDesired Competencies (Technical/Behavioral Competency)Must-HaveSound programming knowledge on PySpark & SQL in terms of processing large amount of semi structured &...

  • Hadoop Developer

    1 month ago

    Bengaluru, India Tech Mahindra Full time

    Job Description:Good hands on experience in HadoopStrong technical knowledge in pysparkGood communication to handle client requirement.Must Have Skills: Hadoop + PysparkExperience: 4-6 yearsNotice Period: 0-15 DaysWork Timing: Regular Shift

  • Hadoop Developer

    2 weeks ago

    Bengaluru, India Tech Mahindra Full time

    Job Description:Good hands on experience in HadoopStrong technical knowledge in pysparkGood communication to handle client requirement.Must Have Skills: Hadoop + PysparkExperience: 4-6 yearsNotice Period: 0-15 DaysWork Timing: Regular Shift

  • Hadoop Developer

    3 weeks ago

    Bengaluru, India Tech Mahindra Full time

    Job Description:Good hands on experience in HadoopStrong technical knowledge in pysparkGood communication to handle client requirement.Must Have Skills: Hadoop + PysparkExperience: 4-6 yearsNotice Period: 0-15 DaysWork Timing: Regular Shift

  • Hadoop Developer

    2 weeks ago

    Bengaluru, India Tech Mahindra Full time

    Job Description: Good hands on experience in Hadoop Strong technical knowledge in pyspark Good communication to handle client requirement. Must Have Skills: Hadoop + Pyspark Experience: 4-6 years Notice Period: 0-15 Days Work Timing: Regular Shift

  • Hadoop Developer

    1 month ago

    Bengaluru, India Tech Mahindra Full time

    Job Description:Good hands on experience in HadoopStrong technical knowledge in pysparkGood communication to handle client requirement.Must Have Skills: Hadoop + PysparkExperience: 4-6 yearsNotice Period: 0-15 DaysWork Timing: Regular Shift

  • Pyspark Developer

    4 days ago

    Bengaluru, Karnataka, India Manvision Technologies Full time

    SN Required Information Details 1 Role PySpark Developer 2 Required Technical Skill Set PySpark, Python, HDFS, Hadoop, SQL 3 No of Requirements 02 4 Desired Experience Range 5+ years Location of Requirement India Desired Competencies (Technical/Behavioral Competency) Must-Have Sound...

  • Data Engineer

    2 weeks ago

    Bengaluru, India HNM Solutions Full time

    Note : Only immediate joinerRole : PysparkExperience : 8+ yearsLocation : Hyderabad, ChenniaInterview mode is Walkin driveRequirements :Mandatory skills : (8+ Years of experience in Data engineering with 5+ Years on Pyspark/NoSQL is Mandatory)1. Person should be strong in Pyspark2. Should have hands on in MWAA (Airflow) / AWS EMR(Hadoop, Hive) framework3....

  • Data Engineer

    2 months ago

    Bengaluru, India HNM Solutions Full time

    Note : Only immediate joinerRole : PysparkExperience : 8+ yearsLocation : Hyderabad, ChenniaInterview mode is Walkin driveRequirements :Mandatory skills : (8+ Years of experience in Data engineering with 5+ Years on Pyspark/NoSQL is Mandatory)1. Person should be strong in Pyspark2. Should have hands on in MWAA (Airflow) / AWS EMR(Hadoop, Hive) framework3....

  • Lead PySpark Developer

    3 days ago

    Bengaluru, India LTIMindtree Full time

    Job Description:Primary Skill Set: Lead PySpark DeveloperSeconday Skill Set: SAS toolsLead a team of data engineers in the design and implementation of big data solutions using PySpark.Collaborate with cross-functional teams to understand business needs and develop data-driven solutions.Design and implement scalable and robust data pipelines using...

  • Lead PySpark Developer

    5 days ago

    Bengaluru, India LTIMindtree Full time

    Job Description: Primary Skill Set: Lead PySpark Developer Seconday Skill Set: SAS tools Lead a team of data engineers in the design and implementation of big data solutions using PySpark. Collaborate with cross-functional teams to understand business needs and develop data-driven solutions. Design and implement scalable and robust data pipelines using...

  • Lead PySpark Developer

    3 days ago

    Bengaluru, Karnataka, India LTIMindtree Full time

    Job Description: Primary Skill Set: Lead PySpark Developer Seconday Skill Set: SAS tools Lead a team of data engineers in the design and implementation of big data solutions using PySpark. Collaborate with cross-functional teams to understand business needs and develop data-driven solutions. Design and implement scalable and robust data pipelines using...

  • Lead PySpark Developer

    5 days ago

    Bengaluru, India LTIMindtree Full time

    Job Description: Primary Skill Set: Lead PySpark Developer Seconday Skill Set: SAS tools Lead a team of data engineers in the design and implementation of big data solutions using PySpark. Collaborate with cross-functional teams to understand business needs and develop data-driven solutions. Design and implement scalable and robust data pipelines using...

  • Bigdata + Pyspark

    4 days ago

    Bengaluru, Karnataka, India Impetus Technologies Full time

    BIgdata + Pyspark :Big Data; Hadoop / HDFS,Pyspark,PythonExperience with design and coding across one or more platforms and languages (e.g. , Python/Pyspark/SQL) as appropriateHands-on expertise with application design, software developmentProficient in Big Data technologies Designs, codes, tests, corrects and documents large and/or complex programs and...

  • Bigdata + Pyspark

    4 days ago

    Bengaluru, Karnataka, India Impetus Technologies Full time

    BIgdata + Pyspark : Big Data; Hadoop / HDFS,Pyspark,Python Experience with design and coding across one or more platforms and languages (e.g. , Python/Pyspark/SQL) as appropriateHands-on expertise with application design, software development Proficient in Big Data technologies Designs, codes, tests, corrects and documents large and/or complex programs and...

  • Hadoop Developer

    4 days ago

    Bengaluru, India C5i Full time

    Responsibilities:Design, develop, and maintain scalable data pipelines and systems for data processing.Utilize Hadoop and related technologies to manage large-scale data processing.Perform data ingestion using Sqoop and manage data in Hive using Beeline and HQL.Develop and maintain shell scripts for automation of data processing tasks.Implement full and...

  • Hadoop Developer

    5 days ago

    Bengaluru, India C5i Full time

    Responsibilities:Design, develop, and maintain scalable data pipelines and systems for data processing.Utilize Hadoop and related technologies to manage large-scale data processing.Perform data ingestion using Sqoop and manage data in Hive using Beeline and HQL.Develop and maintain shell scripts for automation of data processing tasks.Implement full and...

  • Python Pyspark Lead

    3 days ago

    Bengaluru, Karnataka, India Virtusa Full time

    Python Pyspark Lead - CREQ187226 Description Design and develop on Hadoop applications,Hands on in developing Jobs in pySpark with Python/ SCALA (Preferred) or Java/ SCALAExperience on Core Java, Experience on Map Reduce programs, Hive programming, Hive queries performance conceptsExperience on source code management with Git repositoriesSecondary...