PySpark/Hadoop Developer

1 month ago

Chennai, India 12542 Citicorp Services India Private Limited Full time

Data Engineer:

The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working with tech leads, product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment.

Responsibilities:

Responsible for design and development of big data solutions. Partner with domain experts, product managers, analyst, and data scientists to develop Big Data pipelines in Hadoop Responsible for moving all legacy workloads to cloud platform Work with data scientist to build Client pipelines using heterogeneous sources and provide engineering services for data science applications Ensure automation through CI/CD across platforms both in cloud and on-premises Define needs around maintainability, testability, performance, security, quality and usability for data platform Drive implementation, consistent patterns, reusable components, and coding standards for data engineering processes Convert SAS based pipelines into languages like PySpark, Scala to execute on Hadoop and non-Hadoop ecosystems Tune Big data applications on Hadoop and non-Hadoop platforms for optimal performance Evaluate new IT developments and evolving business requirements and recommend appropriate systems alternatives and/or enhancements to current systems by analyzing business processes, systems and industry standards. Applies in-depth understanding of how data analytics collectively integrate within the sub-function as well as coordinates and contributes to the objectives of the entire function. Produces detailed analysis of issues where the best course of action is not evident from the information available, but actions must be recommended/taken. Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.

Qualifications:

6-9 years of total IT experience 3+ years of experience with Hadoop (Cloudera)/big data technologies Knowledge of the Hadoop ecosystem and Big Data technologies Hands-on experience with the Hadoop eco-system (HDFS, MapReduce, Hive, Pig, Impala, Spark, Kafka, Kudu, Solr)  Experience on designing and developing Data Pipelines for Data Ingestion or Transformation using Java or Scala or Python. Experience with Spark programming (pyspark or scala or java) Hands-on experience with Python/Pyspark/Scala and basic libraries for machine learning is required; Exposure to containerization and related technologies ( Docker, Kubernetes) Exposure to aspects of DevOps (source control, continuous integration, deployments, etc.) Proficient in programming in Java or Python with prior Apache Beam/Spark experience a plus.  System level understanding - Data structures, algorithms, distributed storage & compute Can-do attitude on solving complex business problems, good interpersonal and teamwork skills Possess team management experience and have led a team of data engineers and analysts.  Experience in Snowflake is a plus.

Education:

Bachelor’s degree/University degree or equivalent experience

------------------------------------------------------

Job Family Group:

Technology

------------------------------------------------------

Job Family:

Data Analytics

------------------------------------------------------

Time Type:

Full time

------------------------------------------------------

Citi is an equal opportunity and affirmative action employer.

Qualified applicants will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.

Citigroup Inc. and its subsidiaries ("Citi”) invite all qualified interested applicants to apply for career opportunities. If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review .

  • Hadoop Developer/PySpark Developer

    1 month ago

    Chennai, India Citi Full time

    Data Engineer:The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working with tech leads, product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment.Responsibilities:Responsible for design and development of big...

  • PySpark/Hadoop Developer

    1 week ago

    Chennai, Tamil Nadu, India 12542 Citicorp Services India Private Limited Full time

    Data Engineer:The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working with tech leads, product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment.Responsibilities:Responsible for design and development of big...

  • PySpark/Hadoop Developer

    3 weeks ago

    chennai, India 12542 Citicorp Services India Private Limited Full time

    Data Engineer: The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working with tech leads, product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment. Responsibilities: Responsible for design and...

  • Hadoop Developer/PySpark Developer

    1 month ago

    Chennai, India Citi Full time

    Data Engineer: The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working with tech leads, product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment. Responsibilities: Responsible for design and development...

  • Hadoop Developer/PySpark Developer

    1 month ago

    chennai, India Citi Full time

    Data Engineer: The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working with tech leads, product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment. Responsibilities: Responsible for design and...

  • Hadoop Developer/PySpark Developer

    1 month ago

    Chennai, India 12542 Citicorp Services India Private Limited Full time

    Data Engineer: The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working with tech leads, product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment. Responsibilities: Responsible for design and development...

  • Hadoop Developer/PySpark Developer

    1 month ago

    chennai, India 12542 Citicorp Services India Private Limited Full time

    Data Engineer: The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working with tech leads, product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment. Responsibilities: Responsible for design and...

  • Big Data Engineer

    3 weeks ago

    Chennai, India Sysvine Technologies Full time

    Experience: 5 - 8 yearsKey Skills: Hadoop, Pyspark, Python, SQL, HiveLocation: ChennaiNotice Period: We prefer candidates who can join immediately or candidates who are on break or candidates who are at end of their notice periodWhat are we looking for?We are looking for experienced Big Data Hadoop Developer expertise with Hadoop, Big Data Ecosystems, Hive,...

  • Pyspark Developer, Pyspark

    7 minutes ago

    Chennai, India Krishpar Technologies Pvt. Ltd Full time

    Dear candidates,We are hiring for one of our client (MNC)Position : Pyspark DeveloperExperience: 4- 6 YaersLocation: Chennai, BangaloreJob Description:Big Data Testing on Hadoop, Hive, etc., using HiveQL, MapReduce or other big data technologies for automated validationImplementing DevOps / CI-CD using Bitbucket, Jenkins, etc.,Implementing Test Automation...

  • Pyspark Developer, Pyspark

    24 hours ago

    chennai, India Krishpar Technologies Pvt. Ltd Full time

    Dear candidates, We are hiring for one of our client (MNC) Position : Pyspark Developer Experience: 4- 6 Yaers Location: Chennai, Bangalore Job Description: Big Data Testing on Hadoop, Hive, etc., using HiveQL, MapReduce or other big data technologies for automated validation Implementing DevOps / CI-CD using Bitbucket, Jenkins, etc., ...

  • Big Data Engineer

    3 weeks ago

    Chennai, India Sysvine Technologies Full time

    Experience: 5 - 8 years Key Skills: Hadoop, Pyspark, Python, SQL, Hive Location: Chennai Notice Period: We prefer candidates who can join immediately or candidates who are on break or candidates who are at end of their notice period What are we looking for? We are looking for experienced Big Data Hadoop Developer expertise with Hadoop, Big Data Ecosystems,...

  • Data Engineer

    2 months ago

    Chennai/Hyderabad/Bangalore, IN HNM Solutions Full time

    Note : Only immediate joinerRole : PysparkExperience : 8+ yearsLocation : Hyderabad, ChenniaInterview mode is Walkin driveRequirements :Mandatory skills : (8+ Years of experience in Data engineering with 5+ Years on Pyspark/NoSQL is Mandatory)1. Person should be strong in Pyspark2. Should have hands on in MWAA (Airflow) / AWS EMR(Hadoop, Hive) framework3....

  • Data Engineer

    2 months ago

    Chennai,Hyderabad,Bangalore, India HNM Solutions Full time

    Note : Only immediate joiner Role : Pyspark Experience : 8+ years Location : Hyderabad, Chennia Interview mode is Walkin drive Requirements : Mandatory skills : (8+ Years of experience in Data engineering with 5+ Years on Pyspark/NoSQL is Mandatory) 1. Person should be strong in Pyspark2. Should have hands on in MWAA (Airflow) / AWS EMR(Hadoop, Hive)...

  • Data Engineer

    3 weeks ago

    Chennai/Hyderabad/Bangalore, India HNM Solutions Full time

    Note : Only immediate joiner Role : Pyspark Experience : 8+ years Location : Hyderabad, Chennia Interview mode is Walkin drive Requirements : Mandatory skills : (8+ Years of experience in Data engineering with 5+ Years on Pyspark/NoSQL is Mandatory) 1. Person should be strong in Pyspark2. Should have hands on in MWAA (Airflow) / AWS EMR(Hadoop, Hive)...

  • Data Engineer

    3 weeks ago

    Chennai/Hyderabad/Bangalore, IN HNM Solutions Full time

    Note : Only immediate joinerRole : PysparkExperience : 8+ yearsLocation : Hyderabad, ChenniaInterview mode is Walkin driveRequirements :Mandatory skills : (8+ Years of experience in Data engineering with 5+ Years on Pyspark/NoSQL is Mandatory)1. Person should be strong in Pyspark2. Should have hands on in MWAA (Airflow) / AWS EMR(Hadoop, Hive) framework3....

  • Pyspark Dev

    1 week ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Pyspark Dev & QA - CREQ188700 DescriptionPyspark Dev & QAMandatory SkillsBIG Data technology mentioned below Hadoop / Big Data (HDFS, PYTHON, SPARK-SQL, MapReduce) with PySparkBuild CI/CD pipelines is required Outstanding coding, debugging and analytical skillsSpark APIs to cleanse, explore, aggregate, transform, store & analyse available dataKnowledge of...

  • PySpark Developers

    1 month ago

    chennai, India Techno Wise Full time

    We are looking for a PySpark developer ( with ETL background ) to be able to design and build solution on one of our customer programs. This to build a data standardized and curation layer that will integrate data across internal and external sources, provide analytical insights and integrate with customer’s critical systems. Roles and...

  • PySpark Developers

    3 weeks ago

    chennai, India Techno Wise Full time

    We are looking for a PySpark developer ( with ETL background ) to be able to design and build solution on one of our customer programs. This to build a data standardized and curation layer that will integrate data across internal and external sources, provide analytical insights and integrate with customer’s critical systems. Roles and...

  • Pyspark Dev

    1 week ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Pyspark Dev & QAMandatory SkillsBIG Data technology mentioned below Hadoop / Big Data (HDFS, PYTHON, SPARK-SQL, MapReduce) with PySparkBuild CI/CD pipelines is required Outstanding coding, debugging and analytical skillsSpark APIs to cleanse, explore, aggregate, transform, store & analyse available dataKnowledge of installing, configuring, debugging and...

  • Apache Pyspark

    1 week ago

    Chennai, Tamil Nadu, India Maneva Consulting Pvt. Ltd Full time

    GreetingsFromManevaJobDescriptionJobTitleApachePysparkLocationChennaiExperience4 10YearsNotice15 30DaysJobRequirements:Mandatory:Primary skill:PysparkEducation: B.Tech/M.Tech/MCA46 years ofexperience in the design and implementation of Big Data systemsusing PySpark database migration transformation and integrationsolutions for any Data...