PySpark Data Engineer

5 hours ago

Delhi, India ITI Data Full time
Location

:

IndiaType

: Full-timeExperience

: 10 – 13 yearsFunctions

: Consulting, Finance, Information Technology, Big Data EngineeringIndustries

: Capital Markets, Investment Banking, Alternative Investments, Financial Services, Management Consulting, Information Technology and Services, Healthcare

Job Description

We are looking for a PySpark solutions developer and data engineer that is able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building a data standardized and curation needs on Hadoop cluster. This is high visibility, fast-paced key initiative will integrate data across internal and external sources, provide analytical insights, and integrate with the customer’s critical systems.

Key ResponsibilitiesAbility to design, build and unit test applications on

Spark framework on Python

.Build PySpark based applications for both

batch and streaming

requirements, which will require in-depth knowledge on majority of

Hadoop and NoSQL

databases as well.Develop and execute

data pipeline

testing processes and validate business rules and policiesOptimize performance of the built Spark applications in Hadoop using configurations around

Spark Context, Spark-SQL, Data Frame, and Pair RDD's

.Experience in

skilled in Hadoop, Kafka, Scala, Spark.Optimize

performance for data access

requirements by choosing the appropriate native Hadoop file formats (

Avro, Parquet, ORC etc

) and compression codec respectively.Ability to design & build real-time applications using

Apache Kafka & Spark Streaming

Build

integrated solutions

leveraging Unix shell scripting, RDBMS, Hive, HDFS File System, HDFS File Types, HDFS compression codec .

Build

data tokenization

libraries and integrate with Hive & Spark for

column-level obfuscation

Experience in processing large amounts of

structured and unstructured

data, including integrating data from multiple sources.Create and maintain integration and regression testing framework on

Jenkins

integrated with

BitBucket

and/or GIT repositoriesParticipate in the

agile development

process, and document and communicate issues and bugs relative to data standards in

scrum meetingsWork

collaboratively

with onsite and offshore team. Develop & review technical

documentation

for artifacts delivered.Ability to solve

complex data-driven

scenarios and triage towards defects and production issuesAbility to

learn-unlearn-relearn

concepts with an open and analytical mindsetParticipate in

code release

and production deployment

  • PySpark Data Engineer

    3 hours ago

    Delhi, India ITI Data Full time

    Location : IndiaType : Full-timeExperience : 10 – 13 yearsFunctions : Consulting, Finance, Information Technology, Big Data EngineeringIndustries : Capital Markets, Investment Banking, Alternative Investments, Financial Services, Management Consulting, Information Technology and Services, HealthcareJob DescriptionWe are looking for a PySpark...

  • ITI Data | PySpark Data Engineer

    8 hours ago

    Delhi, India ITI Data Full time

    Location:IndiaType: Full-timeExperience: 10 – 13 yearsFunctions: Consulting, Finance, Information Technology, Big Data EngineeringIndustries: Capital Markets, Investment Banking, Alternative Investments, Financial Services, Management Consulting, Information Technology and Services, HealthcareJob DescriptionWe are looking for a PySpark solutions developer...

  • Data Engineer for Azure Synapse and Pyspark Development

    1 day ago

    Delhi, Delhi, India Tata Consultancy Services Full time

    About Tata Consultancy ServicesTata Consultancy Services (TCS) is a global leader in IT services, consulting, and business solutions. With over 50 years of experience, TCS has been at the forefront of driving innovation and growth through technology.Job Title: Data Engineer for Azure Synapse and Pyspark DevelopmentThis role offers an exciting opportunity to...

  • Data Engineer

    3 days ago

    Delhi, India Tata Consultancy Services Full time

    Job DescriptionName of the position: Data engineerLocation : Bangalore, Hyderabad,Mumbai, Pune ,Chennai,NCRSkill Requirements & Experience- Overall 8+ years of work experience in Data Warehouse(DWH) Development- Experience in Azure, Python , SQL , Pyspark- Hands on exp in Azure data factory, data bricks- Exposure and experience to Cosmos- Understanding of...

  • AWS Data Engineer

    4 weeks ago

    delhi, India ITI Data Full time

    Job Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....

  • AWS Data Engineer

    4 months ago

    Delhi, India ITI Data Full time

    Job DescriptionWe are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources. There...

  • AWS Data Engineer

    1 month ago

    delhi, India ITI Data Full time

    Job DescriptionWe are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources. There...

  • Data Warehouse Engineer | Data Warehouse Engineer | delhi

    2 weeks ago

    delhi, India Data Warehouse Engineer Full time

    Experience : 2- 5 YearsPrimary Skills : Strong SQL knowledge, Data warehouse concepts and ETL hands on experience, Azure services like Data Factory, Azure Data Lake, and Good analysis skills.Good to have skills: Power BI, Databricks, Python.

  • Data Warehouse Engineer

    2 weeks ago

    Delhi, India Data Warehouse Engineer Full time

    Experience : 2- 5 YearsPrimary Skills : Strong SQL knowledge, Data warehouse concepts and ETL hands on experience, Azure services like Data Factory, Azure Data Lake, and Good analysis skills.Good to have skills: Power BI, Databricks, Python.

  • Python, Pyspark Data Engineer

    3 days ago

    Delhi, India KC Executive Search Full time

    Technical Stack: Python, Spark, AWS, PySparkResponsibilitiesDevelop and enhance data-processing, orchestration, monitoring, and more by leveraging popular open-source software, AWS, and GitLab automation.Collaborate with product and technology teams to design and validate the capabilities of the data platform Identify, design, and implement process...

  • Pyspark/Python Data Engineer

    4 months ago

    Delhi, India Genpact Full time

    Genpact (NYSE: G) is a global professional services and solutions firm delivering outcomes that shape the future. Our 125,000+ people across 30+ countries are driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients. Powered by our purpose – the relentless pursuit of a world that works better for people – we...

  • Pyspark Hadoop Developer

    1 month ago

    Delhi, India Tata Consultancy Services Full time

    Dear Candidate,Greetings from TCS Human Resources Team!!Job Role: Pyspark Hadoop DeveloperJob Locations: BengaluruExperience Range: 5 to 10 years5+yrs of hands on experience in Hadoop Scala, Pyspark, Hive, Spark , Hive/Impala/SQL with Hadoop eco systemGood Data background. Hadoop is the primary skill and PySpark is secondary experience.Experience in design...

  • Pyspark Hadoop Developer

    1 month ago

    Delhi, India Tata Consultancy Services Full time

    Dear Candidate,Greetings from TCS Human Resources Team!!Job Role: Pyspark Hadoop DeveloperJob Locations: BengaluruExperience Range: 5 to 10 years- 5+yrs of hands on experience in Hadoop Scala, Pyspark, Hive, Spark , Hive/Impala/SQL with Hadoop eco system- Good Data background. Hadoop is the primary skill and PySpark is secondary experience.- Experience in...

  • Python, Pyspark Data Engineer

    3 days ago

    Delhi, India KC Executive Search Full time

    Technical Stack: Python, Spark, AWS, PySparkResponsibilitiesDevelop and enhance data-processing, orchestration, monitoring, and more by leveraging popular open-source software, AWS, and GitLab automation.Collaborate with product and technology teams to design and validate the capabilities of the data platform Identify, design, and implement process...

  • Python +pyspark

    6 months ago

    Delhi, India Nityo Infotech Full time

    2 hours ago **Job Code**: JD-19623 **JOB DESCRIPTION**: Mandatory Skills - Python, Pyspark, DataBricks, SQL Primary Skills: - Hands-on Python, PySpark, Databricks, SQL, AWS (S3, Lambda, EC2, RDS), and CI-CD tools. Job description: - 6 or more years of experience developing, testing, and implementing major Information Technology programs or projects that...

  • PySpark Developer

    3 days ago

    Delhi, India Tata Consultancy Services Full time

    Role : PySpark Developer Technical Skill Set : PySpark, Python, HDFS, Hadoop, SQLLocation : Mumbai, Pune ,Chennai, Banglore , NCR, HyderabadMust-Have :Sound programming knowledge on PySpark & SQL in terms of processing large amount of semi structured & unstructured dataAbility to design data pipelines in end to end mannerKnowledge on Avro, Parquet...

  • PySpark Developer

    2 weeks ago

    Delhi, India Tata Consultancy Services Full time

    Role - PySpark DeveloperExperience- 5 TO 10 YRSLocation- Bangalore / HyderabadMust Have- PysparkDesired Competencies (Technical/Behavioral Competency)Must-Have · Hands-on experience in Pyspark including Dataframe core functions sparkSQL and SparkStreaming. · Minimum 3 years of experience HIVE, Hadoop, Kafka, YARN, HBase & MongoDB · Hands on experience in...

  • PySpark Developer

    2 weeks ago

    Delhi, India Tata Consultancy Services Full time

    Role - PySpark DeveloperExperience - 5 TO 10 YRSLocation - Bangalore / HyderabadMust Have - PysparkDesired Competencies (Technical/Behavioral Competency)Must-Have · Hands-on experience in Pyspark including Dataframe core functions sparkSQL and SparkStreaming. · Minimum 3 years of experience HIVE, Hadoop, Kafka, YARN, HBase & MongoDB · Hands on...

  • Pyspark Developer

    2 weeks ago

    Delhi, India Tata Consultancy Services Full time

    Greetings from Tcs !!!We are conducting interviews for Pyspark Developer .Role: Pyspark DeveloperExperience: 6-10 YearsLocation: BangaloreResponsibility of / Expectations from the Role- Minimum 5 years of PySpark Development experience, especially in Spark SQL and Complex Transformations- Minimum 3 years of Python development experience- Minimum 2 years of...

  • Pyspark Developer

    2 weeks ago

    Delhi, India Tata Consultancy Services Full time

    Greetings from Tcs !!!We are conducting interviews for Pyspark Developer .Role: Pyspark DeveloperExperience: 6-10 YearsLocation:BangaloreResponsibility of / Expectations from the RoleMinimum 5 years of PySpark Development experience, especially in Spark SQL and Complex TransformationsMinimum 3 years of Python development experienceMinimum 2 years of...