Business Consulting-Technical analyst with ETL,GCP using Pyspark

2 days ago

Pune, Maharashtra, India NTT DATA, Inc. Full time ₹ 5,00,000 - ₹ 8,00,000 per year

You bring systems design experience with the ability to architect and explain complex systems interactions, data flows, common interfaces and APIs.

You bring a deep understanding of and experience with software development and programming languages such as Java/Kotlin, and Shell scripting.

You have hands-on experience with the following technologies as a senior software developer: Java/Kotlin, Spring, Spring Boot, Wiremock, Docker, Terraform, GCP services (Kubernetes, CloudSQL, PubSub, Storage, Logging, Dashboards), Oracle & amp; Postgres, SQL, PgWeb, Git, Github & amp; Github Actions, GCP Professional Data Engineering certification

Data Pipeline Development:

Designing, implementing, and optimizing data pipelines on GCP using PySpark for efficient and scalable data processing.

ETL Workflow Development:

Building and maintaining ETL workflows for extracting, transforming, and loading data into various GCP services.

GCP Service Utilization:

Leveraging GCP services like BigQuery, Cloud Storage, Dataflow, and Dataproc for data storage, processing, and analysis.

Data Transformation:

Utilizing PySpark for data manipulation, cleansing, enrichment, and validation.

Performance Optimization:

Ensuring the performance and scalability of data processing jobs on GCP.

Collaboration:

Working with data scientists, analysts, and other stakeholders to understand data requirements and translate them into technical solutions.

Data Quality and Governance:

Implementing and maintaining data quality standards, security measures, and compliance with data governance policies on GCP.

Troubleshooting and Support:

Diagnosing and resolving issues related to data pipelines and infrastructure.

Staying Updated:

Keeping abreast of the latest GCP services, PySpark features, and best practices in data engineering.

Required Skills:

GCP Expertise: Strong understanding of GCP services like BigQuery, Cloud Storage, Dataflow, and Dataproc.

PySpark Proficiency: Demonstrated experience in using PySpark for data processing, transformation, and analysis.

Python Programming: Solid Python programming skills for data manipulation and scripting. Data Modeling and ETL: Experience with data modeling, ETL processes, and data warehousing concepts.

SQL: Proficiency in SQL for querying and manipulating data in relational databases.

Big Data Concepts: Understanding of big data principles and distributed computing concepts.

Communication and Collaboration: Ability to effectively communicate technical solutions and collaborate with cross-functional teams

  • GCP Lead

    1 week ago

    Pune, Maharashtra, India Datametica Full time US$ 90,000 - US$ 1,20,000 per year

    Job descriptionLocation: PuneExp: 6-10 YearsWe are looking to hire a Data Engineer with strong hands-on experience in SQL, ETL, Pyspark, GCP,Required Past Experience:3+ Years experience in ETL pipelines experience along with any GCP cloud experience along with pyspark.Experience in Shorked on at least one development project from ETL Perspective.File...

  • Industry Consulting Snr. Consultant

    6 days ago

    Pune, Maharashtra, India NTT Data Full time

    Job DescriptionNTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Industry Consulting Snr. Consultant to join our team in Pune, Mahrshtra (IN-MH), India (IN).You bring systems design...

  • Pyspark Developer

    1 week ago

    Pune, Maharashtra, India Tech Mahindra Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Pyspark DeveloperRequirements:Mandatory: Primary skill: Pyspark, Data Engineering, Azure Data BricksGood Experience of Hadoop, Hive, and Cloudera/ Azure/GCP 3+ years of experience in the design and implementation of Big Data systems using PySpark, database migration, transformation and integration solutions for any Data warehousing project.Must have...

  • GCP Data Developer- BigQuery, Pyspark

    4 days ago

    Pune, Maharashtra, India Swift Strategic Solutions Inc Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Our technology services client is seeking multipleGCP Data Developer- BigQuery, Pysparkto join their team on a Full Time basis. Below are further details about the role:Role: GCP Data Developer- BigQuery, PysparkExperience: 5-8 YearsLocation: Pune, HyderabadNotice Period: Immediate- 30 DaysMandatory Skills: GCP, BigQuery, Pyspark, SQLJob...

  • Senior Business Analytics Specialist

    2 weeks ago

    Pune, Maharashtra, India beBeeData Full time ₹ 1,20,00,000 - ₹ 2,10,00,000

    Job OpportunityThis role involves leading business analysis to drive requirements gathering, solution design documentation, and test scenario execution.The ideal candidate will have 5-8 years of experience in data analysis, business knowledge, and technical product development. They should be able to communicate effectively with stakeholders and team...

  • Data Architecture Specialist

    5 days ago

    Pune, Maharashtra, India beBeeData Full time ₹ 20,00,000 - ₹ 27,00,000

    Job Title:Senior Data Architect Consultant Job Description:We are seeking an experienced Senior Data Architect Consultant to lead our data architecture initiatives in Pune, India. The ideal candidate will have expertise in systems design, software development, and programming languages such as Java/Kotlin, and Shell scripting. Key Responsibilities:Designing,...

  • Data Engineer – Python

    3 days ago

    Pune, Maharashtra, India CBNsense Full time

    Job Title Data Engineer - Python PySpark Experience 3 to 10 years Number of Position 4 Location Pune Client - USA Work Model Full-time hybrid work 3 days week in office Client location Job Type Contract-to-Hire Client Domain Utilities Energy About the Role We are seeking a highly skilled Data Engineer with hands-on expertise in...

  • Pyspark Lead

    1 week ago

    Pune, Maharashtra, India Wipro Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Position OverviewWe are seeking a skilled and experienced Senior PySpark Developer with expertise in Apache spark, Spark Batch, and Spark Streaming to join our dynamic team. The ideal candidate will design, develop, and maintain high-performance, scalable applications for processing large-scale data in batch and real-time environments.Required Skills and...

  • Principal Consultant- Databricks Developer and ETL

    1 week ago

    Pune, Maharashtra, India Genpact Full time ₹ 5,00,000 - ₹ 10,00,000 per year

    Job Description Genpact (NYSE: G) is a global professional services and solutions firm delivering outcomes that shape the future. Our 125,000 people across 30 countries are driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients. Powered by our purpose - the relentless pursuit of a world that works...

  • Officer-Pyspark Developer

    4 days ago

    Pune, Maharashtra, India Citi Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    AtCitiwe're not just building technology, we're building the future of banking. Encompassing a broad range of specialties, roles, and cultures, our teams are creating innovations used across the globe. Citi is constantly growing and progressing through our technology, with laser focused on evolving the ways of doing things. As one of the world's most global...