Business Consulting-Technical analyst with ETL,GCP using Pyspark
1 day ago
You bring systems design experience with the ability to architect and explain complex systems interactions, data flows, common interfaces and APIs.
You bring a deep understanding of and experience with software development and programming languages such as Java/Kotlin, and Shell scripting.
You have hands-on experience with the following technologies as a senior software developer: Java/Kotlin, Spring, Spring Boot, Wiremock, Docker, Terraform, GCP services (Kubernetes, CloudSQL, PubSub, Storage, Logging, Dashboards), Oracle & amp; Postgres, SQL, PgWeb, Git, Github & amp; Github Actions, GCP Professional Data Engineering certification
Data Pipeline Development:
Designing, implementing, and optimizing data pipelines on GCP using PySpark for efficient and scalable data processing.
ETL Workflow Development:
Building and maintaining ETL workflows for extracting, transforming, and loading data into various GCP services.
GCP Service Utilization:
Leveraging GCP services like BigQuery, Cloud Storage, Dataflow, and Dataproc for data storage, processing, and analysis.
Data Transformation:
Utilizing PySpark for data manipulation, cleansing, enrichment, and validation.
Performance Optimization:
Ensuring the performance and scalability of data processing jobs on GCP.
Collaboration:
Working with data scientists, analysts, and other stakeholders to understand data requirements and translate them into technical solutions.
Data Quality and Governance:
Implementing and maintaining data quality standards, security measures, and compliance with data governance policies on GCP.
Troubleshooting and Support:
Diagnosing and resolving issues related to data pipelines and infrastructure.
Staying Updated:
Keeping abreast of the latest GCP services, PySpark features, and best practices in data engineering.
Required Skills:
GCP Expertise: Strong understanding of GCP services like BigQuery, Cloud Storage, Dataflow, and Dataproc.
PySpark Proficiency: Demonstrated experience in using PySpark for data processing, transformation, and analysis.
Python Programming: Solid Python programming skills for data manipulation and scripting. Data Modeling and ETL: Experience with data modeling, ETL processes, and data warehousing concepts.
SQL: Proficiency in SQL for querying and manipulating data in relational databases.
Big Data Concepts: Understanding of big data principles and distributed computing concepts.
Communication and Collaboration: Ability to effectively communicate technical solutions and collaborate with cross-functional teams
-
Business Consulting-Technical Analyst
6 days ago
Pune, Maharashtra, India NTT DATA Global Delivery Services Ltd Full time ₹ 20,00,000 - ₹ 25,00,000 per yearBusiness Consulting-Technical analyst with ETL,GCP using Pyspark Req ID: 335231 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Business Consulting-Technical analyst with...
-
Gcp Data Engineer
1 week ago
Pune, Maharashtra, India TekWissen Full time ₹ 8,00,000 - ₹ 24,00,000 per yearOverview:TekWissen is a global workforce management provider throughout India and many other countries in the world. The below job opportunity is to one of our clients who is a part of a trusted global innovator of IT and business services headquartered in Tokyo. We help clients transform through consulting, industry solutions, business process services, IT...
-
MarsDevs - AI/ML Engineer - ETL/PySpark
2 weeks ago
Pune, Maharashtra, India Marsdevs Full time ₹ 8,00,000 - ₹ 24,00,000 per yearCompany Description : MarsDevs is a remote-first, offshore digital product development company, committed to delivering high-quality code and long-lasting technical architectures. We specialize in building MVPs, mobile and web applications, and SaaS products for startups and SMBs. Our services include systematic project analysis, UI/UX design,...
-
Pyspark Developer
1 week ago
Pune, Maharashtra, India Tech Mahindra Full time ₹ 15,00,000 - ₹ 25,00,000 per yearPyspark DeveloperRequirements:Mandatory: Primary skill: Pyspark, Data Engineering, Azure Data BricksGood Experience of Hadoop, Hive, and Cloudera/ Azure/GCP 3+ years of experience in the design and implementation of Big Data systems using PySpark, database migration, transformation and integration solutions for any Data warehousing project.Must have...
-
PySpark Databricks Engineer
6 days ago
Pune, Maharashtra, India AVE-Promagne Business Solutions Full time ₹ 8,00,000 - ₹ 24,00,000 per yearPrimary Tech skills : AZURE ,Databricks ,Pyspark, SQL Secondary Tech skills : ADF, Synapse Job Description : Roles : - Experience in Pyspark, SQL, Cloud data warehouse like Azure Synapse, databricks - Experience working with structured and unstructured data - Extensive knowledge of Data Warehousing concepts, strategies, methodologies -...
-
Pyspark Lead
1 week ago
Pune, Maharashtra, India Wipro Full time ₹ 12,00,000 - ₹ 36,00,000 per yearPosition OverviewWe are seeking a skilled and experienced Senior PySpark Developer with expertise in Apache spark, Spark Batch, and Spark Streaming to join our dynamic team. The ideal candidate will design, develop, and maintain high-performance, scalable applications for processing large-scale data in batch and real-time environments.Required Skills and...
-
ETL & Data Architect
1 day ago
Pune, Maharashtra, India Vodafone Full time ₹ 20,00,000 - ₹ 25,00,000 per yearDesign and implement ETL processes and data architecture solutions. Develop and enforce data management policies and best practices. Collaborate with stakeholders to define metadata, data models, and integration strategies. Architect solutions using ETL tools like Ab-Initio or equivalent. Ensure alignment with enterprise architecture policies and...
-
ETL Databricks
1 week ago
Pune, Maharashtra, India Virtusa Full time ₹ 20,00,000 - ₹ 25,00,000 per yearLooking for a offshore Techlead with databricks engineer experience and lead the team from offshore.Develop and maintain a metadata driven generic ETL framework for automating ETL codeDesign, build, and optimize ETL/ELT pipelines using Databricks (PySpark/SQL) on AWS .Ingest data from a variety of structured and unstructured sources (APIs, RDBMS, flat files,...
-
Officer-Pyspark Developer
6 days ago
Pune, Maharashtra, India Citi Full time ₹ 15,00,000 - ₹ 20,00,000 per yearAtCitiwe're not just building technology, we're building the future of banking. Encompassing a broad range of specialties, roles, and cultures, our teams are creating innovations used across the globe. Citi is constantly growing and progressing through our technology, with laser focused on evolving the ways of doing things. As one of the world's most global...
-
GCP Data Engineer
2 weeks ago
Pune, Maharashtra, India, Maharashtra Tata Consultancy Services Full timeWe're Hiring: GCP Data EngineersLocations: Pune & BangaloreExperience: 6+ YearsAre you passionate about building scalable data solutions on Google Cloud Platform? Join our team as a GCP Data Engineer and work on cutting-edge data engineering projects using tools like BigQuery, Dataproc, Pub/Sub, and Airflow.Must-Have Skills:Hands-on expertise with GCP...