▷ (Only 24h Left) GCP Data Lead

3 weeks ago

Bengaluru, India Cygnus Professionals Inc. Full time
Role: Google Cloud Platform (GCP) Data Lead

Location: Bangalore, India (Day 1 Onsite)

Job Type: Contract

Job Description:

- Strong GCP experience- Prior experience in building pipelines using dataflow, data fusion, data proc, etc- Very strong python coding skill- Experience in optimising GCP resources for cost and performance- Experience in containerisation of applications- CI/CD using GIThub actions- Exposure to DataPlex, Data Catalog, and metadata management- GCP IAM and role management- Design and implement end-to-end data architectures leveraging GCP services (e.g., Big Query, Cloud Storage, Dataflow, Pub/Sub, Cloud Composer) for large-scale data ingestion and processing.- Build and optimize large-scale data pipelines using Apache Spark on GCP (via Dataproc or other Spark services). Ensure high performance and scalability in Spark-based data processing workloads.- Develop and manage scalable data ingestion pipelines for structured and unstructured data using tools like Cloud Dataflow, Cloud Pub/Sub, and Apache Spark.- Provide guidance for designing secure, scalable, and efficient data solutions on the Google Cloud Platform, integrating with on-premise/cloud systems like SAP S/4HANA.- Implement both real-time streaming and batch processing pipelines using Apache Spark, Dataflow, and other GCP services to meet business requirements.- Implement data governance, access controls, and security best practices to ensure the integrity, confidentiality, and compliance of data across systems.- Collaborate with business stakeholders, data scientists, and engineering teams to define data requirements, ensuring the architecture aligns with business goals.- Optimize Apache Spark jobs for performance, scalability, and cost-efficiency, ensuring that the architecture can handle growing data volumes.- Provide technical leadership to the data engineering team, mentoring junior engineers in data architecture, Apache Spark development, and GCP best practices.

Technical Expertise:

- Expert-level programming proficiency in Python, Java, and Scala.- Extensive hands-on experience with big data technologies, including Spark, Hadoop, Hive, Yarn, MapReduce, Pig, Kafka, and PySpark.- Proficient in Google Cloud Platform services, such as BigQuery, Dataflow, Cloud Storage, Dataproc, and Cloud Composer Google Pub/Sub, and Google Cloud Functions.Expertise in Apache Spark for both batch and real-time processing, as well as proficiency in Apache Beam, Hadoop, or other big data frameworks. Experienced in using Cloud SQL, BigQuery, and Looker Studio (Google Data Studio) for cloud-based data solutions.- Skilled in orchestration and deployment tools like Cloud Composer, Airflow, and Jenkins for continuous integration and deployment (CI/CD).- Expertise in designing and developing integration solutions involving Hadoop/HDFS, real-time systems, data warehouses, and analytics solutions.- Experience with DevOps practices, including version control (Git), CI/CD pipelines, and infrastructure-as-code (e.g., Terraform, Cloud Deployment Manager). Strong background in working with relational databases, NoSQL databases, and in-memory databases.- Experience managing large datasets within Data Lake and Data Fabric architectures.- Strong knowledge of security best practices, IAM, encryption mechanisms, and compliance frameworks (GDPR, HIPAA) within GCP environments.- Experience in implementing data governance, data lineage, and data quality frameworks.- In-depth knowledge of web technologies, application programming languages, OLTP/OLAP technologies, data strategy disciplines, relational databases, data warehouse development, and big data solutions.- Led end-to-end processes for the design, development, deployment, and maintenance of data engineering projects.- Excellent debugging and problem-solving skills.- Retail and e-commerce domain knowledge is a plus.- Positive attitude with strong analytical skills and the ability to guide teams effectively.

Preferred Qualifications:

- GCP Certifications: Such as Professional Data Engineer / Apache Spark and Python certifications.- Experience with Data visualization tools like Tableau, Power BI etc.

Nice to have (but not mandatory):

- Experience in integration with S4 Hana- Experience with integration with Salesforce

  • GCP Lead Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    About the RoleWe are seeking a skilled GCP Lead Engineer to join our team at NTT DATA. This role is ideal for someone with expertise in designing and implementing Google Cloud Platform (GCP) solutions.Key ResponsibilitiesAssess, design, and implement GCP solutions, including compute, network, storage, identity, security, and disaster recovery.Develop and...

  • Data Engineer

    3 weeks ago

    Bengaluru, India Tech Mahindra Full time

    Tech Mahindra is #hiring for GCP Data Engineer .If you're an experienced professional looking for an exciting new challenge.We want to hear from you! Please apply to with your resume explaining your experience and qualifications.Location: Pan IndiaGCP Data Engineer : U2 – up to 3 yearsPrimary Skill : Big Querry, Data flow, Data Composer, SQL, Python.•...

  • GCP Lead Engineer

    1 month ago

    Bengaluru, India NTT DATA Full time

    Job Description Req ID: 298250 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a GCP Lead Engineer to join our team in Bangalore, Karnātaka (IN-KA), India (IN).Mandatory...

  • GCP Lead Engineer

    2 months ago

    Bengaluru, India NTT DATA Services Full time

    Req ID: 298250 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a GCP Lead Engineer to join our team in Bangalore, Karnātaka (IN-KA), India (IN).Mandatory Skills:Technical...

  • Gcp Data Science

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    **Statistical Modeling**: Strong understanding of statistical techniques and experience leveraging BigQuery ML and Vertex AI for model development. **Machine Learning Algorithms**: Experience with GCP's pre-built ML APIs (e.g., Vision AI, Natural Language API) and custom model building using Vertex AI. **Feature Engineering**: Ability to create meaningful...

  • Gcp data lead

    4 weeks ago

    Bengaluru, India Cygnus Professionals Inc. Full time

    Role: Google Cloud Platform (GCP) Data LeadLocation: Bangalore, India (Day 1 Onsite)Job Type: ContractJob Description:Strong GCP experiencePrior experience in building pipelines using dataflow, data fusion, data proc, etcVery strong python coding skillExperience in optimising GCP resources for cost and performanceExperience in containerisation of...

  • GCP Data Lead

    4 weeks ago

    Bengaluru, India Cygnus Professionals Inc. Full time

    Role: Google Cloud Platform (GCP) Data LeadLocation: Bangalore, India (Day 1 Onsite)Job Type: ContractJob Description:Strong GCP experiencePrior experience in building pipelines using dataflow, data fusion, data proc, etcVery strong python coding skillExperience in optimising GCP resources for cost and performanceExperience in containerisation of...

  • GCP Data Lead

    4 weeks ago

    Bengaluru, India Cygnus Professionals Inc. Full time

    Role: Google Cloud Platform (GCP) Data LeadLocation: Bangalore, India (Day 1 Onsite)Job Type: ContractJob Description:Strong GCP experiencePrior experience in building pipelines using dataflow, data fusion, data proc, etcVery strong python coding skillExperience in optimising GCP resources for cost and performanceExperience in containerisation of...

  • GCP Data Lead

    4 weeks ago

    Bengaluru, India Cygnus Professionals Inc. Full time

    Role: Google Cloud Platform (GCP) Data LeadLocation: Bangalore, India (Day 1 Onsite)Job Type: ContractJob Description:Strong GCP experiencePrior experience in building pipelines using dataflow, data fusion, data proc, etcVery strong python coding skillExperience in optimising GCP resources for cost and performanceExperience in containerisation of...

  • GCP Data Lead

    4 weeks ago

    Bengaluru, India Cygnus Professionals Inc. Full time

    Role: Google Cloud Platform (GCP) Data LeadLocation: Bangalore, India (Day 1 Onsite)Job Type: ContractJob Description:Strong GCP experiencePrior experience in building pipelines using dataflow, data fusion, data proc, etcVery strong python coding skillExperience in optimising GCP resources for cost and performanceExperience in containerisation of...

  • Data Architect(Gcp)

    3 months ago

    Bengaluru, Karnataka, India 66degrees Full time

    Overview of 66degrees 66degrees is a leading consulting and professional services company specializing in developing AI-focused, data-led solutions leveraging the latest advancements in cloud technology. With our unmatched engineering capabilities and vast industry experience, we help the world's leading brands transform their business challenges into...

  • ▷ (Only 24h Left) Lead Engineer

    1 month ago

    Bengaluru, India NeerInfo Solutions Full time

    We’re Hiring! Join a Leading Fortune IT Company!Job Title: Engineering Head (Principal - Architecture)Experience Required: 16-19 YearsLocation: Chennai / BangalorePosition Type: Full-TimeResponsibilities:- Lead and mentor a high-performing engineering team, fostering a culture of ownership, innovation, and problem-solving.- Define and drive the company's...

  • Sr Data Engineer | 6+Yrs | Bangalore

    2 months ago

    Bengaluru, India ByteSprint Talent Search LLP Full time

    Job DescriptionSr Data Engineer 6+YrsHybrid ModelBangalore locationMandatory skills: Any cloud(AWS,Azure, GCP)Pyspark or spark with scala combination.Python, SQLserver Role Overview:Develops and maintains scalable data pipelines and builds out new API integrations to support continuing increases in data volume and complexity.Collaborates with analytics and...

  • Data Engineer

    2 months ago

    Bengaluru, India StockGro Full time

    StockGro is a mobile-first cross-platform (Android & iOS, Mobile + Web App) Fintech product that’s empowering 25 million+ users to master the art of trading and investment in a risk-free and gamified manner. At StockGro - India’s First and Largest Social Investment Platform, users indulge in Social Investing and learn various trading strategies by...

  • Lead engineer

    2 weeks ago

    Bengaluru, India Live Connections Full time

    Lead Engineer - GCP Data ProtectionExperience: 12 to 16 YearsLocation: Bangalore / HyderabadNotice Period: Maximum 30 DaysOrganization: One of the leading GCCsCTC: 60 LPA or 30% on current CTC, whichever is lower.We are looking for a Lead Engineer - GCP Data Protection to spearhead cutting-edge cloud transformation initiatives. This role is ideal for...

  • (Only 24h Left) Lead Data Engineer

    5 days ago

    Bengaluru, India Publicis Groupe Full time

    Epsilon is a global data, technology and services company that powers the marketing and advertising ecosystem. For decades, we’ve provided marketers from the world’s leading brands the data, technology and services they need to engage consumers with 1 View, 1 Vision and 1 Voice. 1 View of their universe of potential buyers. 1 Vision for engaging each...

  • ▷ (Only 24h Left) Data Scientist -1

    3 weeks ago

    Bengaluru, India CommerceIQ Full time

    What You’ll Be Doing- As a Data Scientist, you will work closely with Engineering/Product/Operations teams to build state-of- the-art ML based solutions for B2B SaaS products. This entails not only leveraging advanced techniques for predictions, time-series forecasting, topic modelling, optimisation but deep understanding of business and product too.-...

  • Cloud Data Engineer for Big Data Solutions

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    About the Role:As a Cloud Data Engineer, you will be responsible for designing and implementing large-scale data processing systems using GCP's BigQuery, Dataflow, and Dataproc technologies. Your expertise in ETL/ELT pipelines, data warehousing, and lakehouse solutions will enable us to efficiently process structured and unstructured data.Key...

  • GCP Lead Engineer

    3 weeks ago

    Bengaluru, India Innova ESI Full time

    Company Description Innova ESI is a leading IT partner firm dedicated to providing innovative and sustainable IT solutions for digital transformation. With a global presence in USA, Canada, Mexico, India, and Singapore, we offer end-to-end data solutions to enhance business operations and enable growth. Our joint venture of Enterprise Solutions, Inc., USA &...

  • GCP Lead Engineer

    3 weeks ago

    Bengaluru, India Innova ESI Full time

    Company DescriptionInnova ESI is a leading IT partner firm dedicated to providing innovative and sustainable IT solutions for digital transformation. With a global presence in USA, Canada, Mexico, India, and Singapore, we offer end-to-end data solutions to enhance business operations and enable growth. Our joint venture of Enterprise Solutions, Inc., USA &...