Technical Lead-Adf/Spark/Kafka

2 weeks ago

Bangalore City, India Wipro Full time

Tech Lead Python, Spark/Scala, Kafka and Cloud storage on Azure Cloud platform Role Sr.Data Engineer Years of Experience 5-7 Yrs Preferred Location HYD 2 Shift Timing (IST) 11:00 AM 08:30 PM Short Description Develop batch and real-time streaming data pipelines using Python, Spark/Scala, Kafka and Cloud storage on the Azure Cloud platform, as well as implement and integrate APIs using Microservice architecture in SpringBoot/Java Engagement Duration (in Months) 18 Engagement & Project Overview IDStrat Transformation and ClinicalInsights Primary Responsibilities Build batch and real-time streaming data pipelines using Python, Spark/Scala, Kafka and Cloud storage on Azure Cloud platform End-to-End ownership, implement to production deployment Schedule ETL jobs, monitor the pipeline, resolving the issues Contribute in designing the data solutions including integration, transformation and loading Must Have Skills Should have minimum 5-7 years of experience in Data Engineering and ETL Minimum 2 years of experience in building data solutions on Azure Cloud Experience in developing batch and real-time streaming data pipelines using Python, Spark/Scala, Kafka and Cloud storage on Azure Cloud platform Hands on experience in implementing and integrating APIs using Microservice architecture in SpringBoot/Java Experience in Azure Data Factory to streamline orchestration and data movement tasks and integrate with Databricks and optimize Spark ETL jobs Experience in integrating, transforming, and consolidating data from various structured and unstructured data Experience with Azure services such as ADLS, Databricks, AKS, Azure EventHub, CosmosDB Strong hold on SQL and NoSQL databases Translate complex functional and technical requirements into detailed design Expert in CI/CD including automating data pipelines, monitoring data processes, and managing deployments Propose best practices/standards Excellent communication skills. Nice To Have Skills Snowflake

  • Kafka Architect

    2 weeks ago

    bangalore, India Ultrabot Innovations Full time

    Job Description :As a Kafka Architect specializing in Spark and Apache Server, you will play a key role in designing, architecting, and implementing real-time data streaming solutions using Apache Kafka, Apache Spark, and related technologies. You will work closely with our clients and internal teams to understand business requirements, design robust...

  • Kafka Architect

    2 months ago

    Bangalore/Pune/Kolkata/Chennai, IN Ultrabot Innovations Full time

    Job Description :As a Kafka Architect specializing in Spark and Apache Server, you will play a key role in designing, architecting, and implementing real-time data streaming solutions using Apache Kafka, Apache Spark, and related technologies. You will work closely with our clients and internal teams to understand business requirements, design robust...

  • Kafka Architect

    3 weeks ago

    Bangalore/Pune/Kolkata/Chennai, India Ultrabot Innovations Full time

    Job Description :As a Kafka Architect specializing in Spark and Apache Server, you will play a key role in designing, architecting, and implementing real-time data streaming solutions using Apache Kafka, Apache Spark, and related technologies. You will work closely with our clients and internal teams to understand business requirements, design robust...

  • Kafka Architect

    2 months ago

    Bangalore/Pune/Kolkata/Chennai, India Ultrabot Innovations Full time

    Job Description :As a Kafka Architect specializing in Spark and Apache Server, you will play a key role in designing, architecting, and implementing real-time data streaming solutions using Apache Kafka, Apache Spark, and related technologies. You will work closely with our clients and internal teams to understand business requirements, design robust...

  • Kafka Architect

    4 weeks ago

    Bangalore/Pune/Kolkata/Chennai, IN Ultrabot Innovations Full time

    Job Description :As a Kafka Architect specializing in Spark and Apache Server, you will play a key role in designing, architecting, and implementing real-time data streaming solutions using Apache Kafka, Apache Spark, and related technologies. You will work closely with our clients and internal teams to understand business requirements, design robust...

  • Technical Lead

    4 weeks ago

    bangalore, India CHUBB Full time

    Design and implement data pipelines using best practices and industry-leading tools like Databricks and Azure Data Factory (ADF) Extract, transform, and load large datasets from various sources, ensuring data quality and integrity Utilize Python and Spark to perform complex data manipulations and aggregations Write optimized SQL queries to interact...

  • Lead Data Engineering

    3 weeks ago

    Bangalore City, India Apex Systems Full time

    Location: Bangalore and Chennai Over 8 years of experience in Data Engineering leading the design, development and implementation of data pipelines, ETL processes and data models Lead and mentor a team of data engineers, fostering continuous learning and professional development in data technologies. Proficiency in Data Engineering, Data Warehousing, Data...

  • Kafka Technical Lead

    3 weeks ago

    bangalore, India TEKsystems Global Services in India Full time

    We are seeking a highly skilled AWS Senior Technical Lead having an experience of 10+ years with Kafka cluster setup, implementation and a strong hands-on background on Data Streaming, Engineering and Computing. As an AWS Senior Lead, you will play a crucial role in designing, developing, and implementing cloud-based solutions on the AWS platform. The ideal...

  • Kafka Technical Lead

    2 months ago

    bangalore, India TEKsystems Global Services in India Full time

    We are seeking a highly skilled AWS Senior Technical Lead having an experience of 10+ years with Kafka cluster setup, implementation and a strong hands-on background on Data Streaming, Engineering and Computing. As an AWS Senior Lead, you will play a crucial role in designing, developing, and implementing cloud-based solutions on the AWS platform. The ideal...

  • Kafka Technical Lead

    2 months ago

    bangalore, India TEKsystems Global Services in India Full time

    We are seeking a highly skilled AWS Senior Technical Lead having an experience of 10+ years with Kafka cluster setup, implementation and a strong hands-on background on Data Streaming, Engineering and Computing. As an AWS Senior Lead, you will play a crucial role in designing, developing, and implementing cloud-based solutions on the AWS platform. The ideal...

  • Lead Kafka developer with Cloudera

    3 months ago

    bangalore, India Anicalls (Pty) Ltd Full time

    Candidate should have:- Thorough understanding of Hadoop, Spark, and ecosystem components.- Knowledge of Linux and shell scripting is a must.- Sound knowledge on Scala or JAVA.- Experience working with a Streaming platform specifically KAFKA- Knowledge of AWS and Google Cloud Platform and their various components are preferable.- Sound knowledge on Spark,...

  • Technical Lead

    1 week ago

    Bangalore, Karnataka, India Suitable.AI Full time

    Skills required : - Minimum 2+ years in Data Engineering- Last 5 years of experience should be from consulting companies- Skillsets - (Java/Scala/Python) + Big Data stack+ DSA + cloud experience- Proven track record of building and shipping large-scale engineering products and/or knowledge of cloud infrastructure such as GCP/AWS preferred- Must have strong...

  • Technical Lead

    6 days ago

    bangalore, India Suitable.AI Full time

    Skills required : - Minimum 2+ years in Data Engineering - Last 5 years of experience should be from consulting companies - Skillsets - (Java/Scala/Python) + Big Data stack+ DSA + cloud experience - Proven track record of building and shipping large-scale engineering products and/or knowledge of cloud infrastructure such as GCP/AWS preferred - Must...

  • Lead Data Engineer

    2 weeks ago

    bangalore, India CHUBB Full time

    Design and implement data pipelines using best practices and industry-leading tools like Databricks and Azure Data Factory (ADF) Extract, transform, and load large datasets from various sources, ensuring data quality and integrity Utilize Python and Spark to perform complex data manipulations and aggregations Write optimized SQL queries to interact...

  • Data Lead Engineer

    4 weeks ago

    Bangalore/Delhi, IN Squareroot Consulting Pvt Ltd Full time

    Job Description :We are seeking a highly skilled and motivated Data Lead with expertise in Spark, Scala, Kafka, Big Data, and Batch Processing. As a Data Lead, you will play a key role in leading and managing the end-to-end data processing pipeline, ensuring the reliability, scalability, and efficiency of our data systems.Experience : 6+ YearsLocation :...

  • Data Lead Engineer

    2 months ago

    Bangalore/Delhi, IN Squareroot Consulting Pvt Ltd Full time

    Job Description :We are seeking a highly skilled and motivated Data Lead with expertise in Spark, Scala, Kafka, Big Data, and Batch Processing. As a Data Lead, you will play a key role in leading and managing the end-to-end data processing pipeline, ensuring the reliability, scalability, and efficiency of our data systems.Experience : 6+ YearsLocation :...

  • Data Lead Engineer

    2 months ago

    Bangalore/Delhi, India Squareroot Consulting Pvt Ltd Full time

    Job Description : We are seeking a highly skilled and motivated Data Lead with expertise in Spark, Scala, Kafka, Big Data, and Batch Processing. As a Data Lead, you will play a key role in leading and managing the end-to-end data processing pipeline, ensuring the reliability, scalability, and efficiency of our data systems.Experience : 6+ YearsLocation :...

  • Data Lead Engineer

    3 weeks ago

    Bangalore/Delhi, India Squareroot Consulting Pvt Ltd Full time

    Job Description : We are seeking a highly skilled and motivated Data Lead with expertise in Spark, Scala, Kafka, Big Data, and Batch Processing. As a Data Lead, you will play a key role in leading and managing the end-to-end data processing pipeline, ensuring the reliability, scalability, and efficiency of our data systems.Experience : 6+ YearsLocation :...

  • Technical Lead

    4 weeks ago

    Bangalore Urban, India Wipro Full time

    Tech Lead-Python, Spark/Scala, Kafka and Cloud storage on Azure Cloud platform RoleSr.Data EngineerYears of Experience5-7 YrsPreferred LocationHYD 2Shift Timing (IST)11:00 AM - 08:30 PMShort DescriptionDevelop batch and real-time streaming data pipelines using Python, Spark/Scala, Kafka and Cloud storage on the Azure Cloud platform, as well as implement and...

  • Technical Lead

    4 weeks ago

    Bangalore Urban, India Wipro Full time

    Tech Lead-Python, Spark/Scala, Kafka and Cloud storage on Azure Cloud platform RoleSr.Data EngineerYears of Experience5-7 YrsPreferred LocationHYD 2Shift Timing (IST)11:00 AM - 08:30 PMShort DescriptionDevelop batch and real-time streaming data pipelines using Python, Spark/Scala, Kafka and Cloud storage on the Azure Cloud platform, as well as implement and...