Senior PySpark Engineer

2 days ago


delhi, India WiseAnalytics.io Full time
Job Overview
Wise Analytics is seeking a highly skilled and experienced Senior Data Engineer to lead the design and architecture of data engineering solutions for our global clients in the retail and finance domains. The ideal candidate will be an independent contributor capable of managing a technical team, handling stakeholders, and ensuring timely delivery of complex projects. Expertise in technologies such as GCP, Airflow, Databricks, Kafka, and experience with creating data lakes, data warehouses, or lakehouses is essential.
Key Responsibilities
Design and Architecture
Lead the design and architecture of scalable and efficient data engineering solutions on GCP, leveraging technologies such as PySpark, Databricks and Kafka.
Develop data models and ETL/ELT pipelines to support data lakes, data warehouses, and lakehouses for advanced analytics and reporting.
Team Leadership
Manage and mentor a team of data engineers, providing technical guidance and ensuring best practices are followed.
Foster a collaborative and innovative team environment, encouraging continuous learning and improvement.
Project Management
Oversee project timelines, budgets, and resources, ensuring successful delivery of complex data engineering projects.
Collaborate with cross-functional teams to align project goals with business objectives and stakeholder requirements.
Stakeholder Management
Engage with senior stakeholders to understand business needs and translate them into technical requirements.
Present project updates, technical solutions, and data insights to stakeholders in a clear and concise manner.
Technical Expertise
Implement and optimize data processing workflows using Airflow, ensuring reliability and efficiency.
Develop and maintain real-time data streaming solutions with Kafka to support high-velocity data ingestion and processing.
Ensure data quality, security, and governance across all data engineering processes and solutions.
Qualifications
Education
Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
Experience
15+ years of experience in data engineering, with a strong background in designing and implementing data solutions.
Proven experience in a leadership role, managing technical teams and handling complex projects.
Technical Skills
Expertise in cloud platforms, particularly GCP.
Proficiency with PySpark and Databricks for data processing and analytics.
Strong experience with Apache Airflow for orchestrating data workflows.
Extensive knowledge of Apache Kafka for real-time data streaming.
Familiarity with data modeling, ETL/ELT processes, and building data lakes/warehouses/lakehouses.
Soft Skills
Excellent communication and presentation skills, with the ability to articulate technical concepts to non-technical stakeholders.
Strong problem-solving skills and a proactive approach to identifying and addressing challenges.
Ability to manage multiple priorities and deliver high-quality results under tight deadlines.
Preferred Qualifications
Experience in the retail and finance domains.
Certification in GCP or related technologies.
Knowledge of additional data engineering tools and frameworks.
  • Data Engineer

    2 weeks ago


    delhi, India Randstad India Full time

    Job Title - Pyspark DeveloperLocation - Hyderabad, Bangalore, ChennaiExperience - 4 - 8 YearsLooking for Immediate joiners or np with 15 days onlyAs a Data Engineer, you will be responsible for designing, developing, and maintaining data solutions for data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring...


  • delhi, India WiseAnalytics.io Full time

    Job Overview Wise Analytics is seeking a highly skilled and experienced Senior Data Engineer to lead the design and architecture of data engineering solutions for our global clients in the retail and finance domains. The ideal candidate will be an independent contributor capable of managing a technical team, handling stakeholders, and ensuring timely...


  • delhi, India True Tech Professionals Full time

    Senior Data EngineerKey Skills: Snowflake, Python, PysparkExperience: 6+ YearsEmployment Type: Full-timeAvailability: Immediate Joiners PreferredJob Description:We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. The ideal candidate will have a strong background in data engineering, with a focus on Snowflake,...

  • Senior Data Engineer

    1 month ago


    Delhi, India TTEC Digital Full time

    We are seeking a proactive and experienced Senior Data Engineer to join our team. The ideal candidate will have a strong technical background in data engineering and a proven track record of taking ownership of complex data projects. You will leverage your expertise in Azure, PySpark, and Databricks to design, develop, and manage scalable data solutions...

  • Senior Data Engineer

    3 weeks ago


    new delhi, India Insight Global Full time

    Insight Global is looking for a Senior Data Engineer to work remotely in India. Candidates will work IST hours. Candidates will be a part of the clients data modernization project, transitioning from on-premises to cloud infrastructure (Azure). They will utilize programming skills in SQL, PySpark, and Python to support data transformation and migration...

  • AWS Data Engineer

    1 week ago


    Delhi, India Mactores Full time

    Mactores is a trusted leader among businesses in providing modern data platform solutions. Since 2008, Mactores has been enabling businesses to accelerate their value through automation by providing automated, agile, and secure end-to-end data solutions. We collaborate with customers to strategize, navigate, and accelerate an ideal path forward with digital...


  • delhi, India Tata Consultancy Services Full time

    Required Skills: ADF,ADB, PySpark, Scala,MS Azure, SQL Atleast 6+ years of Experience in Data Analytics as Azure Data engineer. Extensive experience on Design and Development of solutions in Azure Data Factory, Databricks, SQL, Synapse, Snowflake, Python, pySpark Good understanding of Data ware house concepts and ETL process's Experience in building...


  • delhi, India Tata Consultancy Services Full time

    Required Skills: ADF,ADB, PySpark, Scala,MS Azure, SQLAtleast 6+ years of Experience in Data Analytics as Azure Data engineer.Extensive experience on Design and Development of solutions in Azure Data Factory, Databricks, SQL, Synapse, Snowflake, Python, pySparkGood understanding of Data ware house concepts and ETL process'sExperience in building pipelines in...

  • Senior Data Engineer

    3 weeks ago


    delhi, India MPM Consulting Full time

    Data EngineerAt MPM we are searching for a highly skilled Data Engineers with a minimum of 5 years of experience in data engineering, specifically within the Life Science/ Pharma domain.PRIMARY RESPONSIBILITIES:Designing, developing, and maintaining data pipelines and systems in Azure Cloud environments, with a strong emphasis on Azure Data Lake, Azure Data...

  • Senior Data Engineer

    2 weeks ago


    delhi, India Truelancer.com Full time

    ✨ Join Us at Truelancer and be part of a team that’s shaping the future!Ready to make an impact? Let’s connect!Job Title: Sr Data EngineerExperience: 5 - 9 yearsLocation: Work from Office, ChennaiMandatory Skills: AWS Data Services + Strong SQL QueriesKey Responsibilities:Design and implement scalable data pipelines using AWS services such as...

  • Data Engineer

    3 weeks ago


    Delhi, India Truelancer.com Full time

    Job Title:Sr Data EngineerExperience:5 - 9 yrsLocation:Work from Office ,ChennaiMandatory Skills:AWS Data Services+Strong in SQL QueriesAs a Senior Data Engineer, you will play a crucial role in designing, developing, and maintaining data pipelines and analytics solutions. You will work with a range of AWS technologies and data tools to ensure our data...

  • Data Engineer

    3 days ago


    delhi, India krtrimaIQ Cognitive Solutions Full time

    Job Title: Data Engineer Location: Bangalore (Hybrid) Experience: 5+ years Key Responsibilities: Design, develop, and maintain scalable data pipelines using PySpark, SQL, Hive, and Hadoop. Collaborate with cross-functional teams to understand data requirements and deliver solutions. Optimize and troubleshoot data workflows to ensure high performance and...

  • Data Engineer

    2 weeks ago


    Delhi, India Deloitte Full time

    Hi, We are looking for Candidates who are having experience as Data Engineer Role.Experience Required: 4+ years.Location: PAN IndiaResponsibilities:Databricks Development: Design, develop, and optimize Databricks workflows and solutions.Python & PySpark: Write, test, and maintain Python and PySpark code for data processing and analytics.SQL Proficiency:...


  • delhi, India HTC Global Services Full time

    8+ years of experience in IT Industry b. 5+ years of experience with Azure Data Engineering Stack (Data Factory, Databricks, Synapse, Event Hub, Cosmos DB, ADLS Gen2, Function app) c. 3+ years of experience with Python / Pyspark d. Good understanding of other Azure services e. Excellent knowledge of SQL f. Good understanding of Data Warehouse Architecture,...

  • Data Engineer

    1 month ago


    Delhi, India Amossys Consulting Services Pvt Ltd Full time

    Position Title:Data EngineerExperience Required:3-5+ YearsEmployment Type:Full-timeEmployment Mode:HybridLocation:Bangalore, Pune, Trivandrum.Job Summary:We are looking for a skilled Data Engineer with hands-on experience in PySpark and Big Data platforms to join our growing team. The ideal candidate will have strong expertise in building data pipelines,...

  • Data Engineer

    3 days ago


    delhi, India Zemoso Technologies Full time

    Experience: 4.5-6 Years data pipelines experience Key Responsibilities: Python Proficiency: Demonstrate a strong command of Python programming language, actively contributing to the development and maintenance of data engineering solutions. Data Engineering Expertise: Set up and maintain efficient data pipelines, ensuring smooth data flow and integration...


  • delhi, India HTC Global Services Full time

    8+ years of experience in IT Industryb. 5+ years of experience with Azure Data Engineering Stack (Data Factory, Databricks, Synapse, Event Hub, Cosmos DB, ADLS Gen2, Function app)c. 3+ years of experience with Python / Pysparkd. Good understanding of other Azure servicese. Excellent knowledge of SQLf. Good understanding of Data Warehouse Architecture, Data...


  • delhi, India HTC Global Services Full time

    8+ years of experience in IT Industryb. 5+ years of experience with Azure Data Engineering Stack (Data Factory, Databricks, Synapse, Event Hub, Cosmos DB, ADLS Gen2, Function app)c. 3+ years of experience with Python / Pysparkd. Good understanding of other Azure servicese. Excellent knowledge of SQLf. Good understanding of Data Warehouse Architecture, Data...

  • Data Engineer

    3 weeks ago


    Delhi, India Zemoso Technologies Full time

    Job Title: Data EngineerExperience:4+ Years data pipelines experienceLocation:HyderabadJob Type: Full-timeKey Responsibilities:● Python Proficiency:Demonstrate a strong command of Python programming language, actively contributing to thedevelopment and maintenance of data engineering solutions.● Data Engineering Expertise:Set up and maintain efficient...

  • Data Engineer

    3 weeks ago


    delhi, India Zemoso Technologies Full time

    Experience: 4+ Years data pipelines experienceKey Responsibilities:Python Proficiency:Demonstrate a strong command of Python programming language, actively contributing to the development and maintenance of data engineering solutions.Data Engineering Expertise:Set up and maintain efficient data pipelines, ensuring smooth data flow and integration across...