Azure with Pyspark

17 hours ago

Thiruvananthapuram Kerala, India UST Global Full time

9 - 12 Years- 4 Openings- Thiruvananthapuram**Role description**:
**Responsibilities**:
**Data Pipeline Development**:

- Design, implement, and optimize end-to-end data pipelines on Azure, focusing on scalability and performance.
- Develop and maintain ETL workflows for seamless data processing.

**Azure Cloud Expertise**:

- Utilize Azure services such as Azure Data Factory, Azure SQL Database, and Azure Databricks for effective data engineering.
- Implement and manage data storage solutions on Azure.

**Data Transformation with PySpark**:

- Leverage PySpark for advanced data transformations, ensuring high-quality and well-structured output.
- Implement data cleansing, enrichment, and validation processes using PySpark.

**Performance Optimization**:

- Optimize data pipelines, queries, and PySpark jobs to enhance overall performance and scalability.
- Identify and address performance bottlenecks within data processing workflows.

**Requirements**:

- Proven experience as a Data Engineer, emphasizing expertise in Azure.
- Proficiency in Azure Data Factory and other relevant Azure services.
- Expertise in PySpark for data processing and analytics is a must.
- Experience with data modeling, ETL processes, and data warehousing.

**Skills**:
Azure, PySpark

**About UST**:
UST is a global digital transformation solutions provider. For more than 20 years, UST has worked side by side with the world’s best companies to make a real impact through transformation. Powered by technology, inspired by people and led by purpose, UST partners with their clients from design to operation. With deep domain expertise and a future-proof philosophy, UST embeds innovation and agility into their clients’ organizations. With over 30,000 employees in 30 countries, UST builds for boundless impact—touching billions of lives in the process.

  • Azure Databricks

    2 days ago

    Kochi, Kerala, India Cognizant Full time

    **Skill: - Azure Databricks** **Experience: 3 to 14 years** **Location: -**Kochi (Walkin on 1st Feb)** **Must Have Skills** - Hands on experience with Azure Databricks, Azure Data Factory, Azure Synapse Analytics, Azure Analysis Services, Blob Storage, Databricks/Spark, Azure SQL DW/Synapse, and Azure functions. - In-depth knowledge of relational database...

  • Azure Cloud Data Architect

    8 hours ago

    Thiruvananthapuram, Kerala, India beBeeData Full time ₹ 18,00,000 - ₹ 25,00,000

    Job TitleWe are seeking a skilled Azure Data Engineer to join our team.Design and Build Robust Data PipelinesDeploy and Manage Scalable Data SolutionsImplement CI/CD Pipelines for Seamless Data WorkflowsKey Skills and Qualifications:Strong SQL SkillsFabric ExperienceStrong Programming Skills in Python (PySpark)Hands-on Experience with Azure Data...

  • Cloud Architect for Benefits Management Solutions

    6 hours ago

    Thiruvananthapuram, Kerala, India beBeeDataEngineer Full time ₹ 12,00,000 - ₹ 30,00,000

    Job Title: Data EngineerWe are seeking a self-driven and motivated professional to develop scalable ETL/ELT pipelines using Azure Databricks, PySpark, and Delta Lake.The role involves working across cross-functional teams to design and implement data solutions, ensuring data governance with Unity Catalog, and supporting real-time and batch processing.Key...

  • Senior Cloud Data Engineering Specialist

    5 hours ago

    Thiruvananthapuram, Kerala, India beBeeDataEngineering Full time ₹ 1,04,000 - ₹ 1,30,878

    Job Role Overview:As a senior data engineering specialist, you will design and develop scalable solutions using Azure Databricks and PySpark. This role involves collaboration with product owners to analyze requirements and create detailed interface designs.Key Responsibilities:Develop and optimize data pipelines using ADF, SQL, and NoSQL...

  • Chief Data Workflow Architect

    21 hours ago

    Thiruvananthapuram, Kerala, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job OverviewAs a Senior Data Engineer, you will be responsible for developing and implementing scalable real-time and batch processing workflows using Azure Databricks, PySpark, and Apache Spark.You will focus on data pre-processing, including cleaning, transformation, deduplication, normalization, encoding, and scaling to ensure high-quality input for...

  • Lead Ii

    4 weeks ago

    Thiruvananthapuram, Kerala, India UST Full time

    Role Proficiency This role requires proficiency in developing data pipelines including coding and testing for ingesting wrangling transforming and joining data from various sources The ideal candidate should be adept in ETL tools like Informatica Glue Databricks and DataProc with strong coding skills in Python PySpark and SQL This position demands...

  • Chief Cloud Migration Expert

    4 hours ago

    Thiruvananthapuram, Kerala, India beBeeCloudMigration Full time ₹ 25,00,000 - ₹ 35,00,000

    Job Title: Chief Cloud Migration ExpertJob Summary: We are seeking a highly skilled and experienced professional to lead large-scale cloud migration projects.About the Role:This role involves designing, developing, and implementing complex data pipelines for migrating data from various on-premise sources to cloud platforms.Main Responsibilities:Design and...

  • Scalable Data Solutions Engineer

    4 days ago

    Thiruvananthapuram, Kerala, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 23,40,000

    Job Title: Data Engineer - Microsoft FabricJob Description: We are seeking an experienced data engineer to join our high-impact team. As a data engineer, you will be responsible for designing and implementing robust, scalable data solutions on the Microsoft Fabric and Azure platforms.Key Responsibilities:Collaborate with stakeholders to convert business...

  • Cloud Data Tester

    7 hours ago

    Thiruvananthapuram, Kerala, India beBeeData Full time ₹ 18,00,000 - ₹ 20,00,000

    We are seeking a Data Analyst to join our team in Bengaluru. This role offers the opportunity to work on meaningful projects, collaborate with talented colleagues, and contribute to the success of a growing organization.Job Description:A minimum of 5-8 years of experience in IT and at least 4+ years' experience in cloud environments.In-depth knowledge of...

  • Highly Skilled Senior Data Professional Wanted

    7 hours ago

    Thiruvananthapuram, Kerala, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Senior Data Engineer and ArchitectRole Summary:We are seeking a highly skilled Senior Data Engineer and Architect to join our team. As a key member of our data engineering team, you will be responsible for designing, building, and maintaining large-scale data processing systems.Key Responsibilities: Design and develop scalable data architectures...