Azure with Pyspark

4 days ago

Thiruvananthapuram Kerala, India UST Global Full time

9 - 12 Years- 4 Openings- Thiruvananthapuram**Role description**:
**Responsibilities**:
**Data Pipeline Development**:

- Design, implement, and optimize end-to-end data pipelines on Azure, focusing on scalability and performance.
- Develop and maintain ETL workflows for seamless data processing.

**Azure Cloud Expertise**:

- Utilize Azure services such as Azure Data Factory, Azure SQL Database, and Azure Databricks for effective data engineering.
- Implement and manage data storage solutions on Azure.

**Data Transformation with PySpark**:

- Leverage PySpark for advanced data transformations, ensuring high-quality and well-structured output.
- Implement data cleansing, enrichment, and validation processes using PySpark.

**Performance Optimization**:

- Optimize data pipelines, queries, and PySpark jobs to enhance overall performance and scalability.
- Identify and address performance bottlenecks within data processing workflows.

**Requirements**:

- Proven experience as a Data Engineer, emphasizing expertise in Azure.
- Proficiency in Azure Data Factory and other relevant Azure services.
- Expertise in PySpark for data processing and analytics is a must.
- Experience with data modeling, ETL processes, and data warehousing.

**Skills**:
Azure, PySpark

**About UST**:
UST is a global digital transformation solutions provider. For more than 20 years, UST has worked side by side with the world’s best companies to make a real impact through transformation. Powered by technology, inspired by people and led by purpose, UST partners with their clients from design to operation. With deep domain expertise and a future-proof philosophy, UST embeds innovation and agility into their clients’ organizations. With over 30,000 employees in 30 countries, UST builds for boundless impact—touching billions of lives in the process.

  • Senior Data Specialist

    1 week ago

    Thiruvananthapuram, Thiruvananthapuram / Trivandrum, India Uplers Full time

    Job Description Experience: 3.00 + years Salary: Confidential (based on experience) Expected Notice Period: 15 Days Shift: (GMT+05:30) Asia/Kolkata (IST) Opportunity Type: Remote Placement Type: Full Time Contract for 12 Months(40 hrs a week/160 hrs a month) (*Note: This is a requirement for one of Uplers client - Mavlers end client from UK - A data-based...

  • Data Engineer

    2 weeks ago

    Thiruvananthapuram, India Neo Aid Full time

    Senior Data Engineer Job DescriptionWe are looking for a highly skilled Senior Data Engineer with 6+ years of experience, specialising in data engineering and data migration projects. The ideal candidate should be proficient in Azure Data Factory (ADF), ETL/ELT processes, Python/PySpark, and have strong experience building scalable data pipelines across...

  • Senior Data Engineer

    5 days ago

    Thiruvananthapuram, India Us Technology International Private Limited Full time

    Senior Data EngineersExperience - 6 +Yrs (5-7 relevant years) Location: TrivandrumMandatory Skill - Pyspark, SQL, Azure DatabricksUK Shift Timing Job Description – Key Accountabilities / Responsibilities:Provide technical direction and leadership to data engineers on data platform initiatives - ensuring adherence to best practices in data modelling, End to...

  • Senior Data Engineer

    5 days ago

    Thiruvananthapuram, India UST Full time

    Senior Data EngineersExperience - 6 +Yrs (5-7 relevant years) Location: TrivandrumMandatory Skill - Pyspark, SQL, Azure DatabricksUK Shift TimingJob Description Key Accountabilities / Responsibilities:Provide technical direction and leadership to data engineers on data platform initiatives - ensuring adherence to best practices in data modelling, End to End...

  • Senior Data Engineer

    5 days ago

    Thiruvananthapuram, India Us Technology International Private Limited Full time

    Senior Data EngineersExperience - 6 +Yrs (5-7 relevant years) Location: TrivandrumMandatory Skill - Pyspark, SQL, Azure DatabricksUK Shift Timing Job Description – Key Accountabilities / Responsibilities:Provide technical direction and leadership to data engineers on data platform initiatives - ensuring adherence to best practices in data modelling, End to...

  • Senior Data Engineer

    5 days ago

    Thiruvananthapuram, India Us Technology International Private Limited Full time

    Senior Data EngineersExperience - 6 +Yrs (5-7 relevant years) Location: TrivandrumMandatory Skill - Pyspark, SQL, Azure DatabricksUK Shift Timing Job Description – Key Accountabilities / Responsibilities:Provide technical direction and leadership to data engineers on data platform initiatives - ensuring adherence to best practices in data modelling, End to...

  • Senior Data Engineer

    3 days ago

    thiruvananthapuram, India UST Full time

    Senior Data EngineersExperience - 6 +Yrs (5-7 relevant years) Location: TrivandrumMandatory Skill - Pyspark, SQL, Azure DatabricksUK Shift TimingJob Description Key Accountabilities / Responsibilities:Provide technical direction and leadership to data engineers on data platform initiatives - ensuring adherence to best practices in data modelling, End to End...

  • Senior Data Engineer

    3 days ago

    thiruvananthapuram, India Us Technology International Private Limited Full time

    Senior Data EngineersExperience - 6 +Yrs (5-7 relevant years) Location: TrivandrumMandatory Skill - Pyspark, SQL, Azure DatabricksUK Shift Timing Job Description – Key Accountabilities / Responsibilities:Provide technical direction and leadership to data engineers on data platform initiatives - ensuring adherence to best practices in data modelling, End to...

  • Senior Data Engineer

    5 days ago

    Thiruvananthapuram, India Us Technology International Private Limited Full time

    Senior Data Engineers Experience - 6 +Yrs (5-7 relevant years) Location: Trivandrum Mandatory Skill - Pyspark, SQL, Azure Databricks UK Shift Timing Job Description – Key Accountabilities / Responsibilities: Provide technical direction and leadership to data engineers on data platform initiatives - ensuring adherence to best practices in data modelling,...

  • Senior Data Engineer

    5 days ago

    Thiruvananthapuram, India UST Full time

    Senior Data Engineers Experience - 6 +Yrs (5-7 relevant years) Location: Trivandrum Mandatory Skill - Pyspark, SQL, Azure Databricks UK Shift Timing Job Description Key Accountabilities / Responsibilities: - Provide technical direction and leadership to data engineers on data platform initiatives - ensuring adherence to best practices in data modelling, End...