Azure with Pyspark

6 months ago

Thiruvananthapuram Kerala, India UST Global Full time

9 - 12 Years- 4 Openings- Thiruvananthapuram**Role description**:
**Responsibilities**:
**Data Pipeline Development**:

- Design, implement, and optimize end-to-end data pipelines on Azure, focusing on scalability and performance.
- Develop and maintain ETL workflows for seamless data processing.

**Azure Cloud Expertise**:

- Utilize Azure services such as Azure Data Factory, Azure SQL Database, and Azure Databricks for effective data engineering.
- Implement and manage data storage solutions on Azure.

**Data Transformation with PySpark**:

- Leverage PySpark for advanced data transformations, ensuring high-quality and well-structured output.
- Implement data cleansing, enrichment, and validation processes using PySpark.

**Performance Optimization**:

- Optimize data pipelines, queries, and PySpark jobs to enhance overall performance and scalability.
- Identify and address performance bottlenecks within data processing workflows.

**Requirements**:

- Proven experience as a Data Engineer, emphasizing expertise in Azure.
- Proficiency in Azure Data Factory and other relevant Azure services.
- Expertise in PySpark for data processing and analytics is a must.
- Experience with data modeling, ETL processes, and data warehousing.

**Skills**:
Azure, PySpark

**About UST**:
UST is a global digital transformation solutions provider. For more than 20 years, UST has worked side by side with the world’s best companies to make a real impact through transformation. Powered by technology, inspired by people and led by purpose, UST partners with their clients from design to operation. With deep domain expertise and a future-proof philosophy, UST embeds innovation and agility into their clients’ organizations. With over 30,000 employees in 30 countries, UST builds for boundless impact—touching billions of lives in the process.

  • Digital : Azure Databricks

    7 days ago

    Thiruvananthapuram, India Tata Consultancy Services Full time

    TCS Hiring!!!! VIRTUAL DRIVE: 28th November 2024NOTE: If the skills/profile matches and interested, please reply to this email by attaching your latest updated CV and with below few details:Name:Contact Number:Email ID:Highest Qualification in: (Eg. B.Tech/B.E./M.Tech/MCA/M.Sc./MS/BCA/B.Sc./Etc.)Current Organization Name:Total IT Experience::3 to 7 years...

  • Data Engineer

    1 month ago

    Bangalore/Pune/Trivandrum/Thiruvananthapuram, India Rao Career Solutions Full time

    Job Summary :- We are looking for a skilled Data Engineer with hands-on experience in PySpark and Big Data platforms to join our growing team. The ideal candidate will have strong expertise in building data pipelines, working with big data technologies like Apache Hive and Parquet, and handling diverse data use cases. You will work closely with data...

  • QBrainX - Data Engineer - Snowflake/Data Bricks/Python/Pyspark/SQL

    2 months ago

    Kerala/Tamil Nadu/Coimbatore, IN QBrainX Full time

    Job title - Data Engineer (Mid-Level)Exp - 3+ to 7 yearsJob Location : Coimbatore (Work from Office)Do you love building the infrastructure that fuels data-driven decision making? Are you passionate about wrangling raw data into a usable format? If so, then we want to hear from you!We are searching for a Mid-Level Data Engineer to join our growing team. In...

  • Databricks Senior Data Engineer

    3 weeks ago

    Thiruvananthapuram, India Luxoft Full time

    Project descriptionWe are hiring for projects in the Middle East. The team comprises front-end and back-end developers, data analysts and data scientists, architects, analysts, and project managers. You will work with modern technologies and be an integral part of continuing to evolve our high-performing platforms. If you are serious about being a part of...

  • QBrainX - Data Engineering Lead - Python/PySpark

    1 month ago

    Coimbatore/Hyderabad/Chennai/Tamil Nadu/Kerala/Andhra Pradesh/Anywhere in India/Multiple Locations, IN QBrainX Full time

    Job title - Data Engineering LeadExp - 6 to 8 yearsJob Location : Coimbatore / Hyderabad (Work from Office)Do you love building the infrastructure that fuels data-driven decision making? Are you passionate about wrangling raw data into a usable format? If so, then we want to hear from you!We are searching for a Data Engineering Lead to join our growing team....

  • Data Scientist

    4 weeks ago

    Kerala/Tamil Nadu/Coimbatore, India QBrainX Full time

    Data Engineer RoleAbout the Position :We are searching for a skilled Data Engineer to join our team at QBrainX. As a Data Engineer, you will play a key part in designing, developing, and maintaining our data pipelines. You will work closely with data scientists, analysts, and business stakeholders to ensure our data infrastructure meets the ever-evolving...

  • Data Engineer

    1 month ago

    Kerala/Tamil Nadu/Coimbatore, India QBrainX Full time

    Job Title - Data Engineer (Mid-Level)Experience - 3+ to 7 yearsJob Location : Coimbatore (Work from Office)Are you passionate about building data infrastructure that fuels data-driven decision making? Do you enjoy wrangling raw data into a usable format?We are seeking a Mid-Level Data Engineer to join our growing team. In this role, you will play a key part...

  • Data Engineering Leadership Role

    4 weeks ago

    Anywhere in India/Multiple Locations/Hyderabad/Chennai/Kerala/Andhra Pradesh/Coimbatore QBrainX Full time

    Data Engineering Lead PositionThe ideal candidate for this role will have extensive experience in designing, developing, and maintaining large-scale data pipelines using Python and SQL. Proficiency in cloud platforms such as AWS, Azure, and GCP is a plus, as well as experience with Terraform and Infrastructure as Code (IaC) principles.Key Responsibilities :-...

  • Data Engineering Lead

    1 month ago

    Anywhere in India/Multiple Locations/Hyderabad/Chennai/Kerala/Andhra Pradesh/Coimbatore QBrainX Full time

    Job Title: Data Engineering LeadWe are seeking a highly skilled Data Engineering Lead to join our team at QBrainX. As a key member of our data engineering team, you will play a crucial role in designing, developing, and maintaining our data pipelines.Key Responsibilities:Design and develop scalable data pipelines using Python and SQL.Develop and implement...