Azure with Azure Data Factory

2 hours ago

Cochin, Kerala, India UST Full time ₹ 12,00,000 - ₹ 24,00,000 per year

Role Description
Job Title: Data Engineer
Experience
: 5-8 years

Job Description
We are seeking a highly skilled
Data Engineer
with expertise in building scalable data pipelines and working with Azure Cloud services. The ideal candidate will be proficient in designing and implementing end-to-end data pipelines, ensuring high performance and scalability. You will leverage Azure services like Azure Data Factory, Azure SQL Database, and Azure Databricks, as well as PySpark for data transformations. This is an excellent opportunity to make a significant impact in an innovative environment.

Key Responsibilities

  • Data Pipeline Development: Design, implement, and optimize end-to-end data pipelines on Azure, focusing on scalability, performance, and reliability.
  • ETL Workflow Development: Develop and maintain ETL workflows to ensure seamless and efficient data processing.
  • Azure Cloud Expertise: Utilize Azure services such as Azure Data Factory, Azure SQL Database, and Azure Databricks for effective data engineering solutions.
  • Data Storage Solutions: Implement and manage data storage solutions on Azure, ensuring optimal performance and scalability.
  • Data Transformation: Use PySpark to perform advanced data transformations, ensuring high-quality and well-structured data outputs.
  • Data Cleansing and Enrichment: Implement data cleansing, enrichment, and validation processes using PySpark.
  • Performance Optimization: Optimize data pipelines, queries, and PySpark jobs to enhance overall system performance and scalability.
  • Bottleneck Identification: Identify and resolve performance bottlenecks within data processing workflows, ensuring efficiency.

Must Have Skills

  • Azure Expertise: Proven experience with Azure Cloud services, especially Azure Data Factory, Azure SQL Database, and Azure Databricks.
  • PySpark Proficiency: Expertise in PySpark for data processing and analytics.
  • Data Engineering: Strong background in building and optimizing data pipelines and workflows.
  • ETL Processes: Solid experience with data modeling, ETL processes, and data warehousing.
  • Performance Tuning: Ability to optimize data pipelines and jobs to ensure scalability and performance.
  • Problem-Solving: Experience troubleshooting and resolving performance bottlenecks in data workflows.
  • Collaboration: Strong communication skills and ability to work collaboratively in cross-functional teams.

Good To Have Skills

  • Azure Certifications: Any relevant Azure certifications will be considered a plus.
  • Advanced Data Transformation: Experience with advanced data transformations using PySpark and other tools.
  • Data Governance: Knowledge of data governance best practices in the context of data engineering and cloud solutions.
  • Big Data Technologies: Familiarity with other big data technologies and tools such as Hadoop, Spark, and Kafka.
  • Data Warehousing: Experience in designing and implementing data warehousing solutions in the Azure environment.

Qualification

  • Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field.
  • 5-8 years of professional experience as a Data Engineer with a focus on Azure technologies.

Skills
Azure,Azure Data Factory,Pyspark

  • Azure Data Architect

    1 week ago

    Cochin, Kerala, India Adfolks Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Role Overview:We are seeking a seasoned Senior Data Architect to lead enterprise data and analytics initiatives on Microsoft Azure and Microsoft Fabric. The role involves architecting, implementing, and governing scalable, secure, and high-performing data solutions that drive business insights and AI innovation.Key Responsibilities:Define and implement...

  • Azure Data Engineer

    4 days ago

    Cochin, Kerala, India Techmax Technologies Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    About Techmax TechnologiesAt Techmax Technologies, we're transforming the automotive industry through cutting-edge data analytics, customer experience innovations, and digital solutions. Since 2020, we've partnered with dealerships and automotive businesses to build data-driven ecosystems that boost efficiency, enhance customer engagement, and accelerate...

  • NET with Azure

    1 week ago

    Cochin, Kerala, India AMUS HIRING Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    NOTE: RELEVANT PROFILES ONLY CAN BE CONTACTEDLocation – Kochi, KeralaNP – Immediate to 15 day who is currently serving their NPWork mode - Full TimeInterview mode - Virtual/F2FResponsibilities include:• Design, develop, enhance, document, and maintain robust applications using .NET Core 6/8+, C#, RESTAPIs, T-SQL, and modern JavaScript/jQuery•...

  • Lead II/Azure Cloud Developer

    2 weeks ago

    Cochin, Kerala, India MSK Brawn Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Description : - Lead II -Azure Cloud Developer Location : - Kochi yrs work experience lacsMUST HAVE : - Notice Period < 30 days (This absolute must, Managable/Negotiable will not work) need clear commitment hence should be able to prove LWD within next 20-25 days.Responsibilities : - Integration Maintenance : Learn all current integrations, ranging from...

  • Azure Architect

    1 week ago

    Cochin, Kerala, India Adfolks Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Location: IndiaExperience: 10+ years (5+ years in Azure)About the RoleWe are seeking an experienced Senior Azure Cloud Architect to lead the design, implementation, and governance of enterprise-grade cloud solutions. You will define cloud strategy, build scalable hybrid/multi-cloud architectures, and provide technical leadership to the cloud engineering...

  • Azure DevOps Engineer

    2 days ago

    Cochin, Kerala, India 41af5716-2e74-4868-93e2-327d4585bcf5 Full time ₹ 3,93,000 - ₹ 20,00,000 per year

    OverviewWe are looking for an experienced Azure DevOps Engineer with strong expertise in managing release pipelines, CI/CD processes, and automation to support Microsoft Power Platform (Dynamics 365, Dataverse, Power Apps, Power Automate) solutions. The role will focus on implementing DevOps best practices, ensuring reliable deployments, and maintaining...

  • Senior Data Engineer

    2 days ago

    Cochin, Kerala, India Milestone Technologies, Inc. Full time

    Role Overview:Seeking an experienced Senior Data Engineer to lead data ingestion, transformation, and optimization initiatives using the modern Apache and Azure data stack. The role involves working on scalable pipelines, large-scale distributed systems, and data lake management.Core Responsibilities:· Build and manage high-volume data pipelines using...

  • Azure DevOps Lead Engineer

    1 week ago

    Cochin, Kerala, India Adfolks Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Role OverviewWe are looking for a skilled Azure DevOps Lead Engineer to drive DevOps transformation initiatives across Azure and multi-cloud environments. The ideal candidate will design and implement CI/CD pipelines, Infrastructure as Code (IaC), automation, and containerization solutions. This role involves setting DevOps strategy, implementing GitOps...

  • Azure DevOps Lead Engineer

    1 week ago

    Cochin, Kerala, India Adfolks Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Role Overview We are looking for a skilled Azure DevOps Lead Engineer to drive DevOps transformation initiatives across Azure and multi-cloud environments. The ideal candidate will design and implement CI/CD pipelines, Infrastructure as Code (IaC), automation, and containerization solutions. This role involves setting DevOps strategy, implementing GitOps...

  • Manager - Data Engineer

    4 hours ago

    Cochin, Kerala, India KPMG Global Services Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Manager – Data Engineer, Tax TechnologyRoles and responsibilitiesThe responsibilities of the role include:Responsible for client interaction to gather the Business challenges and documentation Critical thinking and problem-solving skills are essential for interpreting data. Responsible of presenting the solution and insight to the Clients. Evaluate...