Azure Data Engineer

5 hours ago

Bengaluru Karnataka India, Karnataka Sigmoid Full time

We are looking for a skilled TL Azure Data Engineer with 9+ years of experience in big data technologies, particularly Python, PYSpark, SQL and data lakehouse architectures. The ideal candidate will have a strong background in building scalable data pipelines and experience with modern data storage formats, including Apache Iceberg. You will work closely with cross-functional teams to design and implement efficient data solutions in a cloud-based environment.

Key Responsibilities:

  • Data Pipeline Development:
  • Design, build, and optimize scalable data pipelines using Apache Spark.
  • Implement and manage large-scale data processing solutions across data lakehouses.
  • Data Lakehouse Management:
  • Work with modern data lakehouse platforms (e.g.Apache Iceberg) to handle large datasets.
  • Optimize data storage, partitioning, and versioning to ensure efficient access and querying.
  • SQL & Data Management:
  • Write complex SQL queries to extract, manipulate, and transform data.
  • Develop performance-optimized queries for analytical and reporting purposes.
  • Data Integration:
  • Integrate various structured and unstructured data sources into the lakehouse environment.
  • Work with stakeholders to define data needs and ensure data is available for downstream consumption.
  • Data Governance and Quality:
  • Implement data quality checks and ensure the reliability and accuracy of data.
  • Contribute to metadata management and data cataloging efforts.
  • Performance Tuning:
  • Monitor and optimize the performance of Spark jobs, SQL queries, and overall data infrastructure.
  • Work with cloud infrastructure teams to optimize costs and scale as needed.

Qualifications:

  • Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.
  • 8+ years of experience in data engineering, with a focus on python, Spark and SQL Programming languages.
  • Hands-on experience with Apache Iceberg, Snowflake, or similar technologies.
  • Strong understanding of data lakehouse architectures and data warehousing principles.
  • Proficiency in Azure.
  • Experience with version control systems like Git and CI/CD pipelines.
  • Excellent problem-solving and analytical skills.
  • Strong communication and collaboration skills.

Nice to Have:

  • Experience with containerization (Docker, Kubernetes) and orchestration tools like Airflow.
  • Certifications in Azure cloud technologies

  • Azure Data Engineer

    5 hours ago

    Bengaluru, Karnataka, India, Karnataka E-Solutions Full time

    Job Role: Azure Data EngineerLocation: Bangalore, Chennai , HyderabadType of Hire: PermanentMust Have Skills : Azure Data Factory, Azure Databricks, Pyspark/Scala, Logic App, Azure SQL, ADLS Gen 2.0, Azure key Vault, T-SQL, Stored Procedure. Data Engineer JDDeveloping, managing and optimizing robust and reliable data pipelines using azure native...

  • Azure Data Engineer

    5 hours ago

    Bengaluru, Karnataka, India, Karnataka Tata Consultancy Services Full time

    Greetings from TCS!!TCS is hiring for Azure Data Engineer (Azure Databricks, PySpark)Job Location: Kolkata, Pune, Mumbai, Bangalore, BhubaneshwarDesired Experience Range: 5 to 10 YearsRequired Skill Set: Azure Data Engineer, ADF, Azure Databricks Spark (PySpark or Scala), Python, PL/SQLJob Description:· Strong experience in Azure Data Factory , ADB( Azure...

  • Data Engineer

    5 hours ago

    Bengaluru, Karnataka, India, Karnataka Quess IT Staffing Full time

    We Are Hiring: Data Engineer (Azure)Payroll: Quess Location: Pune / Bangalore Experience: 7+ Years (Minimum 6+ Years Relevant) Shift: 4:00 PM – 1:00 AM IST Notice Period: Immediate – 15 Days Interview Process:• L1 – Face-to-Face (Pune/Bangalore Office)• L2 – Virtual Interview Role & Responsibilities:• Design and maintain Azure Data Pipelines...

  • Azure Data Engineer

    5 hours ago

    Bengaluru, Karnataka, India, Karnataka LTIMindtree Full time

    Job Description:Seeking an experienced Senior Specialist with 7 to 11 years of expertise in Azure Data Factory and Azure SQL to design and manage scalable data integration solutionsDesign develop and maintain scalable data pipelines leveraging Azure Data Factory and Azure SQL Implement data integration workflows to support analytics and business intelligence...

  • Azure Data Engineer

    5 hours ago

    Bengaluru, Karnataka, India, Karnataka LTIMindtree Full time

    Role Senior Data Engineer 8 years of experienceKey responsibilitiesBuild reusable utilities templates and automation pipelinesDesign scalable data engineering frameworks standards and best practicesProvide architectural guidance cost optimization and performance tuning support for Data Engineering solutions mainly on AzureEvaluate and onboard new featuresin...

  • Azure Data Engineer

    5 hours ago

    Bengaluru, Karnataka, India, Karnataka Tata Consultancy Services Full time

    Role - Azure Data EngineerExperience - 6 to 8Location - Pan IndiaRequired Technical Skill Set** - ADLS/ADF/ADBDesired Competencies :1.Lead Back-end development & maintenance of Data Quality Product2. Design and develop Data Pipeline using ADF, Databricks and integrate with other Azure Services 3. Experience in setting up DevOps pipeline4. Databricks...

  • Azure Data Engineer

    5 hours ago

    Bengaluru, Karnataka, India, Karnataka Tata Consultancy Services Full time

    Greetings from TCS!Job Title: Azure Data Engineer With DatabricksRequired Skillset: Azure Databricks, SQL, Python, PySpark, ADF, ADB.Location: HYD, Chennai, BLR, Noida, Pune, KolkataExperience Range: 4-8 yearsJob DescriptionMust-Have**1. Design, develop, and deploy data processing solutions using Azure Databricks.2. Collaborate with data engineers,...

  • Azure Data Engineer

    5 hours ago

    Bengaluru, Karnataka, India, Karnataka Tata Consultancy Services Full time

    Greetings from TCS!Job Title: Azure Data EngineerRequired Skillset: Data Lake architecture, Azure Services – ADLS, ADF, Azure Databricks, SynapseLocation: Kolkata/BangaloreExperience Range: 6-10 yearsJob DescriptionMust-Have**Build the solution for optimal extraction, transformation, and loading of data from a wide variety of data sources using Azure data...

  • Azure Data Engineer

    5 hours ago

    Bengaluru, Karnataka, India, Karnataka Capgemini Invent Full time

    At Capgemini Invent, we believe difference drives change. As inventive transformation consultants, we blend our strategic, creative and scientific capabilities, collaborating closely with clients to deliver cutting-edge solutions. Join us to drive transformation tailored to our client's challenges of today and tomorrow. Informed and validated by science and...

  • Azure Data Engineer

    5 hours ago

    Bengaluru, Karnataka, India, Karnataka Seosaph-infotech Full time

    Position OverviewWe are seeking a highly skilled Azure Data Engineer with Business Intelligence (BI) expertise and 3–6 years of industry experience. The ideal candidate will have strong hands-on development skills, experience in data engineering on Azure, BI solution design, data modeling, and the ability to collaborate with stakeholders to translate...