Data Engineer

1 week ago

Chennai, Tamil Nadu, India XANDER CONSULTING AND ADVISORY PRIVATE LIMITED Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Description :

- Design, develop, and maintain scalable data pipelines using Azure Databricks, Apache Spark, and Delta Lake.

- Implement data ingestion, transformation, and integration from multiple structured and unstructured data sources.

- Optimize ETL/ELT processes for performance, cost-efficiency, and scalability in Azure environments.

- Build and manage data models, ensuring data quality, consistency, and lineage across systems.

- Collaborate with cross-functional teams (data scientists, analysts, business users) to understand requirements and deliver data solutions.

- Integrate Databricks with other Azure services such as Azure Data Lake Storage (ADLS), Synapse Analytics, Azure SQL Database, and Azure Data Factory.

- Develop and manage notebooks, jobs, and clusters within Azure Databricks for batch and streaming workloads.

- Implement CI/CD pipelines for data workflows using tools like Azure DevOps or GitHub Actions.

- Monitor, debug, and tune performance of Spark jobs and clusters.

- Ensure compliance, data security, and governance following organizational and regulatory standards.

- Document technical designs, processes, and best practices.

Required Skills and Qualifications :

- Bachelors or Masters degree in Computer Science, Information Systems, or a related field.

years of experience in data engineering or big data development, with at least 2+ years in Azure Databricks.

- Strong hands-on experience in Apache Spark (PySpark/Scala/SQL) for data transformation and processing.

- Proficiency in Azure Data Lake (ADLS), Azure Data Factory (ADF), Azure Synapse, and Azure SQL Database.

- Experience building Delta Lake architectures and implementing medallion (bronze-silver-gold) data models.

- Solid understanding of ETL/ELT design, orchestration, and performance optimization.

- Experience with CI/CD pipelines, Git, and DevOps principles.

- Familiarity with data security, compliance, and access control within Azure.

- Strong problem-solving skills and the ability to troubleshoot distributed data workflows.

- Excellent communication skills and the ability to work in agile, cross-functional teams.

Preferred Skills :

- Experience with Power BI or other visualization tools for data consumption.

- Exposure to machine learning pipelines or integration with Azure Machine Learning.

- Knowledge of Databricks REST APIs, Unity Catalog, and MLflow.

- Experience with real-time data streaming tools (Kafka, Event Hubs, or Spark Streaming).

- Familiarity with infrastructure-as-code (IaC) using Terraform or ARM templates.

- Understanding of data governance, metadata management, and lineage tracking.

Key Attributes :

- Strong analytical and data-driven mindset.

- Excellent problem-solving and performance-tuning abilities.

- Detail-oriented with a focus on data integrity and reliability.

- Ability to manage multiple data projects simultaneously.

- Passion for continuous learning and adopting modern data technologies

  • Data Engineer

    3 days ago

    Chennai, Tamil Nadu, India Crayon Data Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    Data Engineer – Chennai Build the foundation of data intelligence — design, develop, and optimize data pipelines that power AI-driven insights. Location: Chennai, India Experience: 2–3 years in data engineering or data-driven product development Role Overview As a Data Engineer, you will be responsible for building and...

  • Data Engineer

    3 days ago

    Chennai, Tamil Nadu, India CX Data Labs Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    HiRole - Azure Data EngineerLocation - Chennai, Gurugram (5days Office)Full timeShift timing 2pm to 11pmKey Responsibilities:· Develop, maintain, and optimize scalable ETL/ELT pipelines using PySpark and Databricks.· Collaborate with cross-functional teams to design and implement data models and data integration solutions.· Create and maintain robust SQL...

  • Lead Data Engineer

    4 days ago

    Chennai, Tamil Nadu, India Eucloid Data Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support the development of data infrastructure for our clients by participating in activities that may include starting from upstream and downstream technology selection to designing and building different components....

  • Data Engineer

    3 days ago

    Chennai, Tamil Nadu, India AABM Cloud Data Solution Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    We are seeking a Senior Data Engineer with10+ yearsof experience to join our team.Required Qualifications10+ years of experience in data engineering or similar role.Hands on experience in snowflake, SSIS, Python ( including NumPy and Pandas) is mandatory.Expertise in SQL and any cloud environment (AWS & Azure, etc.) is mandatory.Proficiency in mordern BI...

  • Lead Data Engineer

    1 day ago

    Chennai, Tamil Nadu, India AABM Cloud Data Solutions Full time ₹ 12,00,000 - ₹ 18,00,000 per year

    About the RoleWe are seeking a Lead Data Engineer to join our team. In this role, you will be crucial in designing, building, and maintaining our data infrastructure. This is a hands-on position requiring a strong foundation in data warehousing, ETL processes, and database management. The ideal candidate will have a keen eye for detail and the ability to...

  • Python & SSIS Data Engineer

    1 week ago

    Chennai, Tamil Nadu, India AABM Cloud Data Solutions Full time ₹ 10,00,000 - ₹ 42,00,000 per year

    MANDATORY PREVIOUS WORK EXPERIENCE IN SSIS AND PYTHON. (Candidates without this experience in the past will NOT be considered for this role)JOB RESPONSIBILITIES5+ years previous work experience as a Data Engineer.Design and build data pipelines using SSIS, ADF, Talend, Informatica, Data Stage, or Ab Initio to move and transform data reliably.Manage and...

  • Data & Business Insights Associate

    7 days ago

    Chennai, Tamil Nadu, India NTT DATA Full time ₹ 4,00,000 - ₹ 12,00,000 per year

    Works under supervision with internal and external teams to understand customer requirements Collaborate with Automation Engineers, Machine Learning Engineers, Data Analysts and Business Intelligence Analyst to execute deliverables. Ability to elaborate on the details of a project Appropriate mathematical knowledge Strong communication and critical thinking...

  • Senior Data scientist

    1 week ago

    Chennai, Tamil Nadu, India Crayon Data Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Senior Data Scientist Chennai (AI/ML & BFSI Focus) From models to measurable impact — drive real change with AI.Location: Chennai, India Experience: 6–8 years in Data Science/ML (preferably in BFSI or other data-rich industries) Innovators who don't just build models but drive measurable business impact. Expertise: Fierce desire to use AI in every...

  • Data Architect

    3 days ago

    Chennai, Tamil Nadu, India Unifi Data Management Services Full time ₹ 20,00,000 - ₹ 50,00,000 per year

    Role Summary:The Data Architect is responsible for designing, governing, and implementing enterprise-wide data architecture strategies to enable trusted, scalable, and high-performing data platforms. This role ensures data consistency, security, and accessibility across multiple business domains aligning data initiatives with business goals and digital...

  • Senior Software Engineer

    1 week ago

    Chennai, Tamil Nadu, India Crayon Data Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Senior Software Engineer Chennai (AI/ML Engineering Focus) Build the backbone of AI — engineer the systems that power enterprise intelligenceLocation: Chennai, India Experience: 6–8 years in software engineering (preferably in AI/ML or data-driven product development) Problem solvers who don't just write code but build scalable, impactful AI...