ai - Senior Data Engineer - ETL Pipeline

3 weeks ago

Cochin, Kerala, India DBiz Full time

Position : Senior Data Engineer

We are seeking an experienced Senior Data Engineer to design and implement high-performance data pipelines, ensuring the smooth flow of large datasets across systems.

In this role, you will leverage your expertise in Databricks, Pyspark, and ETL design to build efficient data solutions.

You will also contribute to optimizing database performance and ensure that data systems meet both scalability and security Responsibilities :

- Develop, optimize, and maintain scalable ETL pipelines for processing large datasets using Databricks and Pyspark.

- Apply advanced techniques such as chunking and partitioning to handle large file volumes efficiently.

- Tuning and optimizing databases for better performance and storage efficiency.

- Collaborate with cross-functional teams to ensure the architecture meets business requirements and data quality standards.

- Design and implement solutions with an eye toward scalability and long-term performance.

- Work with Azure services, with exposure to Azure Function Apps, Azure Data Factory, and Cosmos DB (preferred).

- Communicate effectively with stakeholders to align on project goals and Skills and Experience :

- Databricks (advanced-level skills in data engineering workflows)

- Pyspark (intermediate-level skills for processing big data)

- Strong ETL design skills, particularly in partitioning, chunking, and database tuning for large datasets

- Azure experience (a plus, including Function Apps, Cosmos DB, and Azure Data Factory)

- Excellent communication skills for effective collaboration with teams and stakeholders

(ref:hirist.tech)

  • Senior Data Engineer

    6 days ago

    Cochin, Kerala, India DBiz ai solutions Full time ₹ 15,00,000 - ₹ 28,00,000 per year

    About the Company: Dbiz is a high-performing product & engineering company that partners up with organizations to help build out digital solutions with the right technology at the right time, whilst being innovative by using technology in different ways. Requirements:Azure Synapse Analytics/ Azure DatabricksAzure Cloud , Azure Data Lake , Azure SQLData...

  • Data/QA Engineer _ AWS Cloud

    3 days ago

    Cochin, Kerala, India Inforizon Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job Description: QA Engineer (Agentic AI AWS)Key ResponsibilitiesDevelop and execute test plans, test cases, and automation scripts for AI-powered applications on AWS.Test data pipelines (ETL/ELT, streaming, batch) for correctness, completeness, and performance.Validate AI/ML workflows (model training, inference, fine-tuning, and retraining...

  • ETL Tester

    4 weeks ago

    Cochin, Kerala, India SE Mentor Solutions Full time

    Company Description SE-Mentor Solutions is a software engineering company specializing in Software Development, Data Engineering, and Software test automation. We assist clients in refining their software engineering practices and achieving success. We serve top-tier software companies, Government organizations, and educational institutions across various...

  • ETL Tester

    3 weeks ago

    Cochin, Kerala, India SE Mentor Solutions Full time

    Company DescriptionSE-Mentor Solutions is a software engineering company specializing in Software Development, Data Engineering, and Software test automation. We assist clients in refining their software engineering practices and achieving success. We serve top-tier software companies, Government organizations, and educational institutions across various...

  • ETL Tester

    1 day ago

    Cochin, Kerala, India SE Mentor Solutions Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Company DescriptionSE-Mentor Solutions is a software engineering company specializing in Software Development, Data Engineering, and Software test automation. We assist clients in refining their software engineering practices and achieving success. We serve top-tier software companies, Government organizations, and educational institutions across various...

  • Data Engineer

    3 days ago

    Cochin, Kerala, India ZoftSolutions Pvt Ltd Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    Key ResponsibilitiesDesign, build, and maintain ETL/ELT pipelines for structured and unstructured data in AWS.Work with AWS services (e.g., S3, Glue, EMR, Lambda, Kinesis, Step Functions, Redshift, Athena, DynamoDB, RDS) to manage data ingestion, transformation, and storage.Ensure data quality, governance, and security across the full lifecycle.Enable...

  • Data Engineer

    2 days ago

    Cochin, Kerala, India Dynamed Healthcare Solutions Pvt. Ltd Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    About the Role:We are looking for a skilled Data Engineer to design, build, and maintain scalable data pipelines and infrastructure to support AI-driven applications. You will collaborate closely with data scientists, analysts, and software engineers to ensure high-quality, efficient data processing for advanced analytics and machine learning.Key...

  • Data Engineer

    3 days ago

    Cochin, Kerala, India Thomasmount Consulting Pvt ltd Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Senior Data Engineer | 8-13 Years | Cochin / Trivandrum Experience : 8 to 13 Years Location : Cochin / Trivandrum Job Type : Full-Time About the Role : We are hiring a Senior Data Engineer with deep expertise in ETL, data pipelines, AWS cloud services, and modern data warehousing technologies. This is a hands-on engineering role ideal for someone...

  • Etl Developer

    3 days ago

    Cochin, Kerala, India Evoke HR Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Role: ETL DeveloperJob SummaryWe are seeking a seasoned ETL Developer to join our team and support a high-impact data migration initiative. This role will involve designing and building scalable ETL pipelines, developing complex transformation logic in SQL and C#, and collaborating with cross-functional teams including analysts, QA, and business...

  • Data Scientist

    2 weeks ago

    Cochin, Kerala, India Simelabs - Digital, AIML, Automation, Robotics, Gen AI. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    We are seeking a passionate and innovativeAI/ML Engineerto join our team and drive cutting-edge AI solutions across traditional machine learning, deep learning, and generative AI. If you enjoy solving complex problems, experimenting with the latest AI technologies, and building real-world applications, we'd love to connect.Key ResponsibilitiesDesign,...