ai - Senior Data Engineer - ETL Pipeline

23 hours ago


Cochin, Kerala, India DBiz Full time

Position : Senior Data Engineer

We are seeking an experienced Senior Data Engineer to design and implement high-performance data pipelines, ensuring the smooth flow of large datasets across systems.

In this role, you will leverage your expertise in Databricks, Pyspark, and ETL design to build efficient data solutions.

You will also contribute to optimizing database performance and ensure that data systems meet both scalability and security Responsibilities :

- Develop, optimize, and maintain scalable ETL pipelines for processing large datasets using Databricks and Pyspark.

- Apply advanced techniques such as chunking and partitioning to handle large file volumes efficiently.

- Tuning and optimizing databases for better performance and storage efficiency.

- Collaborate with cross-functional teams to ensure the architecture meets business requirements and data quality standards.

- Design and implement solutions with an eye toward scalability and long-term performance.

- Work with Azure services, with exposure to Azure Function Apps, Azure Data Factory, and Cosmos DB (preferred).

- Communicate effectively with stakeholders to align on project goals and Skills and Experience :

- Databricks (advanced-level skills in data engineering workflows)

- Pyspark (intermediate-level skills for processing big data)

- Strong ETL design skills, particularly in partitioning, chunking, and database tuning for large datasets

- Azure experience (a plus, including Function Apps, Cosmos DB, and Azure Data Factory)

- Excellent communication skills for effective collaboration with teams and stakeholders

(ref:hirist.tech)

  • Cochin, Kerala, India beBeeTestEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    About People Prime Worldwide:We are a leader in business transformation, leveraging technology to address a wide range of business needs.Our mission is to unleash human energy through technology for an inclusive and sustainable future.We provide consulting, technology, professional, and outsourcing services.Job Details:location : Pan IndiaMode Of Work :...


  • Cochin, Kerala, India beBeeDataEngineering Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Senior Data Engineering Project LeadWe are seeking an experienced Senior Data Engineering Project Lead to spearhead the delivery of end-to-end data platforms, AI, BI, and analytics projects.Key ResponsibilitiesDevelop comprehensive project plans, roadmaps, and timelines for data ingestion, transformation, governance, AI/ML models, and analytics...

  • Data Pipeline Lead

    5 days ago


    Cochin, Kerala, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Lead Data ArchitectOur team is seeking an accomplished data architect to lead the design and implementation of large-scale data pipelines. As a senior member, you will play a pivotal role in shaping our company's data strategy.Key Responsibilities:Design and implement ETL pipelines using FiveTran, DBT Cloud, and custom frameworksDevelop client-specific data...


  • Cochin, Kerala, India beBeeDataEngineering Full time ₹ 1,80,00,000 - ₹ 2,40,00,000

    Job OverviewWe are seeking an accomplished Technical Project Manager for Data Engineering to lead end-to-end delivery of high-profile projects. Your expertise in technical product management, technical program management, and delivery lead roles will ensure seamless project execution.The ideal candidate possesses hands-on development experience in data...


  • Cochin, Kerala, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Data Engineer Job Summary">At the heart of our business, we need a skilled Data Engineer to build and manage the intricate networks that drive our analytics. With 3+ years of experience in a Data Engineering role, you'll be responsible for creating scalable data pipelines that deliver real-time insights across our global operations.This isn't just about ETL;...

  • Data Engineer

    23 minutes ago


    Cochin, Kerala, India Dynamed Healthcare Solutions Pvt. Ltd Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    About the Role:We are looking for a skilled Data Engineer to design, build, and maintain scalable data pipelines and infrastructure to support AI-driven applications. You will collaborate closely with data scientists, analysts, and software engineers to ensure high-quality, efficient data processing for advanced analytics and machine learning.Key...


  • Cochin, Kerala, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Unlock the Power of Data EngineeringSteps AI is a fast-growing AI startup that's revolutionizing the way businesses interact with customers. We're looking for an AWS-focused Data Engineer to join our team and help us power our platform with real-time, intelligent insights.The ideal candidate has 1–2 years of experience in data engineering and is excited to...

  • ETL Tester

    3 days ago


    Cochin, Kerala, India SE Mentor Solutions Full time

    Company Description SE-Mentor Solutions is a software engineering company specializing in Software Development, Data Engineering, and Software test automation. We assist clients in refining their software engineering practices and achieving success. We serve top-tier software companies, Government organizations, and educational institutions across various...


  • Cochin, Kerala, India Reflections Info Systems Full time

    We are looking for 13+years experienced candidates for this role.Mode: HybridPreferred Location: Cochin/Trivandrum/ChennaiPrimary Skills :- Overall 13+ Years of Experience in IT with 6+ years of experience as a Technical Product Manager, Technical Program Manager, or Delivery Lead.- Hands-on development experience in data engineering, including data...


  • Cochin, Kerala, India beBeeDataEngineer Full time ₹ 12,00,000 - ₹ 20,00,000

    Job Summary:We are seeking an experienced Python data engineer to join our organization. As a key member of our data initiatives, you will be responsible for designing and maintaining scalable, reliable data pipelines using Python and orchestration tools.Main Responsibilities:Create robust and scalable data pipelines using Python.Utilize orchestration and...