Senior Data Engineer

2 days ago

Coimbatore, Tamil Nadu, India ShellKode Full time US$ 1,50,000 - US$ 2,00,000 per year

About The Role
We are seeking a highly skilled
Senior Data Engineer
with expertise in
SQL, Python, and Spark
to join our team. The ideal candidate will have extensive experience in designing, building, and optimizing data pipelines on
AWS
,
Databricks
,
Snowflake
, and
Airflow
, enabling robust analytics and machine learning use cases. You will work closely with data scientists, analysts, and business stakeholders to ensure the availability, reliability, and performance of our data infrastructure.

Key Responsibilities

  • Data Pipeline Development:

  • Design, implement, and maintain scalable ETL/ELT pipelines using PySpark and Airflow.

  • Ingest data from various sources into Snowflake and other AWS-based storage and processing layers.

  • Data Architecture & Modeling:

  • Build and optimize data models for analytics and BI tools.

  • Ensure data consistency, integrity, and quality across multiple systems.

  • Performance Optimization:

  • Optimize Spark jobs and SQL queries for cost and performance.

  • Implement partitioning, caching, and efficient storage formats (Parquet, Delta).

  • Cloud & Platform Engineering:

  • Leverage AWS services such as S3, Lambda, Glue, EMR, and Redshift for large-scale data processing.

  • Work with Databricks to build unified data engineering and ML pipelines.

  • Automation & Orchestration:

  • Automate workflows using Apache Airflow (DAG creation, scheduling, monitoring).

  • Implement CI/CD pipelines for data workflows.

  • Data Governance & Security:

  • Ensure compliance with data governance, privacy, and security policies.

  • Implement monitoring, logging, and alerting for data pipelines.

Required Skills & Experience

  • Core Technical Skills:

  • Strong in SQL (complex joins, window functions, performance tuning).

  • Proficient in Python for data engineering use cases.

  • Expertise in Apache Spark (PySpark or Spark SQL).

  • Platform Expertise:

  • Hands-on experience with AWS data services (S3, Glue, EMR, Lambda, Redshift).

  • Strong working knowledge of Databricks for both ETL and ML workflows.
  • Experience with Snowflake (data loading, modeling, optimization).

  • Proficiency in Apache Airflow for orchestration.

  • Additional Skills:

  • Solid understanding of data lakehouse and warehouse architectures.

  • Experience with Delta Lake, Parquet, or ORC formats.
  • Familiarity with CI/CD and DevOps practices for data.

Preferred Qualifications

  • 4+ years of experience in data engineering.
  • Experience in building real-time/streaming pipelines (Kafka, Kinesis).
  • Knowledge of data governance tools and practices.
  • Exposure to Data Ops and data pipelines.

Skills: data,aws,spark,data engineering,snowflake,databricks

  • Senior Data Engineering Position

    7 days ago

    Coimbatore, Tamil Nadu, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Senior Data EngineerAbout the RoleWe are seeking a highly skilled data engineer to join our team. The ideal candidate will have experience designing and implementing scalable batch processing systems, optimizing database performance, and reducing latency metrics across our data platform.

  • Senior Data Center Engineer

    17 hours ago

    Coimbatore, Tamil Nadu, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Senior Data Center EngineerJob Description:We are seeking a highly skilled and experienced Senior Data Center Engineer to lead our team in designing, implementing, and managing complex data center infrastructure.Key Responsibilities:Develop and execute comprehensive plans for data center deployment, migration, and maintenance.Lead the design and...

  • Highly Skilled Senior Data Engineer

    3 days ago

    Coimbatore, Tamil Nadu, India beBeeData Full time US$ 22,00,000 - US$ 25,00,000

    Senior Data Engineer for AnalyticsWe are seeking a highly skilled and experienced Senior Data Engineer to join our team. As a key member of our analytics group, you will be responsible for designing, building, and maintaining large-scale data systems that drive business decisions.Job Description:In this role, you will work closely with cross-functional teams...

  • Chief Data Engineering Expert

    3 days ago

    Coimbatore, Tamil Nadu, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Senior Data Engineer RoleWe are seeking an experienced Senior Data Engineer to develop and manage ETL pipelines using Azure Data Factory and Fabric toolbox.Design and implement data models to deliver high-quality datasets for analytics across business areas.Collaborate with cross-functional teams to integrate data engineering solutions with existing...

  • Data Engineer

    2 days ago

    Coimbatore, Tamil Nadu, India Augusta HiTech Software Full time US$ 1,50,000 - US$ 2,00,000 per year

    Industry: ITQualification: Any DegreeRequired Skills: SQL, PythonWorking Shift: 2PM to 11PM ISTCity: CoimbatoreCountry: IndiaJob Title: Senior Data EngineerLocation: [Insert Location or "Remote"]Job Type: [Full-time / Contract]Experience Level: 8+ YearsOverviewWe are seeking a highly experienced Data Engineer with a strong background in SQL and Python. The...

  • Cloud Data Engineer

    4 days ago

    Coimbatore, Tamil Nadu, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job OverviewThis role is focused on designing, building, and optimizing data pipelines using cloud-based technologies. As a senior data engineer, you will be responsible for creating scalable ingestion and transformation workflows across multiple cloud platforms.">Key ResponsibilitiesDevelop and maintain ETL/ELT data pipelines using Apache Spark, AWS Glue,...

  • Senior Data Engineering Lead

    16 hours ago

    Coimbatore, Tamil Nadu, India beBeeDataEngineer Full time ₹ 17,33,000 - ₹ 25,11,000

    Job Title:AWS Senior Data Engineer**Key Responsibilities:**">Develop Spark jobs using Scala for new project requirements and enhance existing Spark jobs for ongoing product enhancements.Create new data pipelines using AWS Lambda or Apache Airflow, analyze data processing, integration, modeling, and reporting requirements, and define data loading strategies...

  • Senior Data Engineer

    6 days ago

    Coimbatore, Tamil Nadu, India NovintiX Full time

    Novinti X is seeking a highly skilled and experienced Data Engineers (x4) to join our growing data team focused on delivering impactful solutions in the healthcare and medical technology space.If you're passionate about building scalable pipelines, driving analytics, and working with healthcare data standards—this role is for you???? ????????????

  • Engineer/Senior Engineer

    2 days ago

    Coimbatore, Tamil Nadu, India Bull Machines Pvt Ltd Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Job Description: Senior Engineer – Construction Machinery Company OverviewAt Bull Machines Pvt. Ltd., we pride ourselves on innovation and excellence in the construction equipment industry. As we continue to grow, we are seeking a Senior Engineer to join our dynamic team. This role plays a crucial part in driving our engineering projects forward. Role...

  • Master Data Engineer

    6 days ago

    Coimbatore, Tamil Nadu, India beBeeDatabase Full time ₹ 10,00,000 - ₹ 15,00,000

    Senior Database AdministratorWe are seeking a highly skilled and proactive professional to lead our database operations.This senior role combines traditional database administration responsibilities with modern data engineering tasks to support our comparative data loads and ensure optimal performance of critical database systems.The ideal candidate will...