ETL Specialist

2 weeks ago

Coimbatore, Tamil Nadu, India Responsive Full time

We are seeking a highly skilled ETL Specialist with expertise in building, maintaining, and optimizing data pipelines using Python scripting. The ideal candidate will have experience working in a Linux environment, managing large-scale data ingestion, processing files in S3, and balancing disk space and warehouse storage efficiently. This role will be responsible for ensuring seamless data movement across systems while maintaining performance, scalability, and reliability.

Key Responsibilities:

  • ETL Pipeline Development: Design, develop, and maintain efficient ETL workflows using Python to extract, transform, and load data into structured data warehouses.
  • Data Pipeline Optimization: Monitor and optimize data pipeline performance, ensuring scalability and reliability in handling large data volumes.
  • Linux Server Management: Work in a Linux-based environment, executing command-line operations, managing processes, and troubleshooting system performance issues.
  • File Handling & Storage Management: Efficiently manage data files in Amazon S3, ensuring proper storage organization, retrieval, and archiving of data.
  • Disk Space & Warehouse Balancing: Proactively monitor and manage disk space usage, preventing storage bottlenecks and ensuring warehouse efficiency.
  • Error Handling & Logging: Implement robust error-handling mechanisms and logging systems to monitor data pipeline health.
  • Automation & Scheduling: Automate ETL processes using cron jobs, Airflow, or other workflow orchestration tools.
  • Data Quality & Validation: Ensure data integrity and consistency by implementing validation checks and reconciliation processes.
  • Security & Compliance: Follow best practices in data security, access control, and compliance while handling sensitive data.
  • Collaboration with Teams: Work closely with data engineers, analysts, and product teams to align data processing with business needs.

Skills Required:

  • Proficiency in Python: Strong hands-on experience in writing Python scripts for ETL processes.
  • Linux Expertise: Experience working with Linux servers, command-line operations, and system performance tuning.
  • Cloud Storage Management: Hands-on experience with Amazon S3, including handling file storage, retrieval, and lifecycle policies.
  • Data Pipeline Management: Experience with ETL frameworks, data pipeline automation, and workflow scheduling (e.g., Apache Airflow, Luigi, or Prefect).
  • SQL & Database Handling: Strong SQL skills for data extraction, transformation, and loading into relational databases and data warehouses.
  • Disk Space & Storage Optimization: Ability to manage disk space efficiently, balancing usage across different systems.
  • Error Handling & Debugging: Strong problem-solving skills to troubleshoot ETL failures, debug logs, and resolve data inconsistencies.

Nice to Have:

  • Experience with cloud data warehouses (e.g., Snowflake, Redshift, BigQuery).
  • Knowledge of message queues (Kafka, RabbitMQ) for data streaming.
  • Familiarity with containerization tools (Docker, Kubernetes) for deployment.
  • Exposure to infrastructure automation tools (Terraform, Ansible).

Qualifications:

  • Bachelor's degree in Computer Science, Data Engineering, or a related field.
  • 3+ years of experience in ETL development, data pipeline management, or backend data engineering.
  • Strong analytical mindset and ability to handle large-scale data processing efficiently.
  • Ability to work independently in a fast-paced, product-driven environment.

  • ETL Specialist

    2 days ago

    Coimbatore, Tamil Nadu, India beBeeEtl Full time ₹ 90,00,000 - ₹ 1,20,00,000

    We are seeking a skilled ETL professional to join our team as an ETL Developer. This role involves working with Talend Open Studio, BigQuery, PostgreSQL, Python, and GCP.**Job Summary:**We require an experienced developer who can design and optimize complex ETL pipelines using TMC. Additionally, they should be proficient in BigQuery, PostgreSQL, Python, and...

  • Senior Data Integration Specialist

    2 days ago

    Coimbatore, Tamil Nadu, India beBeeDataIntegration Full time US$ 1,20,000 - US$ 1,70,000

    Job Title: Senior Data EngineerData Integration Specialist Job OverviewWe are seeking a highly skilled and experienced data integration specialist to lead the design, development, and deployment of complex ETL solutions using Informatica IDMC.The ideal candidate will have a strong background in developing, documenting, testing, and maintaining ETL...

  • T24 Data Migration Specialist

    3 hours ago

    Coimbatore, Tamil Nadu, India beBeeData Full time ₹ 1,20,00,000 - ₹ 2,00,00,000

    T24 Data Migration Expert WantedWe are currently seeking a seasoned T24 Data Migration Specialist to spearhead data migration initiatives for core banking system implementations. This expert will be responsible for designing, implementing and executing end-to-end data migration strategies ensuring accuracy, compliance and seamless integration throughout the...

  • High-Level Data Validation Expert

    2 days ago

    Coimbatore, Tamil Nadu, India beBeeDataQuality Full time ₹ 20,00,000 - ₹ 25,00,000

    Lead Data Assurance SpecialistWe are seeking a seasoned professional to spearhead our data validation and quality assurance efforts.The ideal candidate will possess strong technical expertise in Snowflake, ETL, Data Warehousing, and Data Migration testing, combined with exceptional leadership qualities to guide junior engineers.Key Responsibilities:Oversee...

  • Cloud Data Specialist

    3 hours ago

    Coimbatore, Tamil Nadu, India beBeeData Full time ₹ 18,00,000 - ₹ 25,00,000

    Job Title: Cloud Data SpecialistWe are seeking a skilled Cloud Data Specialist to join our team. This is an exciting opportunity to work with cutting-edge technology and contribute to the development of innovative data solutions.As a Cloud Data Specialist, you will be responsible for designing and implementing data pipelines that clean, transform, and...

  • Senior Data Integration Specialist

    2 days ago

    Coimbatore, Tamil Nadu, India beBeeData Full time ₹ 7,50,000 - ₹ 15,00,000

    Informatica DeveloperThis role involves working on data integration and operations teams, utilizing skills in ETL development, support, data cleansing, quality control, and transformation.Data Integration Specialist: Develop and maintain Informatica mappings, workflows, and sessions to ensure seamless data flow between systems.ETL Expert: Extract, transform,...

  • Data Quality Specialist

    15 hours ago

    Coimbatore, Tamil Nadu, India beBeeETL Full time ₹ 9,00,000 - ₹ 12,00,000

    We are seeking a detail-oriented individual with excellent analytical and problem-solving skills to join our team as an ETL Tester. The ideal candidate will have a strong understanding of ETL processes, data warehousing concepts, and SQL, as well as experience working in an Agile environment.Key responsibilities include developing and executing test plans...

  • Data Integration Specialist

    5 days ago

    Coimbatore, Tamil Nadu, India beBeeETL Full time

    Job DescriptionPTR Global is a leader in providing innovative workforce solutions, dedicated to optimizing talent acquisition and management processes.Our commitment to excellence has earned us the trust of businesses looking to enhance their talent strategies.We cultivate a dynamic and collaborative environment that empowers our employees to excel and...

  • Data Governance Specialist

    3 hours ago

    Coimbatore, Tamil Nadu, India beBeeDataGovernance Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data Catalog SpecialistWe are seeking an experienced data catalog specialist to join our team. This role will be responsible for ensuring the accuracy and integrity of our data assets, while also providing expertise on metadata management and data governance principles.Key Responsibilities:Metadata IngestionData Lineage StitchingWorkflow...

  • Senior Data Integration Specialist

    18 hours ago

    Coimbatore, Tamil Nadu, India beBeeInformatician Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job Title: Senior Informatica BDM/DEI Professional">This role is responsible for delivering strategic advisory services through a global network of experts in technology, business, and functional domains. We cater to companies across Asia Pacific, Middle East, and European Union, providing strategic and reselling partnerships in digital, automation, AI, and...