ETL Specialist

2 weeks ago

Coimbatore, India Responsive Full time

We are seeking a highly skilled ETL Specialist with expertise in building, maintaining, and optimizing data pipelines using Python scripting. The ideal candidate will have experience working in a Linux environment, managing large-scale data ingestion, processing files in S3, and balancing disk space and warehouse storage efficiently. This role will be responsible for ensuring seamless data movement across systems while maintaining performance, scalability, and reliability.Key Responsibilities:- ETL Pipeline Development: Design, develop, and maintain efficient ETL workflows using Python to extract, transform, and load data into structured data warehouses.- Data Pipeline Optimization: Monitor and optimize data pipeline performance, ensuring scalability and reliability in handling large data volumes.- Linux Server Management: Work in a Linux-based environment, executing command-line operations, managing processes, and troubleshooting system performance issues.- File Handling & Storage Management: Efficiently manage data files in Amazon S3, ensuring proper storage organization, retrieval, and archiving of data.- Disk Space & Warehouse Balancing: Proactively monitor and manage disk space usage, preventing storage bottlenecks and ensuring warehouse efficiency.- Error Handling & Logging: Implement robust error-handling mechanisms and logging systems to monitor data pipeline health.- Automation & Scheduling: Automate ETL processes using cron jobs, Airflow, or other workflow orchestration tools.- Data Quality & Validation: Ensure data integrity and consistency by implementing validation checks and reconciliation processes.- Security & Compliance: Follow best practices in data security, access control, and compliance while handling sensitive data.- Collaboration with Teams: Work closely with data engineers, analysts, and product teams to align data processing with business needs.Skills Required:- Proficiency in Python: Strong hands-on experience in writing Python scripts for ETL processes.- Linux Expertise: Experience working with Linux servers, command-line operations, and system performance tuning.- Cloud Storage Management: Hands-on experience with Amazon S3, including handling file storage, retrieval, and lifecycle policies.- Data Pipeline Management: Experience with ETL frameworks, data pipeline automation, and workflow scheduling (e.g., Apache Airflow, Luigi, or Prefect).- SQL & Database Handling: Strong SQL skills for data extraction, transformation, and loading into relational databases and data warehouses.- Disk Space & Storage Optimization: Ability to manage disk space efficiently, balancing usage across different systems.- Error Handling & Debugging: Strong problem-solving skills to troubleshoot ETL failures, debug logs, and resolve data inconsistencies.Nice to Have:- Experience with cloud data warehouses (e.g., Snowflake, Redshift, BigQuery).- Knowledge of message queues (Kafka, RabbitMQ) for data streaming.- Familiarity with containerization tools (Docker, Kubernetes) for deployment.- Exposure to infrastructure automation tools (Terraform, Ansible).Qualifications:- Bachelor’s degree in Computer Science, Data Engineering, or a related field.- 3+ years of experience in ETL development, data pipeline management, or backend data engineering.- Strong analytical mindset and ability to handle large-scale data processing efficiently.- Ability to work independently in a fast-paced, product-driven environment.

  • ETL Specialist

    2 weeks ago

    Coimbatore, India Responsive Full time

    We are seeking a highly skilled ETL Specialist with expertise in building, maintaining, and optimizing data pipelines using Python scripting. The ideal candidate will have experience working in a Linux environment, managing large-scale data ingestion, processing files in S3, and balancing disk space and warehouse storage efficiently. This role will be...

  • Etl Specialist

    1 week ago

    Coimbatore, India Responsive Full time

    We are seeking a highly skilled ETL Specialist with expertise in building, maintaining, and optimizing data pipelines using Python scripting. The ideal candidate will have experience working in a Linux environment, managing large-scale data ingestion, processing files in S3, and balancing disk space and warehouse storage efficiently. This role will be...

  • ETL Data Quality Specialist

    1 week ago

    coimbatore, India beBeeDataQuality Full time

    Job DescriptionWe are seeking a skilled professional to fill the role of ETL Data Quality Specialist. This position requires strong expertise in ETL and API testing, as well as solid SQL skills and data flow understanding.ETL / Data Pipeline Testing: Perform end-to-end testing of ETL pipelines in cloud/on-prem environments, validating source-target mappings,...

  • ETL Specialist

    2 weeks ago

    Coimbatore, Tamil Nadu, India, Tamil Nadu Responsive Full time

    We are seeking a highly skilled ETL Specialist with expertise in building, maintaining, and optimizing data pipelines using Python scripting. The ideal candidate will have experience working in a Linux environment, managing large-scale data ingestion, processing files in S3, and balancing disk space and warehouse storage efficiently. This role will be...

  • ETL Data Quality Specialist

    2 weeks ago

    coimbatore, India beBeeDataTransformation Full time

    Ensuring data integrity requires meticulous attention to detail and a deep understanding of data transformations.About the RoleThis position involves testing and validating ETL processes to guarantee accuracy and reliability in data flow from various sources to our database.The ideal candidate will have excellent problem-solving skills, experience in ETL...

  • Senior Data Engineering Specialist

    2 weeks ago

    coimbatore, India beBeeDataValidation Full time

    Job TitleA Data Validation Specialist is required to design, develop and execute data validation test cases for ETL processes. The successful candidate will validate data extraction, transformation and loading processes for accuracy and completeness.ResponsibilitiesDesign and develop ETL test cases, scenarios and scripts for validation and regression...

  • Data Transformation Expert

    4 days ago

    coimbatore, India beBeeEtl Full time

    ETL Specialist OpportunityWe are seeking a skilled ETL specialist to join our team. As an ETL specialist, you will play a vital role in shaping the future of data processing and analysis.Job Description:The successful candidate will be responsible for designing, developing, and executing test cases, scenarios, and scripts for validation and regression...

  • HCM Technical Specialist – Cloud Applications

    2 weeks ago

    coimbatore, India beBeeTechnical Full time

    Job TitleHCM Technical Specialist – Cloud ApplicationsWe are seeking a highly skilled HCM Technical Specialist to join our team and support the development and management of cloud-based applications. This is an exciting opportunity for a talented individual to work with cutting-edge technology and make a significant impact on our business.Key...

  • Job Title: Data Integration Specialist

    3 days ago

    coimbatore, India beBeeDataIntegration Full time

    Job Title: Data Integration SpecialistRole OverviewWe are seeking an experienced Data Integration Specialist to join our team. The ideal candidate will have a strong background in developing and maintaining ETL processes, reports, and database solutions using SSIS, SSRS, and SQL Server.Key ResponsibilitiesDevelopment & Implementation:Design and implement...

  • Unlock Your Potential as an Enterprise Data Specialist

    2 weeks ago

    coimbatore, India beBeeData Full time

    Job OpportunityWe are seeking a highly skilled Enterprise Data Specialist to join our team.Ensure accurate data lineage by analyzing legacy mainframe data flows.Document source-to-target mappings for ETL/ELT migration to Snowflake, collaborating with Data Engineers to validate data pipelines and performance.Partner with Reporting Analysts to guarantee data...