Senior ETL Developer

2 days ago

Salem, Tamil Nadu, India beBeeETL Full time ₹ 20,00,000 - ₹ 40,00,000
Senior ETL Developer

We are seeking a highly skilled Senior ETL Developer to join our team and help us design, develop, and maintain efficient data pipelines.

  • Key Responsibilities:
  • 1. Data Pipeline Development:
  • Design and develop robust data pipelines using Databricks PySpark to extract, transform, and load data from heterogeneous sources into our data lake and data warehouse.
  • Optimize data pipeline workflows for performance and scalability, leveraging Databricks PySpark and Spark SQL to efficiently process large data volumes.
  • Implement robust error handling and monitoring mechanisms to proactively detect and resolve issues within data pipelines.
  • Design and implement data solutions following the Medallion Architecture principles, organizing data into Bronze, Silver, and Gold layers.
  • Ensure data is appropriately cleansed, enriched, and optimized at each stage to support robust analytics and reporting.
  • 2. Data Management:
  • Develop and maintain data pipelines using Databricks PySpark, ensuring data quality, integrity, and reliability throughout the data lifecycle.
  • Collaborate with data engineering, data science, and business intelligence teams to translate data requirements into efficient data pipelines and workflows.
  • 3. Query Optimization:
  • Write and optimize complex SQL queries for data manipulation, aggregation, and analysis within Databricks PySpark applications.
  • 4. Project Coordination and Continuous Improvement:
  • Participate in project planning and coordination activities to ensure timely delivery of data solutions.
  • Stay updated on the latest developments in Databricks PySpark, Spark SQL, and related technologies, recommending and implementing best practices and optimizations.
  • Document data pipelines, data lineage, and metadata to facilitate knowledge sharing and ensure compliance with data governance standards.
  • Required Skills and Qualifications:
  • Bachelor's degree in Computer Science, Engineering, or a related field.
  • Minimum of 3 years of experience as an ETL developer, with a strong focus on Databricks PySpark development.
  • Proficiency in Python programming, with extensive experience in developing and debugging Databricks PySpark applications.
  • In-depth understanding of Spark architecture and internals, with hands-on experience in Spark RDDs, DataFrames, and Spark SQL.
  • Expertise in writing and optimizing complex SQL queries for data manipulation, aggregation, and analysis.
  • Proven experience in working with large-scale data warehousing and ETL frameworks.
  • Strong problem-solving skills and the ability to troubleshoot and resolve data pipeline issues.
  • Excellent communication and collaboration skills, with the ability to work effectively in a team environment.
  • Preferred Skills and Qualifications:
  • Experience with cloud platforms with a preference for AWS.
  • Experience with data platform tools such as DataBricks, Snowflake, and Tableau.
  • Demonstrated ability to implement best practices for data management.
  • Strong understanding of data governance and data quality principles.
  • Relevant certifications in Databricks PySpark, Spark SQL, or related technologies.
Benefits

The ideal candidate will be offered a competitive salary and benefits package, including comprehensive health insurance, retirement plan, and paid time off.

  • ETL Development Specialist

    3 days ago

    Salem, Tamil Nadu, India beBeeInformatica Full time ₹ 12,00,000 - ₹ 20,00,000

    Job Opportunity: ETL DeveloperThe role involves understanding requirements, designing and developing ETL architecture, transformation jobs and workflow schedules for a global company.Key responsibilities include design and development of scalable ETL workflows using Informatica BDM/DEI, integrating data from structured and unstructured sources including big...

  • Senior ETL Testing Specialist

    2 days ago

    Salem, Tamil Nadu, India beBeeData Full time ₹ 60,00,000 - ₹ 1,25,00,000

    Role Summary:We are seeking a skilled ETL Tester to join our organization. As an ETL Tester, you will be responsible for ensuring the integrity of Extract, Transform, and Load processes.Key Responsibilities:Develop comprehensive test plans for ETL processesAnalyze and troubleshoot issues in ETL processesCollaborate with development teams to identify process...

  • Expert ETL Developer Wanted

    4 days ago

    Salem, Tamil Nadu, India beBeeETLDeveloper Full time ₹ 20,00,000 - ₹ 25,00,000

    We are seeking a skilled professional to join our team as an ETL Developer with expertise in Talend Open Studio.Position Details:- Role: ETL Developer- Experience: 8 to 10 years- Work Timings: Flexible work hours, remote available- Duration: Contractual position, minimum 3 months- Location Preference: South Indian Profiles PreferredMandatory Requirements:-...

  • Senior Business Intelligence Developer

    4 days ago

    Salem, Tamil Nadu, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 24,00,000

    Extracting insights from complex data sets is crucial for business growth. As a Senior Data Engineer, you will design and develop efficient ETL pipelines using FiveTran, DBT Cloud, and custom frameworks.Key ResponsibilitiesMaintain high-quality data exchange by implementing client-specific data transformations in HighTouch integration.Collaborate with the...

  • Lead ETL Data Engineer

    2 days ago

    Salem, Tamil Nadu, India beBeeDatastage Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Job Title: ETL Developer - DataStage, AWS, SnowflakeWe are seeking an experienced ETL Lead with expertise in IBM DataStage to design and develop scalable data pipelines using various technologies.This role involves close collaboration with architects, business analysts, and data modelers to ensure timely and accurate delivery of critical data assets...

  • ETL Data Quality Specialist

    2 days ago

    Salem, Tamil Nadu, India beBeeQuality Full time ₹ 24,00,000 - ₹ 30,00,000

    Job Title: Quality Assurance Analyst As a skilled quality assurance professional, you will play a crucial role in ensuring the integrity of our software products. Key Responsibilities:Develop and execute comprehensive test plans to identify defects and improve overall system reliability.Collaborate with cross-functional teams to analyze and resolve complex...

  • Senior Data Warehouse Engineer

    3 weeks ago

    Salem, Tamil Nadu, India Asian Hires Full time

    POSITION REPORTS TO: Senior Manager Application DevelopmentDEPARTMENT: Information TechnologyPOSITION LOCATION: INDIA (REMOTE)COMPANY BUDGET: 18,00,000 - 23,00,000Job SummaryWe are seeking a highly skilled Senior Data Warehouse Engineer to manage a single version of the truth ofdata, convert business entities into facts and dimensions, and integrate data...

  • Senior Data Pipeline Developer

    2 days ago

    Salem, Tamil Nadu, India beBeeDataPipeline Full time ₹ 12,00,000 - ₹ 20,00,000

    Data Pipeline Developer PositionWe are seeking an experienced professional to design and maintain large-scale data pipelines using Python and AWS.Data Pipelines Development: Develop and implement complex data pipelines that efficiently manage and process massive datasets.ETL Processes Implementation: Implement and optimize Extract, Transform, Load processes...

  • Senior Test Lead

    5 days ago

    Salem, Tamil Nadu, India NovintiX Full time

    𝗔𝗯𝗼𝘂𝘁 𝘁𝗵𝗲 𝗖𝗼𝗺𝗽𝗮𝗻𝘆NovintiX is a fast-growing engineering and digital transformation consultancy serving top Life Sciences, MedTech, and Healthcare clients across the US, Ireland, and Europe. We specialise in delivering high-impact digital programs, engineering services, and talent solutions that accelerate...

  • Python Development Position

    2 days ago

    Salem, Tamil Nadu, India beBeeDevelopment Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title:Main Python Developer PositionDescription:We are seeking an accomplished professional in large dataset processing and data automation to design, develop, and execute scalable ETL workflows.This role involves collaborating with cross-functional teams to support the development of a cutting-edge digital platform. The successful candidate will possess...