Real-Time Data Pipeline Specialist

1 week ago

dindigul, India beBeeDataEngineer Full time

Job SummaryWe are seeking a skilled Data Engineer to design and build real-time data ingestion and processing pipelines.Key ResponsibilitiesDesign and implement scalable solutions leveraging Apache NiFi for both structured and unstructured datasets.Integrate Apache NiFi with the Google Cloud Platform (GCP) ecosystem, including BigQuery, Pub/Sub, Dataflow, Dataproc, Cloud Storage, and Composer.Develop and optimize SQL queries for BigQuery and relational databases to support analytics and reporting.Build Python scripts for data transformation, automation, and developing custom NiFi processors.Implement data quality checks, validation rules, deduplication, late-arriving data handling, and robust error-handling frameworks.Collaborate with cross-functional teams to deliver high-quality data products.Monitor, troubleshoot, and optimize pipeline performance to ensure stability and reliability.Required Skills:Apache NiFi developmentGoogle Cloud Platform experienceSQL query optimizationPython scriptingData engineeringBonus Points:Distributed systems experienceCloud security knowledgeAgile methodologies understanding

  • Data Pipeline Specialist

    1 week ago

    dindigul, India beBeeDataQuality Full time

    Data Pipeline Specialist Job DescriptionAs a Data Pipeline Specialist, you will play a crucial role in ensuring the smooth functioning of our data pipelines.The ideal candidate will have a strong background in software testing with experience in ETL tools like Informatica or Talend. Proficiency in programming languages such as Java, Python, or SQL is also...

  • Real-Time Data Solutions Architect

    2 weeks ago

    dindigul, India beBeeDataEngineering Full time

    Senior Data Engineer - Real-Time Data SolutionsAbout the RoleWe are seeking a highly skilled Senior Data Engineer to join our data engineering team. The ideal candidate will have extensive hands-on experience in building and optimizing data ingestion and transformation pipelines using Azure Data Platform, Databricks, Snowflake, and Kafka.You will be...

  • Data Pipelines Specialist

    1 week ago

    dindigul, India beBeeDataEngineeringExpert Full time

    Job Title: Data Engineering ExpertWe are seeking a highly skilled professional with strong experience in large-scale data processing to join our team. You will be responsible for designing, developing, and maintaining scalable data pipelines that drive business insights and operational efficiency.Key Responsibilities:Design and develop ETL/ELT pipelines...

  • Cloud Data Pipeline Architect

    2 weeks ago

    dindigul, India beBeeDataEngineer Full time

    Job Summary:Azure Databricks EngineerAbout the Role:We are seeking a highly skilled data engineer to manage data ingestion pipelines, handle ETL/ELT processes and provide excellent support in incident management, service monitoring and cluster troubleshooting.The ideal candidate will have strong knowledge of Microsoft Azure services, proficiency in...

  • Data Pipeline Specialist

    4 days ago

    dindigul, India beBeeData Full time

    Support Engineering ExpertKey ResponsibilitiesIssue ResolutionEscalation PointMonitoring and Data QualityRoot Cause AnalysisCollaboration and CommunicationWe are seeking a highly skilled Support Engineer to serve as the escalation point for data ingestion, pipeline, and consumption issues. This role requires strong technical skills and ability to...

  • Cloud Data Support Specialist

    2 weeks ago

    dindigul, India beBeeData Full time

    Cloud Data Support SpecialistSeeking a skilled Cloud Data Support Specialist to join our team. The ideal candidate will have a strong foundation in Unix system administration, a solid understanding of cloud-based data services (especially AWS), and hands-on experience with ETL pipelines and production support.Administer and support Unix/Linux-based...

  • High Performance Data Pipelines Specialist

    1 week ago

    dindigul, India beBeeDataEngineer Full time

    Senior Data Engineer RoleWe are seeking a seasoned Data Engineering Expert to craft and optimize high-performance data pipelines and systems. The ideal candidate will possess extensive experience with cloud platforms, ETL/ELT tools, and proficiency in Python, SQL, and distributed frameworks.Key Responsibilities:Design, develop, and maintain scalable ETL/ELT...

  • Azure Data Engineer Specialist

    1 day ago

    dindigul, India beBeeSenior Full time

    We are seeking a senior data engineer to build and maintain our data pipeline for enterprise search applications. The ideal candidate will have experience with Azure Data Factory, Databricks, and other big data technologies.Key Responsibilities:Design and implement data integration feeds according to business requirementsDevelop pipelines between different...

  • Data Integration Specialist

    4 days ago

    dindigul, India beBeeData Full time

    Job Title: Data Integration SpecialistData Migration and Transformation ExpertWe are seeking a skilled data integration specialist to join our team. As an experienced ETL developer, you will be responsible for designing and implementing data migration solutions using strong SQL scripting skills.The ideal candidate will have a solid background in extract,...

  • QA Data Integration Specialist

    1 week ago

    dindigul, India beBeeData Full time

    Job Title:QA Data Integration Specialist We are seeking a skilled professional to lead the testing of our data pipelines. The ideal candidate will have strong hands-on experience in data integration testing, API testing, and master data management (MDM) testing.The successful candidate will be responsible for performing end-to-end testing of data pipelines,...