Data Pipeline Architect

2 weeks ago

pushkar, India beBeeData Full time

Scalable Data Pipeline ArchitectWe're seeking a skilled professional to lead the design, building and optimization of data pipelines. The ideal candidate will have expertise in cloud platforms, modern frameworks and distributed processing technologies.The role requires strong experience with Python, SQL and data processing practices, as well as hands-on experience working with AI/LLM frameworks.As a leader in our organization, you will collaborate with cross-functional teams to drive innovation and growth. You will develop technical solutions to meet business requirements and ensure high availability and reliability of data systems.About the Role:Design and implement scalable data pipelines using cloud platforms and modern frameworks.Develop and maintain robust crawling systems using IP rotation, throttling and dynamic content extraction.Establish pipeline orchestration using Airflow, Celery or similar technologies.Define and enforce data quality, validation and security measures across all data flows and pipelines.Collaborate with product, engineering and analytics teams to translate business requirements into scalable technical solutions.Develop monitoring, logging and performance metrics to ensure high availability and reliability of data systems.

  • Data Architect Specialist

    2 weeks ago

    pushkar, India beBeeData Full time

    Data Architect SpecialistJob Overview:We are seeking a skilled Data Architect Specialist to design and implement efficient data architectures, ensure robust pipeline performance through monitoring and optimization, and drive business growth by leveraging data-driven insights.Key Responsibilities:Data Engineering & Pipeline DevelopmentDesign and develop...

  • Data Architect

    1 week ago

    pushkar, India beBeeData Full time

    We are looking for an experienced Data Architect to design and develop scalable data pipelines that drive business insights.Job Title: Data ArchitectResponsibilities:Create efficient data ingestion processes from various sources including APIs, databases, and third-party platforms.Design and maintain scalable ETL/ELT frameworks using PySpark and AWS...

  • Chief Data Architect

    2 weeks ago

    pushkar, India beBeeDataEngineer Full time

    Job OpportunityWe are seeking an experienced data professional to join our team as a Senior Data Engineer.Data Ingestion and Transformation: Design and develop scalable pipelines in an Azure-based environment.Real-Time Data Streaming: Work with Kafka for efficient message queuing and real-time data processing.ETL/ELT Pipelines: Build high-performance...

  • Chief Data Architect

    1 week ago

    pushkar, India beBeeDataAnalytics Full time

    Job SummarySeeking an experienced Data Analytics Engineer to design and implement scalable data models, architect robust data pipelines, and optimize Snowflake performance.This role involves collaborating with cross-functional teams to deliver insightful dashboards and reports using Sigma Computing.Design and build efficient data models that align with...

  • GCP Data Pipeline Specialist

    1 week ago

    pushkar, India beBeeDataEngineer Full time

    At ThreatXIntel, we empower businesses with reliable and accessible cybersecurity services. Our mission is to monitor and test clients' digital environments to identify vulnerabilities before they can be exploited.We seek a skilled Freelance GCP Data Engineer with expertise in Apache NiFi, SQL, and API integrations to design, implement, and optimize...

  • Senior Data Architect

    2 days ago

    pushkar, India beBeeData Full time

    Key Skills for Data Architect SuccessEpsilon Technologies Group is seeking a skilled and experienced data architect to design, build, and deploy data-centered solutions.Create efficient end-to-end data delivery solutions from acquisition through analysis, visualization, and decision support.Develop scalable ETL/ELT processes using tools such as Matillion,...

  • Data Engine Architect

    2 days ago

    pushkar, India beBeeSpecialist Full time

    Information Retrieval SpecialistWe're seeking an experienced Information Retrieval Specialist to join our team. As a key member of our data engine, you will design and implement search solutions using OpenSearch/Elasticsearch.Key Responsibilities:Develop custom query builders and search pipelines tailored to business requirements.Leverage LLMs and AI for...

  • Enterprise Data Architect

    2 weeks ago

    pushkar, India beBeeData Full time

    Key Role in Data InfrastructureEpsilon seeks a skilled Data Architect to craft and deploy data-centric solutions encompassing sourcing, validation, transformation, analysis, and presentation. This pivotal role will engage with both onshore and offshore teams to shape the future of capital markets technology.Job Responsibilities:Design and implement...

  • Cloud Data Systems Architect

    1 week ago

    pushkar, India beBeeData Full time

    Cloud Data Systems ArchitectAs a Cloud Data Systems Architect, you will design, build, and maintain large-scale data processing systems on Google Cloud Platform. Experience with GCP services such as Dataflow, BigQuery, and Storage Classes is necessary.The ideal candidate has 5+ years of experience in data engineering and proficiency in Python, particularly...

  • Azure Data Engineer for High-Performance Data Pipelines

    1 week ago

    pushkar, India beBeeDataEngineer Full time

    Job OpportunityWe are seeking an experienced data engineer to design and develop ETL/ELT pipelines using Azure Data Factory (ADF) and Databricks (PySpark).Azure Data Engineering Expertise: Strong experience in designing, developing, and maintaining ETL/ELT pipelines for efficient data transformation and integration.Data Integration Specialist: Build and...