AI Driven Data Pipelines Architect

5 days ago

Aurangabad, Maharashtra, India beBeeData Full time ₹ 10,00,000 - ₹ 15,00,000

About Our Platform

We are building a cutting-edge, AI-driven trading platform that ingests vast streams of financial, alternative, and sentiment data and transforms them into actionable insights for informed decision-making.

Our mission is to integrate robust data engineering pipelines with Generative AI and advanced ML models to democratize access to sophisticated trading strategies and optimize business outcomes.

The Role

We are seeking a highly skilled Data Engineer who is passionate about designing and building scalable data pipelines in the cloud to support our growth initiatives.

You will work closely with cross-functional teams to ensure clean, validated, and well-structured data is available for model training, inference, and strategy generation.

Key Responsibilities

  • Design and implement scalable ingestion pipelines to handle high-volume, real-time, and batch data feeds from multiple sources.
  • Write custom code to run within cloud-based function apps for event-driven and scheduled ingestion tasks.
  • Integrate APIs, event streams, and unstructured data into unified pipelines to facilitate data processing and analytics.

Data Transformation & Storage

  • Implement preprocessing, validation, and enrichment logic to prepare datasets for AI/ML models and improve data quality.
  • Leverage cloud-based data lakes, analytics services, and query languages for efficient querying, storage, and OLAP-style analytics.

Collaboration with AI/ML

  • Partner with data scientists to ensure AI-ready datasets (structured, time-series, textual, and alternative data) are available for model development.
  • Enable backtesting and live trading workflows by delivering low-latency, reliable data pipelines that meet business requirements.

Operations & Observability

  • Establish monitoring, alerting, and logging for all pipelines to ensure reliability, scalability, and performance.
  • Design pipelines that can adapt to changing needs, failure recovery, and cost efficiency in the cloud.

Requirements

  • 5–8 years of experience in data engineering or pipeline development, with strong proficiency in programming languages.
  • Hands-on experience building serverless/cloud-native pipelines (cloud-based functions, AWS Lambda, or similar).
  • Strong expertise with cloud-based data stacks (data lakes, analytics services, query languages).
  • Proven ability to design and manage data ingestion, transformation, and storage pipelines at scale.
  • Solid understanding of batch vs. streaming data processing and how to optimize for low-latency use cases.
  • Bonus: familiarity with financial data structures or trading systems.
  • Entrepreneurial mindset with the ability to deliver hands-on solutions in a fast-paced environment.
],

  • Senior Software Architect

    3 weeks ago

    Aurangabad, Maharashtra, India Auric AI Labs Full time

    Build Systems That Don't Exist Yet We're looking for an exceptional architect who thinks from first principles to lead development of a revolutionary real-time battlefield vision intelligence platform.This isn't just another enterprise software job - you'll be solving problems at the intersection of AI, massive-scale data processing, and national...

  • AI Solutions Architect

    1 day ago

    Aurangabad, Maharashtra, India beBeeArtificial Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    AI Solutions ArchitectWe are seeking a highly skilled AI Specialist to contribute to impactful, large-scale AI solutions.This is a unique opportunity to work with an international team and integrate AI solutions into real-world products.As an AI Solutions Architect, you will design and develop custom machine learning and deep learning models for...

  • AI Infra Architect

    2 weeks ago

    Aurangabad, Maharashtra, India LTIMindtree Full time

    Job Description:What You'll DoArchitect the Future: Lead the end-to-end design and development of AI infrastructure, encompassing hardware, software, networking, and multi-cloud environments.Innovate and Evaluate: Assess, select, and implement best-in-class technologies, tools, and frameworks (e.g., TensorFlow, PyTorch, Kubernetes, Docker) to build and...

  • Database Data Architect

    1 day ago

    Aurangabad, Maharashtra, India beBeeDataArchitect Full time ₹ 24,50,000 - ₹ 31,00,000

    We're seeking a highly skilled Database Data Architect to design and implement our data warehouse architecture.About the Role:As a Database Data Architect, you'll work closely with our development team to develop and maintain ETL processes using RDBMS such as MySQL and Oracle SQL. You'll also be responsible for designing and implementing data pipelines using...

  • Full Stack Tech Lead

    4 weeks ago

    Aurangabad, Maharashtra, India S2T AI - AI-Powered Investigations Full time

    As a Technical Team Lead for our Deepfusion product, you will play a key role in shaping the development of an advanced investigative platform. You'll drive key technology decisions, oversee the full development lifecycle, and work to deliver a product that supports investigators by seamlessly integrating data from diverse sources.Key...

  • AI Architect

    2 days ago

    Aurangabad, Maharashtra, India beBeeEngineering Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Industrial AI EngineerWe are driving the future of industrial automation and engineering by developing intelligent AI agents tailored for the manufacturing and automotive sectors.In this role, you will design, build, and optimize ML models for intelligent requirement understanding and automation.You will develop scalable, production-grade AI pipelines and...

  • AI-Driven Solutions Developer

    1 day ago

    Aurangabad, Maharashtra, India beBeeData Full time US$ 5,00,000 - US$ 10,00,000

    Job OpportunityA cutting-edge company is seeking skilled professionals to collaborate on innovative AI-driven projects.This role is ideal for individuals who are passionate about driving technological advancements in AI and data science. In this position, you will work closely with a team of experts to design and implement efficient solutions using...

  • End-To-End Data Pipeline Developer

    39 minutes ago

    Aurangabad, Maharashtra, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    At Xebia, we are seeking a skilled professional to join our team. The successful candidate will have expertise in building end-to-end data pipelines and delivering scalable data solutions on Azure.Key Responsibilities:Design, develop, and run complete E2E Data PipelinesHandle data validation, transformation, and warehousingCollaborate with stakeholders for...

  • Chief Data Pipeline Architect

    2 hours ago

    Aurangabad, Maharashtra, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Data Engineering LeaderJob OverviewWe are seeking an experienced Data Engineering Leader to spearhead the design and implementation of end-to-end data pipelines and integration solutions. About YouA minimum of 8 years of experience in Technical, Solutioning, and Analytical roles.Strong understanding of Cloud platforms like Big Query or Redshift.Experience...

  • Chief Data Architect

    2 hours ago

    Aurangabad, Maharashtra, India beBeeDataArchitect Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Job Overview:We are seeking a highly skilled and innovative Data Architect to lead our data ecosystem strategy. As a key member of the team, you will be responsible for designing and implementing scalable and secure enterprise data architecture solutions that meet business needs.Key Responsibilities:Design and develop end-to-end data pipelines using ETL/ELT...