ai - Senior Data Architect - ETL/PySpark

3 weeks ago

Chandigarh, Chandigarh, India Sprouts Full time

Job Title : Senior Data Architect

Location : Bangalore/Chandigarh

Job Type : Full-time

Experience : 10+ years

Job Summary are looking for an experienced Data Architect to lead the design, development, and optimization of our modern data infrastructure. The ideal candidate will have deep expertise in big data platforms, data lakes, lakehouse architectures, and hands-on experience with modern tools such as Spark clusters, PySpark, Apache Iceberg, the Nessie catalog, and Apache Airflow.

This role will be pivotal in evolving our data platform, including database migrations, scalable data pipelines, and governance-ready architectures for both analytical and operational use cases.

Key Responsibilities :

- Design and implement scalable and reliable data architectures for real-time and batch processing systems

- Evaluate and recommend data tools, frameworks, and infrastructure aligned with company goals

- Develop and optimize complex ETL/ELT pipelines using PySpark and Apache Airflow

- Architect and manage data lakes using Spark on Apache Iceberg and Nessie catalog for

versioned and governed data workflows

- Perform data analysis, data profiling, data quality improvements, and data modeling

- Lead database migration efforts, including planning, execution, and optimization

- Define and enforce data engineering best practices, data governance standards, and schema

evolution strategies

- Collaborate cross-functionally with data scientists, analysts, platform engineers, and business Skills & Qualifications :

- 10+ years of experience in data architecture, data engineering, data security, data

governance, and big data platforms

- Deep understanding of trade-offs between managed services and open-source data stack

tools, including cost, scalability, operational overhead, flexibility, and vendor lock-in

- Strong hands-on experience with PySpark for writing data pipelines and distributed data processing

- Proven expertise with Apache Iceberg, Apache Hudi, and the Nessie catalog for modern table formats and versioned data catalogs

- Experience in scaling and managing Elasticsearch and PostgreSQL clusters

- Strong experience with Apache Airflow for workflow orchestration (or equivalent tools)

- Demonstrated success in database migration projects across multiple cloud providers

- Ability to perform deep data analysis and compare datasets between systems

- Experience handling 100s of terabytes of data or more

- Proficiency in SQL, data modeling, and performance tuning

- Excellent communication and presentation skills, with the ability to lead technical

conversations

Nice to Have :

- Experience in Sales, Marketing, and CRM domains, especially with Accounts and Contacts data

- Knowledge in AI and vector databases.

- Exposure to streaming data frameworks (Kafka, Flink, etc.)

- Ability to support analytics and reporting initiatives

Why Join Us :

- Work on cutting-edge data architectures using modern open-source technologies

- Be part of a team transforming data operations and analytics at scale

- Opportunity to architect high-impact systems from the ground up

- Join a collaborative, innovation-driven culture

(ref:hirist.tech)

  • Senior Data Analyst

    2 weeks ago

    Chandigarh, Chandigarh, India Aspire Talent Innovations Full time

    Role : Senior Analytics Engineer (Customer Success)- Shift : 4 PM - 12 AM (Overlap with USA - Eastern Time)- Experience : 5 - 6 years (startup background strongly preferred)- Focus mix : Data Analytics 50%, Data Engineering 30%, Customer Success 20%Company Overview :We are a unified, financial-services CRM with an AI agent co-pilot that connects fragmented...

  • Senior Data Solutions Developer

    2 weeks ago

    Chandigarh, Chandigarh, India beBeeAbinitio Full time ₹ 19,50,000 - ₹ 25,90,000

    About This Role: We are seeking highly skilled senior ETL developers to join our data engineering team. Key Responsibilities: Lead the design and development of scalable data solutions for business-critical applications in the payments domain. Work with Ab Initio Query It and Metadata Hub for metadata-driven development. Design and implement logical and...

  • Technical lead

    2 weeks ago

    Chandigarh, Chandigarh, India Trantor Full time

    Job ResponsibilitiesLead the development and integration of Python-based applications with LLMs (Open AI, Deep Seek, Anthropic, LLa MA, etc.).Architect and implement LLM pipelines including prompt engineering, retrieval-augmented generation (RAG), fine-tuning, and evaluation.Design scalable microservices and APIs for AI features.Collaborate with MLOps teams...

  • AI & Data Crawling Programmer

    7 days ago

    Chandigarh, Chandigarh, India Savanna HR Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    We are looking for a skilled and motivated AI and Data Crawling Programmer with 12 years of hands-on experience in building data-driven solutions. The ideal candidate will have practical exposure to AI/ML concepts, data crawling, and web scraping, with the ability to design efficient crawlers and contribute to AI model development. Prior experience in...

  • Senior Big Data Engineer

    2 weeks ago

    Chandigarh, Chandigarh, India Firminiq Systems Full time ₹ 2,00,000 - ₹ 20,00,000 per year

    Role OverviewAs a Big Data Engineer, you will be responsible for architecting, developing, and maintaining scalable data platforms. You will work on complex data migration projects, particularly transitioning from MySQL to NoSQL, while leveraging AWS cloud-based big data tools.Key ResponsibilitiesDesign, develop, and maintain large-scale data processing...

  • AI Architect and Engineer

    2 weeks ago

    Chandigarh, Chandigarh, India beBeeLeadership Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Technical Lead Job DescriptionThe Technical Lead will oversee the development and integration of Python-based applications with Large Language Models (LLMs). The successful candidate will have a strong understanding of LLM architecture and experience in designing scalable microservices and APIs for AI features.Job Responsibilities:Lead the development and...

  • Senior Robotics and AI Trainer

    2 days ago

    Chandigarh, Chandigarh, India Focalyt Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Company DescriptionFocalyt is a team of professionals dedicated to empowering youth in rural and urban areas through skill development. By providing education, vocational skills, and hands-on experience, Focalyt aims to enhance employability and reduce unemployment. The company aligns with The Pradhan Mantri Kaushal Vikas Yojana (PMKVY) to offer financial...

  • Technical Lead

    3 weeks ago

    Chandigarh, Chandigarh, India Trantor Full time

    Job Responsibilities Lead the development and integration of Python-based applications with LLMs (OpenAI, DeepSeek, Anthropic, LLaMA, etc.). Architect and implement LLM pipelines including prompt engineering, retrieval-augmented generation (RAG), fine-tuning, and evaluation. Design scalable microservices and APIs for AI features. Collaborate with MLOps...

  • Generative AI Expert

    2 weeks ago

    Chandigarh, Chandigarh, India Cogniter Technologies Full time

    Cogniter Technologies is hiring a Generative AI Expert to join our advanced AI/ML team. In this role, you will design and deliver scalable AI applications powered by Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), AI agents, and vector databases. If youre passionate about building intelligent systems that solve real-world problems, wed...

  • Technical Lead

    4 weeks ago

    Chandigarh, Chandigarh, India Trantor Full time

    Technical Lead – Python & LLM IntegrationLocation: Hybrid, Chandigarh, IndiaExperience: 6+ years in software development (at least 2+ years in Python leadership roles)Employment Type: Full-timeAbout the RoleWe are looking for a hands-on technical lead with strong Python expertise and a deep understanding of Large Language Model (LLM) integration to lead...