Senior Data Pipeline Architect

3 days ago

Tirupati, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

We are seeking an experienced Data Engineer to design, develop and maintain data pipelines and infrastructure. As a key member of our team, you will work closely with data scientists, analysts and other stakeholders to ensure data is accurate, timely and accessible.

Key Responsibilities:
  • Design, develop and implement data pipelines using Azure Data Factory and Databricks to support the ingestion, transformation and movement of data.
  • Develop and optimize ETL (Extract, Transform, Load) processes to ensure efficient data flow and transformation.
  • Develop and maintain Azure Data Lake solutions ensuring efficient storage and retrieval of large datasets.
  • Work with Azure Synapse Analytics to build and manage scalable data warehousing solutions that enable advanced analytics and reporting.
  • Integrate various data sources into MS-Fabric ensuring data consistency, quality and accessibility across different platforms.
  • Optimize data processing workflows and storage solutions to improve performance and reduce costs.
  • Manage and optimize databases (SQL and NoSQL) to support high-performance queries and data storage requirements.
  • Implement data quality checks and monitoring to ensure accuracy and consistency of data.
  • Collaborate with data scientists, analysts and other stakeholders to understand data requirements and deliver actionable insights.
  • Create and maintain comprehensive documentation for data processes, pipelines and infrastructure architecture and best practices.
  • Identify and resolve issues in data pipelines, data lakes and warehousing solutions providing timely support and maintenance.

Requirements:

  • Experience: 2-4 years of experience in data engineering or a related field.
  • Technical Skills:
  1. Proficiency with Azure Data Factory, Azure Synapse Analytics, Databricks and Azure Data Lake
  2. Strong SQL skills and experience with data warehousing concepts (DWH)
  3. Knowledge of data modeling, ETL processes and data integration
  4. Experience with relational databases (e.g., MS-SQL, PostgreSQL, MySQL)
  5. Hands-on experience with ETL tools and frameworks (e.g., Apache Airflow, Talend)
  6. Knowledge of big data technologies (e.g., Hadoop, Spark) is a plus
  7. Familiarity with cloud platforms (e.g., AWS, Azure, Google Cloud) and associated data services (e.g., S3, Redshift, BigQuery)
  8. Familiarity with data visualization tools (e.g., Power BI) and experience with programming languages such as Python, Java or Scala.
  9. Experience with schema design and dimensional data modeling
Analytical Skills: Strong problem-solving abilities and attention to detail.Communication: Excellent verbal and written communication skills with the ability to explain technical concepts to non-technical stakeholders.Education: Bachelor's degree in computer science, Engineering, Mathematics or a related field. Advanced degrees or certifications are a plus.

  • Chief Data Pipeline Architect

    13 hours ago

    Tirupati, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    We're seeking a highly skilled Data Engineer to join our team and drive the design and optimization of our distributed data pipelines, infrastructure, and tools.This is an exciting opportunity for someone with 2-7 years of hands-on experience in data engineering to work on high-impact projects shaping the future of the industry.The ideal candidate will have...

  • Senior Data Systems Architect

    6 days ago

    Tirupati, Andhra Pradesh, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Senior Data Systems ArchitectWe are excited to announce an opportunity for a skilled data architect to join our team.About the Role: As a senior data systems architect, you will play a crucial role in designing, developing, and deploying large-scale data processing systems using cutting-edge technologies like PySpark and...

  • Data Pipeline Specialist

    1 day ago

    Tirupati, Andhra Pradesh, India beBeeDataEngineering Full time ₹ 1,20,00,000 - ₹ 2,00,00,000

    We are seeking a skilled data engineer to join our team as a Data Pipeline Specialist. The successful candidate will be responsible for designing, implementing and optimizing data pipelines using Azure Synapse, Databricks and Apache Spark (PySpark).About the RoleThe Data Pipeline Specialist will work closely with data architects, analysts and product owners...

  • Freelance Data Pipeline Architect

    6 days ago

    Tirupati, Andhra Pradesh, India beBeeAzureDataFactory Full time ₹ 30,00,000 - ₹ 40,00,000

    We are seeking a highly skilled Freelance Azure Data Factory Specialist to support our data integration and ETL needs.About the RoleThe ideal candidate will have strong hands-on expertise in designing, building, and maintaining scalable and secure data pipelines across multiple sources and destinations using Azure Data Factory (ADF).Key...

  • Chief Data Pipeline Architect

    1 week ago

    Tirupati, Andhra Pradesh, India beBeeDataEngineering Full time ₹ 18,00,000 - ₹ 24,00,000

    Job Title:Azure Data EngineerJob Description:We are seeking a skilled data engineer to develop, optimize and maintain automated data pipelines supporting governance and analytics initiatives.This role involves designing, building and deploying ETL/ELT pipelines in Databricks (PySpark, SQL, Delta Lake) to ingest, transform and curate metadata from multiple...

  • Senior AWS Data Pipeline Architect

    1 week ago

    Tirupati, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 25,00,000

    Job SummaryWe are seeking an experienced Data Engineer with expertise in AWS to join our organization. As a Data Engineer, you will design and implement robust data pipelines using AWS, Python, and SQL to support business analytics and decision-making across multiple domains.The ideal candidate will have a strong command of SQL for creating, managing, and...

  • Senior Data Architect

    7 days ago

    Tirupati, Andhra Pradesh, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Title: Data ArchitectA key player in our team, the ideal candidate will design and implement scalable data architectures using Medallion Architecture principles.Main Responsibilities:Develop data lake solutions in Azure Data Lake Storage (ADLS) Gen2 to store and process large datasets.Create optimized SQL queries for data extraction, transformation, and...

  • Senior Data Architect

    13 hours ago

    Tirupati, Andhra Pradesh, India beBeeData Full time ₹ 90,00,000 - ₹ 1,50,00,000

    Job Title:Senior Data ArchitectAbout the Role:We are seeking a skilled and experienced Data Architect to lead our data management initiatives. The successful candidate will have a strong background in data architecture, with 3+ years of experience in HRIS implementation, data governance, or system integration.Key Responsibilities:Data Strategy Development:...

  • Senior Cloud Architect

    15 hours ago

    Tirupati, Andhra Pradesh, India beBeeDevops Full time ₹ 1,80,00,000 - ₹ 2,10,00,000

    We are seeking an accomplished Solution Architect to spearhead the development and maintenance of our Software Configuration Management (SCM) suite and DevOps platform. This role requires a seasoned professional with extensive expertise in Continuous Integration/Continuous Deployment (CI/CD) pipelines, infrastructure automation, security, and identity...

  • Data Architect

    15 hours ago

    Tirupati, Andhra Pradesh, India beBeeData Full time ₹ 80,00,000 - ₹ 2,00,00,000

    As a highly skilled Data Architect, you will be responsible for designing and implementing data pipelines that meet the needs of our organization.Key Responsibilities:Develop data visualizations using business intelligence tools to provide actionable insights to stakeholders.Design and optimize scalable data architectures to ensure efficient data processing...