Chief Data Architect

4 days ago

Tirupati, Andhra Pradesh, India beBeeDataDeveloper Full time ₹ 80,00,000 - ₹ 1,50,00,000
Job Overview

We are seeking a skilled data developer to join our implementation team. As a key member of this team, you will play a pivotal role in designing, developing, and maintaining ETL processes that ensure efficient extraction, transformation, and loading of data from various sources into our data lake and data warehouse.

To succeed in this role, you will work closely with cross-functional teams to build and optimize data workflows that support the project's analytics and reporting needs.

Key Responsibilities:

  • Design and develop ETL processes using Databricks PySpark to extract, transform, and load data from heterogeneous sources into our data lake and data warehouse.
  • Optimize ETL workflows for performance and scalability, leveraging Databricks PySpark and Spark SQL to efficiently process large data volumes.
  • Implement robust error handling and monitoring mechanisms to proactively detect and resolve issues within ETL processes.
  • Design and implement data solutions following the Medallion Architecture principles, organizing data into Bronze, Silver, and Gold layers.
  • Ensure data is appropriately cleansed, enriched, and optimized at each stage to support robust analytics and reporting.
  • Hands On experience in creating advanced data pipelines using databricks workflows.
  • Develop and maintain data pipelines using Databricks PySpark, ensuring data quality, integrity, and reliability throughout the ETL lifecycle.
  • Collaborate with cross-functional teams to translate data requirements into efficient ETL workflows and pipelines.
  • Write and optimize complex SQL queries for data manipulation, aggregation, and analysis within Databricks PySpark applications.
  • Participate in project planning and coordination activities to ensure timely delivery of ETL solutions.
  • Stay updated on the latest developments in Databricks PySpark, Spark SQL, and related technologies, recommending and implementing best practices and optimizations.
  • Document ETL processes, data lineage, and metadata to facilitate knowledge sharing and ensure compliance with data governance standards.
Required Skills and Qualifications
  • Bachelor's degree in Computer Science, Engineering, or a related field.
  • Minimum of 3 years of experience as an ETL developer, with a strong focus on Databricks PySpark development.
  • Proficiency in Python programming, with extensive experience in developing and debugging Databricks PySpark applications.
  • In-depth understanding of Spark architecture and internals, with hands-on experience in Spark RDDs, DataFrames, and Spark SQL.
  • Expertise in writing and optimizing complex SQL queries for data manipulation, aggregation, and analysis.
  • Proven experience in working with large-scale data warehousing and ETL frameworks.
  • Strong problem-solving skills and the ability to troubleshoot and resolve ETL process issues.
  • Excellent communication and collaboration skills, with the ability to work effectively in a team environment.
Benefits

This role offers opportunities for growth and development in a dynamic and collaborative environment.

Preferred Qualifications
  • Experience with cloud platforms with a preference for AWS.
  • Experience with data platform tools such as DataBricks, Snowflake, and Tableau.
  • Demonstrated ability to implement best practices for ETL processes and data management.
  • Strong understanding of data governance and data quality principles.
  • Relevant certifications in Databricks PySpark, Spark SQL, or related technologies.

  • Chief AI Infrastructure Architect

    3 days ago

    Tirupati, Andhra Pradesh, India beBeescalability Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Chief AI Infrastructure ArchitectAbout the Role:In this pivotal position, you will spearhead the design and implementation of our cutting-edge AI infrastructure. Your primary objective is to ensure seamless scalability while aligning with our overarching technical vision.Key Responsibilities:

  • Chief Data Solutions Architect

    4 days ago

    Tirupati, Andhra Pradesh, India beBeeDataArchitect Full time ₹ 12,60,000 - ₹ 20,16,000

    Job Opportunity: Data ArchitectOverview:We are seeking a skilled and motivated Data Architect to design, develop, and maintain scalable data systems that process and transform large volumes of structured and unstructured data. The ideal candidate will have a strong background in data architecture, cloud technologies, and software development, and will...

  • Chief Data Architect

    2 days ago

    Tirupati, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Opportunity: Data Engineer">We are seeking a highly skilled professional to join our team as a Data Engineer.">This role involves designing and implementing a scalable data platform using open-source technologies.">">Design and Build a Scalable Data Warehouse: Develop a data warehouse for analytics and reporting that meets the organization's...

  • Senior Data Infrastructure Specialist

    1 day ago

    Tirupati, Andhra Pradesh, India beBeeCloudNative Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job Role: Chief Data ArchitectWe are seeking a seasoned data expert to lead our cloud-native data infrastructure.About the Position:The successful candidate will have 10+ years of experience in software development and data engineering, with a strong focus on backend and cloud architecture.Key Responsibilities:Design and implement workflow management systems...

  • Chief Data Engineering Specialist

    5 days ago

    Tirupati, Andhra Pradesh, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Data Architect PositionWe seek a seasoned data architect to lead multiple data engagements on GCP Cloud.Key ResponsibilitiesBuild and manage data lakes, data warehouses, data integration, data migration, and business intelligence/artificial intelligence solutions on cloud platforms like GCP/AWS/Azure.Translate business requirements into functional and...

  • Chief Cloud Architect

    3 days ago

    Tirupati, Andhra Pradesh, India beBeeCloud Full time US$ 2,00,000 - US$ 2,50,000

    Job OpportunityWe are seeking an experienced Salesforce Architect to join our team.Key Responsibilities:Lead and participate in solution architecture discussions across multiple Salesforce clouds.Define scalable architecture patterns and best practices for Salesforce implementation and integration.Collaborate with stakeholders to ensure architecture...

  • Data Platform Architect

    6 days ago

    Tirupati, Andhra Pradesh, India beBeeInnovation Full time US$ 1,80,000 - US$ 2,50,000

    Finance Innovation LeadWe're seeking a skilled Finance Innovation Lead to join our mission-driven team. This role involves architecting and developing platforms for financial data, providing technical structure to teams, and working closely with management and stakeholders to define strategic roadmaps.The ideal candidate will have 8+ years of experience in...

  • Senior Data Systems Architect

    3 days ago

    Tirupati, Andhra Pradesh, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Senior Data Systems ArchitectWe are excited to announce an opportunity for a skilled data architect to join our team.About the Role: As a senior data systems architect, you will play a crucial role in designing, developing, and deploying large-scale data processing systems using cutting-edge technologies like PySpark and...

  • Data Architect

    6 days ago

    Tirupati, Andhra Pradesh, India beBeeData Full time ₹ 12,01,000 - ₹ 20,10,000

    Job Title: Data ArchitectWe are seeking a skilled professional to design, build and optimize scalable data pipelines.The ideal candidate will be responsible for enabling seamless data integration, transformation and analytics to support business intelligence and advanced analytics initiatives.Main Responsibilities:Design and maintain ETL/ELT pipelines using...

  • Chief Data Architect

    4 days ago

    Tirupati, Andhra Pradesh, India beBeeDataManagement Full time US$ 90,000 - US$ 1,20,000

    Data Management Role">As a key figure in our organization's data strategy, you will lead the development and governance of data assets across various industries. Your responsibilities will include overseeing the entire data lifecycle management, designing and maintaining complex data models, managing metadata, delivering scalable and secure data products on...