Data Architect

Description : The core responsibilities for the job include the following : Data Architecture and Infrastructure : - Design and build robust, scalable, and high-performance data architectures that enable seamless data access and analysis.- Develop, optimize, and manage data pipelines and ETL processes for data ingestion from diverse data sources.- Lead the design and implementation of cloud-based data warehousing and big data solutions (AWS, GCP, Azure).Data Pipeline Optimization : - Develop, maintain, and optimize data pipelines that efficiently move, transform, and process large volumes of structured and unstructured data.- Implement best practices for data pipeline automation, ensuring real-time data availability and minimal downtime.Advanced Data Solutions : - Work on the integration of advanced data processing frameworks such as Spark, Kafka, and Hadoop for batch and real-time data streaming.- Collaborate with data science teams to deploy machine learning models and pipelines into production.Mentorship and Leadership : - Act as a technical mentor to junior and mid-level data engineers, guiding best practices, code reviews, and technical problem-solving.- Lead by example in coding standards, technology decisions, and system : - Work closely with data scientists, customer success, product teams, and other stakeholders to translate business needs into scalable data solutions.- Ensure seamless data flow and availability for different teams to drive insights, reporting, and and Security : - Ensure data integrity, security, and compliance with company policies and industry standards.- Optimize data storage, retrieval, and scalability to improve performance and reduce costs.Innovation and Continuous Improvement : - Stay up to date with emerging technologies and data engineering methodologies, continuously improving data infrastructure.- Proactively propose innovative solutions to improve the efficiency and reliability of data systems.Requirements : - 12+ years of experience in data engineering or related fields.- Strong experience in designing and building scalable data pipelines and architectures.- Extensive hands-on experience with cloud platforms such as AWS, GCP, or Azure.- Proven experience with big data technologies (e.g., Hadoop, Spark, Kafka, Hive).- Strong experience with data warehousing solutions (e.g., Redshift, Snowflake, BigQuery, etc.) and data modeling.- Must have designed and built at least two scalable systems.Technical Skills : - Proficiency in programming languages like Python, PySpark, or Scala.- Strong experience in designing robust APIs and microservices.- Experience with data pipeline orchestration tools (e.g., Apache Airflow, Kubeflow, etc.)- Deep understanding of ETL/ELT processes and data modeling.Problem Solving and Innovation : - Strong analytical skills with the ability to solve complex technical challenges.- Experience in troubleshooting data systems and optimizing their performance.- Ability to evaluate and recommend new technologies and tools to improve data systems.Collaboration and Communication : - Excellent communication skills, with the ability to explain complex technical issues to both technical and non-technical stakeholders.- Ability to work in a collaborative, cross-functional environment with multiple stakeholders.- Strong interpersonal skills and experience mentoring other engineers.Good To Have : - Hands-on experience with machine learning pipelines and working with data science teams.- Experience with data governance frameworks and compliance (e.g., GDPR, CCPA).- Experience in real-time data streaming and processing technologies (e.g., Kafka, Kinesis, Flink).- Good knowledge of SQL and database technologies.- Familiarity with containerization (e.g., Docker) and orchestration (e.g., Kubernetes) is a plus.- Experience building data pipelines for CPG/Retail industries. (ref:hirist.tech)