Lead Data Engineer

3 months ago

thiruvananthapuram, India CloudQ IT Services Full time

Role Overview :

The Lead Data Engineer will design, implement, and maintain data solutions on Microsoft Azure and SQL Server platforms. You will lead a team of data engineers, collaborate with stakeholders, and ensure efficient data processing, storage, and retrieval.

Technical Responsibilities :

- Design and maintain scalable data pipelines.

- Create solutions using Azure Data Factory and Databricks for ETL processes.

- Develop solutions in SSIS.

- Analyze and optimize the existing data landscape for efficiency and scalability.

- Collaborate with onshore counterparts to address technical gaps and complex scenarios.

- Monitor and troubleshoot data systems to ensure performance and reliability.

- Apply database management principles to ensure data quality.

- Optimize data processes for speed and efficiency.

- Ensure data architecture supports business needs and governance policies.

- Define and execute data engineering strategies aligned with company goals.

- Integrate data from various sources, ensuring quality and consistency.

- Stay updated with emerging technologies and industry trends.

- Translate banking industry business processes into data requirements.

- Manage data migrations across different databases and servers.

- Validate data transformations and support machine learning models.

- Define data requirements and mappings in collaboration with business and IT teams.

- Maintain and test data pipelines for performance validation.

- Assemble large data sets that meet business requirements.

- Identify data gaps and align with business and IT needs.

- Understand the impact of data conversions on operations.

- Manage complex data cases with accuracy and efficiency.

Leadership Responsibilities :

- Provide technical direction to the development team.

- Review and approve code to ensure quality standards.

- Troubleshoot and resolve technical issues.

- Foster a culture of collaboration, innovation, and continuous improvement.

- Optimize data flow for cross-functional teams.

- Work closely with onshore data counterparts, product owners, and stakeholders to understand data needs.

- Collaborate with IT and DevOps teams to align data infrastructure with IT architecture.

- Implement best practices for data security and privacy.

- Drive continuous improvement in data engineering.

Candidate Expectations :

- Design and develop warehouse solutions using Azure Synapse Analytics, ADLS, ADF, Databricks, Power BI, and Azure Analysis Services.

- Proficient in SSIS, SQL, and query optimization.

- Experience as a technical lead mentoring data engineers and managing data projects.

- Experience in an onshore-offshore model handling challenging scenarios.

- Lead data architecture and implementation to optimize data warehouse accuracy and usage.

- Expertise in handling large data sets, building ETL pipelines, and generating insights using Data Science and Analytics.

- Proficiency in Azure, AWS cloud services, and DevOps/CI/CD frameworks.

- Ability to transform ambiguous requirements into deliverables.

- Strong technical and interpersonal skills with excellent communication.

- Drive automation using Infrastructure as Code (IaC), Terraform, Jenkins, and other CI/CD tools.

- Help define architecture frameworks, best practices, and processes.

- Collaborate on data warehouse architecture and technical design.

- Expertise in Azure Data Factory for building ETL pipelines.

- Proficient in SQL and relational databases.

- Proficient in Python for ETL projects.

- Experience with Databricks is an advantage.

- Expertise in data lifecycle management, data ingestion, transformation, loading, validation, and performance tuning.

(ref:hirist.tech)

  • Lead Data Engineer

    3 months ago

    Thiruvananthapuram, India Nissan Motor Full time

    Responsibilities:  Collaborate with product / business owners through the product/project lifecycle and develop data engineering solutions aligning to Nissan's data strategy. Lead a team of data engineers and guide them during the development lifecycle. Work with solution and Data architect to resolve any technical and architectural challenges during...

  • Technical Lead

    3 weeks ago

    Thiruvananthapuram, India Mig Staffing Full time

    Please apply if you are interested in C2HWe are looking for a qualified Data Engineer to join our team in TrivandrumRoles and responsibilities :- Lead Data Engineer will be responsible for designing, implementing, and maintaining data solutions on the Microsoft Azure Data platform and SQL Server (SSIS, SSAS, UC4 Atomic)- Leading a team of data engineers,...

  • Lead Data Scientist

    3 months ago

    Thiruvananthapuram, India Nissan Motor Full time

    Nissan is a pioneer in Innovation and Technology. With a focus on Mobility, Operational Excellence, Value to our Customers and Electrification of vehicles, you can expect to be part of a very exciting journey here at Nissan. Nissan is going after a massive Digital Transformation backed by leading technologies across the organization globally. We are...

  • Data Engineer

    2 months ago

    Thiruvananthapuram, India Wenger & Watson Full time

    Should have a minimum 2+ years in Data Engineering, Data Analytics platform. Should have strong hands-on design and engineering background in AWS, across a wide range of AWS services with the ability to demonstrate working on large engagements. Should be involved in Requirements Gathering and transforming them to into Functionally and technical design....

  • Data Engineer

    1 day ago

    thiruvananthapuram, India Hanuma InfoTech Full time

    Responsibilities:Build ML infrastructure in the cloud environmentDevelop and maintain data pipelines to support machine learning models, including data cleaning, prep-processing data, training and evaluation of modelsSupport Prompt Engineering development that utilizes LLM and deep learning frameworksCollaborate with teams to design and implement solutions,...

  • Lead Engineering Manager

    5 days ago

    thiruvananthapuram, India Armada Full time

    About the Company Armada is an edge computing startup that provides computing infrastructure to remote areas where connectivity and cloud infrastructure is limited, as well as areas where data needs to be processed locally for real-time analytics and AI at the edge. We’re looking to bring on the most brilliant minds to help further our mission of bridging...

  • Lead Software Engineer

    3 months ago

    Thiruvananthapuram, India Nissan Motor Full time

    Responsibilities:  Participate in technical planning & requirements gathering phases including Design, code, test, troubleshoot, and document engineering software applications. Lead a team to achieve the collective goal in optimal time. Mentor junior engineers to help drive the technical roadmap of the organization. Be responsible for ensuring...

  • Data Engineer

    3 months ago

    thiruvananthapuram, India BayOne Solutions Full time

    Work from office - 3 days/Week Location: Bangalore/Gurgaon Job Description: Min 5 years of experience in modern data engineering/data warehousing/data lakes technologies on cloud platforms like Azure, AWS, GCP, Data Bricks etc. Azure experience is preferred over other cloud platforms. 5 years of proven experience with SQL, schema design and dimensional...

  • Senior Data Engineer

    1 month ago

    Thiruvananthapuram, India LyondellBasell Full time

    Basic Functions:We are seeking a skilled and motivated Data Engineer to join our Data Solutions team that is focused on the development, maintenance, and optimization of our data infrastructure. The team specializes in data modeling and Power BI report development to support the objectives of our business stakeholders. The Senior Data Engineer is a core...

  • AI Engineering Manager/Lead

    4 days ago

    thiruvananthapuram, India Armada Full time

    About the Company Armada is an edge computing startup that provides computing infrastructure to remote areas where connectivity and cloud infrastructure is limited, as well as areas where data needs to be processed locally for real-time analytics and AI at the edge. We’re looking to bring on the most brilliant minds to help further our mission of bridging...

  • Lead Machine Learning/Artificial Intelligence Engineer

    4 weeks ago

    Thiruvananthapuram, India Mig Staffing Full time

    We are seeking a skilled and passionate AI/ML Lead with 4-6 years of experience to drive the development and implementation of cutting-edge artificial intelligence and machine learning solutions. As an AI/ML Lead, you will be responsible for leading a team of AI/ML engineers, collaborating with cross-functional teams, and driving innovation in our AI/ML...

  • Data Engineer

    4 days ago

    Thiruvananthapuram, India Vtricks Technologies Full time

    Job Title : GCP Cloud Data Engineer Location : Bangalore / Pune / Trivandrum / Chennai (Hybrid)Notice Period : Immediate to 15 daysExperience Required : 8-12 years (5+ years of relevant experience)Job Overview :We are seeking experienced GCP Cloud Data Engineers with a strong background in cloud migration, data pipelines, and data integrations. The role...

  • Data Engineer

    3 weeks ago

    Thiruvananthapuram, India Spinnaker Analytics Full time

    Profile OverviewWe're seeking talented Data/ML-Ops Engineers to join our team. In this role, you'll play a key part in the data lifecycle, transforming raw data into valuable insights for data analysis and machine learning projects. You'll design and build data pipelines, ensuring data quality and collaborating closely with data scientists and business...

  • AI Engineering Manager/Lead

    2 weeks ago

    Thiruvananthapuram, India Armada Full time

    About the Company Armada is an edge computing startup that provides computing infrastructure to remote areas where connectivity and cloud infrastructure is limited, as well as areas where data needs to be processed locally for real-time analytics and AI at the edge. We’re looking to bring on the most brilliant minds to help further our mission of...

  • AI Engineering Manager/Lead

    5 days ago

    thiruvananthapuram, India Armada Full time

    About the CompanyArmada is an edge computing startup that provides computing infrastructure to remote areas where connectivity and cloud infrastructure is limited, as well as areas where data needs to be processed locally for real-time analytics and AI at the edge. We’re looking to bring on the most brilliant minds to help further our mission of bridging...

  • AI Engineering Manager/Lead

    1 month ago

    Thiruvananthapuram, India Armada Full time

    About the Company Armada is an edge computing startup that provides computing infrastructure to remote areas where connectivity and cloud infrastructure is limited, as well as areas where data needs to be processed locally for real-time analytics and AI at the edge. We’re looking to bring on the most brilliant minds to help further our mission of bridging...

  • AI Engineering Manager/Lead

    1 month ago

    Thiruvananthapuram, India Armada Full time

    About the CompanyArmada is an edge computing startup that provides computing infrastructure to remote areas where connectivity and cloud infrastructure is limited, as well as areas where data needs to be processed locally for real-time analytics and AI at the edge. We’re looking to bring on the most brilliant minds to help further our mission of bridging...

  • AI Engineering Manager/Lead

    1 month ago

    Thiruvananthapuram, India Armada Full time

    About the CompanyArmada is an edge computing startup that provides computing infrastructure to remote areas where connectivity and cloud infrastructure is limited, as well as areas where data needs to be processed locally for real-time analytics and AI at the edge. We’re looking to bring on the most brilliant minds to help further our mission of bridging...

  • Lead Backend Engineer

    3 months ago

    Thiruvananthapuram, India LogicPlum Full time

    Position - Lead Backend Engineer  We are seeking a highly skilled and experienced  Lead Backend Engineer to join our team. The  Lead Backend Engineer will be responsible for designing and implementing scalable, high-performance software solutions for our clients using Python and Angular. This individual will be a key member of our...

  • Associate Lead

    2 days ago

    Thiruvananthapuram, India Envestnet Full time

    Description Apply advanced statistical techniques and machine learning/deep learning algorithms to large datasets to develop models that optimize financial decision-making and operational efficiency.Analyze and interpret complex data from various sources (structured and unstructured) to identify trends, anomalies, and actionable insights.Build, train, and...