Senior Azure Data Engineer

4 days ago

Kozhikode, India Manuh Technologies Full time
  1. Role Summary


The Senior Data Engineer role is responsible for designing, developing and maintaining data pipelines, databases and data infrastructure to enable the efficient data collection, storage and analysis. This role requires collaboration with data scientists, data infrastructure architects and data consumption stakeholders to ensure the availability of high-quality data for insights and decision making is available and will deliver value to the GM strategic vision for the future and meet prioritized business needs. This role drives to deliver innovative solutions with technical fidelity and thrives to work effectively across a cross-functional data and stakeholder ecosystem.

Key Responsibilities:

  • Design, construct, install and maintain data architectures, including database and large-scale processing systems.
  • Develop and maintain ELT (Extract, Load, Transform) processes to collect, cleanse and transform data from various sources inclusive of cloud
  • Design and implement data pipelines to collect, process and transfer data from various sources to storage systems (data warehouses, data lakes, etc.
  • Implement security measures to protect sensitive data and ensure compliance with data privacy regulations.
  • Build data solutions that ensure data quality, integrity and security through data validation, monitoring, and compliance with data governance policies
  • Maintain Master Data, Metadata, Data Management Repositories, Logical Data Models, and Data Standards
  • Troubleshoot and resolve data-related issues affecting data quality fidelity

·Database Management: Design, optimize, and manage databases to ensure data integrity, performance, and scalability.

·Testing and Quality Assurance: Write unit tests, conduct code reviews, and implement automated testing frameworks to ensure the reliability and stability of applications.

·Continuous Integration and Deployment (CI/CD): Implement CI/CD pipelines using Azure DevOps or similar tools to automate build, test, and deployment processes.

·Performance Optimization: Identify and resolve performance bottlenecks, memory leaks, and other issues to ensure optimal application performance.

Qualifications:

  • Strong skills in SQL, Python, Unix and ability to write efficient ETL on Kubernetes and Cloud platforms specifically Azure and Databricks
  • Strong understanding and ability to provide mentorship in the areas of data ETL processes and tools for designing and managing data pipelines
  • Experience with big data frameworks and tools like Apache Hadoop, Apache Spark, or Apache Kafka for processing and analyzing large datasets.
  • Hands on experience with data serialization formats like JSON, Parquet and XML
  • Hands on file processing experience dealing with CSV, SPSS and other standard formats
  • Hands on experience with GIT Repository and Integrating code between Databricks and GIT environments
  • Consistently designs solutions and build data solutions that are highly automated, performant, with quality checks that provide data consistency and accuracy outcomes
  • Experience in actively managing large-scale data engineering projects, including planning, resource allocation, risk management, and ensuring successful project delivery and adjust style for all delivery methods (i.e.: Waterfall, Agile, POD, etc.)
  • Understands data governance principles, data privacy regulations, and experience implementing security measures to protect data
  • Ability to work effectively in cross-functional teams, collaborate with data scientists, analysts, and stakeholders to deliver data solutions
  • Influential communication skills to effectively convey technical concepts to non-technical stakeholders and document data engineering processes
  • Models a mindset of continuous learning, staying updated with the latest advancements in data engineering technologies, and a drive for innovation
  • ·Bachelor’s degree in computer science, Engineering, or a related field. Master's degree preferred.
  • ·5 to 7 years of experience as a Software Engineer with primary focus on Data Engineering.


  • ·Hands-on experience with Azure cloud services, Azure Storage, Azure DevOps, etc.

  • OneNine AI | Senior Data Engineer | kozhikode

    3 weeks ago

    kozhikode, India OneNine AI Full time

    World's leading commercial real estate company is seeking a Senior Data Engineer, with 7+ years of experience, to lead the engineering efforts for a new quantitative research team by building data pipelines, developing ML/AI solutions, and supporting data science models that aid investment decisions in commercial real estate.. The ideal candidate would be an...

  • Advanced Data Engineer Lead

    4 weeks ago

    Kozhikode, Kerala, India Aezion, Inc Full time

    Job Title: Advanced Data Engineer LeadAzure Data Engineering Expertise: We seek an experienced Senior Data Engineer Lead to spearhead our data engineering efforts, leveraging expertise in T-SQL, SSIS, SSRS, and Azure technologies.Key Responsibilities:Design, develop, and deploy complex data extraction pipelines in Python.Collaborate with cross-functional...

  • Azure DevOps Engineer

    1 month ago

    Kozhikode, India Tata Consultancy Servicess Full time

    Job title: Azure DevOps Engineer4to6yrsexperienceAbout the roleAs a DevOps Engineer on the team, you'll work with Architects and the Squads to ensure that sound DevOps practices are in place. You will play a key role in designing these processes as well as implementing them with your teammates.Your responsibilities include:As a DevOps expert, you will be...

  • Cybersecurity Data Engineering Specialist

    4 weeks ago

    Kozhikode, Kerala, India System Two Security Full time

    Key ResponsibilitiesAs a Senior Security Data Engineer at System Two Security, you will play a pivotal role in ensuring the integrity and usability of our data. You will be responsible for generating synthetic data, building validation frameworks for ML models, partnering with data science teams to tailor AI solutions, and overseeing data architecture and...

  • Data Architect

    2 weeks ago

    Kozhikode, India Job Conquers Full time

    Company Descriptionwe suggest you enter details hereRole DescriptionThis is a full-time remote role for a Data Architect. The Data Architect will be responsible for designing, creating, and managing an organization's data architecture, including databases, data lakes, and data warehouses. They will also collaborate with data scientists, analysts, and IT...

  • Director of Engineering

    3 weeks ago

    Kozhikode, India BayOne Solutions Full time

    Director of EngineeringPosition Overview We are looking for a Director of Engineering to manage and expand our office in Gurugram. In this position, you will be responsible for managing teams, leading critical technical initiatives, and cultivating a customer-centric culture. You will ensure seamless collaboration across departments, uphold rigorous...

  • Senior Infrastructure and Cybersecurity Engineer

    4 weeks ago

    Kozhikode, India Celito Tech, Inc. Full time

    Job DescriptionJob Title: Sr. Infrastructure and Cybersecurity EngineerReports To: Director, Infrastructure and CybersecurityEmployment Type: Full-timeTimings: Upto 2.30 AM ISTTHE CELITO TEAMThe Celito Team architects the buildout of simplified, integrated, and compliant technology stacks. With both consulting and products, our expertise can help our...

  • AWS Data Engineer

    2 months ago

    Kozhikode, India Tata Consultancy Services Full time

    Dear Candidate,Greetings from TCS Human Resources Team!!Job Role: AWS Data EngineerJob Locations: HyderabadExperience Range: 7 to 10 yearsJob Description:-Key Responsibilities: Lead the design, implementation, and optimization of data pipelines and architectures using AWS services like S3, Redshift, Glue, Lambda, and Step Functions. Architect data solutions...

  • Lead Engineer

    4 weeks ago

    Kozhikode, India Finanshels Accounting Accounting LLP Full time

    Job Description This is a remote position. Company Overview: Finanshels.com is a pioneering financial technology company headquartered in Dubai, UAE. Our mission is to revolutionize the financial landscape by empowering businesses with cutting-edge solutions that streamline accounting and tax operations, democratize access to skilled finance...

  • Big Data Engineer

    2 hours ago

    Kozhikode, India HCLTech Full time

    We are seeking a talented Scala and Spark Engineer to join our team. The ideal candidate will have a strong background in Scala and Spark development, with experience in building scalable and reliable data processing pipelines. Additionally, familiarity with Google Big Query for backend connectivity is highly desirable. Responsibilities: Develop and maintain...

  • Senior Automation Engineer

    3 months ago

    Kozhikode, India Art Technology and Software Full time

    6+ years of automation experience.Primary ResponsibilitiesPrepare reusability and maintainability of the test scripts.Develop and continuously improve automated tests as new system features and enhancements are developedExecute automated tests and manage the resulting defect log.Assist in the identification, selection, and implementation of tools,...

  • Senior Construction Engineer

    3 weeks ago

    Kozhikode, Kerala, India Savvy Search Full time

    About Savvy Search: A leading construction company in India known for its innovative project management and client satisfaction.Job Overview:Savvy Search is seeking an experienced Senior Construction Engineer to join our team in Kozhikode, Kerala. As a key member of our site engineering team, you will be responsible for overseeing the execution of projects...

  • Senior Lead Engineer

    4 weeks ago

    kozhikode, India Technical Xperts Group (TXG) Full time

    Technical Xperts Group (TXG) is seeking talented BIM professionals to join our team! We are looking for experienced BIM Managers, BIM Coordinators, BIM Leads, Senior BIM Engineers, and BIM modelers to help us deliver high-quality projects to our clients. 1. BIM MANAGERS QUALIFICATIONS: - 10-12 Years of experience in the respected field...

  • Senior Lead Engineer

    4 weeks ago

    kozhikode, India Technical Xperts Group (TXG) Full time

    Technical Xperts Group (TXG) is seeking talented BIM professionals to join our team! We are looking for experienced BIM Managers, BIM Coordinators, BIM Leads, Senior BIM Engineers, and BIM modelers to help us deliver high-quality projects to our clients. 1. BIM MANAGERS QUALIFICATIONS: 10-12 Years of experience in the respected field...

  • Machine Learning Engineer

    4 days ago

    Kozhikode, India Recro Full time

    Our Client is looking for Proficient in Python, TensorFlow, PyTorch, scikit-learn, Pandas; experienced in AI and ML model development, integration, deployment; familiar with Docker, Kubernetes, CI/CD tools, AWS/Azure/GCP; strong in data structures, algorithms, system design; and knowledgeable in SQL, PostgreSQL, and NoSQL databases.Responsibilities: Model...

  • Lead Engineer

    4 weeks ago

    Kozhikode, India Finanshels Accounting Accounting LLP Full time

    This is a remote position. Company Overview: Finanshels is a pioneering financial technology company headquartered in Dubai, UAE. Our mission is to revolutionize the financial landscape by empowering businesses with cutting-edge solutions that streamline accounting and tax operations, democratize access to skilled finance professionals, and deliver...

  • Senior Java Software Engineer

    2 weeks ago

    Kozhikode, Kerala, India Tata Consultancy Services Full time

    Tata Consultancy Services is a leading global IT services company, and we are currently looking for experienced Senior Java Software Engineers to join our team.We offer a competitive salary of ₹1,200,000 - ₹2,000,000 per annum, depending on experience, plus additional benefits.As a Senior Java Software Engineer at TCS, you will be responsible for...

  • Senior Timing Engineer Lead

    1 day ago

    Kozhikode, Kerala, India LeadSoc Technologies Pvt Ltd Full time

    Job Title: Senior Timing Engineer LeadAbout Us:At LeadSoc Technologies Pvt Ltd, we are dedicated to providing innovative solutions for the digital age. As a leading technology company, we strive to push the boundaries of what is possible.Role Overview:We are seeking an experienced Senior Timing Engineer Lead to join our team. The ideal candidate will have a...

  • Senior Back-end Software Engineer

    3 weeks ago

    Kozhikode, Kerala, India Noviindus Technologies Full time

    We are a cutting-edge software development company, Noviindus Technologies, seeking an exceptional Senior Back-end Software Engineer to join our engineering team. This role is perfect for a skilled Python developer who wants to take their career to the next level and contribute to building scalable, high-performance web applications.Job DescriptionThe ideal...

  • Full Stack Engineer

    3 months ago

    kozhikode, India Stealth Startup Full time

    About Us:We are a New York based AI startup revolutionizing the way in which sports content is created and consumed. We are passionate about the intersection of sports and technology, and are looking for an exceptional full-stack engineer to join our team and help bring our vision to life.Job Description:We are seeking a full-stack engineer with a strong...