Lead Data Engineer/Manager

2 weeks ago

GurgaonGurugramBangalore, IN HuQuo Consulting Pvt. Ltd. Full time

Job Title: Lead Data Engineer

Job Summary

Looking for a versatile professional to join our team in the role of Hybrid Data Engineer and Database Administrator. This position requires a blend of expertise in Data Engineering and Database Administration, with a strong foundation in Oracle DBA roles and a demonstrated transition into data engineering. The ideal candidate will possess hands-on experience with database systems such as Oracle, Postgres, Teradata, or Starburst (Trino) in an AWS environment, coupled with proficiency in utilizing Git, Jenkins, and Terraform. Additionally, broader AWS experience including S3, Lambda, VPC, and Security Groups would be highly Should possess hands-on experience in working on some of the relational and non-relational databases (Oracle/SQL Server/DB2/PostgreSQL/MySQL/Golden Gate/Data Guard) on AWS and/or on-prem infrastructure.

- 3+ years' Experience with AWS Services like AWS RDS, Aurora, EC2, S3, DMS, AWS Backups, and DynamoDB.

- 3+ years' Experience with the Design and deployment of Amazon Machine Images (AMI) for Database.

- Experience in database migrations from on-prem to AWS Cloud, replication technologies like Oracle Golden Gate/HVR

- Manage and maintain database systems to ensure optimal performance, reliability, and security.

- Design, develop, and deploy data pipelines and ETL processes for efficient data ingestion, transformation, and analysis.

- Collaborate with cross-functional teams to understand data requirements and develop scalable data solutions.

- Monitor database performance metrics and proactively address any performance or scalability issues.

- Implement version control for database scripts and configurations using Git.

- Utilize Jenkins for continuous integration and deployment of database changes.

- Automate infrastructure provisioning and configuration using Terraform to enhance deployment efficiency.

- Leverage AWS services such as S3, Lambda, VPC, and Security Groups to support data infrastructure.

- Provide guidance on database security best practices, including access control and encryption.

- Stay updated on emerging technologies and industry trends in data engineering and database administration.

Required Qualifications:

- 7-8 Years industry implementation experience with Data Engineering tools.

- Broad experience of working within AWS and must be capable of working from infrastructure (VPC, EC2, Security groups, S3 etc.) to AWS data platforms (Redshift, RDS, Glue etc). In depth experience of Azure/GCP is also of interest.

- Experience with database migrations to AWS Cloud

- Design, deploy and support databases on Ec2, AWS RDS and VMware platform

- Working Knowledge of infrastructure automation with Terraform.

- Understanding of source control management and associated tools such as Git.

- Manage and maintain database systems to ensure optimal performance, reliability, and security.

- Design, develop, and deploy data pipelines and ETL processes for efficient data ingestion, transformation, and analysis.

- Collaborate with cross-functional teams to understand data requirements and develop scalable data solutions.

- Monitor database performance metrics and proactively address any performance or scalability issues.

- Implement version control for database scripts and configurations using Git.

- Utilize Jenkins for continuous integration and deployment of database changes.

- Automate infrastructure provisioning and configuration using Terraform to enhance deployment efficiency.

- Leverage AWS services such as S3, Lambda, VPC, and Security Groups to support data infrastructure.

- Provide guidance on database security best practices, including access control and encryption.

- Stay updated on emerging technologies and industry trends in data engineering and database administration.

- 3-5 years of management experience required

- 3-5 years consulting experience preferred

- Bachelor's degree or equivalent experience, master's degree Preferred

- Strong data warehousing, OLTP systems, data integration and SDLC

- Understanding and experience with major Data Architecture philosophies (Dimensional, ODS, Data Vault, etc.)

- Understanding of modern data warehouse capabilities and technologies such as real-time, cloud, Big Data.

- Understanding of on premises and cloud infrastructure architectures (e.g. AWS, Azure, GCP)

Preferred Skills & Experience:

- Proven experience in Oracle DBA roles, with a solid understanding of database management principles.

- Proficiency in at least one of the following database systems: Oracle, Postgres, Teradata, or Starburst (Trino).

- Hands-on experience with Git, Jenkins, and Terraform for version control and automation.

- Familiarity with AWS cloud services, particularly in an environment involving S3, Lambda, VPC, and Security Groups.

- Strong analytical and problem-solving skills with a proactive approach to troubleshooting.

- Excellent communication and collaboration abilities, with a track record of working effectively in cross-functional teams.

- Knowledge and working experience with Data Integration processes, such as Data Warehousing, EAI, etc.

- Ability to analyze business requirements as they relate to the data movement and transformation processes, research, evaluation and recommendation of alternative solutions.

- Ability to provide technical direction to other team members including contractors and employees.

- Ability to contribute to conceptual data modelling sessions to accurately define business processes, independently of data structures and then combine the two together.

- Demonstrated ability to serve as a trusted advisor that builds influence with client management beyond simply EDM.

- Can create documentation and presentations such that they "stand on their own"

- Hands on experience with AWS Products (e.g. AWS Lambda, AWS Glue, AWS EMR, AWS S3, AWS Sage maker, etc.)

- Designing, building, operationalizing, securing, and monitoring data processing systems on AWS Cloud Platform

- Can contribute to internal/external Data Integration proof of concepts.

- Shows ability to create new and innovative solutions to problems previously not encountered.

- Ability to work independently on projects as well as collaborate effectively across teams

- Strong team building, interpersonal, analytical, problem identification and resolution skills

- Experience working with multi-level business communities

- Can effectively utilize SQL and/or available BI tool to validate/elaborate business rules.

- Effectively influences and, at times, oversees business and data analysis activities to ensure sufficient understanding and quality of data.

- Deals effectively with all team members and builds strong working relationships/support with them.

- Understands modern data warehouse concepts (real-time, cloud, Big Data) and how to enable such capabilities from a reporting and analytic standpoint.

(ref:hirist.tech)

  • Lead Data Engineer/Manager

    2 weeks ago

    Gurgaon/Gurugram/Bangalore, IN HuQuo Consulting Pvt. Ltd. Full time

    Job Title: Lead Data EngineerJob SummaryLooking for a versatile professional to join our team in the role of Hybrid Data Engineer and Database Administrator. This position requires a blend of expertise in Data Engineering and Database Administration, with a strong foundation in Oracle DBA roles and a demonstrated transition into data engineering. The ideal...

  • Lead Data Engineer

    2 weeks ago

    Gurgaon/Gurugram, IN HuQuo Consulting Pvt. Ltd. Full time

    Job Description for this position include :- Provides technical leadership in Big Data space (Hadoop Stack like M/R, HDFS, Pig, Hive, HBase, Flume, Sqoop, .NoSQL stores like Cassandra, HBase etc) across Fractal and contributes to open source Big Data technologies- Write and tune complex Java, MapReduce, Pig and Hive jobs- Adapt quickly to change in...

  • Lead Data Engineer

    2 weeks ago

    Gurgaon/Gurugram, IN HuQuo Consulting Pvt. Ltd. Full time

    Job Description for this position include :- Provides technical leadership in Big Data space (Hadoop Stack like M/R, HDFS, Pig, Hive, HBase, Flume, Sqoop, .NoSQL stores like Cassandra, HBase etc) across Fractal and contributes to open source Big Data technologies- Write and tune complex Java, MapReduce, Pig and Hive jobs- Adapt quickly to change in...

  • Lead Data Engineer

    2 weeks ago

    Gurgaon/Gurugram, IN HuQuo Consulting Pvt. Ltd. Full time

    Job brief :We are looking for a Lead Data Engineer to join our Data Engineering Team. In this senior role, you will be responsible for managing our Enterprise Data Warehouse and ensuring the efficient flow and quality of data.Your tasks will include implementing data processes, managing data architecture, designing ETL processes, and analyzing data for...

  • Lead Data Engineer

    2 weeks ago

    Gurgaon/Gurugram, IN HuQuo Consulting Pvt. Ltd. Full time

    Job brief :We are looking for a Lead Data Engineer to join our Data Engineering Team. In this senior role, you will be responsible for managing our Enterprise Data Warehouse and ensuring the efficient flow and quality of data.Your tasks will include implementing data processes, managing data architecture, designing ETL processes, and analyzing data for...

  • OfBusiness - Lead Engineer - Data Science

    2 weeks ago

    Gurgaon/Gurugram, IN OFB TECH Pvt Ltd Full time

    We are seeking a highly skilled and experienced AI/ML Lead to join our dynamic team. The ideal candidate should have an experience of 5-7 years in artificial intelligence and machine learning, with a proven track record of leading successful projects. As the AI/ML Lead, you will be responsible for driving the development and implementation of cutting-edge AI...

  • OfBusiness - Lead Engineer - Data Science

    2 weeks ago

    Gurgaon/Gurugram, IN OFB TECH Pvt Ltd Full time

    We are seeking a highly skilled and experienced AI/ML Lead to join our dynamic team. The ideal candidate should have an experience of 5-7 years in artificial intelligence and machine learning, with a proven track record of leading successful projects. As the AI/ML Lead, you will be responsible for driving the development and implementation of cutting-edge AI...

  • Publicis Sapient

    2 months ago

    Bangalore/Gurgaon/Gurugram, IN Publicis Sapient Full time

    Job Description :As Manager, Data Engineering, you will be responsible for translating client requirements into design, architecting, and implementing Azure Cloud based big data solutions for clients. Your role will be focused on delivering high quality solutions by independently driving design discussions related to Data Ingestion, Transformation &...

  • Lead Python Engineer

    2 weeks ago

    Gurgaon/Gurugram, IN IT Resource Hunter Full time

    Responsibilities Reporting to the Senior Engineer Manager - Risk and data Technology, and working closely with CIO he/she will focus on designing and developing the key frameworks (service engines, API backed business functions) for business process execution as well as work in Data science projects (planning optimization, pricing/scheduling optimization)...

  • Lead Python Engineer

    2 weeks ago

    Gurgaon/Gurugram, IN IT Resource Hunter Full time

    Responsibilities Reporting to the Senior Engineer Manager - Risk and data Technology, and working closely with CIO he/she will focus on designing and developing the key frameworks (service engines, API backed business functions) for business process execution as well as work in Data science projects (planning optimization, pricing/scheduling optimization)...

  • Manager - Data Engineering

    2 weeks ago

    Gurgaon/Gurugram, IN Spectral Consultant Full time

    Spectral Consultants is hiring for the role of Data Engineering- Manager with one of the financial services organisation.Experience : 10+ yearsShould have experience in - Enterprise Data EnvironmentResponsibilities :- Extensive experience with Azure Cloud strategies & industry best practices- Excellent skills in data management, data modelling, data...

  • Manager - Data Engineering

    2 weeks ago

    Gurgaon/Gurugram, IN Spectral Consultant Full time

    Spectral Consultants is hiring for the role of Data Engineering- Manager with one of the financial services organisation.Experience : 10+ yearsShould have experience in - Enterprise Data EnvironmentResponsibilities :- Extensive experience with Azure Cloud strategies & industry best practices- Excellent skills in data management, data modelling, data...

  • Lead Assistant Manager/Manager/Senior Manager

    2 weeks ago

    Gurgaon/Gurugram, IN Huquo Full time

    Role : Data EngineerLocation : GurgaonDesignation : AM- AVP Work Timings : 1pm- 10pmRole Overview :We are looking for a go getter person who can define :- Analytic strategies to meet the demands of business requirements- Technical requirements of the analytic solutions- Data requirements of the analytic solution processes- The person will be part of the Data...

  • Lead Assistant Manager/Manager/Senior Manager

    2 months ago

    Gurgaon/Gurugram, IN Huquo Full time

    Role : Data EngineerLocation : GurgaonDesignation : AM- AVP Work Timings : 1pm- 10pmRole Overview :We are looking for a go getter person who can define :- Analytic strategies to meet the demands of business requirements- Technical requirements of the analytic solutions- Data requirements of the analytic solution processes- The person will be part of the Data...

  • Senior Data Engineer

    2 weeks ago

    Gurgaon/Gurugram, IN Optivalue tek consulting Full time

    Company Description : OptiValueTek is a leading provider of IT solutions specializing in Application & Process Integration, API Management, Data Engineering, Data Science/MLOps, DevOps & SRE, Cloud Solutions, and Web/Mobile & Application Development. With a focus on innovation, we have developed in-house tools/accelerators for Test Automation, API...

  • Senior Data Engineer

    2 weeks ago

    Gurgaon/Gurugram, IN Optivalue tek consulting Full time

    Company Description : OptiValueTek is a leading provider of IT solutions specializing in Application & Process Integration, API Management, Data Engineering, Data Science/MLOps, DevOps & SRE, Cloud Solutions, and Web/Mobile & Application Development. With a focus on innovation, we have developed in-house tools/accelerators for Test Automation, API...

  • Lead GCP Data Engineer

    2 weeks ago

    Pune/Gurgaon/Gurugram, IN IFLOWTECH SOLUTIONS PRIVATE LIMITED Full time

    Location : Pune, GurgaonExperience : 8 to 10 yearNotice period : Immediate to 15 DaysJob Description : Skills :- 8+ years of proven experience as a Lead in Big Data Engineering. - Mandatory Technical Proficiency - Hands-on experience with Python, PySpark, Google Cloud Full Stack, SQL, DevOps, Databricks, DBT & Terraform. - Experience with Google Cloud...

  • Data Engineering Manager

    2 weeks ago

    Gurgaon/Gurugram, IN Spectral Consultant Full time

    Spectral Consultants is hiring for the role of Data Engineering- Manager with one of the financial services organisation.Experience- 10+ yearsShould have experience in - Enterprise Data Extensive experience with Azure Cloud strategies & industry best practices- Excellent skills in data management, data modelling, data services, reporting, and analytics as...

  • Data Engineering Manager

    2 weeks ago

    Gurgaon/Gurugram, IN Spectral Consultant Full time

    Spectral Consultants is hiring for the role of Data Engineering- Manager with one of the financial services organisation.Experience- 10+ yearsShould have experience in - Enterprise Data Extensive experience with Azure Cloud strategies & industry best practices- Excellent skills in data management, data modelling, data services, reporting, and analytics as...

  • Lead Assistant Manager

    2 weeks ago

    Delhi NCR/Gurgaon/Gurugram, IN HuQuo Consulting Pvt. Ltd. Full time

    - Proficient in SQL, data environments (Big Query, Google Analytics, Snowflake) and data transformation (Python) - Advanced knowledge in analytics and statistical software languages (preferably Python)- Good understanding of ETL data pipelines with multiple data sources including APIs and databases- Strong experience with Google Analytics tools- Hands on...