Lead Big Data Engineer

We are seeking a talented Lead Big Data Engineer to deliver roadmap features which would helpcustomers to Measure, Communicate and Eliminate Cyber Risks.Working with a team of engineers and architects, you will be responsible for prototyping, designing, developing and supporting a highly scalable, distributed SaaS based Security Risk Prioritization product.Responsibilities :- Be the thought leader in data platform and pipeline along with Risk Evaluation.- Provide technical leadership to the engineering organization on data platform design, roll out and evolution.- Liason to product teams, professional services and sales engineers on solution and trade-off reviews and represent engineering in such conversations.- Drive technology explorations and roadmaps.- Serve as a technical lead on our most demanding, cross-functional departments.- Ensure the quality of architecture and design of systems.- Functionally decompose complex problems into simple, straight-forward solutions.- Fully and completely understand system interdependencies and limitations.- Possess expert knowledge in performance, scalability, enterprise system architecture, and engineering best practices.- Leverage knowledge of internal and industry prior art in design decisions.- Effectively research and benchmark cloud technology against other competing systems in the industry.- Able to document the details so it will be easy for developers to understand the requirements.- Assisting developers with proper requirements and directions.- Assist in the career development of others, actively mentoring individuals and the community on advanced technical issues and helping managers guide the career growth of their team members.- Exert technical influence over multiple teams, increasing their productivity and effectiveness by sharing your deep knowledge and experience.- Able to share knowledge and train Bachelors degree in computer science or equivalent- 8+ years of total experience.- 4+ years of relevant experience in design and architecture Big Data solutions using Spark- 3+ years experience in working with engineering resources for innovation.- 4+ years experience in understanding Big Data events flow pipeline.- 3+ years experience in performance testing for large infrastructure.- 3+ In depth experience in understanding various search solutions solr/elastic.- 3+ years experience in Kafka- In depth experience in Data lakes and related ecosystems.- In depth experience of messing queue- In depth experience in giving requirements to build a scalable architecture for Big data and Micro-services environments.- In depth experience in understanding caching components or services- Knowledge in Presto technology.- Knowledge in Airflow.- Hands-on experience in scripting and automation- In depth understanding of RDBMS/NoSQL, Oracle , Cassandra , Kafka , Redis, Hadoop, lambda architecture, kappa , kappa ++ architectures with flink data streaming and rule engines- Experience in working with ML models engineering and related deployment.- Design and implement secure big data clusters to meet many compliances and regulatory requirements.- Experience in leading the delivery of large-scale systems focused on managing the infrastructure layer of the technology stack.- Strong experience in doing performance benchmarking testing for Big data technologies.- Strong troubleshooting skills.- Experience leading development life cycle process and best practices- Experience in Big Data services administration would be added value.- Experience with Agile Management (SCRUM, RUP, XP), OO Modeling, working on internet, UNIX, Mid (ref:hirist.tech)