Lead- Data Engineer

Who we are?

Kaleidofin is a fintech focused on the mission to build a digital ecosystem ensuring finance for everyone, everywhere. We provide the ki credit platform, combining credit health assessment (ki score), middleware rails and best-in-class risk management (ki view) to enable the financial ecosystem to serve customers better. Via credit health scores for impactful finance and innovative debt structures to scale, Kaleidofin is driving towards its goal to unlock access to tailored finance for women, agri and nano entrepreneurs – in India, Bangladesh and East Africa and expanding. The platform has unlocked debt capital of cumulatively more than USD 3 billion for more than 5 million customers and small enterprises working in the informal economy.

Over the years, Kaleidofin has raised capital of USD 37 million from marquee investors, including Rabo Partnerships, the Bill & Melinda Gates Foundation, the Michael & Susan Dell Foundation, Oiko Credit, Flourish, Omidyar Network, Blume Ventures, and Bharat Fund, who share our commitment to financial inclusion and impact.

Kaleidofin has received exceptional international recognition in being featured as one of only six Indian companies on the Forbes 100 To Watch in Asia list and in being selected to present at the United Nations General Assembly Special Task Force.The company won Amazon AI Conclave award for Fintech, was one of only ten startups chosen for Google Launch Pad Accelerator program in 2019 and was recognized as India's Most Innovative Wealth, Asset and Investment Management Service/Product by IAMAI. Kaleidofin won the Swanari TechSprint 2023 organized by the RBI Innovation Hub, was recognized as a finalist in IFTA 2023 for 'Best Use of AI in Fintech', as a finalist at the prestigious Global SME Finance Awards 2024 hosted by the SME Finance Forum and IFC. Moreover, Kaleidofin emerged as a winner at the prestigious G20 TechSprint Technology for the Planet.

Kaleidofin aims to become a leading player in enabling finance for everyone, everywhere. We also invest in culture and team growth, reflective in our selection as a Great Place to Work for multiple years.

To know more about Kaleidofin, do visit our site

What you'll do?

• You will take on the role of technical expert and a lead member of our Engineering team.
• Designing and building data pipelines from data ingestion to consumption within a hybrid big data architecture, using Cloud Native AWS, NOSQL, SQL etc.
• Responsible to design and develop distributed, high volume, high velocity multi-threaded event processing systems.
• Designing, building, and operationalizing large-scale enterprise data solutions using Hadoop based technologies along with AWS, Spark, Hive, & Data Lake, using Hive, PySpark and Python Programming.
• Develop efficient software code for multiple use cases leveraging Python and Big Data technologies for various use cases built on the platform.
• Responsible to Ingest data from files, streams and databases and process the data with Hadoop, Scala, SQL/NOSQL Database, Spark, ML, IoT.
• Develop programs in Scala and Python as part of data cleaning and processing.
• It includes data modelling, data ingestion, transformation, data consumption patterns and optimizing complex queries, creating efficient UDFs to extend the functionalities.
• You will be involved in product feature development and will be working in close partnership with other engineering teams.
• You will be responsible for mentoring other team members and ensuring high availability platform and stability for batch and stream processing systems.

Who you need to be?

• Experience - Min. 6 Years.
• Candidate should have Data Processing ability (ETL techniques) using hive scripting experience.
• Native Data engineer with 3 to 4 years of hands-on experience with SQL and NOSQL databases such as HBase, Cassandra or MongoDB.
• Extensive 2+ years' experience working with Hadoop and related processing frameworks such as Spark, Hive, Kafka etc.
• Experience worked with REST and SOAP based APIs to extract data for data pipelines.
• Experience with cloud-native ETL languages/frameworks (Scala, Python, Databricks, AWS Glue).
• Experience in working with Real time data streams and Kafka Platform.
• Working knowledge with workflow orchestration tools like Apache Airflow design and deploy DAGs.
• Bachelor's degree in Computer Science or Computing related discipline from premier institutes