Kaleidofin - Lead Data Engineer - ETL/PySpark

2 weeks ago

Bengaluru, India Kaleidofin Full time

What youll do ?- You will take on the role of technical expert and a lead member of our Engineering team.- Designing and building data pipelines from data ingestion to consumption within a hybrid big data architecture, using Cloud Native AWS, NOSQL, SQL etc.- Responsible to design and develop distributed, high volume, high velocity multi-threaded event processing systems.- Designing, building, and operationalizing large-scale enterprise data solutions using Hadoop based technologies along with AWS, Spark, Hive, & Data Lake, using Hive, PySpark and Python Programming.- Develop efficient software code for multiple use cases leveraging Python and Big Data technologies for various use cases built on the platform.- Responsible to Ingest data from files, streams and databases and process the data with Hadoop,Scala, SQL/NOSQL Database, Spark, ML, IoT.- Develop programs in Scala and Python as part of data cleaning and processing.- It includes data modelling, data ingestion, transformation, data consumption patterns and optimizing complex queries, creating efficient UDFs to extend the functionalities.- You will be involved in product feature development and will be working in close partnership with other engineering teams.- You will be responsible for mentoring other team members and ensuring high availability platform and stability for batch and stream processing systems.Who you need to be ?- Experience - Min. 6 Years.- Bachelor's degree in computer science or computing-related discipline from premier institutes.- The candidate should have data processing ability (ETL techniques) using have scripting experience.- Native Data engineer with 3 to 8 years of hands-on experience with SQL and NOSQL databases such as HBase, Cassandra or MongoDB.- Extensive 2+ years experience working with Hadoop and related processing frameworks such as Spark, Hive, Kafka etc.- Experience working with REST and SOAP based APIs to extract data for data pipelines.- Experience with cloud-native ETL languages/frameworks (Scala, Python, Databricks, AWS Glue).- Experience in working with Real time data streams and Kafka Platform.- Working knowledge with workflow orchestration tools like Apache Airflow design and deploy DAGs.- Location : Bangalore. (ref:hirist.tech)

  • Kaleidofin - Lead Data Engineer - ETL/PySpark

    6 days ago

    Bengaluru, Karnataka, India Kaleidofin Private Limited Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    What youll do ?You will take on the role of technical expert and a lead member of our Engineering team.Designing and building data pipelines from data ingestion to consumption within a hybrid big data architecture, using Cloud Native AWS, NOSQL, SQL etc.Responsible to design and develop distributed, high volume, high velocity multi-threaded event processing...

  • Lead- Data Engineer

    6 days ago

    Greater Bengaluru Area, India Kaleidofin Private Limited Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Who we are?Kaleidofin is a fintech focused on the mission to build a digital ecosystem ensuring finance for everyone, everywhere. We provide the ki credit platform, combining credit health assessment (ki score), middleware rails and best-in-class risk management (ki view) to enable the financial ecosystem to serve customers better. Via credit health scores...

  • Data Engineer- 1

    1 week ago

    Bengaluru, Karnataka, India Databricks PySpark Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Position: Senior Data Engineer We are seeking an experienced Senior Data Engineer to design and implement high-performance data pipelines, ensuring the smooth flow of large datasets across systems. In this role, you will leverage your expertise in Databricks, Pyspark, and ETL design to build efficient data solutions. You will also contribute to...

  • Kaleidofin - Lead Software Engineer - Java/Python

    3 days ago

    Bengaluru, India Kaleidofin Full time

    Description :What youll do :As a Software Development Manager, you will be responsible for shaping product features and driving the technical execution of the product roadmap. You will work closely with your team to build systems that stress scalability, accuracy, and robustness. This is a hands-on leadership role you will actively participate in design,...

  • Lead Software Engineer

    6 days ago

    Bengaluru, Karnataka, India Kaleidofin Private Limited Full time ₹ 1,20,000 - ₹ 2,60,000 per year

    Lead -Software Engineer- BangaloreWho are we?Kaleidofin is a fintech focused on the mission to build a digital ecosystem ensuring finance for everyone, everywhere. We provide the ki credit platform, combining credit health assessment (ki score), middleware rails and best-in-class risk management (ki view) to enable the financial ecosystem to serve customers...

  • Lead software engineer

    3 weeks ago

    Bengaluru, India Kaleidofin Private Limited Full time

    Lead -Software Engineer- BangaloreWho are we?Kaleidofin is a fintech focused on the mission to build a digital ecosystem ensuring finance for everyone, everywhere. We provide the ki credit platform, combining credit health assessment (ki score), middleware rails and best-in-class risk management (ki view) to enable the financial ecosystem to serve customers...

  • Data Engineer with SQL/ETL/Pyspark

    2 weeks ago

    Bengaluru, Karnataka, India Infosys Finacle Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Mandate Skills - SQL, ETL ,hadoop, pyspark, Apache, KafkaRequired Skills5+ years of relevant Data engineering experience using Python or Java as tech be well versed into HadoopDevelopment expertise with 2+ years of experience, in SQL, ETL processes and other similar technologiesSkilled in Identifying and optimizing database queriesStrong understanding of...

  • Senior Data Scientist

    3 weeks ago

    Bengaluru, India Kaleidofin Private Limited Full time

    Who we are? Kaleidofin is a fintech focused on the mission to build a digital ecosystem ensuring finance for everyone, everywhere. We provide the ki credit platform, combining credit health assessment (ki score), middleware rails and best-in-class risk management (ki view) to enable the financial ecosystem to serve customers better. Via credit health scores...

  • Pyspark Lead

    1 week ago

    Bengaluru, Karnataka, India Krazy Mantra HR Solutions Pvt. Ltd Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    We are looking for a skilled PySpark Lead with 5 to 8 years of experience, based in Bangalore. The ideal candidate will have expertise in PySpark, SQL, ETL/Data Engineering Pipeline, Cloud understanding, CI/CD, Data Modelling, Airflow, Kafka, EMR/Databricks, Snowflake, and Hive.Roles and ResponsibilityDesign and develop scalable data pipelines using PySpark...

  • 15h Left: Senior Data Scientist

    4 weeks ago

    Bengaluru, India Kaleidofin Private Limited Full time

    Who we are? Kaleidofin is a fintech focused on the mission to build a digital ecosystem ensuring finance for everyone, everywhere. We provide the ki credit platform, combining credit health assessment (ki score), middleware rails and best-in-class risk management (ki view) to enable the financial ecosystem to serve customers better. Via credit health scores...