Data Squad Engineer

2 months ago


Bangalore, Karnataka, India Akshaya IT Business Solutions Full time

Responsibilities :

  • Develop and maintain data pipelines using PySpark for efficient data processing.
  • Utilize AWS services, especially S3 for storage and data processing tasks.
  • Manage data storage and retrieval using HDFS and Hive.
  • Implement Spark for large-scale data processing and analytics.
  • Work with both relational (MySQL) and non-relational (NoSQL) databases for data storage and retrieval.
  • Explore and implement Kafka for real-time data streaming.
  • Utilize MongoDB for handling semi-structured and unstructured data.
  • Leverage ThoughtSpot for advanced data analytics and visualization.
  • Handle data ingestion from various sources and ensure data quality.
  • Execute data migration tasks efficiently and accurately.
  • Implement automation and orchestration tasks using RunDeck.
Requirements :
  • Proficiency in PySpark.
  • Strong knowledge of AWS, particularly S3 and other cloud services.
  • Experience with Hadoop Distributed File System (HDFS).
  • Familiarity with Hive and Spark.
  • Advanced proficiency in Python.
  • Hands-on experience with both MySQL and NoSQL technologies.
  • Good to have experience with Kafka and MongoDB.
  • Familiarity with the ThoughtSpot analytics platform.
  • Experience in data ingestion and data migration processes.
  • Knowledge of RunDeck for automation and orchestration.

(ref:hirist.tech)

  • Bangalore, Karnataka, India HECTADATA INDIA TECHNOLOGY CONSULTING PRIVATE LIMI Full time

    Job title : Cyber Security Engineer. Years of experience : 5-10 years. Location : Bangalore. What will you be doing : - You will work with the DLP squad supporting Engineering & Operational tasks involved in data security services, ranging from service delivery, task automation, DevOps tasks and supporting an on-call rota.- Working with the DLP squad & wider...


  • Bangalore, Karnataka, India HECTADATA INDIA TECHNOLOGY CONSULTING PRIVATE LIMI Full time

    Job title : Cyber Security Engineer. Years of experience : 5-10 years. Location : Bangalore. What will you be doing : - You will work with the DLP squad supporting Engineering & Operational tasks involved in data security services, ranging from service delivery, task automation, DevOps tasks and supporting an on-call rota.- Working with the DLP squad & wider...


  • Bangalore, Karnataka, India CIEL HR Full time

    Job Description (Job Summary/Roles & Responsibilities): - Implements and oversees the test automation approach to deliver the Test Automation Strategy including CICD, nonfunctional testing - performance, security engineering, compatibility, accessibility testing- Drives out improvements and innovation around tooling, frameworks and approaches working on...

  • Data Modeler

    2 months ago


    Bangalore, Karnataka, India Codersbrain technology pvt ltd Full time

    Job Description :The Data Modeler would perform data analysis of various systems and business context for defining requirements for data use cases across the Risk portfolios.- This will involve facilitating workshops, liaising with external and internal technical teams, initiative squads, business stakeholders, other data modelers and architecture/design...

  • Aptus Data Labs

    4 weeks ago


    Bangalore, Karnataka, India Aptus Data Labs Full time

    Description :We are seeking an experienced Snowflake Lead/Architect with expertise in AWS to join our dynamic team. As a Snowflake Lead/Architect, you will play a critical role in designing and implementing scalable data architectures using Snowflake within the AWS ecosystem. If you have a passion for cloud-based data warehousing and a strong understanding...

  • Aptus Data Labs

    3 weeks ago


    Bangalore, Karnataka, India Aptus Data Labs Full time

    Description :We are seeking an experienced Snowflake Lead/Architect with expertise in AWS to join our dynamic team. As a Snowflake Lead/Architect, you will play a critical role in designing and implementing scalable data architectures using Snowflake within the AWS ecosystem. If you have a passion for cloud-based data warehousing and a strong understanding...

  • Data Engineer

    3 weeks ago


    Bangalore, Karnataka, India Magna Hire Full time

    Responsibilities :Support the data engineering team in implementing analytics solutions, ensuring adherence to project timelines and quality standards.Participate in at least one full implementation lifecycle of dbt (data build tool), contributing to design, development, and deployment phases.Assist senior team members in optimizing dbt models and...

  • Data Engineer

    2 months ago


    Bangalore, Karnataka, India Magna Hire Full time

    Responsibilities :Support the data engineering team in implementing analytics solutions, ensuring adherence to project timelines and quality standards.Participate in at least one full implementation lifecycle of dbt (data build tool), contributing to design, development, and deployment phases.Assist senior team members in optimizing dbt models and...

  • Data Engineer

    4 weeks ago


    Bangalore, Karnataka, India VY Systems Pvt Ltd. Full time

    Job Description : The role described here is for a seasoned data engineer with 8 to 13 years of experience, showcasing proficiency across various aspects of data management and engineering. Let's delve deeper into each requirement :1. Data Engineering Experience :- Candidates should possess a solid foundation in data engineering, indicating a substantial...

  • Data Engineer

    3 weeks ago


    Bangalore, Karnataka, India VY Systems Pvt Ltd. Full time

    Job Description : The role described here is for a seasoned data engineer with 8 to 13 years of experience, showcasing proficiency across various aspects of data management and engineering. Let's delve deeper into each requirement :1. Data Engineering Experience :- Candidates should possess a solid foundation in data engineering, indicating a substantial...

  • Staff Engineer

    2 months ago


    Bangalore, Karnataka, India Prudential Manpower Pvt.lTD Full time

    We are urgently hiring for our client for the position of Staff Engineer - Data Engineering. Location : Anywhere in India.Np : max 30 days.What you'll do : As the Staff Engineer of the data engineering team, you will be entrusted with the delivery of scalable services that empower our customers to maximize return on brand outreach and recruitment. Staff...

  • Staff Engineer

    3 weeks ago


    Bangalore, Karnataka, India Prudential Manpower Pvt.lTD Full time

    We are urgently hiring for our client for the position of Staff Engineer - Data Engineering. Location : Anywhere in India.Np : max 30 days.What you'll do : As the Staff Engineer of the data engineering team, you will be entrusted with the delivery of scalable services that empower our customers to maximize return on brand outreach and recruitment. Staff...

  • Data Engineer

    2 weeks ago


    Bangalore, Karnataka, India Programming.com Full time

    Job Description :This is a full-time on-site role as an AWS Data Engineer located in Bengaluru. The AWS Data Engineer will be responsible for daily tasks including data engineering, data modeling, Extract Transform Load (ETL), data warehousing, and data analytics. Qualifications :- Data Engineering and Data Modeling skills- Experience with Extract Transform...


  • Bangalore, Karnataka, India FENL IT SERVICES FINANCIAL ENGINES LLP Full time

    About the job :At Edelman Financial Engines, we believe every hardworking American deserves to move their financial life forward, and we're growing our team to help more clients every day. We know that assets have the power to fund goals. Those numbers represent individual lifetimes filled with hard work and dreams for our clients and generations to...


  • Bangalore, Karnataka, India FENL IT SERVICES FINANCIAL ENGINES LLP Full time

    About the job :At Edelman Financial Engines, we believe every hardworking American deserves to move their financial life forward, and we're growing our team to help more clients every day. We know that assets have the power to fund goals. Those numbers represent individual lifetimes filled with hard work and dreams for our clients and generations to...

  • Kafka Data Engineer

    2 months ago


    Bangalore, Karnataka, India Codersbrain technology pvt ltd Full time

    Mandatory skills : - Azure, Kafka, Data Engineering or any Data Platforms background- Experience with Azure/MS Data Platforms: Postgres Single and Flex, Azure SQL, MS SQL server, Kubernetes- Terraform (Preferred not Compulsory), Bicep or other IaC.- Helm (helm deployments as well writing charts).- Azure DevOps pipelines.- Engineer, Data Engineer, or similar...

  • Kafka Data Engineer

    3 weeks ago


    Bangalore, Karnataka, India Codersbrain technology pvt ltd Full time

    Mandatory skills : - Azure, Kafka, Data Engineering or any Data Platforms background- Experience with Azure/MS Data Platforms: Postgres Single and Flex, Azure SQL, MS SQL server, Kubernetes- Terraform (Preferred not Compulsory), Bicep or other IaC.- Helm (helm deployments as well writing charts).- Azure DevOps pipelines.- Engineer, Data Engineer, or similar...

  • Senior Data Engineer

    2 months ago


    Bangalore, Karnataka, India Squareroot Consulting Pvt Ltd Full time

    We are seeking a highly skilled and experienced Senior Data Engineer to join our team.Location : Bangalore (Work from Office all 5 days)Experience : 6+ YearsRole : Senior Data EngineerThe ideal candidate will have a strong background in big data technologies and extensive hands-on experience with Cloudera, Spark, Hive, HDFS, Flink, Data Lake, and...

  • Senior Data Engineer

    3 weeks ago


    Bangalore, Karnataka, India Squareroot Consulting Pvt Ltd Full time

    We are seeking a highly skilled and experienced Senior Data Engineer to join our team.Location : Bangalore (Work from Office all 5 days)Experience : 6+ YearsRole : Senior Data EngineerThe ideal candidate will have a strong background in big data technologies and extensive hands-on experience with Cloudera, Spark, Hive, HDFS, Flink, Data Lake, and...

  • Data Engineer

    3 weeks ago


    Bangalore, Karnataka, India Smartgig Technologies Private Limited Full time

    Data Engineer (6+ Years Experience)Location : Bangalore, HybridAvailability : Immediate to 15 DaysAbout the Role :We are seeking a highly skilled Data Engineer with over 6 years of experience to join our dynamic team in Bangalore. The ideal candidate will have extensive experience with Python and PySpark, along with a strong understanding of data engineering...