Senior Data Engineer

3 weeks ago

Bangalore, Karnataka, India Consulting Pandits Full time

Roles and Responsibilities :

- Owning the design, operations and improvements for the Customer Data Warehouse infrastructure

- Engineering solutions to aggregate and automate large scale data flows from varying sources

- Collaborate with others to construct complex data sources for algorithms and machine learning models

- Gather business requirements from business stakeholders and deliver a solution.

- Participate in weekly status meetings to provide updates to stakeholders and provide a strategic plan for upcoming tasks.

- Explore and learn the latest Azure technologies to provide new capabilities and increase efficiency

- Build real-time streaming from internal and external sources to provide insights into the business

- Help continually improve ongoing reporting and analysis processes, automating or simplifying self-service support for customers.

- Create and maintain workflow/technical documents for all the development activities

Specific skills required are :

- Experience in implementing data solutions in Azure including Azure SQL, Azure Synapse, Cosmos DB, Databricks.

- Hands-on experience configuring Delta Lake on Azure Databricks.

- Experience in designing data solutions in Azure including data distributions and partitions, scalability, disaster recovery and high availability.

- Hands on Development or Architecture experience using Azure SQL or SQL DWH (synapse)

- Knowledge of batch and streaming data architectures

- Experience using big data technologies (Hadoop, Hive, HBase, Spark and others).

- Strong knowledge of SQL (MSSQL or MySQL or PostgreSQL or Presto).

- Demonstrated strength in data modeling, ETL development, and data warehousing

- Hands-on experience in Data Engineering using Python.

- Good knowledge of data lake optimization, partitioning, cron job scheduling and debugging.

- Good knowledge of file formats like parquet, CSV and Hadoop file systems.

General Requirements :

- Bachelors or Master's Degree from a reputed college

- 5+ Years of experience in the Data Engineering domain; Worked in the controlled yet agile setup using proven project management methodologies

- Experience in all stages of DE project work (requirements and logical design, physical design,
implementation, testing and deployment)

- Experience communicating with senior management as well as with colleagues from engineering, analytics, and business backgrounds

- Capable of performing under minimal supervision, multi-tasking, meeting deadlines as an individual contributor, and coaching a team.

(ref:hirist.tech)

  • Senior Data Engineer

    2 months ago

    Bangalore, Karnataka, India Squareroot Consulting Pvt Ltd Full time

    We are seeking a highly skilled and experienced Senior Data Engineer to join our team.Location : Bangalore (Work from Office all 5 days)Experience : 6+ YearsRole : Senior Data EngineerThe ideal candidate will have a strong background in big data technologies and extensive hands-on experience with Cloudera, Spark, Hive, HDFS, Flink, Data Lake, and...

  • Senior Data Engineer

    3 weeks ago

    Bangalore, Karnataka, India Squareroot Consulting Pvt Ltd Full time

    We are seeking a highly skilled and experienced Senior Data Engineer to join our team.Location : Bangalore (Work from Office all 5 days)Experience : 6+ YearsRole : Senior Data EngineerThe ideal candidate will have a strong background in big data technologies and extensive hands-on experience with Cloudera, Spark, Hive, HDFS, Flink, Data Lake, and...

  • Senior Engineer

    2 months ago

    Bangalore, Karnataka, India Prudential Manpower Pvt.lTD Full time

    Immediately hiring for our client for the role of Senior Engineer Data & Analytics. Total Exp : 8+ years. Location: Anywhere in India. What you'll do:As the senior manager of the data engineering team, you will be entrusted with the delivery of scalable services that empower our customers to maximize return on brand outreach and recruitment.- Data...

  • Senior Engineer

    3 weeks ago

    Bangalore, Karnataka, India Prudential Manpower Pvt.lTD Full time

    Immediately hiring for our client for the role of Senior Engineer Data & Analytics. Total Exp : 8+ years. Location: Anywhere in India. What you'll do:As the senior manager of the data engineering team, you will be entrusted with the delivery of scalable services that empower our customers to maximize return on brand outreach and recruitment.- Data...

  • Data Engineer

    3 weeks ago

    Bangalore, Karnataka, India Magna Hire Full time

    Responsibilities :Support the data engineering team in implementing analytics solutions, ensuring adherence to project timelines and quality standards.Participate in at least one full implementation lifecycle of dbt (data build tool), contributing to design, development, and deployment phases.Assist senior team members in optimizing dbt models and...

  • Data Engineer

    2 months ago

    Bangalore, Karnataka, India Magna Hire Full time

    Responsibilities :Support the data engineering team in implementing analytics solutions, ensuring adherence to project timelines and quality standards.Participate in at least one full implementation lifecycle of dbt (data build tool), contributing to design, development, and deployment phases.Assist senior team members in optimizing dbt models and...

  • Senior Data Engineer

    2 months ago

    Bangalore, Karnataka, India TecHHire Global Full time

    Job Description :We are looking for a Senior Data engineer-GCP who can design and build production data pipelines from ingestion to consumption within a big data architecture using Python, Scala, PySpark, Java implementing data engineering, ingestion, and curation functions on GCP :Must be able to write quality code and build secure, highly available...

  • Senior Data Engineer

    3 weeks ago

    Bangalore, Karnataka, India TecHHire Global Full time

    Job Description :We are looking for a Senior Data engineer-GCP who can design and build production data pipelines from ingestion to consumption within a big data architecture using Python, Scala, PySpark, Java implementing data engineering, ingestion, and curation functions on GCP :Must be able to write quality code and build secure, highly available...

  • Senior Director

    2 months ago

    Bangalore, Karnataka, India Recruise India Consulting Full time

    Role : Senior Director - Data Engineering & AnalyticsResponsibilities :- Provide strategic leadership and vision for the data engineering and analytics function, aligning it with the overall business goals.- Lead and manage a diverse team of data engineers, data architects, data scientists, and cloud operations specialists.- Foster a culture of innovation,...

  • Senior Director

    3 weeks ago

    Bangalore, Karnataka, India Recruise India Consulting Full time

    Role : Senior Director - Data Engineering & AnalyticsResponsibilities :- Provide strategic leadership and vision for the data engineering and analytics function, aligning it with the overall business goals.- Lead and manage a diverse team of data engineers, data architects, data scientists, and cloud operations specialists.- Foster a culture of innovation,...

  • Senior Data Engineer

    2 months ago

    Bangalore, Karnataka, India Infimetrics Info Solutions Full time

    Position : Senior Java (Data Engineer). Reports to : Technical Lead. Responsible for : Contributing to software development as per business requirements.Purpose of Role :- We are a data driven asset management firm with a very wide variety of datasets with varying frequency of updates that essentially drive our research models which ultimately determine the...

  • Senior Data Engineer

    3 weeks ago

    Bangalore, Karnataka, India Infimetrics Info Solutions Full time

    Position : Senior Java (Data Engineer). Reports to : Technical Lead. Responsible for : Contributing to software development as per business requirements.Purpose of Role :- We are a data driven asset management firm with a very wide variety of datasets with varying frequency of updates that essentially drive our research models which ultimately determine the...

  • Senior Data Engineer

    1 month ago

    Bangalore, Karnataka, India Scaling Theory Full time

    As a Senior Data Engineer, you will partner with Software Engineers, Data Scientists, and Business Intelligence Engineers. You will gain a deep understanding of our services and the data they produce and become our resident expert in how to transform that data into a format that is useful for analytics and business intelligence. You will proactively help to...

  • Senior Data Engineer

    3 weeks ago

    Bangalore, Karnataka, India Scaling Theory Full time

    As a Senior Data Engineer, you will partner with Software Engineers, Data Scientists, and Business Intelligence Engineers. You will gain a deep understanding of our services and the data they produce and become our resident expert in how to transform that data into a format that is useful for analytics and business intelligence. You will proactively help to...

  • Senior Data Engineer

    3 weeks ago

    Bangalore, Karnataka, India SPN Globe Full time

    Must Have Skills : Spark streaming, Kafka, AWS, Python, Data Engineering, PySpark, CI/CD, Kubernetes, Docker, Dynamo DB.Job Description :1) Proven experience as a Senior Data Engineer.2) A well-rounded engineer with good appetite for learning and ramping up on cutting edge technologies.3) Good analytical and debugging skills with self-starting capabilities...

  • Senior Data Engineer

    2 months ago

    Bangalore, Karnataka, India SPN Globe Full time

    Must Have Skills : Spark streaming, Kafka, AWS, Python, Data Engineering, PySpark, CI/CD, Kubernetes, Docker, Dynamo DB.Job Description :1) Proven experience as a Senior Data Engineer.2) A well-rounded engineer with good appetite for learning and ramping up on cutting edge technologies.3) Good analytical and debugging skills with self-starting capabilities...

  • Senior Data Engineer

    1 week ago

    Bangalore, Karnataka, India Arting Digital Full time

    Job Description :Posting title : Senior Data Engineer Experience : 8+ Years Location : Bangalore Work mode : Hybrid Primary skills : Azure services such as Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure Storage, etc.Qualification : Any Engineering/ Computers degreeRoles & Responsibilities :- Design, implement, and manage Azure-based...

  • Senior Lead

    4 weeks ago

    Bangalore, Karnataka, India Peepal Technology and Management Consultant Full time

    Key Responsibilities :Data Engineer Senior Lead role is to coordinate, and execute all activities related to the requirements interpretation, design, and implementation of Data Analytics applications. This individual will apply proven industry and technology experience as well as communication skills, problem-solving skills, and knowledge of best practices...

  • Senior Lead

    3 weeks ago

    Bangalore, Karnataka, India Peepal Technology and Management Consultant Full time

    Key Responsibilities :Data Engineer Senior Lead role is to coordinate, and execute all activities related to the requirements interpretation, design, and implementation of Data Analytics applications. This individual will apply proven industry and technology experience as well as communication skills, problem-solving skills, and knowledge of best practices...

  • Senior Data Engineer

    2 months ago

    Bangalore/Hyderabad, Karnataka, India Sam manpower services & Career LLP Full time

    As a Senior Data Engineer, you will play a critical role in our organization by leading the design, development, and maintenance of our data pipeline infrastructure. You will work closely with cross-functional teams to ensure seamless data integration and accessibility for analytical and operational purposes.Key responsibilities:- Design, construct, and...