Lead Data Platform Engineer

1 week ago

Bangalore, India Digitopcs consulting Full time

Job Description :

Primary responsibilities :

- Lead the design and engineering of a scalable, secure, and cost-optimized Streaming Data Platform.

- Lead role in evolving from (current) on-prem to hybrid cloud infrastructure with Kafka as the core technology.

- Lead platform upgrades and installs.

- Perform the procedures in Development.

- Collaborate with Operations team in production.

- Develop automation of platform components (infrastructure as code).

- Integrate platform metrics into the Syniverse monitoring stack.

- Execute DB performance analysis.

- Collaborate with Development teams to provide results and recommendations.

- This includes queries, DB schema design, and aggregation layers.

- Perform Capacity analysis to be applied directly in planning infrastructure design/purchases.

- Communicate results/recommendations to leadership team.

Qualifications :

- Bachelor's Degree in Computer Science or equivalent.

- 10+ years relevant experience managing Database, Analytics, and Streaming Platforms.

- 7+ years of programming with Java, python, or Go.

- 5+ years of Linux experience in both scripting and administrative role.

- 4+ years Kafka with a preference for Confluent platform:.

- Deploying and managing Brokers, Zookeeper, Schema Registry, ksqlDB, and Connectors.

- Setup mirroring of Kafka topics data across clusters, DCs, and Cloud.

Cloud skills :

- Production experience with Public Cloud.

- AWS strongly preferred.

- Production experience deploying Kafka in cloud and on-prem.

- Production experience deploying cloud native DB services in the cloud.

- Experience performing assessment of Cloud and on-prem data system.

- DBA experience with preference for PostgreSQL or similar relational database.

- Production experience tuning queries and optimizing performance on large scale Analytic DBs.

- Experience developing automation scripts for infrastructure and platform with Ansible strongly preferred.

- Demonstrates understanding of Data Model design and tradeoffs.

- Demonstrates ability to work across teams including Operations, Development, DBA, and Infrastructure admins.

- Demonstrates clear communication to peers and leadership.

Desired :

- Production experience with Impala (Cloudera) or Snowflake DB including data ingestion, schema design, and tuning.

- Experience setting up Grafana and Prometheus for metrics capture.

- Development of Stream processing applications using Confluent (ksqlDB and Kafka Streams).

- Experience in DBA role on TimescaleDB.

- Data pipeline experience using python, Spark Streaming, Streamsets, Kafka Connect, and ksqlDB.

- Experience deploying on Kubernetes using Containers.

- OLTP DB data model and workload tuning experience.

- Previous experience managing Cloudera CDH clusters.

- Logging using Logstash, Fluentd or equivalent

(ref:hirist.tech)

  • Data engineer

    3 weeks ago

    Bangalore, India Data Labs - LANDMARK GROUP Full time

    Job Title: Lead Data Engineer Location: Yemalur, Bangalore Your Key Responsibilities: Lead our engineering team successfully build and deploy cutting edge analytics products for consumption by different stakeholders. Lead the architecture, design, and delivery of enterprise grade microservices. Deliver assigned projects within budgeted time, cost...

  • Data Engineer

    3 weeks ago

    Bangalore, India Data Labs - LANDMARK GROUP Full time

    Job Title: Lead Data Engineer Location: Yemalur, Bangalore Your Key Responsibilities: Lead our engineering team successfully build and deploy cutting edge analytics products for consumption by different stakeholders. Lead the architecture, design, and delivery of enterprise grade microservices. Deliver assigned projects within budgeted time, cost and...

  • Data Engineer

    3 days ago

    bangalore, India Data Labs - LANDMARK GROUP Full time

    Job Title: Lead Data EngineerLocation: Yemalur, BangaloreYour Key Responsibilities:Lead our engineering team successfully build and deploy cutting edge analytics products for consumption by different stakeholders.Lead the architecture, design, and delivery of enterprise grade microservices.Deliver assigned projects within budgeted time, cost and agreed...

  • Data Engineer

    1 month ago

    bangalore, India Data Labs - LANDMARK GROUP Full time

    Job Title: Lead Data Engineer Location: Yemalur, Bangalore Your Key Responsibilities: Lead our engineering team successfully build and deploy cutting edge analytics products for consumption by different stakeholders.Lead the architecture, design, and delivery of enterprise grade microservices. Deliver assigned projects within budgeted time, cost and agreed...

  • Data Engineer

    4 weeks ago

    bangalore, India Data Labs - LANDMARK GROUP Full time

    Job Title: Lead Data Engineer Location: Yemalur, Bangalore Your Key Responsibilities: Lead our engineering team successfully build and deploy cutting edge analytics products for consumption by different stakeholders. Lead the architecture, design, and delivery of enterprise grade microservices. Deliver assigned projects within budgeted time, cost and...

  • Data Engineer

    3 weeks ago

    bangalore, India Data Labs - LANDMARK GROUP Full time

    Job Title: Lead Data Engineer Location: Yemalur, Bangalore Your Key Responsibilities: Lead our engineering team successfully build and deploy cutting edge analytics products for consumption by different stakeholders.Lead the architecture, design, and delivery of enterprise grade microservices. Deliver assigned projects within budgeted time, cost and agreed...

  • Data engineer

    4 days ago

    Bangalore City, India Data Labs - LANDMARK GROUP Full time

    Job Title: Lead Data Engineer Location: Yemalur, Bangalore Your Key Responsibilities: Lead our engineering team successfully build and deploy cutting edge analytics products for consumption by different stakeholders. Lead the architecture, design, and delivery of enterprise grade microservices. Deliver assigned projects within budgeted time, cost and agreed...

  • Lead Data Platform Architect

    3 weeks ago

    Bangalore, India Digitopcs consulting Full time

    Job Title: Lead Data Platform EngineerJob Summary:We are seeking a highly skilled Lead Data Platform Engineer to lead the design and engineering of a scalable, secure, and cost-optimized Streaming Data Platform. The successful candidate will have a strong background in leading the design and implementation of complex data platforms, with a focus on...

  • Data platform engineer

    3 weeks ago

    Bangalore, India ApTask Full time

    Primary responsibilities: Lead the design and engineering of a scalable, secure, and cost-optimized Streaming Data Platform Lead role in evolving from (current) on-prem to hybrid cloud infrastructure with Kafka as the core technology Lead platform upgrades and installs. Perform the procedures in Development. Collaborate with Operations team in...

  • Data Platform Lead

    12 hours ago

    bangalore, India Minus Zero Full time

    About this Role: - Technical leader of the Data Platform, connect with stakeholder, build partnerships and grow the Data Platform team in Bangalore - Own the APIs that services at Toast use to query and retrieve data - Operate and maintain our batch and streaming data processing workflows in production - Own the APIs teams at Toast use to access, process...

  • Data Platform Lead

    4 days ago

    bangalore, India Minus Zero Full time

    About this Role:- Technical leader of the Data Platform, connect with stakeholder, build partnerships and grow the Data Platform team in Bangalore- Own the APIs that services at Toast use to query and retrieve data- Operate and maintain our batch and streaming data processing workflows in production- Own the APIs teams at Toast use to access, process and...

  • Data Platform Engineer

    3 weeks ago

    bangalore, India ApTask Full time

    Primary responsibilities:Lead the design and engineering of a scalable, secure, and cost-optimized Streaming Data PlatformLead role in evolving from (current) on-prem to hybrid cloud infrastructure with Kafka as the core technologyLead platform upgrades and installs. Perform the procedures in Development. Collaborate with Operations team in...

  • Data Platform Lead

    5 days ago

    bangalore, India Minus Zero Full time

    About this Role:Technical leader of the Data Platform, connect with stakeholder, build partnerships and grow the Data Platform team in BangaloreOwn the APIs that services at Toast use to query and retrieve dataOperate and maintain our batch and streaming data processing workflows in productionOwn the APIs teams at Toast use to access, process and retrieve...

  • Data Platform Lead

    4 days ago

    bangalore, India Minus Zero Full time

    About this Role: Technical leader of the Data Platform, connect with stakeholder, build partnerships and grow the Data Platform team in Bangalore Own the APIs that services at Toast use to query and retrieve data Operate and maintain our batch and streaming data processing workflows in production Own the APIs teams at Toast use to access, process and...

  • Senior Data Platform Engineer

    3 weeks ago

    Bangalore, India e-Hireo Full time

    Lead Data EngineerWe are seeking a highly skilled Lead Data Engineer to join our team at e-Hireo. As a key member of our data engineering team, you will be responsible for designing, building, and maintaining large-scale data platforms on AWS.Key Responsibilities:Architect and implement data pipelines using AWS services such as Glue, Kinesis, and...

  • Senior Data Platform Engineer

    3 weeks ago

    Bangalore/Mumbai, India Xalt Consulting Full time

    Job SummaryWe are seeking a highly skilled Data Engineer II to lead and manage our team of data engineers, providing guidance, mentorship, and support. The ideal candidate will have a strong background in large-scale data platforms and data products, with experience in big data technologies such as Hadoop, Spark, and Hive.Key ResponsibilitiesLead and manage...

  • Solution Engineer

    1 month ago

    Bangalore, Karnataka, India HEVO DATA Full time

    Overview :. We're seeking a talented Solutions Engineer with strong background in Data Domain to join our growing Sales team. You'll work one-on-one with the sales team to help discover our customers' business and technical needs and demonstrate how Hevo's platform can solve their real-time data replication needs. In this role, you will have...

  • Senior Data Platform Engineer

    3 weeks ago

    Bangalore, India CodeX Tech-IT LLC Full time

    About the RoleWe are seeking a highly skilled Senior Data Platform Engineer to join our team at CodeX Tech-IT LLC. As a key member of our data engineering team, you will play a critical role in building and scaling our data platform to support complex data processing and analytics needs.Key Responsibilities- Design and develop a scalable and reliable data...

  • Data Engineering Lead

    3 weeks ago

    Bangalore, India e-Hireo Full time

    Lead Data EngineerWe are seeking a highly skilled Lead Data Engineer to join our team at e-Hireo. As a key member of our data engineering team, you will be responsible for designing, building, and maintaining large-scale data platforms that support our business operations.Key Responsibilities:Design and implement data pipelines using AWS services such as...

  • Data Platform Architect

    3 weeks ago

    Bangalore, India Rock interviews Full time

    Principal Data EngineerWe are seeking a highly skilled Principal Data Engineer to join our team at Rock Interviews. As a key member of our data engineering team, you will be responsible for designing, building, and maintaining our enterprise data platform.Key Responsibilities:Architect and develop end-to-end data platforms on Google Cloud Platform (GCP),...