Data Platform Engineer
1 month ago
Primary responsibilities:
Lead the design and engineering of a scalable, secure, and cost-optimized Streaming Data Platform
Lead role in evolving from (current) on-prem to hybrid cloud infrastructure with Kafka as the core technology
Lead platform upgrades and installs. Perform the procedures in Development. Collaborate with Operations team in production.
Develop automation of platform components (infrastructure as code)
Integrate platform metrics into the Syniverse monitoring stack
Execute DB performance analysis. Collaborate with Development teams to provide results and recommendations. This includes queries, DB schema design, and aggregation layers.
Perform Capacity analysis to be applied directly in planning infrastructure design/purchases
Communicate results/recommendations to leadership team
Qualifications :
Bachelor’s Degree in Computer Science or equivalent
10+ years relevant experience managing Database, Analytics, and Streaming Platforms.
7+ years of programming with Java, python, or Go
5+ years of Linux experience in both scripting and administrative role
4+ years Kafka with a preference for Confluent platform:
Deploying and managing Brokers, Zookeeper, Schema Registry, ksqlDB, and Connectors
Setup mirroring of Kafka topics data across clusters, DCs, and Cloud
Cloud skills:
Production experience with Public Cloud. AWS strongly preferred.
Production experience deploying Kafka in cloud and on-prem
Production experience deploying cloud native DB services in the cloud
Experience performing assessment of Cloud and on-prem data system
DBA experience with preference for PostgreSQL or similar relational database
Production experience tuning queries and optimizing performance on large scale Analytic DBs
Experience developing automation scripts for infrastructure and platform with Ansible strongly preferred
Demonstrates understanding of Data Model design and tradeoffs
Demonstrates ability to work across teams including Operations, Development, DBA, and Infrastructure admins
Demonstrates clear communication to peers and leadership
-
Data Platform Engineer
3 days ago
India GeoTech Full timeLocation: (Remote, with occasional travel to Riyadh, Saudi Arabia) Department: Data and AI Employment Type: Full-time About GeoTech GeoTech is at the forefront of data-driven innovation, combining cutting-edge technology with advanced geospatial analytics. Our solutions empower businesses with actionable insights, leveraging the power of AI,...
-
Data Platform Engineer
1 week ago
India GeoTech Full timeLocation: (Remote, with occasional travel to Riyadh, Saudi Arabia) Department: Data and AI Employment Type: Full-time About GeoTech GeoTech is at the forefront of data-driven innovation, combining cutting-edge technology with advanced geospatial analytics. Our solutions empower businesses with actionable insights, leveraging the power of AI, computer...
-
Data Platform Engineer
1 week ago
India GeoTech Full timeLocation: (Remote, with occasional travel to Riyadh, Saudi Arabia) Department: Data and AI Employment Type: Full-time About GeoTech GeoTech is at the forefront of data-driven innovation, combining cutting-edge technology with advanced geospatial analytics. Our solutions empower businesses with actionable insights, leveraging the power of AI,...
-
Data Platform Engineer
1 week ago
India GeoTech Full timeLocation: [Remote, with occasional travel to Riyadh, Saudi Arabia]Department: Data and AIEmployment Type: Full-timeAbout GeoTechGeoTech is at the forefront of data-driven innovation, combining cutting-edge technology with advanced geospatial analytics. Our solutions empower businesses with actionable insights, leveraging the power of AI, computer vision, and...
-
Data Platform Engineer
3 weeks ago
India Luxoft Full timeJob Overview:As a Data Platform Engineer at Luxoft, you will be responsible for designing and implementing a scalable data collection, storage, and distribution platform to house data from various sources. This platform will provide data to systematic and fundamental PMs, as well as enterprise functions, including Ops, Risk, Trading, and Compliance. You will...
-
Data Platform Engineer
1 month ago
India ApTask Full timePrimary responsibilities: Lead the design and engineering of a scalable, secure, and cost-optimized Streaming Data Platform Lead role in evolving from (current) on-prem to hybrid cloud infrastructure with Kafka as the core technology Lead platform upgrades and installs. Perform the procedures in Development. Collaborate with Operations team in production....
-
Lead data engineer
3 days ago
India Wavicle Data Solutions Full timeJob Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, Py Spark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key...
-
Data Engineer
17 hours ago
India Atlas - the Recruitment Platform Full timeBefore you start reading, please read this carefully! I've spent a good amount of time thinking about what to write :D Who we are Atlas Technologies is an anti-CRM built solely for recruitment professionals. A customer relationship management tool is a required software to run any business. The problem is that data entry is extremely...
-
Data Engineer
1 day ago
India Atlas - the Recruitment Platform Full timeBefore you start reading, please read this carefully! I've spent a good amount of time thinking about what to write :D Who we are Atlas Technologies is an anti-CRM built solely for recruitment professionals. A customer relationship management tool is a required software to run any business. The problem is that data entry is extremely onerous and these...
-
Data Engineer
20 hours ago
India Atlas - the Recruitment Platform Full timeBefore you start reading, please read this carefully! I've spent a good amount of time thinking about what to write :DWho we areAtlas Technologies is an anti-CRM built solely for recruitment professionals.A customer relationship management tool is a required software to run any business. The problem is that data entry is extremely onerous and these systems...
-
Lead Data Engineer
1 week ago
India Wavicle Data Solutions Full timeJob Description:We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering.As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions.Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success.Key Responsibilities:Lead...
-
Lead Data Engineer
1 week ago
India Wavicle Data Solutions Full timeJob Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key...
-
Lead Data Engineer: Strategic Data Architect
1 week ago
India Wavicle Data Solutions Full timeWavicle Data Solutions is seeking an experienced Lead Data Engineer to play a pivotal role in architecting and implementing data solutions.About the RoleThis position offers a competitive salary of $180,000 - $220,000 per annum, commensurate with experience. Key ResponsibilitiesDesign and develop scalable and efficient data pipelines using Python and...
-
AWS Data Engineer
3 weeks ago
india ITI Data Full timeJob Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....
-
AWS Data Engineer
3 months ago
India ITI Data Full timeJob Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....
-
AWS Data Engineer
3 months ago
India ITI Data Full timeJob Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....
-
AWS Data Engineer
2 weeks ago
India ITI Data Full timeJob Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....
-
AWS Data Engineer
3 months ago
India ITI Data Full timeJob DescriptionWe are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources. There...
-
india Wavicle Data Solutions Full timeJob Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key Responsibilities:...
-
india Wavicle Data Solutions Full timeJob Description:We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering.As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions.Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success.Key Responsibilities:Lead...