Big data engineer with devops
2 weeks ago
Position – Big Data Engineer with Devops Exp - 6-10 Yrs Job Location- Bangalore ( 3 days hybrid working ) Job Description As a Software Development Engineer you will be responsible for expanding and optimizing our data and data pipeline architecture as well as optimising data flow and collection for cross-functional teams. The ideal candidate is an experienced data pipeline designer and data wrangler who enjoys optimizing data systems and building them from the ground up. The Data Engineer will lead our software developers on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. They must be self-directed and comfortable supporting the data needs of multiple teams systems and products. The right candidate will be excited by the prospect of optimizing or even re-designing our company’s data architecture to support our next generation of products and data initiatives. Responsibilities Create and maintain optimal data pipeline architecture Assemble large complex data sets that meet functional / non-functional business requirements. Identify design and implement internal process improvements: automating manual processes optimizing data delivery, coordinating to re-design infrastructure for greater scalability etc. Work with stakeholders including the Executive Product Data and Design teams to assist with data-related technical issues and support their data infrastructure needs. Keep our data separated and secure Work with data and analytics experts to strive for greater functionality in our data systems. Support PROD systems Qualifications Must have About 6 - 11 years and at least 3 years relevant experience with Bigdata. Must have Experience in building highly scalable business applications, which involve implementing large complex business flows and dealing with huge amount of data. Must have experience in Hadoop, Hive, Spark with Scala with good experience in performance tuning and debugging issues. Hands-on experience in one of the cloud provider (AWS/Azure) is must Good to have any stream processing Spark/Java Kafka. Must have experience in design and development of Big data projects. Good knowledge in Functional programming and OOP concepts, SOLID principles, design patterns for developing scalable applications. Familiarity with build tools like Maven. Must have experience with any RDBMS and at least one No SQL database preferably Postgres SQL Must have experience writing unit and integration tests using scaliest Must have experience using any versioning control system - Git Must have experience with CI / CD pipeline – Jenkins is a plus Databricks Spark certification is a plus.
-
Data Engineer
1 month ago
Bangalore, India Coders Brain Technology Private Limited Full timeQualifications : - Bachelor's degree in Computer Science/Engineering or similar - 5-7 years of data engineering experience with focus on designing and implementing scalable big data transformations and pipelines - Proficiency with Cloud Platforms like Azure(preferred), AWS, and/or Google cloud, including knowledge of core services like Azure: Data...
-
Big Data Engineer
1 month ago
Bangalore, India Tavant Full timeKey Responsibilities :- Design and implement scalable and efficient data processing pipelines using Apache Spark and Java.- Develop APIs and services for data ingestion, transformation, and aggregation to support analytics and machine learning models.- Collaborate with data scientists, analysts, and cross-functional teams to understand data requirements and...
-
bangalore, India Damco Solutions Full timePosition – Big Data Engineer with DevopsExp - 6-10 YrsJob Location- Bangalore ( 3 days hybrid working )Job DescriptionAs a Software Development Engineer you will be responsible for expanding and optimizing our data and data pipeline architecture as well as optimising data flow and collection for cross-functional teams. The ideal candidate is an experienced...
-
Senior Software Engineer
2 months ago
Bangalore, India Talpro Full timeJob Description : Key Responsibilities :Design and Development :- Architect, design, and develop scalable and high-performance data platforms.- Implement data ingestion, transformation, and processing pipelines using technologies such as Apache Kafka, Hadoop, Spark, and Flink.- Develop real-time data processing and stream processing applications.Data...
-
bangalore, India Damco Solutions Full timePosition – Big Data Engineer with DevopsExp - 6-10 YrsJob Location- Bangalore ( 3 days hybrid working )Job DescriptionAs a Software Development Engineer you will be responsible for expanding and optimizing our data and data pipeline architecture as well as optimising data flow and collection for cross-functional teams. The ideal candidate is an experienced...
-
bangalore, India Damco Solutions Full timePosition – Big Data Engineer with Devops Exp - 6-10 Yrs Job Location- Bangalore ( 3 days hybrid working ) Job Description As a Software Development Engineer you will be responsible for expanding and optimizing our data and data pipeline architecture as well as optimising data flow and collection for cross-functional teams. The ideal candidate is an...
-
DevOps Engineer
2 months ago
bangalore, India Sigmoid Full timeAs a DevOps Enginner-2/Lead, you will be responsible for managing, design highly scalable and Available solution for data pipelines that provides the foundation for collecting, storing, modelling, and analysing massive data sets from multiple channels.This position reports to DevOps Architect.Responsibilities:Align Sigmoid with key Client...
-
Cloud big data engineer
2 weeks ago
Bangalore, India Lakshya Software Technologies Pvt. Ltd. Full timeTitle: Cloud Big Data Engineer Exp - 5+ years Location: Anywhere in India Work Mode: HYBRID Mandatory Skills required: Big Data (Hadoop, Hive), Python, Spark, Py Spark, Cloud Platform, Azure Job description: Understanding of Big data Ecosystem with hands on experience working on Apache Spark, Hadoop (HDFS , Hive), Parquet, Elastic Search ...
-
Big Data Engineer
1 month ago
Bangalore, India Eximietas Design Full time6+ years of experience in Big data engineering with a focus on data pipeline setup and management. ● Strong development experience in Java ● Hands-on experience with Kafka , S3, CDC, Spark, and SQL. ● Proficient with container technologies like Docker. ● Experience working in cloud environments and handling large datasets. ● Solid...
-
Big Data Developer
2 months ago
Bangalore, India Luxoft Full timeProject Description: The Cirrus project is developing cloud-based compliance state-of-the-art archive products to archive and retain real-time communications data in line with internal and external regulatory requirements. The product is developed in-house in an agile development setup, based on business requirements from multiple stakeholder parties. By...
-
Big data engineer
2 weeks ago
Bangalore, India Capco Full timeJob Title: Big Data Engineer About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount . With our presence across 32 cities across globe, we support 100+ clients...
-
Senior Data Engineering
1 month ago
Bangalore, India FxConsulting Full timeJob Title : Lead Data EngineerWe are seeking a highly skilled Lead Data Engineer with expertise in Big Data technologies to join our dynamic team. As a Lead Data Engineer, you will play a pivotal role in designing, building, and optimizing large-scale data pipelines and systems to drive business insights and innovation.Key Responsibilities : - Design,...
-
Cloud Big Data Engineer
2 weeks ago
Bangalore, India Lakshya Software Technologies Pvt. Ltd. Full timeTitle: Cloud Big Data Engineer Exp - 5+ years Location: Anywhere in India Work Mode: HYBRID Mandatory Skills required: Big Data (Hadoop, Hive), Python, Spark, PySpark, Cloud Platform, Azure Job description: Understanding of Big data Ecosystem with hands on experience working on Apache Spark, Hadoop (HDFS , Hive), Parquet, Elastic Search ...
-
Big data developer
2 weeks ago
Bangalore, India Impetus Full timeBig Data Developer Experience- 3 to 12 years Locations- Indore, Noida, Hyderabad, Bangalore, Gurgaon Job Description We are looking for a Big Data Developer to design and oversee the implementation of large-scale data solutions, lead a talented team, and ensure the efficient processing and analysis of massive datasets. Roles &...
-
Big Data Engineer
2 weeks ago
Bangalore, India Capco Full timeJob Title: Big Data Engineer About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount . With our presence across 32 cities across globe, we support 100+...
-
Solution Engineer
1 month ago
Bangalore, India HEVO DATA Full timeOverview :. We're seeking a talented Solutions Engineer with strong background in Data Domain to join our growing Sales team. You'll work one-on-one with the sales team to help discover our customers' business and technical needs and demonstrate how Hevo's platform can solve their real-time data replication needs. In this role, you will...
-
Data Scientist/Engineer
3 days ago
Bangalore/Remote, India Skima Innovations Pvt ltd Full timeAbout the Company : We are a stealth company revolutionizing the marketing landscape with our AI-powered platform. Our team has pioneered AI-based marketing systems at some of the most iconic companies in the world including Expedia, Amazon, Snapchat, and BCG to transform how businesses direct their marketing spend to drive unprecedented growth and...
-
Big Data Engineer
3 days ago
Chennai/Bangalore/Hyderabad, India Ascent Force Full timeJob Title : Big Data Engineer Location : Chennai, Bangalore, Hyderabad Key Responsibilities : - Design, develop, and implement data pipelines, leveraging Azure Data Factory, Azure Synapse Analytics, Azure SQL, and Azure Data Lake. - Create scalable, efficient, and robust data solutions to manage data ingestion through APIs and streaming data sources,...
-
Chennai/Bangalore, India Globalemployees Full timeAbout UsGlobalemployees is a cutting-edge technology company that leverages the power of big data to drive innovation.Job DescriptionWe are seeking a highly skilled Data Engineer to join our team. As a key member of our engineering team, you will design, build, and maintain large-scale data processing systems using cloud-based technologies like...
-
Big data platform engineer
2 weeks ago
Bangalore, India Prodapt Full timeRoles and Responsibilities: Understand the requirements from Cloud user (Developers/Data scientists/Data analysts) and work with platform/infra teams to onboard their solutions in Big Query or Data Proc platforms. Create BQ datasets, Compute projects, Hive tables, GCS buckets and Service accounts etc. Grant access to BQ datasets, GCS buckets and Hive...