Big Data Systems Architect and DevOps Lead
5 days ago
o9 Solutions, Inc. is a leading provider of AI-driven platforms for enterprises, aiming to digitally transform planning and decision-making for the planet. With a strong entrepreneurial culture and flat organization, we offer a unique opportunity to join our mission.
About the RoleWe are seeking an experienced Big Data Systems Architect and DevOps Lead to join our team in Dallas or Amsterdam. As a key member of our architecture team, you will design and deploy large-scale, data-intensive solutions using Spark, Delta Lake, and Airflow schedulers. Your expertise in designing and optimizing data queries within Spark/Delta Lake environments, using tools such as Apache Hive, HDFS, and Apache Spark, will be crucial in ensuring the successful execution of projects.
Responsibilities- Collaborate with customers and technical consultants to design and recommend big data architecture solutions tailored to customer requirements.
- Analyze complex distributed production environments and provide actionable recommendations for performance optimization.
- Develop and present detailed architectural documentation for customers' technical teams.
- Partner with o9's Development, DevOps, and project teams to ensure the successful execution of projects.
- Design and implement distributed systems and configurations for Spark, Delta Lake, Hadoop, and related technologies, particularly in cloud-based deployments.
- Produce and maintain comprehensive technical documentation and user manuals for customers.
- Stay current with the latest advancements in the distributed big data ecosystem and cloud technologies.
We require 8+ years of experience in DevOps, with a focus on architecting large-scale storage, data processing, and globally distributed solutions. Proven expertise in designing and deploying large-scale, data-intensive solutions using Spark, Delta Lake, and Airflow schedulers is essential. Strong experience designing and optimizing data queries within Spark/Delta Lake environments, using tools such as Apache Hive, HDFS, Apache Spark, and Hadoop, is also required. Additionally, you should have strong understanding of containerization and container orchestration using K8s to manage Spark jobs and other data-intensive applications.
BenefitsWe offer a competitive salary of $150,000 - $200,000 per year, depending on location and experience. Our benefits package includes health insurance, retirement plan, paid time off, and opportunities for professional growth and development. We prioritize teamwork across all levels and functions, promoting open communication and collaboration.
Roadmap for Success- Develop and maintain tools that ensure smooth operations of o9's cloud-based solution.
- Develop and deploy best-in-class tools that support continuous integration and continuous delivery processes.
- Design and optimize Bigdata platform using Delta lake, Spark/pyspark (Distributed Architecture) using spark code and spark Architecture.
- Perform system and network administration; manage o9 platform deployment at AWS, Azure, Google Cloud; Use Dockers to deploy and manage containers; Monitor infrastructure health using ELK cluster; Administer databases that are part of o9 platforms using MS SQL, Mongo.
- Develop and debug Custom Integration service tasks.
-
Senior Big Data Architect
5 days ago
Bengaluru, Karnataka, India TalentXO Full timeSenior Big Data ArchitectTalentXO is a leading-edge company that seeks a highly skilled and experienced Senior Big Data Architect to join their team. The ideal candidate will have a strong background in designing and implementing scalable big data transformations and pipelines using cloud-based platforms like Azure.The successful candidate will possess...
-
Big Data Architect Leader
2 weeks ago
Bengaluru, Karnataka, India o9 Solutions, Inc. Full timeAbout the RoleAs a Big Data Architect at o9 Solutions, you will play a key role in designing and deploying large-scale, data-intensive solutions using Spark, Delta Lake, and Airflow schedulers.Key Responsibilities:Collaborate with customers and technical consultants to design and recommend big data architecture solutions tailored to customer...
-
Big Data Solutions Architect
5 days ago
Bengaluru, Karnataka, India XHire Full time**About the Role:**XHire is seeking a Big Data Solutions Architect to lead the design, development, and implementation of our data platforms. As a key member of our data organization, you will collaborate with cross-functional teams to deliver high-quality results and drive business growth.**Compensation Package:** $120,000 - $150,000 per year, depending on...
-
Data Architect for Big Data Management
4 weeks ago
Bengaluru, Karnataka, India Synechron Full timeJob Title: Data Architect for Big Data ManagementAbout the Role:We are seeking a highly skilled Data Architect to join our team at Synechron. The ideal candidate will have extensive experience in Informatica Big Data Management and be well-versed in design patterns, Agile, DevOps, and CI/CD principles.Key Responsibilities:The successful candidate will be...
-
Data Architect Lead for Big Data Innovation
2 weeks ago
Bengaluru, Karnataka, India FxConsulting Full timeCompany OverviewFxConsulting is a dynamic and innovative company seeking a highly skilled Data Architect Lead to join our team. As a Data Architect Lead, you will play a pivotal role in designing, building, and optimizing large-scale data pipelines and systems to drive business insights and innovation.Estimated Salary$150,000 - $200,000 per year.Job...
-
Big Data Architect and Engineering Lead
2 weeks ago
Bengaluru, Karnataka, India Prime Search Consultant Full timeJob DescriptionWe are seeking a highly skilled Big Data Architect and Engineering Lead to join our team at Prime Search Consultant. This role will be responsible for architecting and developing large-scale data systems, leading cross-functional teams, and collaborating with stakeholders to drive business outcomes.About the Role:This is an exciting...
-
Data Architect for Scalable Big Data Solutions
4 weeks ago
Bengaluru, Karnataka, India Zeta Global Full timeJob SummaryWe are seeking a highly skilled Data Architect for Scalable Big Data Solutions to join our team at Zeta Global. As a key member of our Data Cloud team, you will be responsible for designing and growing our existing data infrastructure to democratize data access, enable complex data analyses, and automate optimization workflows for business and...
-
Big Data Solutions Architect
4 weeks ago
Bengaluru, Karnataka, India Covenant Consultants Full timeCovenant Consultants is a renowned company seeking an experienced Big Data Solutions Architect to spearhead the design and development of cutting-edge data pipelines. The ideal candidate will possess in-depth knowledge of Databricks, Python, Big Data, Apache Spark, and SQL, with experience in building data governance solutions like Unity Catalog and...
-
Big Data Architect
3 weeks ago
Bengaluru, Karnataka, India Telescopp Full timeJob OverviewWe are seeking a seasoned Big Data Architect to lead the development of scalable big data solutions in Bengaluru. This is a mid-level, full-time position requiring 4-6 years of relevant experience. As a Big Data Architect at Telescopp, you will design and build robust big data infrastructure, develop ETL processes, collaborate with data...
-
Java Architect for Big Data
5 days ago
Bengaluru, Karnataka, India EduRun Full timeJob Title: Java Architect for Big DataJob Summary:We are seeking an experienced Java Architect for Big Data to join our team at EduRun. As a key member of our architecture team, you will be responsible for designing and implementing scalable and performant Java applications for big data processing.Key Responsibilities:- Design and develop efficient Java code...
-
Big Data Solutions Architect
6 days ago
Bengaluru, Karnataka, India Tavant Full timeJob OverviewTavant seeks a highly skilled Big Data Solutions Architect to join our team. As a key member of our data engineering team, you will design and implement scalable and efficient data processing pipelines using Apache Spark and Java.About the RoleIn this role, you will develop APIs and services for data ingestion, transformation, and aggregation to...
-
Big Data Engineer
18 hours ago
Bengaluru, Karnataka, India Tata Consultancy Services Full timeJob Title: Big Data Engineer - Synapse ArchitectOverview:Tata Consultancy Services is a leading IT services company, and we are looking for a highly skilled Big Data Engineer to join our team. As a Big Data Engineer, you will play a key role in designing and implementing scalable data solutions using Azure Synapse.About the Role:We are seeking an experienced...
-
Cloud-Based Big Data Solutions Architect
3 days ago
Bengaluru, Karnataka, India o9 Solutions, Inc. Full timeWe are seeking a skilled Cloud-Based Big Data Solutions Architect to join our team at o9 Solutions, Inc.As a Cloud-Based Big Data Solutions Architect, you will be responsible for designing and implementing large-scale cloud-based big data solutions using Spark, Delta Lake, and Airflow schedulers.Key responsibilities include collaborating with customers and...
-
Big Data Solutions Architect
1 month ago
Bengaluru, Karnataka, India NETSACH GLOBAL Full timeJob Title: Big Data Engineer/ArchitectNetsach Global is seeking a highly skilled Big Data Engineer/Architect to design, develop, and maintain large-scale data processing systems and architecture. The ideal candidate will work closely with data scientists, analysts, and other stakeholders to ensure data systems are scalable, secure, and efficient, enabling...
-
Senior Big Data Architect
4 weeks ago
Bengaluru, Karnataka, India Visionyle Solutions Full timeJoin Visionyle Solutions as a Senior Big Data Architect and take on the exciting challenge of designing and implementing cutting-edge big data solutions. Expected Salary: $120,000 - $150,000 per year.About the Role:In this role, you will be working with our globally distributed team to develop innovative Hadoop-based solutions that meet business needs and...
-
Lead Data Architect
2 weeks ago
Bengaluru, Karnataka, India Data Labs - LANDMARK GROUP Full timeJob Title: Lead Data ArchitectLocation: Yemalur, BangaloreAt Data Labs - LANDMARK GROUP, we are seeking a skilled Lead Data Architect to spearhead our engineering team in successfully designing and deploying cutting-edge analytics products.Main Responsibilities:Lead the architecture, design, and delivery of enterprise-grade microservices.Partner with Product...
-
Data Architect
1 month ago
Bengaluru, Karnataka, India Unison Consulting Pte Ltd Full timeAbout the RoleWe are seeking an experienced Data Architect to lead our big data solutions. As a key member of our team, you will design, develop, and maintain large-scale big data architectures to enable data-driven decision making and analytics capabilities for our clients.
-
Big Data Architect
5 days ago
Bengaluru, Karnataka, India Prodcon Tech Services Pvt Ltd Full timeJob SummaryProdcon Tech Services Pvt Ltd is hiring a Big Data Architect to lead the development of our data infrastructure and ensure the efficient management of large datasets.The ideal candidate will have experience with Transact-SQL (TSQL), Extract, Transform, Load (ETL) processes, and Data Warehousing concepts, as well as a deep understanding of cloud...
-
Lead Big Data Engineer
5 days ago
Bengaluru, Karnataka, India Freelance Full timeData Architecture Expert WantedEstimated Salary: $140,000 - $200,000 per yearWe are seeking a seasoned Data Architect to lead our big data initiatives. The ideal candidate will have extensive experience in designing, developing, and maintaining large-scale data systems using Spark/Scala.Responsibilities:Design, develop, and maintain data architectures for...
-
Principal Big Data Architect
2 weeks ago
Bengaluru, Karnataka, India FxConsulting Full timeEstimated Salary: $120,000 - $180,000 per year.About FxConsultingWe are a leading consulting firm seeking an experienced Principal Big Data Architect to join our team. Our dynamic team focuses on delivering cutting-edge data solutions that drive business growth and innovation.Job DescriptionIn this role, you will play a pivotal part in designing, building,...