Data Engineering Leader for Big Data Platforms
3 weeks ago
We are seeking a seasoned Data Engineer to spearhead the development of our big data platforms at Recro. This is an excellent opportunity for an experienced professional to join our team and make a significant impact.
About the JobIn this role, you will design, build, and maintain large-scale distributed data processing systems that handle 100s of terabytes of data daily. Your expertise in Apache Spark, HDFS, YARN, Map-Reduce, Hive, Kafka, Airflow, Presto, and other big data technologies will be crucial in driving our data-driven decisions across products.
Key Responsibilities- Data Platform Development: Build highly scalable, available, and fault-tolerant distributed data processing systems (batch and streaming systems) that process petabyte-sized data warehouse and Elasticsearch clusters.
- Quality Data Solutions: Design and implement quality data solutions, refine existing diverse datasets, and encourage self-service by simplifying models.
- Data Pipelines: Create data pipelines that optimize on data quality and are resilient to poor-quality data sources.
- System Debugging and Optimization: Perform low-level system debugging, performance measurement, and optimization on large production clusters.
- Architecture and Roadmap: Participate in architecture discussions, influence product roadmaps, and take ownership of new projects.
To succeed in this role, you should have:
- Proficiency in Python and PySpark.
- Deep understanding of Apache Spark, Spark tuning, creating RDDs, and building data frames.
- Experience in big data technologies like HDFS, YARN, Map-Reduce, Hive, Kafka, Spark, Airflow, Presto, etc.
- Experience in building distributed environments using any of Kafka, Spark, Hive, Hadoop, etc.
- Good understanding of the architecture and functioning of Distributed database systems.
- Experience working with various file formats like Parquet, Avro, etc., for large volumes of data.
- Experience with one or more NoSQL databases.
- Experience with AWS and GCP.
- 5+ years of professional experience as a Data or Software Engineer.
-
Data Platform Leader
4 weeks ago
Bengaluru, Karnataka, India Zyoin Full timeAbout the RoleRoku is modernizing its Big Data Platform, and we need a seasoned leader to help us build a state-of-the-art data platform. As our Senior Engineering Manager - Cloud Infrastructure, you will be responsible for building and onboarding a strong 8-10 people Big Data platform team in Bangalore.You will lead the team roadmap, quarterly planning, and...
-
Data Engineering Lead with Scalable Platforms
4 weeks ago
Bengaluru, Karnataka, India HEVO DATA Full timeUnlock Career Growth as a Data Engineering LeadWe're seeking an exceptional Data Engineering Lead to join our team at Hevo Data. In this role, you'll have the opportunity to own and build scalable platform services that support thousands of customers' data pipelines and ensure load data reliability to cloud warehouses.The Hevo platform core services will...
-
Data Platform Solutions Expert
1 month ago
Bengaluru, Karnataka, India HEVO DATA Full timeAbout the RoleWe're seeking a skilled Data Platform Solutions Expert to join our Sales team at HEVO DATA. This is an exciting opportunity for someone with a strong background in Data Domain to work closely with sales professionals, helping them understand customers' business and technical needs and demonstrate how our platform can solve their real-time data...
-
Big Data Engineering Leader
4 weeks ago
Bengaluru, Karnataka, India Publicis Sapient Full timeJob SummaryWe are seeking a seasoned Senior Associate Data Engineering Level 2 to join our team at Publicis Sapient. As a Senior Associate, you will play a critical role in translating client requirements into technical design and implementing components for data engineering solutions.The ideal candidate will have a deep understanding of data integration and...
-
Big Data Platform Engineer
1 week ago
Bengaluru, Karnataka, India Sigmoid Full timeAt Sigmoid, we're building a highly-scalable and extensible big data platform. We're seeking a skilled engineer to assist our teams in various roles.As an expert in Java, Spring boot, and Microservices, you'll be responsible for developing this platform, which collects, stores, models, and analyzes massive data sets from multiple channels.We're looking for...
-
Data Engineering Leader
2 weeks ago
Bengaluru, Karnataka, India Data Inc Full timeJob SummaryWe are seeking a highly skilled and experienced Data Architect to lead our team at Data Inc. This is a key role in shaping the future of our data architecture and driving business growth.About the RoleThe successful candidate will be responsible for overseeing the development and implementation of our data architecture strategy, ensuring alignment...
-
Enterprise Data Architect Leader
4 weeks ago
Bengaluru, Karnataka, India Data Inc Full timeAbout This RoleWe are seeking a highly skilled Enterprise Data Architect Leader to join our team at Data Inc. In this role, you will be responsible for leading the development and implementation of architecture strategy and standards pertinent to the domain.
-
Data Engineering Leader
4 weeks ago
Bengaluru, Karnataka, India Anzyglobal Full timeAt Anzyglobal, we are seeking an experienced Data Engineering Leader to lead and manage a team of data engineers. The successful candidate will have at least 5 years of experience in data engineering, with a strong focus on large-scale data platforms and data products.The ideal candidate will be proficient in either Scala or Java programming language and...
-
Big Data Engineer
4 weeks ago
Bengaluru, Karnataka, India NetAnalytiks Technologies Full timeJob Title: Big Data EngineerWe are seeking an experienced Big Data Engineer to join our team at NetAnalytiks Technologies. As a key member of our data engineering team, you will play a crucial role in designing and implementing large-scale data processing systems.Key Responsibilities:Design and develop scalable data pipelines using Apache Spark and...
-
Big Data Engineer
17 hours ago
Bengaluru, Karnataka, India Tata Consultancy Services Full timeOverviewTata Consultancy Services (TCS) is a global leader in IT services and consulting, with a rich history of innovation and excellence. We are looking for a talented Big Data Engineer - Data Scientist to join our team.Job DescriptionWe are seeking a highly skilled Big Data Engineer - Data Scientist to design and develop high-quality data solutions that...
-
Data Engineering Director
4 weeks ago
Bengaluru, Karnataka, India HEVO DATA Full timeEngineering Leadership OpportunityThis is a unique chance to own and build scalable platform services that support thousands of customers' data pipelines, ensuring load data reliability to cloud warehouses.The Hevo platform core services will provide no-code-low pipeline building capabilities, making a significant impact on the company's business.About the...
-
Big Data Engineer
4 weeks ago
Bengaluru, Karnataka, India Worksconsultancy Full timeCompany OverviewWe are Worksconsultancy, a leading provider of innovative solutions in the field of data engineering.About the RoleWe are seeking an experienced Data Engineer to join our team. The successful candidate will be responsible for designing, building, and maintaining large-scale data systems using big data technologies such as Apache Spark,...
-
Big Data Engineering Leader
3 weeks ago
Bengaluru, Karnataka, India Publicis Sapient Full timeAbout UsPublicis Sapient is a leading digital transformation partner that helps established organizations drive growth and innovation in the digital age. We combine strategy, consulting, and customer experience with agile engineering and problem-solving creativity to unlock value for our clients.Job DescriptionWe are seeking a Senior Associate Data...
-
Data Engineering Leader
17 hours ago
Bengaluru, Karnataka, India Giant Eagle GCC Full timeCompany Overview:Giant Eagle GCC is a leading global food retailer and distributor with a strong presence in the US and India. With over 37,000 team members and $9.7 billion in revenue, we are committed to investing in people, technology, and data to elevate our customer's experience across multiple touchpoints.Job Description:The Data Engineering Manager...
-
Big Data Engineer Specialist
1 month ago
Bengaluru, Karnataka, India LTIMindtree Full timeLTIMindtree is a global leader in digital transformation consulting and technology services. We are seeking an experienced Big Data Engineer to join our team in Bengaluru, NCR Noida, Gurugram, or Delhi.The ideal candidate will have 5+ years of experience in working with big data technologies such as Pyspark, SQL, and Hive. The role involves designing,...
-
Big Data Engineering Expert
4 weeks ago
Bengaluru, Karnataka, India Grid Dynamics Full timeJob Title: Big Data Engineering ExpertCompany Overview: Grid Dynamics is a leading provider of technology consulting, platform and product engineering, and advanced analytics services. We are seeking an experienced Big Data Engineering Expert to join our team.Job Description: As a Big Data Engineering Expert at Grid Dynamics, you will be responsible for...
-
Data Platform Strategist
2 weeks ago
Bengaluru, Karnataka, India UST Full timeAbout USTUST is a global company that delivers value to its clients by leveraging technology and innovation.Job Title: Big Data Engineer LeaderSalary:$150,000 - $200,000 per annum, depending on location and experienceJob Description:We are seeking an experienced Big Data Engineer Leader to join our team at UST. The successful candidate will lead the design...
-
Big Data Engineer
4 weeks ago
Bengaluru, Karnataka, India HR Consulting Full timeAbout the RoleWe are seeking a skilled Senior Data Engineer to join our team at HR Consulting in Bengaluru. This is an exciting opportunity to work on deep data engineering projects and help define our technical strategy and team culture.This role involves designing and building data platforms and services while managing our data infrastructure in cloud...
-
Data Scientist and Engineer for Big Data Analytics
17 hours ago
Bengaluru, Karnataka, India Tata Consultancy Services Full timeJob Overview:">">We are looking for a highly skilled Data Scientist and Engineer to join our team.">">About the Role:">The successful candidate will be responsible for designing, developing, and maintaining high-quality data analytics solutions using big data technologies such as Hadoop, Hive, and Spark.">Key Responsibilities:">">Design and develop...
-
Data Engineering Leader
16 hours ago
Bengaluru, Karnataka, India Recro Full timeAbout the RoleWe are seeking a seasoned Data Engineering Leader to join our team at Recro. As a key member of our engineering organization, you will play a vital role in designing and implementing scalable data processing systems that drive business insights.Key ResponsibilitiesDesign and build large-scale distributed data processing systems using...