Big Data Engineer
4 weeks ago
This role is for a Data Engineering Lead to work on the Vanguard Big Data Platform. The team is responsible for the maintenance and development of leading Big Data initiatives and use cases providing business value.
Job Background/context:
The technology team is responsible for building and maintaining a multitude of applications including:
Real-time ingestion/stream processing and data distribution via Big Data APIs
Build out canonical models and data conformance
Implement best-in-class data management and data ingestion.
Leverage new storage engines like Kudu that enable analytics on fast-changing data
Leverage GPU implementation to enable advanced Machine Learning
Enhance Self-Service capability for data science and ML practitioners
This is a hands-on development role that will offer exposure to the full development cycle, whilst working closely with our business and technology stakeholders.
Analysis and development across Lines of business including Payments, Digital Channels, Liquidities, Trade
Cross-train and fertilize functional and technical knowledge,
Align to Engineering Excellence Development principles and standards.
Promote and increase our Development Productivity scores for coding
Fully adhere to and evangelize a full Continuous Integration and Continuous Deploy pipeline
Development Value:
This role would open career opportunities for the successful individual to establish their profile in the Data Innovation and Architecture organization.
It provides an opportunity in Institutional Client banking to work closely with the business to provide valuable data solutions
Knowledge/Experience:
· 5+ years of experience in Hadoop/big data technologies.
· 3+ years of hands-on experience as a Scala developer (with a previous Java background)
· Experience with Spark/Storm/Kafka or equivalent streaming/batch processing and event-based messaging
· Experience with API development and use of JSON/XML/Hypermedia data formats
· Relational and NoSQL database integration and data distribution principles experience
· Unix/Linux
· Experience with containerization and related technologies (e.g. Docker, Kubernetes)
· Experience with all aspects of DevOps (source control, continuous integration, deployments, etc.)
· Comprehensive knowledge of the principles of software engineering and data analytics
· Advanced knowledge of the Hadoop ecosystem and Big Data technologies Hands-on experience with the Hadoop eco-system (HDFS, MapReduce, Hive, Pig, Impala, Spark, Kafka, Kudu, Solr)
· Knowledge of agile(scrum) development methodology is a plus
· Strong development/automation skills
· Cloudera/Hortonworks/AWS EMR, S3 experience a plus
Qualifications:
Strong academic record, ideally with a good degree and a mathematical or scientific background.
· Strong Communication skills
· Self Motivated
· Willingness to learn
· Excellent planning and organizational skills
-
Big Data Engineer
4 weeks ago
India TekJobs Full timeThis role is for a Data Engineering Lead to work on the Vanguard Big Data Platform. The team is responsible for the maintenance and development of leading Big Data initiatives and use cases providing business value.Job Background/context:The technology team is responsible for building and maintaining a multitude of applications including:Real-time...
-
Big Data Engineer
3 days ago
india NeoStats Full timeJob Description About Neostats Analytics: NeoStats is a new-age, Data & Analytics firm offering contemporary solutions and infinite possibilities. Our mission is to create lasting competitive advantage for our clients by transforming them into world-class, data-driven organizations. Established in 2022 to provide End to End Data & Analytics Services, we...
-
Data Engineer
2 days ago
india Evnek Technologies Pvt Ltd Full timeJob Description This is a remote position. Role: Data Engineer Experience: 2+Years Notice period: Immediate Location: Anywhere from India Job Type: Full Time Responsibilities: Designing & building data architectures including the design & set-up of scalable (big) data lakes (full cloud or hybrid cloud), scheduled transformations &...
-
AWS Big Data Engineer
1 month ago
india Bosch Full timeJob Description Technical skills ETL development experience with overall 4 to 7 years in AWS , Big data engineering related application Minimum 2 end to end ETL project implementation Proficient in PySpark, Python4. Proficient in Big Data technologies - Hadoop, Spark, Hive, Databricks5. Expert in AWS to develop, deploy and maintain ETL...
-
Data Engineer –
4 weeks ago
india HARMAN International Full timeHARMAN’s engineers and designers are creative, purposeful and agile. As part of this team, you’ll combine your technical expertise with innovative ideas to help drive cutting-edge solutions in the car, enterprise and connected ecosystem. Every day, you will push the boundaries of creative design, and HARMAN is committed to providing you with the...
-
Senior Data Engineer
1 month ago
india SWAI TECHNOLOGIES PRIVATE LIMITED Full timeJob Description : We are seeking a highly skilled Senior Data Engineer with 5 - 12 years of experience in building scalable data solutions. The ideal candidate should be proficient in Python programming and SQL, with extensive experience in utilizing various AWS services for data processing and storage. Experience with big data processing environments,...
-
Big Data Engineer
1 month ago
india Careerfit.ai Full timeResponsibilities :Solution Design :- Design, develop, and maintain scalable big data solutions and data pipelines for efficient data ingestion, processing, storage, and analysis.- Collaborate with data scientists, analysts, and business stakeholders to identify data requirements, use cases, and business objectives.- Translate business needs into technical...
-
Big Data Architect
3 weeks ago
India Bloom Consulting Services Full timeNA Experience **12 - 16 years** Offered Salary Notice Period **Not Disclosed** **Role: Big Data Architect** **Location: Bangalore** **Skills required**: Total 12 to 16 years of experience in BI, analytics, visualization, **data** integration, designing and developing **data **warehouses (ETL, **data**bases, front-end tools). 9+ years of...
-
Big Data Architect
3 weeks ago
India Bloom Consulting Services Full timeNot Disclosed Experience **12 - 16 Years** Offered Salary **Lakh** Notice Period **Not Disclosed** **Role: Big Data Architect** **Location: Bangalore** **Skills required**: Total 12 to 16 years of experience in BI, analytics, visualization, **data** integration, designing and developing **data **warehouses (ETL, **data**bases, front-end...
-
Big Data Engineer
1 month ago
india ATech Full timeDesignation: BIG DATA ENGINEERJob Description:Your Role and Responsibilities:- Understand a data warehousing solution and able to work independently in such an environment- Responsible in Project development and delivery experience of a few good size projects- Design, build, optimize and support new and existing data models and ETL processes based on our...
-
Big Data Analyst
1 month ago
india Royal Cyber Inc. Full time1) 5+ years of experience managing and deploying CDP or HDP clusters.2) Experience configuring and setup of big data environments.3) Experience with Kerberos security and LDAP setup.4) Experience in setting up monitoring and alerting pipelines for big data clusters.
-
Big Data Quality Assurance Engineer
4 weeks ago
Bangalore/Anywhere in India/Multiple Locations, IN Leadway Resource Full timeWe are seeking a highly skilled and experienced Big Data QA Engineer to join our team. You will play a critical role in ensuring the quality of our big data platform and data solutions. You will leverage your expertise in Big Data testing methodologies, automation frameworks, and scripting languages to design, develop, and execute comprehensive test...
-
Big Data Quality Assurance Engineer
1 month ago
Bangalore,Anywhere in India,Multiple Locations Leadway Resource Full timeWe are seeking a highly skilled and experienced Big Data QA Engineer to join our team. You will play a critical role in ensuring the quality of our big data platform and data solutions. You will leverage your expertise in Big Data testing methodologies, automation frameworks, and scripting languages to design, develop, and execute comprehensive test...
-
Big Data Developer
3 weeks ago
india RiskInsight Consulting Pvt Ltd Full timeWe are seeking a skilled Data Engineer with 4-8 years of experience, proficient in Big Data technologies and data processing frameworks. The ideal candidate will have strong expertise in Hadoop ecosystem components such as HDFS, HiveQL, and experience in developing and optimizing data processing workflows using PySpark. Proficiency in SQL for data...
-
india Digihelic Solutions Pvt. Ltd. Full timeJob Summary :We are seeking a highly skilled and motivated Product Solution Architect with extensive expertise in designing and implementing cutting-edge big-data solutions on the Amazon Web Services (AWS) platform.As a key member of our dynamic team, you will play a pivotal role in driving the success of our data-driven initiatives, leveraging AWS services...
-
Lead Big Data Engineer
3 weeks ago
Anywhere in India/Multiple Locations, IN Grownex HR Solution Full timeRole : Lead Big Data EngineerWork Mode : Remote(5 Days working)The Opportunity - Development team is establishing a new offshore product development team for its partner , that is building next-generation Big Data, Cloud-Based Business Operation Support technology for utilities, retail energy suppliers and Community Choice Aggregators (CCA). The candidate...
-
Lead Big Data Engineer
3 weeks ago
Anywhere in India,Multiple Locations Grownex HR Solution Full timeRole : Lead Big Data Engineer Work Mode : Remote(5 Days working) The Opportunity - Development team is establishing a new offshore product development team for its partner , that is building next-generation Big Data, Cloud-Based Business Operation Support technology for utilities, retail energy suppliers and Community Choice Aggregators (CCA). The...
-
Big Data Engineer
4 weeks ago
Bangalore/Anywhere in India/Multiple Locations, IN Connectio IT Pvt Ltd Full timeKey Skills : Java, Apache Spark, Apache Kafka, Hadoop, AWSJob Description :- Designing, installing, testing, and maintaining scalable data management systems- Ensuring systems meet business requirements and industry standards- Creating custom software components and analytics applications- Research data acquisition opportunities and new uses for existing...
-
Big Data Engineer
1 month ago
Bangalore,Anywhere in India,Multiple Locations Connectio IT Pvt Ltd Full timeKey Skills : Java, Apache Spark, Apache Kafka, Hadoop, AWS Job Description : - Designing, installing, testing, and maintaining scalable data management systems- Ensuring systems meet business requirements and industry standards- Creating custom software components and analytics applications- Research data acquisition opportunities and new uses for existing...
-
Senior AWS Data Engineer
4 days ago
india LevelUP HCS Full timeAWS Data Engineer- Kafka We are looking for a Data Engineer (AWS Data Platform) to join our busy and dynamic business intelligence – data platform team. If you are a passionate Data Engineer you have the chance to be part of a team that looks to build out our next-generation data platform, leveraging new technologies and techniques to maximize the...