Big Data Engineer

3 weeks ago


India TekJobs Full time

This role is for a Data Engineering Lead to work on the Vanguard Big Data Platform. The team is responsible for the maintenance and development of leading Big Data initiatives and use cases providing business value.


Job Background/context:


The technology team is responsible for building and maintaining a multitude of applications including:


Real-time ingestion/stream processing and data distribution via Big Data APIs

Build out canonical models and data conformance

Implement best-in-class data management and data ingestion.

Leverage new storage engines like Kudu that enable analytics on fast-changing data

Leverage GPU implementation to enable advanced Machine Learning

Enhance Self-Service capability for data science and ML practitioners


This is a hands-on development role that will offer exposure to the full development cycle, whilst working closely with our business and technology stakeholders.


Analysis and development across Lines of business including Payments, Digital Channels, Liquidities, Trade

Cross-train and fertilize functional and technical knowledge,

Align to Engineering Excellence Development principles and standards.

Promote and increase our Development Productivity scores for coding

Fully adhere to and evangelize a full Continuous Integration and Continuous Deploy pipeline


Development Value:

This role would open career opportunities for the successful individual to establish their profile in the Data Innovation and Architecture organization.


It provides an opportunity in Institutional Client banking to work closely with the business to provide valuable data solutions


Knowledge/Experience:

· 5+ years of experience in Hadoop/big data technologies.


· 3+ years of hands-on experience as a Scala developer (with a previous Java background)


· Experience with Spark/Storm/Kafka or equivalent streaming/batch processing and event-based messaging


· Experience with API development and use of JSON/XML/Hypermedia data formats


· Relational and NoSQL database integration and data distribution principles experience


· Unix/Linux


· Experience with containerization and related technologies (e.g. Docker, Kubernetes)


· Experience with all aspects of DevOps (source control, continuous integration, deployments, etc.)


· Comprehensive knowledge of the principles of software engineering and data analytics


· Advanced knowledge of the Hadoop ecosystem and Big Data technologies Hands-on experience with the Hadoop eco-system (HDFS, MapReduce, Hive, Pig, Impala, Spark, Kafka, Kudu, Solr)


· Knowledge of agile(scrum) development methodology is a plus


· Strong development/automation skills


· Cloudera/Hortonworks/AWS EMR, S3 experience a plus


Qualifications:

Strong academic record, ideally with a good degree and a mathematical or scientific background.


· Strong Communication skills


· Self Motivated


· Willingness to learn


· Excellent planning and organizational skills


  • Big Data Engineer

    4 weeks ago


    india TekJobs Full time

    This role is for a Data Engineering Lead to work on the Vanguard Big Data Platform. The team is responsible for the maintenance and development of leading Big Data initiatives and use cases providing business value. Job Background/context: The technology team is responsible for building and maintaining a multitude of applications including: Real-time...

  • Big Data Engineer

    2 days ago


    india NeoStats Full time

    Job Description About Neostats Analytics: NeoStats is a new-age, Data & Analytics firm offering contemporary solutions and infinite possibilities. Our mission is to create lasting competitive advantage for our clients by transforming them into world-class, data-driven organizations. Established in 2022 to provide End to End Data & Analytics Services, we...

  • Data Engineer

    2 days ago


    india Evnek Technologies Pvt Ltd Full time

    Job Description This is a remote position. Role: Data Engineer  Experience: 2+Years Notice period: Immediate Location: Anywhere from India Job Type: Full Time Responsibilities: Designing & building data architectures including the design & set-up of scalable (big) data lakes (full cloud or hybrid cloud), scheduled transformations &...

  • AWS Big Data Engineer

    1 month ago


    india Bosch Full time

    Job Description Technical skills ETL development experience with overall 4 to 7 years in AWS , Big data engineering related application Minimum 2 end to end ETL project implementation Proficient in PySpark, Python4. Proficient in Big Data technologies - Hadoop, Spark, Hive, Databricks5. Expert in AWS to develop, deploy and maintain ETL...

  • Data Engineer –

    4 weeks ago


    india HARMAN International Full time

    HARMAN’s engineers and designers are creative, purposeful and agile. As part of this team, you’ll combine your technical expertise with innovative ideas to help drive cutting-edge solutions in the car, enterprise and connected ecosystem. Every day, you will push the boundaries of creative design, and HARMAN is committed to providing you with the...

  • Senior Data Engineer

    1 month ago


    india SWAI TECHNOLOGIES PRIVATE LIMITED Full time

    Job Description : We are seeking a highly skilled Senior Data Engineer with 5 - 12 years of experience in building scalable data solutions. The ideal candidate should be proficient in Python programming and SQL, with extensive experience in utilizing various AWS services for data processing and storage. Experience with big data processing environments,...

  • Big Data Engineer

    1 month ago


    india Careerfit.ai Full time

    Responsibilities :Solution Design :- Design, develop, and maintain scalable big data solutions and data pipelines for efficient data ingestion, processing, storage, and analysis.- Collaborate with data scientists, analysts, and business stakeholders to identify data requirements, use cases, and business objectives.- Translate business needs into technical...

  • Big Data Architect

    3 weeks ago


    India Bloom Consulting Services Full time

    NA Experience **12 - 16 years** Offered Salary Notice Period **Not Disclosed** **Role: Big Data Architect** **Location: Bangalore** **Skills required**: Total 12 to 16 years of experience in BI, analytics, visualization, **data** integration, designing and developing **data **warehouses (ETL, **data**bases, front-end tools). 9+ years of...

  • Big Data Architect

    3 weeks ago


    India Bloom Consulting Services Full time

    Not Disclosed Experience **12 - 16 Years** Offered Salary **Lakh** Notice Period **Not Disclosed** **Role: Big Data Architect** **Location: Bangalore** **Skills required**: Total 12 to 16 years of experience in BI, analytics, visualization, **data** integration, designing and developing **data **warehouses (ETL, **data**bases, front-end...

  • Big Data Engineer

    1 month ago


    india ATech Full time

    Designation: BIG DATA ENGINEERJob Description:Your Role and Responsibilities:- Understand a data warehousing solution and able to work independently in such an environment- Responsible in Project development and delivery experience of a few good size projects- Design, build, optimize and support new and existing data models and ETL processes based on our...

  • Big Data Analyst

    1 month ago


    india Royal Cyber ​​Inc. Full time

    1) 5+ years of experience managing and deploying CDP or HDP clusters.2) Experience configuring and setup of big data environments.3) Experience with Kerberos security and LDAP setup.4) Experience in setting up monitoring and alerting pipelines for big data clusters.


  • Bangalore/Anywhere in India/Multiple Locations, IN Leadway Resource Full time

    We are seeking a highly skilled and experienced Big Data QA Engineer to join our team. You will play a critical role in ensuring the quality of our big data platform and data solutions. You will leverage your expertise in Big Data testing methodologies, automation frameworks, and scripting languages to design, develop, and execute comprehensive test...


  • Bangalore,Anywhere in India,Multiple Locations Leadway Resource Full time

    We are seeking a highly skilled and experienced Big Data QA Engineer to join our team. You will play a critical role in ensuring the quality of our big data platform and data solutions. You will leverage your expertise in Big Data testing methodologies, automation frameworks, and scripting languages to design, develop, and execute comprehensive test...

  • Big Data Developer

    3 weeks ago


    india RiskInsight Consulting Pvt Ltd Full time

    We are seeking a skilled Data Engineer with 4-8 years of experience, proficient in Big Data technologies and data processing frameworks. The ideal candidate will have strong expertise in Hadoop ecosystem components such as HDFS, HiveQL, and experience in developing and optimizing data processing workflows using PySpark. Proficiency in SQL for data...


  • india Digihelic Solutions Pvt. Ltd. Full time

    Job Summary :We are seeking a highly skilled and motivated Product Solution Architect with extensive expertise in designing and implementing cutting-edge big-data solutions on the Amazon Web Services (AWS) platform.As a key member of our dynamic team, you will play a pivotal role in driving the success of our data-driven initiatives, leveraging AWS services...


  • Anywhere in India/Multiple Locations, IN Grownex HR Solution Full time

    Role : Lead Big Data EngineerWork Mode : Remote(5 Days working)The Opportunity - Development team is establishing a new offshore product development team for its partner , that is building next-generation Big Data, Cloud-Based Business Operation Support technology for utilities, retail energy suppliers and Community Choice Aggregators (CCA). The candidate...


  • Anywhere in India,Multiple Locations Grownex HR Solution Full time

    Role : Lead Big Data Engineer Work Mode : Remote(5 Days working) The Opportunity - Development team is establishing a new offshore product development team for its partner , that is building next-generation Big Data, Cloud-Based Business Operation Support technology for utilities, retail energy suppliers and Community Choice Aggregators (CCA). The...

  • Big Data Engineer

    4 weeks ago


    Bangalore/Anywhere in India/Multiple Locations, IN Connectio IT Pvt Ltd Full time

    Key Skills : Java, Apache Spark, Apache Kafka, Hadoop, AWSJob Description :- Designing, installing, testing, and maintaining scalable data management systems- Ensuring systems meet business requirements and industry standards- Creating custom software components and analytics applications- Research data acquisition opportunities and new uses for existing...

  • Big Data Engineer

    1 month ago


    Bangalore,Anywhere in India,Multiple Locations Connectio IT Pvt Ltd Full time

    Key Skills : Java, Apache Spark, Apache Kafka, Hadoop, AWS Job Description : - Designing, installing, testing, and maintaining scalable data management systems- Ensuring systems meet business requirements and industry standards- Creating custom software components and analytics applications- Research data acquisition opportunities and new uses for existing...


  • india LevelUP HCS Full time

    AWS Data Engineer- Kafka We are looking for a Data Engineer (AWS Data Platform) to join our busy and dynamic business intelligence – data platform team. If you are a passionate Data Engineer you have the chance to be part of a team that looks to build out our next-generation data platform, leveraging new technologies and techniques to maximize the...