Big Data Lead Developer

2 days ago

Hyderabad, Telangana, India HARP Technologies and Services Full time

Experience : 8+ years

Location : Hyderabad ( Hybrid)

Notice Period : Immediate to 3 days candidates preferred

Mandate Skills : Big Data, Hadoop, Aws, Pyspark, ETL, SQL

Job Description :


- Assist in designing and implementing scalable and robust processes for ingesting and transforming complex datasets.


- Designs, develops, constructs, maintains and supports data pipelines for ETL from a multitude of sources.


- Creates blueprints for data management systems to centralize, protect, and maintain data sources.


- Focused on data stewardship and curation, the data engineer enables the data scientist to run their models and analyses to achieve the desired business outcomes


- Ingest large, complex data sets that meet functional and non-functional requirements.


- Enable the business to solve the problem of working with large volumes of data in diverse formats, and in doing so, enable innovative solutions.


- Design and build bulk and delta data lift patterns for optimal extraction, transformation, and loading of data.


- Supports the organisation's cloud strategy and aligns to the data achitecture and governance including the implementation of these data governance practices.


- Engineer data in the appropriate formats for downstream customers, risk and product analytics or enterprise applications.


- Assist in identifying, designing and implementing robust process improvement activities to drive efficiency and automation for greater scalability. This includes looking at new solutions and new ways of working and being on the forefront of emerging technologies.


- Work with various stakeholders across the organization to understand data requirements and apply technical knowledge of data management to solve key business problems.


- Provide support in the operational environment with all relevant support teams for data services.


- Provide input into the management of demand across the various data streams and use cases.


- Create and maintain functional requirements and system specifications in support of data architecture and detailed design specifications for current and future designs.


- Support test and deployment of new services and features.


- Provides technical leadership to junior data engineers in the team


Total Experience :


- 8+ years of data engineering experience


- 6+ years of experience with any data warehouse technical architectures, ETL/ELT, and reporting/analytics tools including , but not limited to , any of the following combinations :


(1) SSIS ,SSRS or something similar


(2) ETL Frameworks


(3) Spark


(4) AWS data builds


- The candidate having DBA ability and knowledge across at least 2 platforms (example: TSQL, SAS, PSQL, IBM VSAM, DynamoDB and DB2) will also be beneficial.


- Should be at least at a proficient level in at least one of Python or Java


- Should be Proficient in bash SQL


- Some experience with R, AWS, XML, json, cron will be beneficial


- Experience with designing and implementing Cloud (AWS) solutions including use of APIs available.


- Some experience with Dev/OPS architecture, implementation and operation would be advantageous.


- Knowledge of Engineering and Operational Excellence using standard methodologies.


- Best practices in software engineering, data management, data storage, data computing and distributed systems to solve business problems with data.


- Some experience in applying SAFe/Scrum/Kanban methodologies.


- Knowledge and understanding of business process management lifecycle which covers the design, modelling, execution, monitoring, and optimization as well as business process re-engineering.


- Good problem solving skills: The ability to exercise judgment in solving technical, operational, and organizational challenges, to identify issues proactively, to present solutions and options leading to resolution

(ref:hirist.tech)

  • Big Data Engineer, Data Modeling

    2 weeks ago

    Hyderabad, Telangana, India data Full time

    What can you tell your friends when they ask you what you do? We're looking for an experienced Big Data Engineer who can create innovative new products in the analytics and data space. You will participate in the development that creates the world's #1 mobile app analytics service. Together with the team, you will build out new product features and...

  • Big Data Engineer, Data Modeling

    2 weeks ago

    Hyderabad, Telangana, India data Full time

    What can you tell your friendswhen they ask you what you do?We're looking for an experienced Big Data Engineer who can create innovative new products in the analytics and data space. You will participate in the development that creates the world's #1 mobile app analytics service. Together with the team, you will build out new product features and...

  • Big Data Engineer, Data Analysis

    2 weeks ago

    Hyderabad, Telangana, India data Full time

    What can you tell your friends when they ask you what you do? "As a Data Analyst at , I help analyze, monitor, and deliver comprehensive and high-quality app market data to app professionals. I am empowered to work independently with the support of team members spread across APAC, AMER, and EMEA. I am the key contributor to the new and existing product...

  • Big Data Developer

    2 weeks ago

    Hyderabad, Telangana, India Virtusa Full time

    Big Data Developer - CREQ189736 Description Job Description:5+ years of hands-on experience in designing, building and supporting Data Applications using Spark, Sqoop and HiveBachelors or masters degree in Computer Science or related fieldStrong knowledge of working with large data sets and high-capacity big data processing platformStrong experience in Unix...

  • Big Data Developer

    2 weeks ago

    Hyderabad, Telangana, India Virtusa Full time

    Big Data Developer - CREQ188135 Description Job Description: 5+ years of hands-on experience in designing, building and supporting Data Applications using Spark, Sqoop and Hive Bachelors or masters degree in Computer Science or related field Strong knowledge of working with large data sets and high-capacity big data processing platform Strong experience in...

  • Big data Developer

    2 weeks ago

    Hyderabad, Telangana, India State Street Full time

    Job TitleTechnical Manager - Big data technologiesRole Summary &Role DescriptionTechnical Manager with specific Hadoop stack - Impala , Hive, HBase, Spark, Sqoop ,Java, Oracle, and Micro services skills for trading application development. Analyzes, designs, develops and maintains software applications to support business units.Requires knowledge of Hadoop...

  • Opening for Big data Developer

    2 weeks ago

    Hyderabad, Telangana, India @ Six Sigma Full time

    Job DetailsRequirement Type Permanent Job Title Opening for Big data Developer Job Level Executive - Non-Managerial Job Description Big data Developer No. of Openings 6 Job Domain IT Experience - Minimum - Maximum data Expected Date Of Joining Joining Time 5 Days Salary 54,167 Work Locations HyderabadRequired skillsBig data

  • Scala Developer

    2 months ago

    Hyderabad, Telangana, India Trinity Infotech INC Full time

    Job Description :- 5+ years of experience working in a software product development organization building modern and scalable big data applications.- Expertise with Scala functional programming (intermediate/advanced level is a mandate).- Data structures and algorithms in Scala.- Fluency in Hadoop technologies around handling big data processing e.g. Spark.-...

  • Scala Developer

    4 weeks ago

    Hyderabad, Telangana, India Trinity Infotech INC Full time

    Job Description :- 5+ years of experience working in a software product development organization building modern and scalable big data applications.- Expertise with Scala functional programming (intermediate/advanced level is a mandate).- Data structures and algorithms in Scala.- Fluency in Hadoop technologies around handling big data processing e.g. Spark.-...

  • Big data Developer

    2 weeks ago

    Hyderabad, Telangana, India State Street Full time

    Job Title Technical Manager - Big data technologies Role Summary &Role Description Technical Manager with specific Hadoop stack - Impala , Hive, HBase, Spark, Sqoop ,Java, Oracle, and Micro services skills for trading application development. Analyzes, designs, develops and maintains software applications to support business units. Requires...

  • Big data engineer

    2 weeks ago

    Hyderabad, Telangana, India Virtusa Full time

    Big data engineer - CREQ189307 Description Position: Big data engineer Primary skills: Big data, aws Location: HYD Create Trigger based automation framework for Data Migration Identify roles/access needed for data migration from federated bucket to managed bucket and Build APIs for the same Integrate CDMS framework with Lake and Data bridge API Data...

  • Big Data Developer

    2 weeks ago

    Hyderabad, Telangana, India Corporate Resources Full time

    Job Description Responsibilities ::: We are looking for a Sr. Big Data Developer for large-scale, high-speed, and low-latency data solutions in the areas of: Large scale data manipulation / ETL . Mentor junior developers in SDLC process. Long-term data storage / data warehousing. Low-latency retrieval systems, i.e. real-time reporting and analytics...

  • Big data spark

    2 weeks ago

    Hyderabad, Telangana, India Virtusa Full time

    Big data spark - CREQ178115 Description Position: Big data spark developer Primary skills: Big data spark & Java Location: Hyderabad & Bangalore Essential:5+ years of IT experience in Bigdata tech stack (Java Spark, Hadoop, Hive)Experience in writing SQL for any one relation databaseExtensive knowledge in HiveSQL Hadoop, SQLExperience in DevOps...

  • Big data spark developer

    2 weeks ago

    Hyderabad, Telangana, India Virtusa Full time

    Big data spark developer - CREQ180562 Description Essential:5 years of IT experience in Bigdata tech stack (Java Spark, Hadoop, Hive)Experience in writing SQL for any one relation databaseExtensive knowledge in HiveSQL Hadoop, SQLExperience in DevOps process (CICD, Jira, Bitbucket)Primary Location Hyderabad, Andhra Pradesh, India Job Type Experienced Primary...

  • Big data spark developer

    2 weeks ago

    Hyderabad, Telangana, India Virtusa Full time

    Big data spark developer - CREQ180562 Description Essential:5 years of IT experience in Bigdata tech stack (Java Spark, Hadoop, Hive)Experience in writing SQL for any one relation databaseExtensive knowledge in HiveSQL Hadoop, SQLExperience in DevOps process (CICD, Jira, Bitbucket) Primary Location Hyderabad, Andhra Pradesh, India Job Type Experienced...

  • Big data spark

    2 weeks ago

    Hyderabad, Telangana, India Virtusa Full time

    Big data spark - CREQ178115 DescriptionPosition: Big data spark developerPrimary skills: Big data spark & JavaLocation: Hyderabad & BangaloreEssential:5+ years of IT experience in Bigdata tech stack (Java Spark, Hadoop, Hive)Experience in writing SQL for any one relation databaseExtensive knowledge in HiveSQL Hadoop, SQLExperience in DevOps process (CICD,...

  • Big Data Faculty- China

    2 weeks ago

    Hyderabad, Telangana, India NIIT Limited Full time

    About The Company:NIIT is a leading Skills and Talent Development Corporation that is building a manpower pool for global industry requirements. The company, which was set up in 1981 to help the nascent IT industry overcome its human resource challenges, today ranks among the world's leading training companies owing to its vast and comprehensive array of...

  • Big Data with Data Modeling

    2 weeks ago

    Hyderabad, Telangana, India Virtusa Full time

    Big Data with Data Modeling CREQ186896 Description EIS DATA JD Tier 2Cloud Data Lead with very strong in Experience in Bigdata (Kafka, Elastic search,Logstash,Kibana)Required experience with Data Engineering and ETL/EDW Design Process and Practices.Experience in Data ModelingExperience in CDC process (Added Advantage if knowledge in Debezium or any CDC...

  • Java Developer

    2 months ago

    Hyderabad/Telangana, Telangana, India RiskInsight Consulting Pvt Ltd Full time

    Responsibilities :- Develop and implement efficient Java code to optimize and enhance functionalities within our Trino (or Presto) platform.- Demonstrate excellent understanding of Java programming language and object-oriented programming (OOP) principles.- Possess strong code comprehension skills to effectively read and analyze open-source code like Trino.-...

  • Java Developer

    4 weeks ago

    Hyderabad/Telangana, Telangana, India RiskInsight Consulting Pvt Ltd Full time

    Responsibilities :- Develop and implement efficient Java code to optimize and enhance functionalities within our Trino (or Presto) platform.- Demonstrate excellent understanding of Java programming language and object-oriented programming (OOP) principles.- Possess strong code comprehension skills to effectively read and analyze open-source code like Trino.-...