Big data Developer

3 weeks ago

Hyderabad, Telangana, India State Street Full time

Job Title

Technical Manager - Big data technologies

Role Summary &Role Description

Technical Manager with specific Hadoop stack - Impala , Hive, HBase, Spark, Sqoop ,Java, Oracle, and Micro services skills for trading application development. Analyzes, designs, develops and maintains software applications to support business units.

Requires knowledge of Hadoop stack - Impala , Hive, HBase, Spark, Sqoop , Scala, Core Java, Micro services , Kubernetes, REST, Test Driven Design, Oracle,

Expected to spend 80% of the time on hands-on development, design and architecture and remaining 20% on guiding the team on technology and removing other impediments Capital Markets Projects experience preferred Provides advanced technical expertise in analyzing, designing, estimating, and developing software applications to project schedule. Oversees systems design and implementation of most complex design components. Creates project plans and deliverables and monitors task deadlines. Oversees, maintains and supports existing software applications. Provides subject matter expertise in reviewing, analyzing, and resolving complex issues. Designs and executes end to end system tests of new installations and/or software prior to release to minimize failures and impact to business and end users. Responsible for resolution, communication, and escalation of critical technical issues. Prepares user and systems documentation as needed. Identifies and recommends Industry best practices. Serves as a mentor to junior staff.

  • Acts as a technical lead/mentor for developers in day to day and overall project areas.
  • Ability to lead a team of agile developers.
  • Worked in a complex deadline driven projects with minimal supervision. Ability to architect/design/develop with minimum requirements by effectively coordinating activities between business analysts, scrum leads, developers and managers.
  • Ability to provide agile status notes on day to day project tasks.

Technical Skills

  • 5+ year of experience with Hadoop stack - Impala , Hive, HBase, Spark, Sqoop
  • 2+ experience on Apache Kafka streaming processing and Python scripting.
  • Exposure and good understanding in Big data and Data lake technologies
  • Experienced, highly skilled in the following technologies as an architect/design lead and developer level. Core Java, Tomcat or another J2EE application server, Oracle and Scripting languages and other open source technologies etc..
  • End to end understanding of software architecture, design, development and implementation.
  • Strong Practical experience using Scrum, Agile modelling and adaptive software development.
  • Continuous integration and build process, test automation and deployment experience
  • Ability to understand and grasp the big picture of system components.
  • Experience building environment and architecture and design guides and architecture and application blueprints.
  • Nice to have knowledge on Public clouds and Data bricks etc..

Core/Must have skills

Hadoop stack - Impala , Hive, HBase, Spark, Sqoop , Scala, Core Java, Micro services , Kubernetes, REST, Test Driven Design, Oracle

Good to have skills

Apache Kafka, public cloud and Data bricks

Work Schedule

12 PM IST to 9 PM IST

Keywords (If any)

Impala , Hive, Spark, Sqoop, Scala

Why this role is important to us

Our technology function, Global Technology Services (GTS), is vital to State Street and is the key enabler for our business to deliver data and insights to our clients. We're driving the company's digital transformation and expanding business capabilities using industry best practices and advanced technologies such as cloud, artificial intelligence and robotics process automation.

We offer a collaborative environment where technology skills and innovation are valued in a global organization. We're looking for top technical talent to join our team and deliver creative technology solutions that help us become an end-to-end, next-generation financial services company.

Join us if you want to grow your technical skills, solve real problems and make your mark on our industry.

About State Street

What we do. State Street is one of the largest custodian banks, asset managers and asset intelligence companies in the world. From technology to product innovation, we're making our mark on the financial services industry. For more than two centuries, we've been helping our clients safeguard and steward the investments of millions of people. We provide investment servicing, data & analytics, investment research & trading and investment management to institutional clients.

Work, Live and Grow. We make all efforts to create a great work environment. Our benefits packages are competitive and comprehensive. Details vary by location, but you may expect generous medical care, insurance and savings plans, among other perks. You'll have access to flexible Work Programs to help you match your needs. And our wealth of development programs and educational support will help you reach your full potential.

Inclusion, Diversity and Social Responsibility. We truly believe our employees' diverse backgrounds, experiences and perspectives are a powerful contributor to creating an inclusive environment where everyone can thrive and reach their maximum potential while adding value to both our organization and our clients. We warmly welcome candidates of diverse origin, background, ability, age, sexual orientation, gender identity and personality. Another fundamental value at State Street is active engagement with our communities around the world, both as a partner and a leader. You will have tools to help balance your professional and personal life, paid volunteer days, matching gift programs and access to employee networks that help you stay connected to what matters to you.

State Street is an equal opportunity and affirmative action employer.

Discover more at

  • Procal - Big Data Engineer - Scala/Python

    2 weeks ago

    Hyderabad, Telangana, India procal tech Full time

    Job Title : Big Data EngineerNo. of Positions : 1Experience : 7-15 YearsJob Type : Full-timeJob Overview :The Sr. Big Data Engineer is a pivotal role in our team, responsible for designing and implementing scalable processing and storage solutions for large datasets. This position requires proficiency in big data technologies, containerization, cloud...

  • AppSierra Solutions

    3 weeks ago

    Hyderabad, Telangana, India AppSierra Solutions Pvt Ltd Full time

    Job description :Essential Duties and Responsibilities :- Build and maintain real-time and batch data pipelines across the advanced analytics platform.- Design, develop and orchestrate highly robust and scalable ETL pipelines.- Design and implement Dimensional and NoSQL data modelling as per the business requirements.- Develop highly optimal codebase and...

  • Big Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India RENOIRCE TECHNOLOGIES PRIVATE LIMITED Full time

    Job Description :Primary & mandatory : PYSPARKSecondary : GCP- At least 5 years of experience in Big Data, Hadoop Data Platform (HDP) and ETL and capable of configuring data pipelines.- Possess the following technical skills - SQL, Python, Pyspark, Hive, Unix, ETL, Control-M (or similar)- At least 2 years of experience in Cloudera Data Platform (CDP)- Skills...

  • Lead/Software Development Engineer

    2 weeks ago

    Hyderabad, Telangana, India MarketScope Full time

    Key Skills : - Spark Core/Streaming with Any programming language Scala/ Java/ Python- Good Understanding of Data Structures, Algorithms, Logical and problem solving skills- Data Transformation, Data Ingestion, Optimization mechanism/techniques- Good understanding of Big Data (Hadoop, MapReduce, Kafka, Cassandra) Technologies, AWS, Postgresql.Education :...

  • Engineering Manager

    2 weeks ago

    Hyderabad, Telangana, India Questsphere Private ltd. Full time

    Job Description :Skills Required : - Experience in project management and Agile/Scrum product development life cycles- Work with product management, design, support and other dev teams to create high-quality solutions- Manage best-of-class engineering teamRequirement (To be successful in this role you have) :- 10+ years of experience with technologies...

  • Data Science Engineer

    6 days ago

    Hyderabad, Telangana, India JMJ CONSULTANTS Full time

    Our client is and R&D Services company head-quarterd in USA having presence in seven countries in the world. Our client is looking for a Data Scientist for Hyderabad location.Overview :As a Data Science Engineer, you will be an integral part of our cutting-edge technology team, responsible for designing, implementing, and maintaining data pipelines and...

  • AIFALABS - Python Developer - ETL/Data Warehousing

    3 weeks ago

    Hyderabad, Telangana, India AIFALABS Full time

    Python DeveloperAIFA India is Seeking a Full-Time Strong Python Developer (Onsite)Are you fascinated by the power of data to unlock insights and drive innovation? Do you excel in Python development, data analysis, and engineering solutions? If harnessing data to propel AI and machine learning projects forward excites you, AIFA India invites you to join our...

  • Data Engineer

    3 weeks ago

    Hyderabad, Telangana, India Black Turtle Full time

    A highly reputed MNC is looking for DATA ENGINEERJob Description : - As a Data Architect, you will be the key technical expert overseeing Co's data product build & operations and drive a strong vision for how data engineering can proactively create a positive impact on the business.- As a member of the data engineering team, you will help lead the...

  • Data Engineer

    4 weeks ago

    Hyderabad, Telangana, India Swiss Re Full time

    About the Role:As a Data Engineer, you will be responsible for designing and implementing complex data pipelines and analytics solutions to support key decision-making business processes in our Property & Casualty business domain. You will gain exposure to a project that is leveraging cutting edge technology that applies Big Data and Machine Learning to...

  • Vassar Labs

    3 weeks ago

    Hyderabad, Telangana, India Vassar Labs IT Solutions Private Limited Full time

    Job Title : Data ModelerVassar Labs IT Solutions Private Limited is a technology company focusing on delivering solutions into Primary sectors like Water, Agriculture, Smart City and Education by using emerging technologies like IoT, GIS, Telemetry Sensors, Crowdsourcing, Big Data, Machine Learning, Satellite Data, Drone Data etc. We provide real-time...

  • Senior Software Engineer

    3 weeks ago

    Hyderabad, Telangana, India Whitefield Network Full time

    Job Description :- In this role, you will play a pivotal role in building and maintaining our data validation platforms, ensuring they operate at scale.- You'll bridge the gap between the software engineering team and users, optimizing software usage and maximizing benefits.- This role offers the exciting opportunity to work with cutting-edge...

  • Centific - Digital Technology Expert - Data Science/Artificial Intelligence

    3 weeks ago

    Hyderabad, Telangana, India Centific Global Technologies Full time

    Excellent Opportunity @ Centific - COE/DAC/Data Science/AI/MLRole Description :The Digital Technology Expert is a senior-level role responsible for driving digital transformation, innovation, and technological excellence within the organization. This individual will lead and execute strategic initiatives related to enterprise software product R&D, Data...

  • Azure Data Engineer

    1 week ago

    Hyderabad, Telangana, India Insightsquared Full time

    Responsibilities :- Design and develop business-critical backend systems using high-quality data pipelines.- Assemble large, complex data sets that meet functional / non-functional business requirements.- Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Azure databricks...

  • Expert Data Engineer

    3 weeks ago

    Hyderabad, Telangana, India Swiss Re Full time

    About the Role:As an Expert Data Engineer, you will be responsible for leading the design and implementation of complex data pipelines and analytics solutions to support key decision-making business processes in our Reinsurance Client Solutions business domain. You will lead the development of strategic use cases within our Data Integration and Analytics...

  • Senior Data Engineer

    4 weeks ago

    Hyderabad, Telangana, India Swiss Re Full time

    About the Role:As a Senior Data Engineer, you will be responsible for designing and implementing complex data pipelines and analytics solutions to support key decision-making business processes in our Property & Casualty business domain. You will gain exposure to a project that is leveraging cutting edge technology that applies Big Data and Machine Learning...

  • Big Data Engineer

    3 days ago

    Hyderabad, Telangana, India Programming.com Full time

    Required technical skills:PySpark, Hive, HDFS, Sqoop, Shell scripting, Kafka, SQL, UNIX, Linux,AWS S3, TWS, Airflow, Bit - Monitor EOD jobs, Kafka notifications and application status - Rerun any failure jobs (TWS/Airflow/Jobserver) - Alert respective stakeholders about delay or issues - Conduct RCA for incidents received for failures or from users - Liaise...

  • Sales Manager

    4 weeks ago

    Hyderabad, Telangana, India Idesign Data Solutions Full time

    Job Requirements Job Title: Sales ManagerCompany Name: iDesign Data Solutions Pvt LtdLocation: Jubilee Hills, Hyderabad, TelanganaSalary: Competitive CompensationQualification: GraduationJob Description: Join Our Team as a Sales ManagerAre you a seasoned sales professional with a track record of building strong client relationships and achieving sales...

  • CarbyneTech - Data Science Engineer - Machine/Deep Learning Models - R/Python/Scala

    14 hours ago

    Hyderabad, Telangana, India CarbyneTech India Pvt Ltd Full time

    Job Requirements :- Experience in research and development. Conference papers at reputed AI/ML platforms (ICML, NIPS, AAAI) will be a big plus,- Experience in Time series modeling (classical methods like ARIMA, STL Forecast). Understanding and implementing active research in Time series forecasting (Bayesian Forecasting, Hierarchical Forecasting),- Hands-on...

  • Data Analyst, ABCS Analytics

    3 days ago

    Hyderabad, Telangana, India Amazon Full time

    DescriptionAmazon Business is dedicated to serving the needs of business customers and we are focused on building the largest and most innovative Business-to-Business (B2B) marketplace in the world. We are recruiting a handful of the best and brightest to make this vision a reality.Amazon Business represents an incredible opportunity to address a vast new...

  • Quality & Performance Specialist

    1 day ago

    Hyderabad, Telangana, India NTT DATA Services Full time

    ETL Tester Req. Experience: 7+ YearsProficient enough in SQL to interrogate databases to test the ETL on Microsoft SQL Server. Previous experience of testing large scale BI solutions integrating with multiple source and target systems. Understand the mappings of source systems to data models in the target data warehouse schema and analytical cubes. Previous...