Big data engineer
2 days ago
Identify roles/access needed for data migration from federated bucket to managed bucket and Build APIs for the same
Integrate CDMS framework with Lake and Data bridge API
Data migration from S3 Managed to Hadoop On prem
Jobs for Daily and Bulk loads
Test support for AVRO to test lake features
Test support for Compression types like LZO, .ENC to test lake features
ABINITIO integration: Build feature to create operation trigger for ABI pipeline
Movement to new datacenter -SQL server migration
Carlstadt to Ashburn (DR switchover)
Develop and maintain data platforms using Python.
Work with AWS and Big Data, design and implement data pipelines, and ensure data quality and integrity.
Collaborate with cross functional teams to understand data requirements and design solutions that meet business needs .
Implement and manage agents for monitoring, logging, and automation within AWS environments.
Handling migration from PySpark to AWS.
(Secondary) Resource must have hands on development experience with various Ab Initio components such as Rollup Scan join Partition by key Partition by Round Robin Gather Merge Interleave Lookup etc.
Must have experience with SQL database programming SQL performance tuning relational model analysis.
Good knowledge in developing UNIX scripts Oracle SQLPLSQL.
Leverage internal tools and SDKs, utilize AWS services such as S3, Athena, and Glue, and integrate with our internal Archival Service Platform for efficient data purging.
Lead the integration efforts with the internal Archival Service Platform for seamless data purging and lifecycle management.
Collaborate with the data engineering team to continuously improve data integration pipelines, ensuring adaptability to evolving business needs. Primary Location Hyderabad, Andhra Pradesh, India Job Type Experienced Primary Skills Big Data, Python, Spark Years of Experience 12 Qualification Education: Any degree or equivalent Experience: 6+ years Travel No
-
Big Data Engineer, Data Modeling
2 weeks ago
Hyderabad, Telangana, India data Full timeWhat can you tell your friendswhen they ask you what you do?We're looking for an experienced Big Data Engineer who can create innovative new products in the analytics and data space. You will participate in the development that creates the world's #1 mobile app analytics service. Together with the team, you will build out new product features and...
-
Big Data Engineer, Data Modeling
2 weeks ago
Hyderabad, Telangana, India data Full timeWhat can you tell your friends when they ask you what you do? We're looking for an experienced Big Data Engineer who can create innovative new products in the analytics and data space. You will participate in the development that creates the world's #1 mobile app analytics service. Together with the team, you will build out new product features and...
-
Big Data Engineer, Data Analysis
1 week ago
Hyderabad, Telangana, India data Full timeWhat can you tell your friends when they ask you what you do? "As a Data Analyst at , I help analyze, monitor, and deliver comprehensive and high-quality app market data to app professionals. I am empowered to work independently with the support of team members spread across APAC, AMER, and EMEA. I am the key contributor to the new and existing product...
-
Big data engineer
1 month ago
Hyderabad, India Virtusa Full timeBig data engineer - CREQ189307 Description Position: Big data engineer Primary skills: Big data, aws Location: HYD Create Trigger based automation framework for Data Migration Identify roles/access needed for data migration from federated bucket to managed bucket and Build APIs for the same Integrate CDMS framework with Lake and Data bridge API Data...
-
Big data engineer
4 weeks ago
Hyderabad, India Virtusa Full timeBig data engineer - CREQ189307 Description Position: Big data engineer Primary skills: Big data, aws Location: HYD Create Trigger based automation framework for Data Migration Identify roles/access needed for data migration from federated bucket to managed bucket and Build APIs for the same Integrate CDMS framework with Lake and Data bridge API Data...
-
Big data engineer
1 week ago
Hyderabad, Telangana, India Virtusa Full timeBig data engineer - CREQ189307 Description Position: Big data engineer Primary skills: Big data, aws Location: HYD Create Trigger based automation framework for Data Migration Identify roles/access needed for data migration from federated bucket to managed bucket and Build APIs for the same Integrate CDMS framework with Lake and Data bridge API Data...
-
Big data engineer
3 days ago
Hyderabad, India Virtusa Full timeBig data engineer - CREQ189284 Description Position: Big data engineer Primary: big data concepts, aws, pyspark Location: HYD Create Trigger based automation framework for Data Migration Identify roles/access needed for data migration from federated bucket to managed bucket and Build APIs for the same Integrate CDMS framework with Lake and Data bridge...
-
Big Data Engineer
2 weeks ago
Hyderabad, Telangana, India TechSophy Full timeJob Description: Roles&Responsibilities : 1. As an Engineer, you are responsible to analyze large data sets to identify errors in the map.2. As an Engineer, you are responsible to design and implement a complex algorithm for resolving the issue, review the solution with a team of engineers and analysts, and integrate the resulting solution into the...
-
Big Data Engineer
2 weeks ago
Hyderabad, Telangana, India Cielo Talent Full timeClient: IBM, This represents a direct and enduring opportunity within IBM, with Cielo as the primary recruitment partner.Title: Big Data EngineerLocation: Hyderabad (Hybrid)Experience: 5 to 10 YearsMust have skills: Big data, Hive, Hadoop, Scala, Spark and Py SparkRequired Technical and Professional Expertise:First and most important – Sound understanding...
-
Big Data Engineer
1 week ago
Hyderabad, Telangana, India TechSophy Full timeJob Description:Roles&Responsibilities :1. As an Engineer, you are responsible to analyze large data sets to identify errors in the map.2. As an Engineer, you are responsible to design and implement a complex algorithm for resolving the issue, review the solution with a team of engineers and analysts, and integrate the resulting solution into the data...
-
Big Data Engineer
3 months ago
Hyderabad, India TechSophy Full timeJob Description: Roles&Responsibilities : 1. As an Engineer, you are responsible to analyze large data sets to identify errors in the map. 2. As an Engineer, you are responsible to design and implement a complex algorithm for resolving the issue, review the solution with a team of engineers and analysts, and integrate the resulting solution into...
-
Big Data Engineer
3 months ago
hyderabad, India TechSophy Full timeJob Description: Roles&Responsibilities : 1. As an Engineer, you are responsible to analyze large data sets to identify errors in the map.2. As an Engineer, you are responsible to design and implement a complex algorithm for resolving the issue, review the solution with a team of engineers and analysts, and integrate the resulting solution into...
-
Big Data Engineer
3 weeks ago
hyderabad, India TechSophy Full timeJob Description: Roles&Responsibilities : 1. As an Engineer, you are responsible to analyze large data sets to identify errors in the map.2. As an Engineer, you are responsible to design and implement a complex algorithm for resolving the issue, review the solution with a team of engineers and analysts, and integrate the resulting solution into...
-
Big Data Engineer
3 weeks ago
Hyderabad, India TechSophy Full timeJob Description: Roles&Responsibilities : 1. As an Engineer, you are responsible to analyze large data sets to identify errors in the map. 2. As an Engineer, you are responsible to design and implement a complex algorithm for resolving the issue, review the solution with a team of engineers and analysts, and integrate the resulting solution into...
-
AWS Big Data Engineer
6 days ago
Hyderabad, India Tiger Analytics Full timeJob DescriptionAbout the role:We are looking for a Senior Data Engineer to be based out of our Chennai, Bangalore & Hyderabad offices. This role involves a combination of hands-on contribution, customer engagement, and technical team management. As a Senior Data Engineer, you will Design and build solutions for near real-time stream processing as well as...
-
AWS Big Data Engineer
2 days ago
Hyderabad, India Tiger Analytics Full timeJob DescriptionWe are looking for a Senior Data Engineer to be based out of our Chennai, Bangalore & Hyderabad offices. This role involves a combination of hands-on contribution, customer engagement, and technical team management. As a Senior Data Engineer, you will Design and build solutions for near real-time stream processing as well as batch processing...
-
Big Data Engineer
2 weeks ago
Hyderabad, India Cielo Talent Full timeClient: IBM, This represents a direct and enduring opportunity within IBM, with Cielo as the primary recruitment partner.Title: Big Data EngineerLocation: Hyderabad (Hybrid)Experience: 5 to 10 YearsMust have skills: Big data, Hive, Hadoop, Scala, Spark and PySparkRequired Technical and Professional Expertise:First and most important – Sound understanding...
-
Big Data Engineer
2 weeks ago
hyderabad, India Cielo Talent Full timeClient: IBM, This represents a direct and enduring opportunity within IBM, with Cielo as the primary recruitment partner. Title: Big Data Engineer Location: Hyderabad (Hybrid) Experience: 5 to 10 Years Must have skills: Big data, Hive, Hadoop, Scala, Spark and PySpark Required Technical and Professional Expertise: First and most important – Sound...
-
Big Data Engineer
2 days ago
Hyderabad, India Cielo Talent Full timeClient: IBM, This represents a direct and enduring opportunity within IBM, with Cielo as the primary recruitment partner.Title: Big Data EngineerLocation: Hyderabad (Hybrid)Experience: 5 to 10 YearsMust have skills: Big data, Scala(Mandatory), Hive, Hadoop, Spark and PySparkRequired Technical and Professional Expertise:First and most important – Sound...
-
Big Data Engineer
2 days ago
Hyderabad, India Cielo Talent Full timeClient: IBM, This represents a direct and enduring opportunity within IBM, with Cielo as the primary recruitment partner.Title: Big Data EngineerLocation: Hyderabad (Hybrid)Experience: 5 to 10 YearsMust have skills: Big data, Scala(Mandatory), Hive, Hadoop, Spark and PySparkRequired Technical and Professional Expertise:First and most important – Sound...
