Big Data- AWS
6 months ago
Job Description
Job description Role and responsibilities : - Creating Project Technical Documentation - Designing Solution architecture, and work on Data Ingestion, Preparation and Transformation. Debugging the production failures and identifying the solution. - Developing efficient frameworks for development and testing using (AWS Dynamo DB, EKS, Kafka) to enable seamless data ingestion process on to the Hadoop platform. - Enabling Data Governance and Data Discovery on Hadoop Platform - Building data processing framework using Spark, HQL - Exposure of Security Framework with Kerberos, Ranger, Atlas - Exposure of Data Pipeline Automation using DevOps tools - Exposure of Job Monitoring framework along validations automation - Exposure of handling structured, Un Structured and Streaming data Technical skills requirements : - Solid hands-on and Solution Architecting experience in Big-Data Technologies (AWS preferred) - Hands on experience in: AWS Dynamo DB, EKS, Kafka, Kinesis, Glue, EMR - Hands-on experience of programming language like Scala with Spark. - Good command and working experience on Hadoop Map Reduce, HDFS, Hive, HBase, and/or No-SQL Databases - Hands on working experience on any of the data engineering analytics platform (Hortonworks Cloudera MapR AWS), AWS preferred - Hands-on experience on Data Ingestion Apache Nifi, Apache Airflow, Sqoop, and Oozie - Hands on working experience of data processing at scale with event driven systems, message queues (Kafka FlinkSpark Streaming) - Hands on working Experience with AWS Services like EMR, Kinesis, S3, CloudFormation, Glue, API Gateway, Lake Foundation - Hands on working Experience with AWS Athena - Data Warehouse exposure on Apache Nifi, Apache Airflow, Kylo - Operationalization of ML models on AWS (e.g. deployment, scheduling, model monitoring etc.) - Feature Engineering Data Processing to be used for Model development - Experience gathering and processing raw data at scale (including writing scripts, web scraping, calling APIs, write SQL queries, etc.) - Experience building data pipelines for structured unstructured, real-time batch, events synchronous asynchronous using MQ, Kafka, Steam processing - Hands-on working experience in analysing source system data and data flows, working with structured and unstructured data - Must be very strong in writing SQL queries - Strengthen the Data engineering team with Big Data solutions - Strong technical, analytical, and problem-solving skills
Posted OnJob description Role and responsibilities : - Creating Project Technical Documentation - Designing Solution architecture, and work on Data Ingestion, Preparation and Transformation. Debugging the production failures and identifying the solution. - Developing efficient frameworks for development and testing using (AWS Dynamo DB, EKS, Kafka) to enable seamless data ingestion process on to the Hadoop platform. - Enabling Data Governance and Data Discovery on Hadoop Platform - Building data processing framework using Spark, HQL - Exposure of Security Framework with Kerberos, Ranger, Atlas - Exposure of Data Pipeline Automation using DevOps tools - Exposure of Job Monitoring framework along validations automation - Exposure of handling structured, Un Structured and Streaming data Technical skills requirements : - Solid hands-on and Solution Architecting experience in Big-Data Technologies (AWS preferred) - Hands on experience in: AWS Dynamo DB, EKS, Kafka, Kinesis, Glue, EMR - Hands-on experience of programming language like Scala with Spark. - Good command and working experience on Hadoop Map Reduce, HDFS, Hive, HBase, and/or No-SQL Databases - Hands on working experience on any of the data engineering analytics platform (Hortonworks Cloudera MapR AWS), AWS preferred - Hands-on experience on Data Ingestion Apache Nifi, Apache Airflow, Sqoop, and Oozie - Hands on working experience of data processing at scale with event driven systems, message queues (Kafka FlinkSpark Streaming) - Hands on working Experience with AWS Services like EMR, Kinesis, S3, CloudFormation, Glue, API Gateway, Lake Foundation - Hands on working Experience with AWS Athena - Data Warehouse exposure on Apache Nifi, Apache Airflow, Kylo - Operationalization of ML models on AWS (e.g. deployment, scheduling, model monitoring etc.) - Feature Engineering Data Processing to be used for Model development - Experience gathering and processing raw data at scale (including writing scripts, web scraping, calling APIs, write SQL queries, etc.) - Experience building data pipelines for structured unstructured, real-time batch, events synchronous asynchronous using MQ, Kafka, Steam processing - Hands-on working experience in analysing source system data and data flows, working with structured and unstructured data - Must be very strong in writing SQL queries - Strengthen the Data engineering team with Big Data solutions - Strong technical, analytical, and problem-solving skills
Skills RequiredAWS EMR
Posted On AWS EMR Roley
Posted Ony
LocationGreater Noida
Posted On Greater Noida-
Big Data Solutions Architect
2 weeks ago
Greater Noida, Uttar Pradesh, India Coforge Full timeCoforge is seeking a seasoned Big Data Solutions Architect to lead the design and implementation of large-scale data processing architectures on AWS. This role requires expertise in building efficient frameworks for development and testing, as well as hands-on experience with tools like Apache NiFi, Apache Airflow, Sqoop, and Oozie.Key...
-
Greater Noida, Uttar Pradesh, India Coforge Full timeAs a Senior Big Data Engineer at Coforge, you will play a pivotal role in designing and developing scalable solutions on the Amazon Web Services (AWS) platform. Your primary responsibility will be to architect and implement large-scale data processing systems that cater to our clients' ever-growing data needs.Candidates with hands-on experience in Big-Data...
-
Big Data Architect with Cloud Expertise
3 weeks ago
Greater Noida, Uttar Pradesh, India Coforge Full timeJob Description:Coforge is seeking a highly skilled Big Data Architect to lead our cloud-based data engineering efforts. In this role, you will design and implement scalable data architectures using AWS technologies such as Dynamo DB, EKS, and Kafka.You will work closely with our development teams to ensure seamless data ingestion, processing, and storage on...
-
Big Data Architect
7 months ago
Noida, India Skillety Technologies Full time**MapReduce.** **experience working with cloud or on-prem Big Data/MPP analytics platform (i.e. AWS Glue/EMR, Google BigQuery,** **Azure Data Warehouse, Databricks, Snowflake, Hortonworks/Cloudera Clusters)** **exp in writing Advanced SQLs** **exp with building dimensional data models to improve accessibility, efficiency and quality of data** **exp in custom...
-
Aws Expert
6 months ago
Greater Noida, India Coforge Ltd. Full time**Must Have**: 1. 5-6 years of experience in Multi Cloud environment (AWS is must) 2. Understand functional requirements and help Analyst for technical feasibility. 3. Design and implementation of AWS Data Analytics platform (EMR) 4. Should able to capture Technical requirements and design. Help team to implement. 5. Able to build PoC to show sample...
-
Big Data Developer
3 weeks ago
Greater Bengaluru Area, India Pixomania Full timeJob Title: Big Data Developer (Scala + Spark)Location: RemoteExperience: 5+ yearsJob Type: Contractual RemoteAbout Us:We are a forward-thinking technology company committed to delivering innovative solutions in Big Data analytics. Join our talented team as a Big Data Developer with expertise in Scala and Apache Spark to drive impactful projects and...
-
Big Data Developer
3 weeks ago
Greater Bengaluru Area, India Pixomania Full timeJob Title: Big Data Developer (Scala + Spark)Location: RemoteExperience: 5+ yearsJob Type: Contractual RemoteAbout Us:We are a forward-thinking technology company committed to delivering innovative solutions in Big Data analytics. Join our talented team as a Big Data Developer with expertise in Scala and Apache Spark to drive impactful projects and...
-
AWS Data Scientist
1 month ago
Bangalore/Pune/Greater Noida, India HyrEzy Talent Solutions Full timeJob Title : AWS Data Engineer (en)Location : VariousKey Responsibilities :Design, develop, and maintain large-scale data architectures on the AWS cloud platform.Develop and implement efficient data pipelines for ingesting, processing, and transforming massive data volumes from diverse sources.Design and optimize data models for efficient storage, retrieval,...
-
Aws Data Architect
6 months ago
Noida, India Skillety Technologies Full time**Overall 9+ years of exp in software industry** **5+ years of experience working with cloud or on-prem Big Data/MPP analytics platform (i.e. AWS Glue/EMR, Google** **BigQuery, Azure Data Warehouse, Databricks, Snowflake, Hortonworks/Cloudera Clusters)** **3+ years of exp in writing Advanced SQLs** **3+ years of exp with building dimensional data models to...
-
Big Data Developer
3 weeks ago
Greater Bengaluru Area, India Pixomania Full timeAs a Big Data Developer, you will design, develop, and optimize distributed data processing systems using Apache Spark and Hive. You will work on large-scale data platforms and ensure the efficient processing of massive datasets in real-time and batch environments. Your experience with Kafka will be an added advantage in building robust and scalable...
-
Big Data Developer
3 weeks ago
Greater Bengaluru Area, India Pixomania Full timeAs a Big Data Developer, you will design, develop, and optimize distributed data processing systems using Apache Spark and Hive. You will work on large-scale data platforms and ensure the efficient processing of massive datasets in real-time and batch environments. Your experience with Kafka will be an added advantage in building robust and scalable...
-
Big Data Technology Specialist
1 week ago
Noida, Uttar Pradesh, India Impetus Full timeImpetus is a renowned company in the field of data engineering, looking for an experienced Big Data Technology Specialist to join our team.We offer an attractive salary package of $120,000 - $180,000 per annum, depending on experience and qualifications. The ideal candidate will have at least 4-8 years of hands-on experience with Big Data technologies,...
-
Big Data Developer Lead
4 days ago
Noida, Uttar Pradesh, India Impetus Full timeJoin Impetus, a leading IT solutions company, as a Big Data Developer in Indore, Pune, Noida, or Gurugram. With an estimated salary of 1200000 INR per year, this role offers exciting opportunities for professional growth and development.Company OverviewAbout Impetus: We are a forward-thinking organization that thrives on innovation and collaboration. Our...
-
Big Data Engineer
5 days ago
Noida, India Tekshapers Software Solutions (P) Limited Full timeJob Description :As a Big Data Engineer, you will be responsible for designing, developing, and maintaining data pipelines to extract, transform, and load (ETL) data from various sources into target systems. You will work closely with data analysts and data scientists to ensure data quality, accuracy, and consistency.Key Responsibilities :- Design and...
-
Big Oh Tech | DevOps Engineer | noida
4 weeks ago
noida, India Big Oh Tech Full timeDevOps At Big Oh (Noida) Roles and Responsibilities: 1 years of DevOps Engineering and Service Delivery: Proven expertise in CI/CD and containerization with hands-on experience in cloud environments such as AWS, Microsoft Azure, and others. Automation and Cloud Integration: Skilled in designing and delivering automated CI/CD solutions in cloud...
-
AWS Data Engineer
2 months ago
Noida, India Firstcareercentre Full timeJob Summary :We are seeking a highly skilled and experienced AWS Data Engineer to join our data team. The AWS Data Engineer will be responsible for designing, developing, and optimizing data pipelines, integrating various data sources, and implementing AWS cloud solutions. The ideal candidate should have a strong background in data engineering, cloud...
-
Big data developer
1 week ago
Noida, India Impetus Full timeLead Software Engineer Locations- Indore, Noida, Hyderabad Job Description We are looking for a Lead Big Data Developer to design and oversee the implementation of large-scale data solutions, lead a talented team, and ensure the efficient processing and analysis of massive datasets. Roles & Responsibilities Key Responsibilities Architect...
-
Big Data Developer
3 weeks ago
Noida, India Impetus Full timeLead Software EngineerLocations- Indore, Noida, HyderabadJob DescriptionWe are looking for a Lead Big Data Developer to design and oversee the implementation of large-scale data solutions, lead a talented team, and ensure the efficient processing and analysis of massive datasets.Roles & ResponsibilitiesKey ResponsibilitiesArchitect Data Solutions: Design...
-
Big data developer
3 weeks ago
Noida, India Impetus Full timeLead Software EngineerLocations- Indore, Noida, HyderabadJob DescriptionWe are looking for a Lead Big Data Developer to design and oversee the implementation of large-scale data solutions, lead a talented team, and ensure the efficient processing and analysis of massive datasets.Roles & ResponsibilitiesKey ResponsibilitiesArchitect Data Solutions: Design...
-
Big Data Developer
3 weeks ago
Noida, India Impetus Full timeLead Software EngineerLocations- Indore, Noida, HyderabadJob DescriptionWe are looking for aLead Big Data Developerto design and oversee the implementation of large-scale data solutions, lead a talented team, and ensure the efficient processing and analysis of massive datasets.Roles & ResponsibilitiesKey ResponsibilitiesArchitect Data Solutions:Design and...