BIG DATA ENGINEER
1 month ago
Job description Role and responsibilities : -Creating Project Technical Documentation - Designing Solution architecture, and work on Data Ingestion, Preparation and Transformation.Debugging the production failures and identifying the solution.- Developing efficient frameworks for development and testing using (AWS Dynamo DB, EKS, Kafka) to enable seamless data ingestion process on to the Hadoop platform.- Enabling Data Governance and Data Discovery on Hadoop Platform- Building data processing framework using Spark, HQL- Exposure of Security Framework with Kerberos, Ranger, Atlas- Exposure of Data Pipeline Automation using DevOps tools- Exposure of Job Monitoring framework along validations automation- Exposure of handling structured, Un Structured and Streaming data Technical skills requirements : - Solid hands-on and Solution Architecting experience in Big-Data Technologies (AWS preferred)- Hands on experience in: AWS Dynamo DB, EKS, Kafka, Kinesis, Glue, EMR- Hands-on experience of programming language like Scala with Spark.- Good command and working experience on Hadoop Map Reduce, HDFS, Hive, HBase, and/or No-SQL Databases- Hands on working experience on any of the data engineering analytics platform (Hortonworks Cloudera MapR AWS), AWS preferred- Hands-on experience on Data Ingestion Apache Nifi, Apache Airflow, Sqoop, and Oozie- Hands on working experience of data processing at scale with event driven systems, message queues (Kafka FlinkSpark Streaming)- Hands on working Experience with AWS Services like EMR, Kinesis, S3, CloudFormation, Glue, API Gateway, Lake Foundation- Hands on working Experience with AWS Athena - Data Warehouse exposure on Apache Nifi, Apache Airflow, Kylo - Operationalization of ML models on AWS (e.g. deployment, scheduling, model monitoring etc.)- Feature Engineering Data Processing to be used for Model development- Experience gathering and processing raw data at scale (including writing scripts, web scraping, calling APIs, write SQL queries, etc.)- Experience building data pipelines for structured unstructured, real-time batch, events synchronous asynchronous using MQ, Kafka, Steam processing- Hands-on working experience in analysing source system data and data flows, working with structured and unstructured data- Must be very strong in writing SQL queries- Strengthen the Data engineering team with Big Data solutions- Strong technical, analytical, and problem-solving skillsPosted On
Job description Role and responsibilities : -Creating Project Technical Documentation - Designing Solution architecture, and work on Data Ingestion, Preparation and Transformation.Debugging the production failures and identifying the solution.- Developing efficient frameworks for development and testing using (AWS Dynamo DB, EKS, Kafka) to enable seamless data ingestion process on to the Hadoop platform.- Enabling Data Governance and Data Discovery on Hadoop Platform- Building data processing framework using Spark, HQL- Exposure of Security Framework with Kerberos, Ranger, Atlas- Exposure of Data Pipeline Automation using DevOps tools- Exposure of Job Monitoring framework along validations automation- Exposure of handling structured, Un Structured and Streaming data Technical skills requirements : - Solid hands-on and Solution Architecting experience in Big-Data Technologies (AWS preferred)- Hands on experience in: AWS Dynamo DB, EKS, Kafka, Kinesis, Glue, EMR- Hands-on experience of programming language like Scala with Spark.- Good command and working experience on Hadoop Map Reduce, HDFS, Hive, HBase, and/or No-SQL Databases- Hands on working experience on any of the data engineering analytics platform (Hortonworks Cloudera MapR AWS), AWS preferred- Hands-on experience on Data Ingestion Apache Nifi, Apache Airflow, Sqoop, and Oozie- Hands on working experience of data processing at scale with event driven systems, message queues (Kafka FlinkSpark Streaming)- Hands on working Experience with AWS Services like EMR, Kinesis, S3, CloudFormation, Glue, API Gateway, Lake Foundation- Hands on working Experience with AWS Athena - Data Warehouse exposure on Apache Nifi, Apache Airflow, Kylo - Operationalization of ML models on AWS (e.g. deployment, scheduling, model monitoring etc.)- Feature Engineering Data Processing to be used for Model development- Experience gathering and processing raw data at scale (including writing scripts, web scraping, calling APIs, write SQL queries, etc.)- Experience building data pipelines for structured unstructured, real-time batch, events synchronous asynchronous using MQ, Kafka, Steam processing- Hands-on working experience in analysing source system data and data flows, working with structured and unstructured data- Must be very strong in writing SQL queries- Strengthen the Data engineering team with Big Data solutions- Strong technical, analytical, and problem-solving skills
Skills Required
AWS EMRPosted On AWS EMR Location
Bangalore / Greater Noida / Pune / HyderabadPosted On Bangalore / Greater Noida / Pune / Hyderabad
-
Big Data Engineer
2 weeks ago
Bengaluru, India Virtusa Full timeBig Data Engineer - CREQ188642 DescriptionLooking for a Senior Software Engineer with strong analytical skills to help build and enhance our Big data application. This individual's day-to-day work will involve using Java, Scala, SQL, Big Data, Linux, git, and maven/gradle to design and build software solutions that are fast, scalable, and flexible.Bachelor's...
-
Big Data Engineer
2 weeks ago
Bengaluru, India Virtusa Full timeBig Data Engineer - CREQ188642 Description Looking for a Senior Software Engineer with strong analytical skills to help build and enhance our Big data application. This individual's day-to-day work will involve using Java, Scala, SQL, Big Data, Linux, git, and maven/gradle to design and build software solutions that are fast, scalable, and flexible....
-
Big Data Engineer
1 month ago
Bengaluru, India Adobe Full timeOur CompanyChanging the world through digital experiences is what Adobe’s all about. We give everyone—from emerging artists to global brands—everything they need to design and deliver exceptional digital experiences! We’re passionate about empowering people to create beautiful and powerful images, videos, and apps, and transform how companies...
-
Big Data Engineer
2 weeks ago
Bengaluru, Karnataka, India Adobe Full timeJOB LEVELMid-level roleEMPLOYEE ROLEIndividual ContributorResponsibilities:Build and maintain cutting-edge data platform and data processing pipelines.Challenge the status quo by innovating and creating a cost-effective big data platform.Collaborate across teams to develop AI-based experiences.Establish a culture of scientific processes and customer...
-
Big Data Engineer
4 days ago
Bengaluru, India Global Pharma Tek Full timeAuto req IDBRTitleBig Data EngineerYour Role and ResponsibilitiesUnderstand a data warehousing solution and able to work independently in such an environmentResponsible in Project development and delivery experience of a few good size projectsPosting CountryIndiaState / ProvinceHaryānaCity / Township / VillageGurgaonTravel RequiredUp to % or day a...
-
Big Data Engineer
1 month ago
Bengaluru, India Hexaware Technologies Full timeDescriptionData Engineer:Primary : ADF, SQLSeconday : Databricks, Pyspark / Scala, CICD, Data Security, encryptionResponsibilities:•Work closely with scrum master, technical leads and provide daily progress updates on the tasks assigned.•Provide support in defining the scope and sizing of the work.•Working closely with various enterprise architects,...
-
Big Data Engineer
2 months ago
Bengaluru, India Hexaware Technologies Full timeDescription Data Engineer: Primary : ADF, SQL Seconday : Databricks, Pyspark / Scala, CICD, Data Security, encryption Responsibilities: •Work closely with scrum master, technical leads and provide daily progress updates on the tasks assigned. •Provide support in defining the scope and sizing of the work. •Working closely with various...
-
Big Data Engineer
2 months ago
Bengaluru, Karnataka, India Adobe Full timeJOB LEVELP50EMPLOYEE ROLEIndividual ContributorP50 (ESDEP50) Big Data Engineer What You'll Do Build the underneath data platform and maintain data processing pipelines using best in class technologies. Special focus on R&D to challenge status-quo and build the next generation big data platform that is efficient and cost effective. Translate complex...
-
Big Data Engineer
1 week ago
Bengaluru, India Infogain Full timeBig Data Engineer (Lead) with skills Big Data, Data Warehouse, Azure Datalake for location Any Infogain Base Location (Noida, Gurugram, Bangalore, Mumbai, Pune)Posted on: May 22, Share on Linkedin Share on Twitter Share on FacebookROLES & RESPONSIBILITIESResponsibilities:· Strong experience ofminimum 5+ years in Snowflake & Snowpipehaving over all 9 to 12...
-
Big Data Engineer
7 days ago
Bengaluru, India Infogain Full timeBig Data Engineer (Lead) with skills Big Data, Data Warehouse, Azure Datalake for location Any Infogain Base Location (Noida, Gurugram, Bangalore, Mumbai, Pune)Posted on: May 23, Share on Linkedin Share on Twitter Share on FacebookROLES & RESPONSIBILITIESResponsibilities:· Strong experience ofminimum 5+ years in Snowflake & Snowpipehaving over all 9 to 12...
-
Big Data Engineer
6 days ago
Bengaluru, India Infogain Full timeBig Data Engineer (Lead) with skills Big Data, Data Warehouse, Azure Datalake for location Any Infogain Base Location (Noida, Gurugram, Bangalore, Mumbai, Pune)Posted on: May 24, Share on Linkedin Share on Twitter Share on FacebookROLES & RESPONSIBILITIESResponsibilities:· Strong experience ofminimum 5+ years in Snowflake & Snowpipehaving over all 9 to 12...
-
Big Data Engineer
5 days ago
Bengaluru, India Global Pharma Tek Full timeAuto req ID BR Title Big Data Engineer Your Role and Responsibilities Understand a data warehousing solution and able to work independently in such an environment Responsible in Project development and delivery experience of a few good size projects Posting Country India State / Province Haryāna City / Township / Village Gurgaon Travel...
-
Big Data Engineer
2 weeks ago
Bengaluru, India Infogain Full timeBig Data Engineer (Lead) with skills Big Data, Data Warehouse, Azure Datalake for location Any Infogain Base Location (Noida, Gurugram, Bangalore, Mumbai, Pune) Posted on: May 19, Share on Linkedin Share on Twitter Share on Facebook ROLES & RESPONSIBILITIES Responsibilities: · Strong experience of minimum 5+ years in Snowflake & Snowpipe ...
-
Big Data Engineer
2 weeks ago
Bengaluru, India Infogain Full timeBig Data Engineer (Lead) with skills Big Data, Data Warehouse, Azure Datalake for location Any Infogain Base Location (Noida, Gurugram, Bangalore, Mumbai, Pune) Posted on: May 20, Share on Linkedin Share on Twitter Share on Facebook ROLES & RESPONSIBILITIES Responsibilities: · Strong experience of minimum 5+ years in Snowflake & Snowpipe ...
-
Big Data Engineer
1 week ago
Bengaluru, India Infogain Full timeBig Data Engineer (Lead) with skills Big Data, Data Warehouse, Azure Datalake for location Any Infogain Base Location (Noida, Gurugram, Bangalore, Mumbai, Pune) Posted on: May 22, Share on Linkedin Share on Twitter Share on Facebook ROLES & RESPONSIBILITIES Responsibilities: · Strong experience of minimum 5+ years in Snowflake & Snowpipe ...
-
Big Data Engineer
1 week ago
Bengaluru, India Infogain Full timeBig Data Engineer (Lead) with skills Big Data, Data Warehouse, Azure Datalake for location Any Infogain Base Location (Noida, Gurugram, Bangalore, Mumbai, Pune) Posted on: May 23, Share on Linkedin Share on Twitter Share on Facebook ROLES & RESPONSIBILITIES Responsibilities: · Strong experience of minimum 5+ years in Snowflake & Snowpipe ...
-
Big Data Engineer
6 days ago
Bengaluru, India Infogain Full timeBig Data Engineer (Lead) with skills Big Data, Data Warehouse, Azure Datalake for location Any Infogain Base Location (Noida, Gurugram, Bangalore, Mumbai, Pune) Posted on: May 24, Share on Linkedin Share on Twitter Share on Facebook ROLES & RESPONSIBILITIES Responsibilities: · Strong experience of minimum 5+ years in Snowflake & Snowpipe ...
-
Big Data Engineer
5 days ago
Bengaluru, India Infogain Full timeBig Data Engineer (Lead) with skills Big Data, Data Warehouse, Azure Datalake for location Any Infogain Base Location (Noida, Gurugram, Bangalore, Mumbai, Pune) Posted on: May 26, Share on Linkedin Share on Twitter Share on Facebook ROLES & RESPONSIBILITIES Responsibilities: · Strong experience of minimum 5+ years in Snowflake & Snowpipe ...
-
Big Data Engineer
3 days ago
Bengaluru, India Infogain Full timeBig Data Engineer (Lead) with skills Big Data, Data Warehouse, Azure Datalake for location Any Infogain Base Location (Noida, Gurugram, Bangalore, Mumbai, Pune) Posted on: May 29, Share on Linkedin Share on Twitter Share on Facebook ROLES & RESPONSIBILITIES Responsibilities: · Strong experience of minimum 5+ years in Snowflake & Snowpipe ...
-
Big Data Engineer
3 days ago
Bengaluru, India Infogain Full timeBig Data Engineer (Lead) with skills Big Data, Data Warehouse, Azure Datalake for location Any Infogain Base Location (Noida, Gurugram, Bangalore, Mumbai, Pune) Posted on: May 28, Share on Linkedin Share on Twitter Share on Facebook ROLES & RESPONSIBILITIES Responsibilities: · Strong experience of minimum 5+ years in Snowflake & Snowpipe ...
