Big Data Engineer
1 week ago
Job Description :
- 6 to 9 Years hands on knowledge on Bigdata, Scala, Spark SQL as well as SQL/NoSQL databases
- Proficient in Hadoop Distributed File System (HDFS), Spark, scala, the Hadoop ecosystem, including Yarn, Hive, and Kafka.
- Highly skilled in writing Spark programs using RDD, Spark SQL, Spark Datasets, and DataFrames.
- Proficient in Scala core concepts.
- Experience in camunda flow.
- Exceptional expertise in complex SQL queries.
- Good experience in working with Hive, Impala and other hadoop ecosystem.
- Experience on writing Unix Scripts.
- Expertise in creating complex and on demand reports for business users.
- Familiarity with distributed computing concepts and cluster management (YARN, Mesos).
- Developed custom Spark jobs and Scala scripts for data transformation, aggregation, and cleansing.
- Built real-time data processing pipelines using Apache Kafka to ingest, process, and analyze streaming data.
- Managed and optimized Hadoop clusters for efficient data storage and processing.
- Optimized Spark jobs and ETL processes to improve performance and reduce processing times.
- Experienced Spark Scala Hadoop Kafka ETL with 10 years of expertise in designing, developing, and optimizing data processing pipelines for large-scale data systems.
- Strong knowledge of SQL, including joins, subqueries, set operations, and advanced OLAP functions.
- Proficient in the Linux and UNIX operating systems and Unix commands.
- Experienced in Agile methodology.
- Expertise in creating pipeline and scheduling it for data transition from source system to target system.
- Basics of python and azure cloud.
- Experience in kafka in spring boot.
- Experience in elastics for real time logging.
- Good understanding of CI/CD process, Git, Jenkins
-
Big Data Engineer
2 weeks ago
Chennai, Tamil Nadu, India Krishpar Technologies Pvt. Ltd Full timeWe are seeking an experienced Big Data Engineer to join our team at Krishpar Technologies Pvt. Ltd.**Salary:** ₹15,00,000 - ₹25,00,000 per annumAs a Big Data Engineer, you will be responsible for designing and developing scalable data pipelines using Pyspark on Hadoop, Hive, and Kafka.**Required Skills and Qualifications: 4+ years of experience in Big...
-
Big Data Engineer
5 days ago
Chennai, Tamil Nadu, India Tata Consultancy Services Full timeJob Title: Big Data EngineerTata Consultancy Services is seeking a skilled Big Data Engineer to join our team. As a leading global consulting and IT services company, we empower our clients to become digital businesses.The estimated salary for this role is ₹1,200,000 - ₹2,500,000 per annum, based on location and experience.About the Role:We are looking...
-
Big Data Engineer
1 week ago
Chennai, Tamil Nadu, India Virtusa Full timeJob Title: Big Data EngineerAbout the Job:Virtusa is seeking a skilled Big Data Engineer to join our team. As a key member of our data analytics team, you will be responsible for designing, building, and supporting data applications using Spark, Sqoop, and Hive.Responsibilities:• Develop components, application interfaces, and solution enablers while...
-
Big Data Engineer
2 months ago
Chennai, India TapTalent.ai Full timeAboutCompany: Our Company is a globaltechnology consulting and digital solutions company that enablesenterprises to reimagine business models and accelerate innovationthrough digital technologies. Powered by more than 84000entrepreneurial professionals across more than 30 countries catersto over 700 clients. Our extensive domain and technology expertisehelps...
-
Big Data Engineer
1 month ago
Chennai/Pune, India Pan Asia Man power Consultants Recruiting agency Full timeBig Data EngineerWe are seeking a skilled Big Data Engineer to contribute to building backend data applications and services. This role requires expertise in various technologies to drive our data initiatives.Key Responsibilities:Design, develop, and maintain backend data applications and services.Collaborate with product owners and team members to gather...
-
Big Data Engineer
2 weeks ago
Coimbatore, Chennai, India Talodyn Networks Private Limited Full timeDescriptionWe are seeking a highly skilled Big Data Engineer with 5-9 years of experience to join our team in India. The ideal candidate should have a strong technical background in big data technologies and be able to design, develop, and maintain large-scale data processing systems. The candidate should also be able to work collaboratively with...
-
Big Data Analytics Engineer
1 month ago
Chennai, Tamil Nadu, India FIS Full timeAbout the RoleFIS is seeking a talented Big Data Analytics Engineer to join our team. As a Big Data Analytics Engineer, you will design, architect, and help maintain enterprise solutions on the big data analytics platform.Key ResponsibilitiesDesign, architect, and help maintain enterprise solutions on the big data analytics platform.Lead systems...
-
Big Data Engineer
1 month ago
Chennai, India Mavinsys Full timeKey Skills and ResponsibilitiesWe are seeking a highly skilled Big Data Developer to design and develop scalable data pipelines using Scala and Spark. The ideal candidate will have extensive experience with Big Data technologies, including Hadoop, Spark, and Kafka, as well as strong proficiency in Scala and Spark APIs.Key ResponsibilitiesDesign and develop...
-
Senior Big Data Engineer
2 weeks ago
Chennai, Tamil Nadu, India InfoCentroid Software Solutions Pvt. Ltd Full timeJob Title: Senior Big Data EngineerAbout the Role:We are seeking an experienced Senior Big Data Engineer to join our team at InfoCentroid Software Solutions Pvt. Ltd in Chennai. The ideal candidate will have a strong background in data engineering, a deep understanding of Google Cloud Platform (GCP) services, and hands-on experience with SQL, Python,...
-
Big Data Developer
1 month ago
Chennai, India ZF Group Full timeJob Description:We are currently seeking dynamic individuals to join our team and contribute to the success for our organization.Experience - 5 to 10 yrsRequirementBachelor’s degree in Computer Science, Engineering, Information Technology, or a related field.Demonstrable experience as a Big Data Developer, Data Engineer, or similar role with a minimum of 5...
-
Big Data Developer
1 month ago
Chennai, India ZF Group Full timeJob Description:We are currently seeking dynamic individuals to join our team and contribute to the success for our organization.Experience - 5 to 10 yrsRequirement- Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field.- Demonstrable experience as a Big Data Developer, Data Engineer, or similar role with a minimum...
-
Data Engineer for Scalable Big Data Solutions
2 weeks ago
Chennai, Tamil Nadu, India RiskInsight Consulting Pvt Ltd Full timeWe are seeking an experienced Data Engineer to join our team at RiskInsight Consulting Pvt Ltd. This role involves designing, developing, and deploying scalable Big Data applications using Apache Spark.ResponsibilitiesCollaborate with data scientists and business analysts to understand requirements and translate them into technical solutions.Write efficient...
-
Data Engineer for Big Data Solutions
2 weeks ago
Chennai, Tamil Nadu, India Wipro Ltd Full timeAbout the RoleWipro Ltd is seeking an experienced Data Engineer to join our team. In this role, you will be responsible for designing and developing batch and streaming solutions using various Big Data technologies.You will work closely with our architect to design secure and scalable solutions that meet the needs of our business. Additionally, you will be...
-
Big data developer
2 months ago
Chennai, India Tata Consultancy Services Full timeGreetings from TCS! TCS is hiring for Big Data Developer Desired Experience: 5+ Years Job Location: Chennai/ Bangalore / Pune / Mumbai Require skillset: Big Data, Hadoop, Spark, Scala, Hive Job Description:- Must Have Technical Qualifications: Spark Hadoop Hive Scala Nice to Have Qualifications: Data Engineer Certification Experience with...
-
Data Engineer
4 weeks ago
Chennai, India Innovalus Full timeJob Title : Data Engineering DeveloperLocation : ChennaiExperience : 4-8 YearsJob Responsibilities :- Design, develop and maintain the data platform and data-driven monitoring tools for risk and data science solutions in Big Data platform, using cutting edge technologies.- Work closely with the partner team to build solution dependency and implement the...
-
Big Data Developer
1 month ago
chennai, India ZF Group Full timeJob Description: We are currently seeking dynamic individuals to join our team and contribute to the success for our organization. Experience - 5 to 10 yrs Requirement Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field. Demonstrable experience as a Big Data Developer, Data Engineer, or similar role with a...
-
Big Data Developer
1 month ago
Chennai, India ZF Group Full timeJob Description:We are currently seeking dynamic individuals to join our team and contribute to the success for our organization.Experience - 5 to 10 yrsRequirementBachelor’s degree in Computer Science, Engineering, Information Technology, or a related field.Demonstrable experience as a Big Data Developer, Data Engineer, or similar role with a minimum of 5...
-
Big Data Developer
1 month ago
Chennai, India ZF Group Full timeJob Description:We are currently seeking dynamic individuals to join our team and contribute to the success for our organization.Experience - 5 to 10 yrsRequirementBachelor’s degree in Computer Science, Engineering, Information Technology, or a related field.Demonstrable experience as a Big Data Developer, Data Engineer, or similar role with a minimum of 5...
-
Big Data Developer
1 month ago
chennai, India ZF Group Full timeJob Description:We are currently seeking dynamic individuals to join our team and contribute to the success for our organization.Experience - 5 to 10 yrsRequirementBachelor’s degree in Computer Science, Engineering, Information Technology, or a related field.Demonstrable experience as a Big Data Developer, Data Engineer, or similar role with a minimum of 5...
-
Big Data Developer
1 month ago
Chennai, India ZF Group Full timeJob Description: We are currently seeking dynamic individuals to join our team and contribute to the success for our organization. Experience - 5 to 10 yrs Requirement Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field. Demonstrable experience as a Big Data Developer, Data Engineer, or similar role with a...