![INTELLIGENCE INDIA.COM LIMITED](https://media.trabajo.org/img/noimg.jpg)
IICL - Senior Big Data Engineer - Spark/Apache Flink
2 days ago
Job Description :
Role : Senior Data Engineer (Flink + Spark + Kafka)
- Minimum 7 years of experience
- B.Sc. Of Computer Science or similar
- Proficiency in programming languages Java, Python or R
- Experience with design and development of ELT workflows and data pipelines for high-volume data systems.
- Experience in Data Engineering and implementation of Real-time data processing pipelines using kafka and spark streams.
- Proven Hands-on experience in Flink.
- Good analytical skills with excellent knowledge of SQL, data management and relational databases like PostgreSQL, Oracle, MySQL
- Experience working with Big Data Tools/Technologies and DataLakes (Hadoop, Map/Reduce, Spark, Hive)
- Knowledge/Experience working with In-memory/NoSQL databases - Redis, HBase, Cassandra, MongoDB, CouchBase, CouchDB
- Knowledge/experience with Apache Nifi/Airflow for Data Pipelines Orchestrations.
- Proven expertise with linux administration and unix scripting for the automation of tasks, Containerzation Kubernetes, Docker, Openshift
- Knowledge and/or experience with using or building CI/CD tools (Jenkins)
- Experience with version control systems like Git
- AI and Machine Learning knowledge, Database Administration, Big Data Storage, Data Analytics and Feature Engineering, ELK Stack Knowledge
- Familiar with cloud platforms like AWS, Google Cloud, or Microsoft Azure
MANDATORY : Java/Python or R, ETL/ELT, Big Data, Hadoop, HBase, Hive, Spark, Flink, Kafka, SQL, Couchbase/CouchDB or REDIS, RDBMS, Spark Streaming,No-SQL DB (MongoDB), Shell/BASH Scripting
PLUS : ELK, Core Java, Apache Nifi / Apache AirFlow, Docker, Kubernetes, Jenkins, Git
-
Overseas/International/Dubai, IN INTELLIGENCE INDIA.COM LIMITED Full timeJob Description :- Proficient in AI and machine learning tools and technologies including Python, R, Panadas, Keras, Tensorflow 0r Thanos, numpy, sklearn, Notebooks, Regression, Classification, Prediction, Neural Networks, LSTMs - Recurrent Neural Networks, Reinforced Learning, Big Data Machine learning and AI Models and Algorithms.- A very good knowledge of...
-
Data Platform Support Engineer
4 weeks ago
Singapore/Overseas/International, IN Sourceone Staffing & Consulting Pvt Ltd Full timeData Platform Support EngineerLocation : SingaporeTeam/ Competency : Production SupportCore Skillsets : Service Delivery, Batch scheduling and monitoring, ETL Development, SQLHigh Level Job Scope : Production SupportJob Description : Big Data Support EngineerSkillset :- Up to 9 years of IT experience & 5+ Years on Hadoop and K8s Eco Systems- Last 3 years of...
-
Senior Data Engineer
2 months ago
Overseas/International, IN Freelancer HR Full timeJob Description Responsibilities :- Understand the factories , manufacturing process , data availability and avenues for improvement- Brainstorm , together with engineering, manufacturing and quality problems that can be solved using the acquired data in the data lake platform.- Define what data is required to create a solution and work with connectivity...
-
Senior Data Engineer
4 weeks ago
Overseas/International, IN Freelancer HR Full timeJob Description Responsibilities :- Understand the factories , manufacturing process , data availability and avenues for improvement- Brainstorm , together with engineering, manufacturing and quality problems that can be solved using the acquired data in the data lake platform.- Define what data is required to create a solution and work with connectivity...
-
Senior Data Engineer
1 month ago
Riyadh/Overseas/International, IN Syllogistics.ai Full timeResponsibilities : - Identifying and extracting data from various sources (such as databases, APIs, logs, files, and external systems).- Ensuring data quality during extraction and handling any anomalies.- Transforming raw data into a structured format suitable for analysis and reporting.- Cleaning, aggregating, and enriching data using ETL tools and...
-
Diatoz Solutions
2 months ago
Dubai/Overseas/International, IN Diatoz Solutions Pvt Ltd Full timeRole : Senior Software EngineerExperience : 5-8 years Skill : DevOpsDescription : The ideal candidate will have experience in implementing automated CI/CD pipelines for software delivery and deployment processes on cloud : - Design and implement CI/CD pipelines to automate software delivery and deployment processes- Design, implement, and manage scalable and...
-
Diatoz Solutions
4 weeks ago
Dubai/Overseas/International, IN Diatoz Solutions Pvt Ltd Full timeRole : Senior Software EngineerExperience : 5-8 years Skill : DevOpsDescription : The ideal candidate will have experience in implementing automated CI/CD pipelines for software delivery and deployment processes on cloud : - Design and implement CI/CD pipelines to automate software delivery and deployment processes- Design, implement, and manage scalable and...
-
Overseas/International/Africa/Nigeria, IN INTELLIGENCE INDIA.COM LIMITED Full timeSkills : Hadoop, Greenplum & ClouderaLocation : Onsite Nigeria, LagosDuration : 1 Year ExtendableJob Description : - Experience in installing, configuring, supporting and Managing Hadoop clusters using Cloudera & Horton works Hadoop distributions.- Experience in Migrating data from Greenplum to Hadoop.- Installing, Configuring, Upgrading and Maintaining Hawq...
-
Diatoz Solutions
2 months ago
Dubai/Overseas/International, IN Diatoz Solutions Pvt Ltd Full timeList of Skills Required :- Experience in Oracle SQL and PL/SQL.- Advanced PLSQL Programming, Collections and Bulk processing, Analytical functions - SQL query tuning and PL/SQL Performance optimization- Good data analysis skills.- Should be able to understand business requirements and do solution design.- Senior PL/SQL Developer (Database) - Senior Database...
-
Diatoz Solutions
4 weeks ago
Dubai/Overseas/International, IN Diatoz Solutions Pvt Ltd Full timeList of Skills Required :- Experience in Oracle SQL and PL/SQL.- Advanced PLSQL Programming, Collections and Bulk processing, Analytical functions - SQL query tuning and PL/SQL Performance optimization- Good data analysis skills.- Should be able to understand business requirements and do solution design.- Senior PL/SQL Developer (Database) - Senior Database...
-
Diatoz Solutions
2 months ago
Dubai/Overseas/International, IN Diatoz Solutions Pvt Ltd Full timeKey Responsibilities : - To manage end to end software development, integration, testing, implementation, and maintenance of multiple projects assigned under the portfolio.- Manages and controls the status and successful delivery of all technology projects assigned under the line of business, ensuring maximum business value is delivered through IT...
-
Diatoz Solutions
4 weeks ago
Dubai/Overseas/International, IN Diatoz Solutions Pvt Ltd Full timeKey Responsibilities : - To manage end to end software development, integration, testing, implementation, and maintenance of multiple projects assigned under the portfolio.- Manages and controls the status and successful delivery of all technology projects assigned under the line of business, ensuring maximum business value is delivered through IT...
-
JTek Software Solutions
1 month ago
Overseas/International, IN JTek Software Solutions Pvt Ltd Full timeRole : Data StewardLocation : USExperience : 5-8 YrsWhat You'll Do :We're setting up a new role to support the improvement and quality of our data in the North America region. The North America Data Steward will act as the go-to person to ensure data quality, completeness, and accuracy from several data sources.Key Responsibilities :This person will...
-
JTek Software Solutions
4 weeks ago
Overseas/International, IN JTek Software Solutions Pvt Ltd Full timeRole : Data StewardLocation : USExperience : 5-8 YrsWhat You'll Do :We're setting up a new role to support the improvement and quality of our data in the North America region. The North America Data Steward will act as the go-to person to ensure data quality, completeness, and accuracy from several data sources.Key Responsibilities :This person will...
-
Chief Product Officer
2 weeks ago
Dubai/Overseas/International, IN Prime Infosoft Full timePosition OverviewAs the Chief Product Officer (CPO), you will be a key member of our executive team, responsible for defining and executing our product strategy, roadmap, and vision. You will play a pivotal role in shaping the future of our company by leading the product team and ensuring that our products align with our business goals and customer needs....
-
Functional Test Engineer
2 weeks ago
Overseas/International, IN Growel Softech Pvt. Ltd. Full timeJob Description :Mandatory Skills : Functional Testing, Manhattan Configuration, WMS Functional Analyst.Location - Any UST location.WMS Functional/Reports Analyst : - WMS : Warehouse Management System SUMMARY Albertsons Company Incorporated is looking for the Manhattan WMS (Warehouse Management System) - Analyst responsible for the testing, configuration,...
-
PL-SQL Developer
1 month ago
Overseas/International, IN KARPIRA Full timeOccupations: PL/SQL Developer wanted for Germany Job Description: On behalf of the Client, an Industrial group in Germany, we seek a high-achieving PL/SQL Developer. Responsibilities: - ETL Design and Development - Further development of complex calculating engines with Oracle PL/SQL in the automotive field - SQL performance tuning Requirements: - A...
-
Senior Business Analyst
2 months ago
Chennai/Overseas/International/Thailand, IN Scripting Resumes Full timeSenior Business Analyst-Insurance Domain, a Full Time role in Chennai and willing to travel onsite to Thailand in understanding the project requirements ,client looking for immediate joiner for this role Role : Senior Business Analyst - Insurance Domain Job Type : Full Time Job Positions : 1Location : Chennai/Thailand Job Description :- 8+ years of total...
-
Chief Information Officer
2 months ago
Overseas/International, IN Sampoorna Consultants Pvt. Ltd Full timeJob Location : FIJIJob Description :Our client if looking for Chief Information Officer who will be responsible for management, implementation, and usability of information and computer technologies, including :- Development and implementation of Group strategies and policies on Information Technology aligned with business strategy with a long-term vision.-...
-
Chief Information Officer
4 weeks ago
Overseas/International, IN Sampoorna Consultants Pvt. Ltd Full timeJob Location : FIJIJob Description :Our client if looking for Chief Information Officer who will be responsible for management, implementation, and usability of information and computer technologies, including :- Development and implementation of Group strategies and policies on Information Technology aligned with business strategy with a long-term vision.-...