IICL - Senior Big Data Engineer - Spark/Apache Flink

4 days ago

india INTELLIGENCE INDIA.COM LIMITED Full time

Job Description :

Role : Senior Data Engineer (Flink + Spark + Kafka)

- Minimum 7 years of experience

- B.Sc. Of Computer Science or similar

- Proficiency in programming languages Java, Python or R

- Experience with design and development of ELT workflows and data pipelines for high-volume data systems.

- Experience in Data Engineering and implementation of Real-time data processing pipelines using kafka and spark streams.

- Proven Hands-on experience in Flink.

- Good analytical skills with excellent knowledge of SQL, data management and relational databases like PostgreSQL, Oracle, MySQL

- Experience working with Big Data Tools/Technologies and DataLakes (Hadoop, Map/Reduce, Spark, Hive)

- Knowledge/Experience working with In-memory/NoSQL databases - Redis, HBase, Cassandra, MongoDB, CouchBase, CouchDB

- Knowledge/experience with Apache Nifi/Airflow for Data Pipelines Orchestrations.

- Proven expertise with linux administration and unix scripting for the automation of tasks, Containerzation Kubernetes, Docker, Openshift

- Knowledge and/or experience with using or building CI/CD tools (Jenkins)

- Experience with version control systems like Git

- AI and Machine Learning knowledge, Database Administration, Big Data Storage, Data Analytics and Feature Engineering, ELK Stack Knowledge

- Familiar with cloud platforms like AWS, Google Cloud, or Microsoft Azure

MANDATORY : Java/Python or R, ETL/ELT, Big Data, Hadoop, HBase, Hive, Spark, Flink, Kafka, SQL, Couchbase/CouchDB or REDIS, RDBMS, Spark Streaming,No-SQL DB (MongoDB), Shell/BASH Scripting

PLUS : ELK, Core Java, Apache Nifi / Apache AirFlow, Docker, Kubernetes, Jenkins, Git

(ref:hirist.tech)

  • Java Developer

    4 days ago

    india Gama Advisory Services LLP Full time

    Overview :We are looking for a talented Java Developer with 2-3 years of experience in Big Data technologies to join our team. As a Java Developer specializing in Big Data, you will be responsible for designing, developing, and maintaining scalable applications and data pipelines using Java and Big Data frameworks. Your expertise in Java programming and Big...

  • Big Data Architect

    1 month ago

    india Swift Strategic Staff Solutions INC Full time

    Responsibilities : Data Ingestion & Processing : Design and build robust data pipelines using tools like Apache Spark, Flink, or Kafka to ingest, transform, and process massive datasets from various sources.Big Data Technologies : Utilize your expertise in big data frameworks like Hadoop, YARN, and HDFS to manage, store, and analyze large volumes of data...

  • Data Engineer

    2 weeks ago

    India Tech Mahindra Full time

    Over all exp : 3-5yrs Notice : Immediate Must Have Skills: 3-5 yrs, of Strong expertise in Python for data engineering tasks (ETL/ELT). 3-5yrs, of Experience with big data tools is desirable: Spark, Spark-Streaming, Kafka, Debezium etc. experience with two or more of this big data streaming technology is mandatory: Apache Spark Streaming, Apache Flink,...

  • Senior Data Engineer

    2 days ago

    india Coffeee.io Full time

    Job Description: Are you a seasoned Scala developer with a passion for data streaming? Do you have at least 5 years of experience working on data streaming frameworks like Apache Kafka, Apache Flink, or Apache Spark Streaming? If so, we want to hear from you! Key Responsibilities: Lead the design, development, and maintenance of real-time data streaming...

  • Data Engineer –

    2 months ago

    india HARMAN International Full time

    HARMAN’s engineers and designers are creative, purposeful and agile. As part of this team, you’ll combine your technical expertise with innovative ideas to help drive cutting-edge solutions in the car, enterprise and connected ecosystem. Every day, you will push the boundaries of creative design, and HARMAN is committed to providing you with the...

  • Data Engineer –

    4 weeks ago

    india HARMAN International Full time

    HARMAN’s engineers and designers are creative, purposeful and agile. As part of this team, you’ll combine your technical expertise with innovative ideas to help drive cutting-edge solutions in the car, enterprise and connected ecosystem. Every day, you will push the boundaries of creative design, and HARMAN is committed to providing you with the...

  • Spark/Scala Developer

    4 days ago

    india Gama Advisory Services LLP Full time

    Overview :We are seeking a talented Spark/Scala Developer with 2-3 years of experience in Big Data technologies to join our team. As a Spark/Scala Developer, you will be responsible for designing, implementing, and maintaining scalable data pipelines and applications using Apache Spark and Scala. Your expertise in Big Data processing and Scala programming...

  • Senior Data Engineer

    2 months ago

    Anywhere in India/Multiple Locations VAST Consulting Full time

    Job Description : - Should have 5+ years of working experience.- Bachelor's or Master's degree in Computer Science, Engineering, or related field.- 5+ years of experience in a data engineering role with strong understanding of graph databases, streaming and batch analytics, cloud data platforms, data visualizations and big data processing.- Deep...

  • Big Data Architect

    2 weeks ago

    India Swift Strategic Staff Solutions INC Full time

    Responsibilities :Data Ingestion & Processing : Design and build robust data pipelines using tools like Apache Spark, Flink, or Kafka to ingest, transform, and process massive datasets from various sources.Big Data Technologies : Utilize your expertise in big data frameworks like Hadoop, YARN, and HDFS to manage, store, and analyze large volumes of data...

  • Java AWS Apache Flink Expert

    4 days ago

    india Synechron Full time

    We are looking to hire a Java AWS Apache Flink Expert to join our dynamic team! About Synechron At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and innovative technology to deliver industry-leading digital solutions. Synechron’s progressive technologies and...

  • Java AWS Apache Flink Expert

    4 days ago

    india Synechron Full time

    We are looking to hire a Java AWS Apache Flink Expert to join our dynamic team! About Synechron At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and innovative technology to deliver industry-leading digital solutions. Synechron’s progressive technologies and...

  • Senior Data Engineer

    1 month ago

    india PrimEra Medical Technologies Full time

    Job Description: Senior Data Engineer About PrimEra: PrimEra is a leading innovator in the healthcare industry, committed to transforming healthcare operations through advanced data solutions. We foster a collaborative and inclusive environment where professionals can grow and make a significant impact. Position Overview: We are seeking a highly skilled...

  • Big Data Architect

    1 month ago

    Anywhere in India/Multiple Locations, IN Swift Strategic Staff Solutions INC Full time

    Responsibilities : Data Ingestion & Processing : Design and build robust data pipelines using tools like Apache Spark, Flink, or Kafka to ingest, transform, and process massive datasets from various sources.Big Data Technologies : Utilize your expertise in big data frameworks like Hadoop, YARN, and HDFS to manage, store, and analyze large volumes of data...

  • Big Data Architect

    4 weeks ago

    Anywhere in India/Multiple Locations, IN Swift Strategic Staff Solutions INC Full time

    Responsibilities : Data Ingestion & Processing : Design and build robust data pipelines using tools like Apache Spark, Flink, or Kafka to ingest, transform, and process massive datasets from various sources.Big Data Technologies : Utilize your expertise in big data frameworks like Hadoop, YARN, and HDFS to manage, store, and analyze large volumes of data...

  • Big Data Architect

    4 weeks ago

    Anywhere in India/Multiple Locations Swift Strategic Staff Solutions INC Full time

    Responsibilities : Data Ingestion & Processing : Design and build robust data pipelines using tools like Apache Spark, Flink, or Kafka to ingest, transform, and process massive datasets from various sources.Big Data Technologies : Utilize your expertise in big data frameworks like Hadoop, YARN, and HDFS to manage, store, and analyze large volumes of data...

  • Data Engineer

    4 weeks ago

    india Oracle Full time

    Responsibilities: Data Engineering: 1. Design, develop, and maintain scalable data pipelines and ETL processes to collect, process, and store large volumes of data from diverse sources. 2. Implement and optimize distributed computing frameworks such as Apache Hadoop, Apache Spark, or Apache Flink to handle big data processing tasks efficiently. 3....

  • Data Engineer

    2 weeks ago

    India Oracle Full time

    Responsibilities: Data Engineering: 1. Design, develop, and maintain scalable data pipelines and ETL processes to collect, process, and store large volumes of data from diverse sources. 2. Implement and optimize distributed computing frameworks such as Apache Hadoop, Apache Spark, or Apache Flink to handle big data processing tasks efficiently. 3....

  • Spark/Scala Developer

    2 days ago

    Anywhere in India/Multiple Locations, IN Gama Advisory Services LLP Full time

    Overview :We are seeking a talented Spark/Scala Developer with 2-3 years of experience in Big Data technologies to join our team. As a Spark/Scala Developer, you will be responsible for designing, implementing, and maintaining scalable data pipelines and applications using Apache Spark and Scala. Your expertise in Big Data processing and Scala programming...

  • Spark/Scala Developer

    2 days ago

    Anywhere in India/Multiple Locations Gama Advisory Services LLP Full time

    Overview :We are seeking a talented Spark/Scala Developer with 2-3 years of experience in Big Data technologies to join our team. As a Spark/Scala Developer, you will be responsible for designing, implementing, and maintaining scalable data pipelines and applications using Apache Spark and Scala. Your expertise in Big Data processing and Scala programming...

  • Senior Data Engineer

    2 weeks ago

    Anywhere in India/Multiple Locations VAST Consulting Full time

    Job Description :Should have 5+ years of working experience. Bachelor's or Master's degree in Computer Science, Engineering, or related field. 5+ years of experience in a data engineering role with strong understanding of graph databases, streaming and batch analytics, cloud data platforms, data visualizations and big data processing. Deep understanding of...