Bigdata/hadoop (with Kafka) Support Expert

1 month ago

Bengaluru, India CGI Group, Inc. Full time

**Position Description**:

- 7 to 8 years working experience on Big Data Engineering and other open source technologies
- Strong Knowledge of Python, Apache Spark (PySpark), Azure Data Lake (Gen 2), PySParkSQL, Spark Streaming
- Strong knowledge of ETL/Data warehosuing concepts.
- Working knowledge and experience with MapReduce, Spark Streaming, Spark Core, Spark SQL, HQL
- Good knowledge in MySQL, PostgreSQL, Cassandra, Elastic Search and MongoDB
- Excellent skills in DB Schema design and NoSQL development
- Work experience in cloud services Azure, AWS
- Strong knowledge of building Big Data Applications with high-performance..
- Ability to think at a high level and see the big picture as well as the technical detail
- Manage teams towards resolutions of technical issues during deployments/integrations with multiple stakeholders
- Good experience in implementation of Design Concepts, Patterns and Principles
- Solid knowledge of software development life cycle.
- Ability to engage with the client in technology discussions.
- Excellent leadership and communication skills.
- Big Data Cloud migration with Confluent/Kafka with 7+yrs of experience
- Proven Expertise on Confluent
- Expertise on Kafka
- Big Data Administration
- Analyzes, designs, creates and implements Big Data infrastructures, including access methods, device allocations, validation checks, organization and security.
- Designs data models, logical and physical infrastructure designs, etc.
- Assists in system planning, scheduling, and implementation
- Initiates corrective actions to stay on schedule
- Install, configure, upgrades Hadoop Clusters including Spark, Impala, Hive, Scala, etc.
- Experience in spark query tuning
- Develops and implements recovery plans and procedures.
- Extremely strong Unix/Linux skills
- Solid administrative knowledge of Apache Hadoop a must (Cloudera distribution a plus)
- BI Tool integrations with Hadoop
- DBA experience HBASE
- Experience with database replication and scaling
- Design, install, and maintain highly available systems (including monitoring, security, backup, and performance tuning)
- Scripting experience
- Very good communication and work management
- Must possess good analytics and problem solving skills
- Must be willing to work flexible hours
- Excellent technical leadership, teamwork and Work with cross-functional teams to set up a production environment and enable solutions for business

**Skills**:

- Hadoop Ecosystem (HDFS)
- Big Data
- Big Data & Analytics Operation
- Data Analysis
- Data Architecture

**What you can expect from us**:
**Insights you can act on**

While technology is at the heart of our clients' digital transformation, we understand that people are at the heart of business success.

When you join CGI, you become a trusted advisor, collaborating with colleagues and clients to bring forward actionable insights that deliver meaningful and sustainable outcomes. We call our employees "members" because they are CGI shareholders and owners, and, as owners, we enjoy working and growing together to build a company we are proud of. This has been our Dream since 1976, and it has brought us to where we are today-one of the world's largest independent providers of IT and business consulting services.

At CGI, we recognize the richness that diversity brings. We strive to create a work culture where everyone belongs, and we collaborate with clients in building more inclusive communities. As an equal opportunity employer, we empower all our members to succeed and grow. If you require an accommodation at any point during the recruitment process, please let us know. We will be happy to assist.

Ready to become part of our success story? Join CGI-where your ideas and actions make a difference.

  • Kafka

    4 weeks ago

    Bengaluru, India Capleo Global Solutions Pvt LTD Full time

    Job Description - Kafka Streaming - Bigdata, Hadoop, HDFS and Hive.

  • Sdet - Bigdata

    1 month ago

    Bengaluru, India Uptycs Full time

    Strong experience on testing Hadoop based Bigdata components running in a microservices environment. Strong programming skills in python is a must. Expertise in Kafka, Zookeeper, Postgres, Redis, Trino/impala, Http proxy etc Expertise in ETL, AWS, Kubernetes, databases, SQL and Linux. Expertise in Go, nodejs is a plus. Excellent troubleshooting skills and a...

  • Unison Consulting

    4 weeks ago

    Bengaluru, India Unison consulting pte ltd Full time

    We are seeking a talented Big Data Engineer to join our team and play a key role in designing, developing, and maintaining our big data infrastructure. You will leverage your expertise in Java, Hadoop, and Kafka to build efficient data pipelines, handle real-time data streams, and ensure high-quality data processing for our data analytics...

  • Hadoop Developer

    2 weeks ago

    Bengaluru, Karnataka, India timesjobs Full time

    Hadoop DeveloperBangalore Mar - 6Gain a good working knowledge for Java Applications including its flow, configuration and other technical aspects.Maintain the configuration for these Software Application.4 to 6 years of ETL Experience with ETL tools like Talend, Informatica etc. Strong understanding of SQL, relational database concepts , data modeling ,...

  • Gcp + Bigdata

    1 week ago

    Bengaluru, India LTIMindtree Full time

    Primary Skills: Hands on experience is mandatory for the GCP, Python/Java, Spark, SQL and Bigdata. Expereince- 5 to 12 years Location- Gurgaon & Bangalore • Must Have Qualifications: o Masters in computer applications or equivalent OR Bachelor degree in engineering or computer science or equivalent. o Deep understanding of Hadoop and Spark Architecture...

  • Hadoop Developer

    1 week ago

    Bengaluru/ Bangalore, India timesjobs Full time

    Hadoop DeveloperBangalore Mar - 6Gain a good working knowledge for Java Applications including its flow, configuration and other technical aspects.Maintain the configuration for these Software Application.4 to 6 years of ETL Experience with ETL tools like Talend, Informatica etc. Strong understanding of SQL, relational database concepts , data modeling ,...

  • GCP + Bigdata

    2 months ago

    Bengaluru, India LTIMindtree Full time

    Primary Skills: Hands on experience is mandatory for the GCP, Python/Java, Spark, SQL and Bigdata.Expereince- 5 to 12 yearsLocation- Gurgaon & Bangalore• Must Have Qualifications:o Masters in computer applications or equivalent OR Bachelor degree in engineering or computer science or equivalent.o Deep understanding of Hadoop and Spark Architecture and its...

  • GCP + Bigdata

    2 months ago

    Bengaluru, India LTIMindtree Full time

    Primary Skills: Hands on experience is mandatory for the GCP, Python/Java, Spark, SQL and Bigdata. Expereince- 5 to 12 years Location- Gurgaon & Bangalore • Must Have Qualifications: o Masters in computer applications or equivalent OR Bachelor degree in engineering or computer science or equivalent. o Deep understanding of Hadoop and Spark Architecture and...

  • Bigdata ETL-Testing

    4 weeks ago

    Bengaluru, India RiskInsight Consulting Pvt Ltd. Full time

    Hands on with In-depth knowledge in BigData ETL Testing Experience in Hadoop Hive is a must ETL test automation experience Cloudera experience is good to have Python/spark experience is added advantage Plan and execute activities in relation to test preparation, automation, execution, and reporting Develop Test specifications based on various requirement...

  • BigData Engineer

    1 month ago

    Bengaluru, India Fractal Full time

    It's fun to work in a company where people truly BELIEVE in what they are doing!The BigData Engineers have expertise in building horizontally scalable applications using distributed technologies like NoSQL Dbs/Hadoop/Spark and others and we execute projects on-premise and cloud based systems. The AI-Engineers and MLOps Engineers work on scaling AI-systems...

  • BigData-Hadoop/Scala Engineer

    4 days ago

    Bengaluru, India F335 Deutsche India Private Limited, Bangalore Branch Full time

    DescriptionSenior Engineer is responsible for developing and delivering elements of engineering solutions to accomplish business goals. Awareness is expected of the important engineering principles of the Bank. Root cause analysis skills, developed through addressing enhancements and fixes to products. Build reliability and resiliency into solutions through...

  • Senior Data Engineer

    2 weeks ago

    Bengaluru, India Prescience Decision Solutions Full time

    Job DescriptionContribute to the development and maintenance of ETL jobs and components utilizing Apache Spark, Python, Pyspark, Talend, Kafka, Hadoop, Airflow, and GCP technologies.Engage in requirement gathering, analysis, and translation of functional requirements into technical tasks using Spark/Airflow.Produce comprehensive Technical ETL design...

  • Cloud Support Eng. I

    1 month ago

    Bengaluru, Karnataka, India AWS India - Karnataka Full time

    Bachelor’s degree OR equivalent experience in a technical position - Requires minimum of 2+ yrs experience on Hadoop/ML/Large Dataset handling with DB/DW experience - Advanced experience in Apache Hadoop, Apache Spark, Apache Hive, and Presto or ETL skills with working experience on Databases Oracle/SQL Server/ PostgreSQL/TerraData - Experience with...

  • Confluent Kafka

    1 month ago

    Bengaluru, India CIEL HR Services Full time

    **Looking for Immediate Joiners to 20 days NP** **Years of Experience **9+ years **Job Description**: - Must have 9+ years of experience with client facing and Team handling Capabilities. - Must Have Very good Communication Skills in order understand client requirements and implement the same along with the team. - Provide expertise and hands on...

  • Java+Kafka Expert

    6 days ago

    Bengaluru, India Direct Client Full time

    Java + Kafka expertBengaluru, DigitalJOB DESCRIPTIONRoles & Responsibilities· 6 plus years of experience with at least 3 years of relevant experience· Hands-on experience in big data computation technologies such as Kafka Streaming, Apache Kafka, Confluent, with a focus oncreating and managing multiple topics on Kafka clusters· Strong knowledge of data...

  • Java+Kafka Expert

    6 days ago

    Bengaluru, India Direct Client Full time

    Java + Kafka expertBengaluru, DigitalJOB DESCRIPTIONRoles & Responsibilities· 6 plus years of experience with at least 3 years of relevant experience· Hands-on experience in big data computation technologies such as Kafka Streaming, Apache Kafka, Confluent, with a focus oncreating and managing multiple topics on Kafka clusters· Strong knowledge of data...

  • Hadoop Admin

    2 months ago

    Bengaluru, India Nityo Infotech Full time

    Experience of Hadoop Admin Activities for Cloudera Platform.Experience of setting up continuous integration/deployment of spark jobs to EMR clustersKnowledge of scheduling spark applications in AWS EMR cluster.Understanding the differences between Hadoop Mapreduce and Apache SparkProficient understanding of code versioning tools as Git, SVNDesign,...

  • Cloud Support Eng. I

    1 month ago

    Bengaluru, Karnataka, India AWS India - Karnataka Full time

    Bachelor’s degree OR equivalent experience in a technical position - Requires minimum of 2+ yrs experience on Hadoop/ML/Large Dataset handling with DB/DW experience - Advanced experience in Apache Hadoop, Apache Spark, Apache Hive, and Presto or ETL skills with working experience on Databases Oracle/SQL Server/ PostgreSQL/TerraData - Experience with...

  • BigData-Hadoop/Scala Engineer

    5 days ago

    Bengaluru, India F335 Deutsche India Private Limited, Bangalore Branch Full time

    Description Senior Engineer is responsible for developing and delivering elements of engineering solutions to accomplish business goals. Awareness is expected of the important engineering principles of the Bank. Root cause analysis skills, developed through addressing enhancements and fixes to products. Build reliability and resiliency into solutions...

  • Kafka Developer

    4 days ago

    Bengaluru, India Horigine Consulting Pvt. Ltd Full time

    DescriptionWe are seeking a highly skilled Kafka Developer to join our team. As a Kafka Developer, you will be responsible for designing, developing, and implementing Kafka-based solutions for our organization. You will collaborate with cross-functional teams to understand business requirements, architect robust Kafka solutions, and ensure the reliable and...