Sr. Spark/ Hadoop/Scala Engineer...

1 day ago

Bengaluru, India Anicalls (Pty) Ltd Full time
Candidate should be able to:
Coordinate Development, Integration, and Production deployments.
Optimize Spark code, Impala queries, and Hive partitioning strategy for better scalability, reliability, and performance.
Build applications using Maven, SBT and integrated with continuous integration servers like Jenkins to build jobs.
Execute Hadoop ecosystem and Applications through Apache HUE
Build Machine Learning Algorithms using Spark.
Perform migration from Legacy Databases RDBMS to Hadoop Ecosystem
Create mapping documents to outline data flow from source to target.
Use Cloudera Manager, an end-to-end tool to manage Hadoop operations in Cloudera Cluster
Design and deploy enterprise-wide scalable operations
Work on leading BI technologies like MSTR, Tableau over Hadoop Ecosystem through ODBC/JDBC connection
Perform Performance tuning of Impala queries
Work on hive performance optimizations like using distributed cache for small datasets, Partition, Bucketing in Hive, and Map Side joins.
Create various database objects like tables, views, functions, and triggers using SQL
Understand business needs, analyze functional specifications and map those to development and designing Apache Spark programs and algorithms.
Install, configure, and use Hadoop components like Spark, Spark Job server, Spark Thrift server, Phoenix on HBase, Flume, Sqoop
Write SPARK jobs to fetch large data volumes from the source
preparing technical specifications, analyzing functional specs, development, and maintenance of code
Develop end to end data pipeline using Spark, Hive, and Impala
Design and documented operational problems by following standards and procedures using software reporting tool JIRA
Use Rest services to access HBASE data and used the data for further processing in the downstream systems
Data wrangling and creating workable datasets and work on different file formats like Parquet, ORC, Sequence files, and different serialization formats like Avro
Perform Feasibility Analysis (For the deliverables) - Evaluating the feasibility of the requirements against complexity and timelines.
complete the full lifecycle of software development and deliver on time
work with end-users to gather requirements and convert them to working documents
interface with various solution/business areas to understand the requirements and prepare documentation to support development
work in a fast-paced, team-oriented environment
Candidate should have:
Experience in Hadoop, HBase, MongoDB, or other NoSQL platforms
Experience with Spark and Spark SQL
Excellent communication skills with both Technical and Business audience
hands-on experience in Java, Spark, Scala, AKKA, Hive, Maven/SBT, Amazon S3
Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala
Good experience in debugging issues using the Hadoop, Spark Log files
Knowledge in Sqoop, Flume preferred
Experience in Kafka, ReST services is a plus.
Experience in Apache Phoenix, Text Search (Solr, ElasticSearch, CloudSearch)
Expertise in Shell-Scripts, Cron Automation, and Regular Expressions
3+ years strong native SQL skills
1+ years experience with Hadoop, Hive, Impala, HBase, and related technologies, MapReduce/YARN, Lambda architectures, MPP shared-nothing database systems, and NoSQL systems
3+ years experience with Scala, Spark, Linux

  • Data Engineer

    2 weeks ago

    Bengaluru, India Tata Consultancy Services Full time

    Dear CandidateGreetings from TATA Consultancy ServicesJob Openings at TCSSkill - Data Engineer ( Spark Scala)Exp range – 5 yrs – 10 yrsLocation- Chennai /Pune/Bangalore/Hyderabad /DelhiNotice period – 30 daysPls find the Job Description below.Big Data Hadoop - Hive and Spark/Scala solid experienceSQL advance knowledge - Been able to test changes and...

  • Data engineer

    5 days ago

    Bengaluru, India Tata Consultancy Services Full time

    Dear CandidateGreetings from TATA Consultancy ServicesJob Openings at TCSSkill - Data Engineer (Spark Scala)Exp range – 5 yrs – 10 yrsLocation- Chennai /Pune/Bangalore/Hyderabad /DelhiNotice period – 30 daysPls find the Job Description below.Big Data Hadoop - Hive and Spark/Scala solid experienceSQL advance knowledge - Been able to test changes and...

  • Data Engineer

    1 week ago

    Bengaluru, India Tata Consultancy Services Full time

    Dear CandidateGreetings from TATA Consultancy ServicesJob Openings at TCSSkill - Data Engineer (Spark Scala)Exp range – 5 yrs – 10 yrsLocation- Chennai /Pune/Bangalore/Hyderabad /DelhiNotice period – 30 daysPls find the Job Description below.- Big Data Hadoop - Hive and Spark/Scala solid experience- SQL advance knowledge - Been able to test changes and...

  • Data Engineer

    1 week ago

    Bengaluru, India Tata Consultancy Services Full time

    Dear CandidateGreetings from TATA Consultancy ServicesJob Openings at TCSSkill - Data Engineer (Spark Scala)Exp range – 5 yrs – 10 yrsLocation- Chennai /Pune/Bangalore/Hyderabad /DelhiNotice period – 30 daysPls find the Job Description below.- Big Data Hadoop - Hive and Spark/Scala solid experience- SQL advance knowledge - Been able to test changes and...

  • Data Engineer

    1 week ago

    Bengaluru, India Tata Consultancy Services Full time

    Dear CandidateGreetings from TATA Consultancy ServicesJob Openings at TCSSkill - Data Engineer ( Spark Scala)Exp range – 5 yrs – 10 yrsLocation- Chennai /Pune/Bangalore/Hyderabad /DelhiNotice period – 30 daysPls find the Job Description below.Big Data Hadoop - Hive and Spark/Scala solid experienceSQL advance knowledge - Been able to test changes and...

  • Spark/Scala Developer

    1 day ago

    Bengaluru, India Maneva Consulting Pvt. Ltd Full time

    Greetings from Maneva! Job Description Job Title - Spark/Scala Developer Experience - 7 -15 Years Location - Bangalore Notice - Immediate to 15 days Requirements:- - Excellent Knowledge on Spark; The professional must have a thorough understanding Spark framework, Performance Tuning etc. - Excellent Knowledge and hands-on experience of at least 4+...

  • Spark-Scala Position

    4 days ago

    Bengaluru, Chennai, Pune, India Orcapod Consulting Services Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Position: Spark-ScalaWork location : Bangalore , Chennai, Hyderabad, Pune, Chandigarh.Type: C2H (Convert into Full-Time Employment (FTE) within 90 days)Experience: 8-10 yrs.Skillsets:Mandatory skills - Spark & ScalaDesired skills - Spark Streaming, Hadoop, Hive, SQL, Sqoop, ImpalaJoining: Immediate joiner or 15 to 30 days

  • ▷ (Urgent) Data Engineer (Spark Scala)

    2 weeks ago

    Bengaluru, India Tata Consultancy Services Full time

    Dear Candidate Greetings from TATA Consultancy Services Job Openings at TCS Skill - Data Engineer (Spark Scala) Exp range – 5 yrs – 10 yrs Location- Chennai /Pune/Bangalore/Hyderabad /Delhi Notice period – 30 days Pls find the Job Description below. - Big Data Hadoop - Hive and Spark/Scala solid experience - SQL advance knowledge - Been able to...

  • Scala Data Engineer

    2 weeks ago

    Bengaluru, India Insbytech Full time

    Hiring Scala Data Engineer (7+ yrs, Bangalore Hybrid) – Must have Scala, Spark, SQL, DS & Algo, Cloud (Azure). Preferred: Kafka, Hadoop, CI/CD, Airflow, Iceberg, NoSQL, Data Governance. Required Candidate profile Scala Data Engineer (7+ yrs) with expertise in Scala, Spark, SQL, DS & Algo, Azure. Skilled in Kafka, Hadoop, CI/CD, Airflow, NoSQL, Data...

  • Scala Data Engineer

    4 days ago

    Bengaluru, Karnataka, India Insbytech Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Hiring Scala Data Engineer (7+ yrs, Bangalore Hybrid) – Must have Scala, Spark, SQL, DS & Algo, Cloud (Azure). Preferred: Kafka, Hadoop, CI/CD, Airflow, Iceberg, NoSQL, Data Governance. Required Candidate profileScala Data Engineer (7+ yrs) with expertise in Scala, Spark, SQL, DS & Algo, Azure. Skilled in Kafka, Hadoop, CI/CD, Airflow, NoSQL, Data...