Spark/Scala
1 day ago
Role:Senior Developer
Required Technical Skill Set: Spark/Scala/Unix
Experience Range: 5-8 years
Location of Requirement : Pune
Must-Have
Minimum 4+ years of experience in development of Spark Scala
Experience in designing and development of solutions for Big Data using Hadoop ecosystem technologies such as with Hadoop Bigdata components like HDFS, Spark, Hive Parquet File format, YARN, MapReduce, Sqoop
Good Experience in writing and optimizing Spark Jobs, Spark SQL etc. Should have worked on both batch and streaming data processing.
Experience in writing and optimizing complex Hive and SQL queries to process huge data. good with UDFs, tables, joins, Views etc
Experience in debugging the Spark code
Working knowledge of basic UNIX commands and shell script
Experience of Autosys, Gradle
Good-to-Have 1. Good analytical and debugging skills
Ability to coordinate with SMEs, stakeholders, manage timelines, escalation & provide on time status
Write clear and precise documentation / specification
Work in an agile environment
Create documentation and document all developed mappings
SN Responsibility of / Expectations from the Role
1 Create Scala/Spark jobs for data transformation and aggregation
2 Produce unit tests for Spark transformations and helper methods
3 Write Scaladoc-style documentation with all code
4 Design data processing pipelines
Location
Pune
Job Function
TECHNOLOGY
Role
Engineer
Job Id
381936
Desired Skills
Scala | Spark
Desired Candidate Profile
Qualifications : BACHELOR OF ENGINEERING
-
Sr. Scala Engineer
2 weeks ago
Pune, Maharashtra, India Persistent Full timeAbout Position:As a Senior Scala Engineer, youll be part of a team of smart, highly skilled technologists who are passionate about learning and supporting utting-edge technologies such as Spark, Scala, Pyspark, Databricks, Airflow, SQL, Docker, Kubernetes, and other Data engineering tools.These technologies are deployed using DevOps pipelines leveraging...
-
Bigdata Scala Engineer Vice president
2 weeks ago
Pune, Maharashtra, India Citi Full timeLead the architecture, design, and development of high-performance, scalable, and reliable big data processing systems using Scala and Apache Spark. Drive technical vision and strategy for big data initiatives, ensuring alignment with overall company goals and industry best practices. Evaluate and recommend new technologies and tools to enhance our big data...
-
Hadoop,Spark Data Engineer
1 week ago
Pune, Maharashtra, India HSBC Full timeSome careers shine brighter than others.If you're looking for a career that will help you stand out, join HSBC and fulfil your potential. Whether you want a career that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities, support and rewards that will take you further.HSBC is one of the largest banking and...
-
Data Engineer/Spark Engineer
2 days ago
Pune, Maharashtra, India OPUS Technologies Full timeWho are we?Opus Technologies is a technology company with a focus on shaping the future of payment technology. With experience building highly innovative solutions and products, we combine our deep technology proficiency with unmatched domain expertise in payments and fintech, enabling us to deliver unparalleled quality and value in everything we do. For the...
-
Bigdata Scala Engineer Vice president
1 week ago
Pune, Maharashtra, India Citi Full timeVice President, Big Data Scala EngineerLocation: Pune About the Role:We are seeking an experienced and highly skilled Vice President, Big Data Scala Engineer to lead the design, development, and implementation of our next-generation big data platforms and applications. The ideal candidate will be a technical leader with a strong background in Scala, Spark,...
-
Sr. Spark Big Data Developer
2 weeks ago
Pune, Maharashtra, India CRUTZ LEELA ENTERPRISES Full timeSr. Spark Big Data DeveloperPFB the detailed JD:�� Experience: 5 to 7+ years (Hands-on)�� Notice Period: 0-30 DaysMust Have: Proficiency in at least one of the following programming languages:Java/Scala/Python Good understanding of SQL Experience of development and deployment of at least one end-to-end datastorage/processing pipeline...
-
Lead Data Engineer
6 days ago
Pune, Maharashtra, India CG-VAK Software & Exports Ltd. Full timeJob Title:Data Engineer (Scala, Spark, Pyspark, Databricks)Remark - final round will be the F2FExperience: 10+ Years (Lead Role)ResponsibilitiesDesign, develop, and maintain robust and scalable data pipelines using Apache Spark and Scala on the Databricks platform.Implement ETL (Extract, Transform, Load) processes for various data sources, ensuring data...
-
Big Data Engineer
2 weeks ago
Pune, Maharashtra, India Digital Placement Full timePrimary – Scala with Spark Hdfs ,Spark with scala , Scala (Functional Programming ), Hbase , Sql , Sqoop ,Shell Script Optional : Apache Kafka , Akka Http , TeamCity , U-deploy , Autosys , SBT Good to have : Pivotal Gemfire 7+ years of strong experience as big data developer. Hands on knowledge of Shell Scripting. Familiar with...
-
Aws Data Engineer
1 week ago
Pune, Maharashtra, India Tech Mahindra Full timeAWS Data EngineerExperience: 3-8 YearsLocation: Pune (Kharadi)About the RoleWe are looking for a highly skilled Spark SQL Developer with a solid background in Data Engineering. The candidate should have hands-on experience in building and optimizing large-scale data pipelines using Apache Spark, along with strong expertise in SQL and ETL processes.Key...
-
Data Engineer
1 week ago
Pune, Maharashtra, India HSBC Full timeSome careers shine brighter than others.If you're looking for a career that will help you stand out, join HSBC and fulfil your potential. Whether you want a career that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities, support and rewards that will take you further.HSBC is one of the largest banking and...