Sr. Spark

2 weeks ago

Hyderabad, Telangana, India Anicalls (Pty) Ltd Full time

Candidate should be able to:
Coordinate Development, Integration, and Production deployments.
Optimize Spark code, Impala queries, and Hive partitioning strategy for better scalability, reliability, and performance.
Build applications using Maven, SBT and integrated with continuous integration servers like Jenkins to build jobs.
Execute Hadoop ecosystem and Applications through Apache HUE
Build Machine Learning Algorithms using Spark.
Perform migration from Legacy Databases RDBMS to Hadoop Ecosystem
Create mapping documents to outline data flow from source to target.
Use Cloudera Manager, an end-to-end tool to manage Hadoop operations in Cloudera Cluster
Design and deploy enterprise-wide scalable operations
Work on leading BI technologies like MSTR, Tableau over Hadoop Ecosystem through ODBC/JDBC connection
Perform Performance tuning of Impala queries

Work on hive performance optimizations like using distributed cache for small datasets, Partition, Bucketing in Hive, and Map Side joins.

Create various database objects like tables, views, functions, and triggers using SQL
Understand business needs, analyze functional specifications and map those to development and designing Apache Spark programs and algorithms.
Install, configure, and use Hadoop components like Spark, Spark Job server, Spark Thrift server, Phoenix on HBase, Flume, Sqoop
Write SPARK jobs to fetch large data volumes from the source
preparing technical specifications, analyzing functional specs, development, and maintenance of code
Develop end to end data pipeline using Spark, Hive, and Impala
Design and documented operational problems by following standards and procedures using software reporting tool JIRA
Use Rest services to access HBASE data and used the data for further processing in the downstream systems
Data wrangling and creating workable datasets and work on different file formats like Parquet, ORC, Sequence files, and different serialization formats like Avro
Perform Feasibility Analysis (For the deliverables) - Evaluating the feasibility of the requirements against complexity and timelines.
complete the full lifecycle of software development and deliver on time
work with end-users to gather requirements and convert them to working documents
interface with various solution/business areas to understand the requirements and prepare documentation to support development
work in a fast-paced, team-oriented environment

Candidate should have:
Experience in Hadoop, HBase, MongoDB, or other NoSQL platforms
Experience with Spark and Spark SQL
Excellent communication skills with both Technical and Business audience
hands-on experience in Java, Spark, Scala, AKKA, Hive, Maven/SBT, Amazon S3
Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala
Good experience in debugging issues using the Hadoop, Spark Log files
Knowledge in Sqoop, Flume preferred
Experience in Kafka, ReST services is a plus.
Experience in Apache Phoenix, Text Search (Solr, ElasticSearch, CloudSearch)
Expertise in Shell-Scripts, Cron Automation, and Regular Expressions
3+ years strong native SQL skills
1+ years experience with Hadoop, Hive, Impala, HBase, and related technologies, MapReduce/YARN, Lambda architectures, MPP shared-nothing database systems, and NoSQL systems
3+ years experience with Scala, Spark, Linux

  • Revrr - Data Engineer - Scala/Spark

    2 weeks ago

    Hyderabad, Telangana, India REVRR TECHNOLOGIES PRIVATE LIMITED Full time

    Company : Revrr TechnologiesEmployment Type : Full-TimeRole : Software Development EngineerPosition : Developer, Senior Developer, and Lead DeveloperProject : Maps and Advertising Platforms ProductExperience: SrSDE:4-8 YearsKey Skills : Spark Core/Streaming with Any programming language Scala/ Java/ Python, Good Understanding of Data Structures, Algorithms,...

  • Sr Engineer Data Engineering

    2 weeks ago

    Hyderabad, Telangana, India TransUnion LLC Full time

    What We Bring to the Table: Looking for a Data Engineer to merge big data, marketing analysis, and database tech for top-notch insights. Need someone hands-on with data, keen on database product development, and good at teamwork. If you're curious about various industries, passionate about data analytics, a SQL lover, and skilled in creating efficient...

  • Sr Engineer Data Engineering

    2 weeks ago

    Hyderabad, Telangana, India TransUnion LLC Full time

    What We'll Bring:We are looking for Data Engineer to bring big data, marketing analytics, and database technology together to deliver best-in-class insights. You must be a hands-on data guru who's passionate about data, database product development, and play well with others. If you have a curiosity for different industries and companies, a passion for data...

  • Hadoop Developer

    2 weeks ago

    Hyderabad, Telangana, India Virtual Labs Inc. Full time

    Job Title: Sr Hadoop DeveloperLocation: Hyderabad, India (Hybrid)Experience: 4 to 8 yrs.•Hands on exp Hadoop, SQL, PY Spark and HDFSClient is looking for a candidate who is more proficient in Hadoop development with some testing experience

  • Sr Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Syniverse Full time

    Syniverse is the world's most connected company. Whether we're developing the technology that enables intelligent cars to safely react to traffic changes or freeing travelers to explore by keeping their devices online wherever they go, we believe in leading the world forward.Which is why we work with some of the world's most recognized brands. Eight of the...

  • Sr Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Syniverse Full time

    Syniverse is the world's most connected company. Whether we're developing the technology that enables intelligent cars to safely react to traffic changes or freeing travelers to explore by keeping their devices online wherever they go, we believe in leading the world forward.Which is why we work with some of the world's most recognized brands. Eight of the...

  • Digital Engineering Sr. Engineer

    4 weeks ago

    Hyderabad, Telangana, India NTT DATA Services Full time

    Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Digital Engineering Sr. Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN).• Experience in...

  • Digital Engineering Sr. Engineer

    1 month ago

    Hyderabad, Telangana, India NTT DATA Services Full time

    Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Digital Engineering Sr. Engineer to join our team in Hyderabad, Telangana (IN-TG), India (IN).• Experience in...

  • Sr. Software Engineer

    2 weeks ago

    Hyderabad, Telangana, India YASH Technologies Full time

    YASH Technologies is a leading technology integrator specializing in helping clients reimagine operating models, enhance competitiveness, optimize costs, foster exceptional stakeholder experiences, and drive business transformation.At YASH, we're a cluster of the brightest stars working with cutting-edge technologies. Our purpose is anchored in a single...

  • Sr. Staff Software Engineer

    2 weeks ago

    Hyderabad, Telangana, India 451 Discovery Comm. India Full time

    Who We Are... When we say, "the stuff dreams are made of," we're not just referring to the world of wizards, dragons and superheroes, or even to the wonders of Planet Earth. Behind WBD's vast portfolio of iconic content and beloved brands, are the bringing our characters to life, the bringing them to your living rooms and the creating what's next... ...

  • Sr. Staff Software Engineer

    4 weeks ago

    Hyderabad, Telangana, India WARNERMEDIA Full time

    Every great story has a new beginning, and yours starts here.Welcome to Warner Bros. Discovery... the stuff dreams are made of.Who We Are... When we say, "the stuff dreams are made of," we're not just referring to the world of wizards, dragons and superheroes, or even to the wonders of Planet Earth. Behind WBD's vast portfolio of iconic content and beloved...

  • Sr. Staff Software Engineer

    4 weeks ago

    Hyderabad, Telangana, India WARNERMEDIA Full time

    Welcome to Warner Bros. Discovery... the stuff dreams are made of.Who We Are... When we say, "the stuff dreams are made of," we're not just referring to the world of wizards, dragons and superheroes, or even to the wonders of Planet Earth. Behind WBD's vast portfolio of iconic content and beloved brands, are the storytellers bringing our characters to life,...

  • Sr. Staff Software Engineer

    1 month ago

    Hyderabad, Telangana, India WARNERMEDIA Full time

    Every great story has a new beginning, and yours starts here.Welcome to Warner Bros. Discovery... the stuff dreams are made of.Who We Are... When we say, "the stuff dreams are made of," we're not just referring to the world of wizards, dragons and superheroes, or even to the wonders of Planet Earth. Behind WBD's vast portfolio of iconic content and beloved...

  • Sr. Staff Software Engineer

    1 month ago

    Hyderabad, Telangana, India WARNERMEDIA Full time

    Welcome to Warner Bros. Discovery... the stuff dreams are made of.Who We Are... When we say, "the stuff dreams are made of," we're not just referring to the world of wizards, dragons and superheroes, or even to the wonders of Planet Earth. Behind WBD's vast portfolio of iconic content and beloved brands, are the storytellers bringing our characters to life,...

  • Sr. Software Engineer

    2 weeks ago

    Hyderabad, Telangana, India CDK Global Full time

    Position Title: Sr Data EngineerLocation - Hyderabad, IndiaAbout CDK Global:CDK Global is the largest global provider of integrated information technology and digital marketing solutions to automotive dealerships and manufacturers in more than 100 countries worldwide for the world's biggest car brands.Although we operate on a global scale, we are small by...

  • Sr. Software Engineer

    2 weeks ago

    Hyderabad, Telangana, India CDK Global Full time

    Position Title: Sr Data EngineerLocation - Hyderabad, IndiaAbout CDK Global:CDK Global is the largest global provider of integrated information technology and digital marketing solutions to automotive dealerships and manufacturers in more than 100 countries worldwide for the world's biggest car brands. Although we operate on a global scale, we are small by...

  • Sr GCP Data Engineer, SMAI

    2 weeks ago

    Hyderabad, Telangana, India Micron Full time

    Our vision is to transform how the world uses information to enrich life for all. Micron Technology is a world leader in innovating memory and storage solutions that accelerate the transformation of information into intelligence, inspiring the world to learn, communicate and advance faster than ever. JR47217 Sr GCP Data Engineer, SMAI As a Sr. Data...

  • Sr. Analytics Engineer

    2 weeks ago

    Hyderabad, Telangana, India 451 Discovery Comm. India Full time

    Who We Are...When we say, "the stuff dreams are made of," we're not just referring to the world of wizards, dragons and superheroes, or even to the wonders of Planet Earth. Behind WBD's vast portfolio of iconic content and beloved brands, are the bringing our characters to life, the bringing them to your living rooms and the creating what's next...From...

  • Sr. Staff Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India 451 Discovery Comm. India Full time

    Who We Are...When we say, "the stuff dreams are made of," we're not just referring to the world of wizards, dragons and superheroes, or even to the wonders of Planet Earth. Behind WBD's vast portfolio of iconic content and beloved brands, are the bringing our characters to life, the bringing them to your living rooms and the creating what's next...From...

  • Senior Cloud Operations Engineer

    2 weeks ago

    Hyderabad, Telangana, India Dun & Bradstree Full time

    GCP - Sr. DevOps Engineer– Job Description Key responsibilities: 1) Outline and create application architectures for re-platform or re-architect strategies and implement proposals and patterns for common application architectures.2) Propose and design solutions to meet the technical performance, scalability, flexibility, operations and capability needs3)...