Bigdata Specialist

5 days ago

Chennai, Tamil Nadu, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000
Data Engineer Role Summary

We are seeking an experienced Data Engineer to join our team. The ideal candidate will have a strong background in designing, building, and supporting data applications using big data technologies such as Spark, Sqoop, and Hive.

The Data Engineer will be responsible for developing and delivering complex software requirements to accomplish business goals. This includes ensuring that software is developed to meet functional, non-functional, and compliance requirements.

The role involves collaborating with data scientists, analysts, and engineers to implement scalable solutions. It also requires analyzing large datasets to extract meaningful insights and patterns, and ensuring data quality, security, and governance in all processes.

A key responsibility of the Data Engineer is to design and optimize data pipelines and workflows for big data systems. They must develop components, application interfaces, and solution enablers while ensuring principal architecture integrity is maintained.

Furthermore, the Data Engineer will participate in estimating work necessary to realize a story requirement through the software delivery lifecycle. They must ensure solutions are well-designed with maintainability, ease of integration, and testing built-in from the outset.

Required Skills:
- 5 years of hands-on experience in designing, building, and supporting data applications using Spark, Sqoop, and Hive
- Bachelor or masters degree in computer science or related field
- Strong knowledge of working with large data sets and high-capacity big data processing platforms
- Advanced knowledge of the Hadoop ecosystem and its components
- In-depth knowledge of Hive, Shell scripting, Python, Spark, and MapReduce jobs
- Experience using Job Schedulers like Autosys
- Good knowledge on Hadoop Architecture and HDFS
- Strong experience in Unix and Shell scripting
- Experience with Jenkins for Continuous Integration
- Experience using Source Code and Version Control Systems like Bitbucket and Git

Nice to Have Skills:
- Experience in international and multicultural environments
- AML Domain knowledge

  • Data Testing

    4 weeks ago

    Chennai, Tamil Nadu, India Citigroup Full time

    The Testing Analyst 2 is a developing professional role Applies specialty area knowledge in monitoring assessing analyzing and or evaluating processes and data Identifies policy gaps and formulates policies Interprets data and makes recommendations Researches and interprets factual information Identifies inconsistencies in data or results defines...

  • 3 Days Left Python Automation Testing

    3 weeks ago

    Chennai, Tamil Nadu, India Citigroup Full time

    The IT Quality Analyst 2 is a developing professional role Applies specialty area knowledge in monitoring assessing analyzing and or evaluating processes and data Identifies policy gaps and formulates policies Interprets data and makes recommendations Researches and interprets factual information Identifies inconsistencies in data or results defines...

  • Bigdata Developer Specialist

    4 days ago

    Gurgaon / Gurugram, Pune, Chennai, India beBeeData Full time ₹ 8,00,000 - ₹ 20,00,000

    Job RoleWe are seeking a motivated Data Analyst to join our dynamic team.The ideal candidate will play a key role in developing and maintaining data solutions, focusing on data processing and analytics to drive business insights.ResponsibilitiesDesigning and implementing data pipelines using Big Data technologies.Developing and optimizing data models to...

  • Data Scientist

    2 days ago

    Chennai, India Luxoft Full time

    **Project** Description**: As part of our strategic partnership with one of the biggest financial institutions in the World, we are hiring various IT specialists, who will become part of their new IT Service Center. The bank is an international organization dedicated to providing financing, advice, and research to developing nations to aid their economic...

  • Data Architectural Specialist

    4 days ago

    Hyderabad / Secunderabad, Telangana, Pune, Chennai, India beBeeDatabricksfocused Full time

    Job Title: Databricks DeveloperOverviewAs a highly skilled Databricks Developer, you will play a key role in driving our big data strategy and delivering innovative solutions to drive business growth.Key ResponsibilitiesDesign, develop, and deploy scalable and efficient Databricks workloads using Python, Spark, and Hadoop platforms.Collaborate with...