Big Data

14 hours ago


Noida, India Iris software Full time

JD:

- Minimum EAP experience - 7 years.- Responsibilities for Big Data Engineer- Perform data cleaning, integration, validation and analysis
- Extensive experience of developing(must), deploying (must) and maintaining(must) the Big Data Ecosystem
- Create and maintain optimal data pipeline architecture
- Assemble large, complex data sets that meet functional / non-functional business requirements
- Create complex data processing jobs in PySpark to load data from RDBMS (Oracle) & process based on complex business rules (must)
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
- Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues (must) and support their data infrastructure needs.
- Qualifications for Big Data Engineer:
- Strong track record of building and optimizing ‘big data’ data pipelines, architectures and data sets.
- Working SQL knowledge and experience working with relational databases
- Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
- Strong analytic skills related to working with structured datasets.
- Strong track record of building in build processes supporting data transformation, data structures, metadata, dependency and workload management.
- Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores (optional)
- Experience in supporting and working with cross-functional teams in a dynamic environment.
- Tech Stack
- Experience with big data tools:

- Hadoop Ecosytem (must),Hive (must)
- Impala (must)
- Sqoop (must)
- Python (must)
- Spark with Python (must)
- PySPARK (must) - experience must in Spark Tuning as well
- SPARK-SQL (must)- Any one RDBMS (good to have)
- Any cloud platform (S3 or Azure)
- Rich experience in scripting language Python (must)
- Rich experience in building optimal pySpark jobs handling huge data load (must)
- Rich experience in performing capacity management and gauging memory needs for huge data processing (must)
- Experience with stream-processing systems: Storm, Spark-Streaming, etc. (good to have)

**Job Summary**:
**Role Based Competencies**:

- Tech - Requirement Management S/W- Tech - NFR- Tech - High level design- Tech - Code quality and coding standards- Beh - Result Orientation- Tech - Code Reviews- Tech - Build Management- Tech - Unit Testing- Beh - Information Seeking- Tech - Agile Methodology- Tech - Analytical \ Problem Solving- Beh - Communication- Beh - Customer Orientation- Beh - Collaboration**Mandatory Competencies**:

- Big Data - Hadoop- Big Data - Hive- Big Data - Impala- Big Data - PySpark**Good to Have Competencies**:


  • Big Data

    5 hours ago


    Noida, Uttar Pradesh, India Iris Software Inc. Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Why Join Us?Are you inspired to grow your career at one ofIndia's Top 25 Best Workplaces in IT industry?Do you want to do the best work of your life at one of thefastest growing IT services companies? Do you aspire to thrive in an award-winning work culture thatvalues your talent and career aspirations? It'shappening right hereat Iris Software.About Iris...

  • Big Data Developer

    4 days ago


    Noida, India Coforge Full time

    We are seeking a highly skilled and experienced Senior Big Data Engineer with strong expertise in Streaming Data and Big Data Technologies, AWS & Scala Mandatory. The ideal candidate will have hands-on experience in architecting and implementing scalable data solutions using modern cloud-native tools and frameworks.Key Responsibilities:- Architect and...

  • Big Data Developer

    5 days ago


    Noida, India Coforge Full time

    We are seeking a highly skilled and experienced Senior Big Data Engineer with strong expertise in Streaming Data and Big Data Technologies , AWS & Scala Mandatory . The ideal candidate will have hands-on experience in architecting and implementing scalable data solutions using modern cloud-native tools and frameworks. Key Responsibilities: Architect and...

  • Big Data Developer

    3 days ago


    Noida, India Coforge Full time

    We are seeking a highly skilled and experienced Senior Big Data Engineer with strong expertise in Streaming Data and Big Data Technologies , AWS & Scala Mandatory . The ideal candidate will have hands-on experience in architecting and implementing scalable data solutions using modern cloud-native tools and frameworks. Key Responsibilities: Architect and...

  • Big Data Developer

    5 days ago


    Noida, India Coforge Full time

    We are seeking a highly skilled and experienced Senior Big Data Engineer with strong expertise in Streaming Data and Big Data Technologies , AWS & Scala Mandatory . The ideal candidate will have hands-on experience in architecting and implementing scalable data solutions using modern cloud-native tools and frameworks.Key Responsibilities:Architect and...

  • Big Data Developer

    3 days ago


    Noida, India Coforge Full time

    We are seeking a highly skilled and experienced Senior Big Data Engineer with strong expertise in Streaming Data and Big Data Technologies , AWS & Scala Mandatory . The ideal candidate will have hands-on experience in architecting and implementing scalable data solutions using modern cloud-native tools and frameworks.Key Responsibilities:Architect and...

  • Big Data Developer

    5 days ago


    Noida, India Coforge Full time

    We are seeking a highly skilled and experienced Senior Big Data Engineer with strong expertise in Streaming Data and Big Data Technologies, AWS & Scala Mandatory. The ideal candidate will have hands-on experience in architecting and implementing scalable data solutions using modern cloud-native tools and frameworks.Key Responsibilities:Architect and...

  • Big data developer

    4 days ago


    Noida, India Coforge Full time

    We are seeking a highly skilled and experienced Senior Big Data Engineer with strong expertise in Streaming Data and Big Data Technologies , AWS & Scala Mandatory. The ideal candidate will have hands-on experience in architecting and implementing scalable data solutions using modern cloud-native tools and frameworks.Key Responsibilities:Architect and...

  • Big data developer

    5 days ago


    Noida, India Coforge Full time

    We are seeking a highly skilled and experienced Senior Big Data Engineer with strong expertise in Streaming Data and Big Data Technologies , AWS & Scala Mandatory. The ideal candidate will have hands-on experience in architecting and implementing scalable data solutions using modern cloud-native tools and frameworks.Key Responsibilities:Architect and...

  • Big Data Developer

    5 days ago


    Noida, India Coforge Full time

    We are seeking a highly skilled and experienced Senior Big Data Engineer with strong expertise in Streaming Data and Big Data Technologies, AWS & Scala Mandatory. The ideal candidate will have hands-on experience in architecting and implementing scalable data solutions using modern cloud-native tools and frameworks.Key Responsibilities:Architect and...