Big Data

15 hours ago

Noida, India Iris software Full time

JD:

- Minimum EAP experience - 7 years.- Responsibilities for Big Data Engineer- Perform data cleaning, integration, validation and analysis
- Extensive experience of developing(must), deploying (must) and maintaining(must) the Big Data Ecosystem
- Create and maintain optimal data pipeline architecture
- Assemble large, complex data sets that meet functional / non-functional business requirements
- Create complex data processing jobs in PySpark to load data from RDBMS (Oracle) & process based on complex business rules (must)
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
- Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues (must) and support their data infrastructure needs.
- Qualifications for Big Data Engineer:
- Strong track record of building and optimizing ‘big data’ data pipelines, architectures and data sets.
- Working SQL knowledge and experience working with relational databases
- Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
- Strong analytic skills related to working with structured datasets.
- Strong track record of building in build processes supporting data transformation, data structures, metadata, dependency and workload management.
- Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores (optional)
- Experience in supporting and working with cross-functional teams in a dynamic environment.
- Tech Stack
- Experience with big data tools:

- Hadoop Ecosytem (must),Hive (must)
- Impala (must)
- Sqoop (must)
- Python (must)
- Spark with Python (must)
- PySPARK (must) - experience must in Spark Tuning as well
- SPARK-SQL (must)- Any one RDBMS (good to have)
- Any cloud platform (S3 or Azure)
- Rich experience in scripting language Python (must)
- Rich experience in building optimal pySpark jobs handling huge data load (must)
- Rich experience in performing capacity management and gauging memory needs for huge data processing (must)
- Experience with stream-processing systems: Storm, Spark-Streaming, etc. (good to have)

**Job Summary**:
**Role Based Competencies**:

- Tech - Requirement Management S/W- Tech - NFR- Tech - High level design- Tech - Code quality and coding standards- Beh - Result Orientation- Tech - Code Reviews- Tech - Build Management- Tech - Unit Testing- Beh - Information Seeking- Tech - Agile Methodology- Tech - Analytical \ Problem Solving- Beh - Communication- Beh - Customer Orientation- Beh - Collaboration**Mandatory Competencies**:

- Big Data - Hadoop- Big Data - Hive- Big Data - Impala- Big Data - PySpark**Good to Have Competencies**:

  • Big Data Developer

    3 weeks ago

    Noida, India Coforge Full time

    We are seeking a highly skilled and experienced Senior Big Data Engineer with strong expertise in Streaming Data and Big Data Technologies , AWS & Scala Mandatory . The ideal candidate will have hands-on experience in architecting and implementing scalable data solutions using modern cloud-native tools and frameworks. Key Responsibilities: Architect and...

  • Govt Sales(Big Data/it)

    15 hours ago

    Noida, India SA Technologies Inc. Full time

    We are looking 10+ years’ experience for this role Location: Noida Position Summary: You will play a pivotal role of building and driving initiatives & responsible to achieve sales target, P&L management etc. This role will also require you to managing internal and external stakeholders related to the India business and also asks for significant...

  • Big Data Engineering

    1 week ago

    Noida, Uttar Pradesh, India prudentialmanp Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Bachelor's degree in computer science, or similar technical fieldMinimum 5 years of experience in Big Data Engineering/Data AnalysisPreferred Skills and Qualifications:Proficiency in Python, SQL and Apache SparkStrong Experience on Snowflake AWS services such as EMR, Glue (serverless architecture), S3, Athena, IAM, Lambda and CloudWatchCore Spark,...

  • Java + Big Data

    15 hours ago

    Noida, India Iris software Full time

    **Mandatory Skills**: - Java 8, Spring, Hibernate - Big Data preferably in Spark **Job Summary**: Job Code - 14-D-50 Exp - 4-5 Years Location - NCR **Role Based Competencies**: - Tech - Requirement Management S/W- Tech - Code quality and coding standards- Beh - Result Orientation- Tech - Code Reviews- Tech - Build Management- Tech - Unit Testing- Beh -...

  • Java + Big Data

    15 hours ago

    Noida, India Iris software Full time

    **Mandatory Skills**: - Java 8, Spring, Hibernate - Big Data preferably in Spark **Job Summary**: Job Code - 24-D-00 Exp - 7-8 Years Location - NCR **Role Based Competencies**: - Tech - Requirement Management S/W- Tech - NFR- Tech - High level design- Tech - Code quality and coding standards- Beh - Result Orientation- Tech - Code Reviews- Tech - Build...

  • Big Data Cloud Architect

    2 weeks ago

    Noida, India Adobe Full time

    Our Company Changing the world through digital experiences is what Adobe’s all about. We give everyone—from emerging artists to global brands—everything they need to design and deliver exceptional digital experiences! We’re passionate about empowering people to create beautiful and powerful images, videos, and apps, and transform how companies...

  • Big Data Lead

    1 week ago

    Noida, Uttar Pradesh, India Briskwin IT Solutions Full time ₹ 1,20,000 - ₹ 6,00,000 per year

    Exp: 6-9yrs**Work location : Pune/Chennai/Mumbai/Noida/Coimbatore/BangaloreWork Mode: Hybrid**Notice Period : Imm – 30 daysMandatory skills : SSIS + MongoDB + SQLJob Title: Data Engineer Experience: 5+ yearsJob Description: We are looking for an experienced Data Engineer with strong expertise in building and managing data pipelines, integrating diverse...

  • QA Engineer-Big Data

    3 days ago

    Noida, India BOLD Limited Full time

    BOLD is an established and fast-growing product company that transforms work lives. Since 2005, BOLD has delivered award-winning career services that have a meaningful and positive impact on job seekers and employers. BOLD’s robust product line includes a professional resume and cover letter writing services, scientifically validated career tests, and...

  • Senior Big Data Engineer

    4 hours ago

    Noida, Uttar Pradesh, India VARAISYS PVT. LTD. Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Role:Big Data EngineerLocation:Noida (Work From Office)Experience:5 YearsRole OverviewWe're looking for aBig Data Engineerwho thrives on scale — the kind whereterabytes of data move every single day. You'll be building and optimizing high-throughput data pipelines that fuel analytics, AI, and business intelligence across the org.Key...

  • Big Oh Tech

    3 weeks ago

    Noida, India Big Oh Tech Full time

    Description :Big Oh Tech is a global leader in IT solutions including software development, enterprise app development, staff augmentation, offshore outsourcing, and technical recruitment consulting.With a team of over 100 technology enthusiasts, we deliver scalable solutions to complex challenges, enhancing business revenue for our clients.Big Oh Tech is...