Big Data Engineer

4 weeks ago


Bangalore, India Zelus India Full time

Job Description :


- Design, develop, and maintain scalable and high-performance data processing applications using Apache Spark and Scala, leveraging distributed computing frameworks for processing large volumes of data.

- Data Ingestion and Transformation : Implement data ingestion pipelines to ingest data from various sources into Azure Data Lake Storage or Azure Blob Storage, and transform the data using Spark RDDs, DataFrames, and Datasets.

- Spark Development : Develop Spark applications using Scala programming language, utilizing Spark APIs for data manipulation, transformation, aggregation, and analysis

- Integrate Spark applications with other Azure services such as Azure Databricks, Azure Synapse Analytics, Azure HDInsight, and Azure SQL Database for end-to-end data processing and analytics workflows.

- Streaming Data Processing : Implement real-time data processing solutions using Spark Streaming or Azure Stream Analytics, enabling real-time insights and analytics on streaming data sources.

- Data Orchestration : Design and implement data orchestration workflows using Apache Airflow or Azure Data Factory to schedule and manage Spark jobs and data pipelines.

- Experience with cloud platforms, particularly Microsoft Azure, and familiarity with Azure services such as Azure Data Lake Storage, Azure Blob Storage, Azure Databricks, Azure Synapse Analytics, and Azure Data Factory.

- Strong understanding of distributed computing principles and big data processing concepts, including data partitioning, parallel processing, and fault tolerance.

(ref:hirist.tech)
  • Big Data Engineer

    2 days ago


    bangalore, India Virtusa Full time

    Big Data Engineer - CREQ188642 Description Looking for a Senior Software Engineer with strong analytical skills to help build and enhance our Big data application. This individual's day-to-day work will involve using Java, Scala, SQL, Big Data, Linux, git, and maven/gradle to design and build software solutions that are fast, scalable, and...

  • Big Data Engineer

    1 month ago


    bangalore, India Virtusa Full time

    Big Data Engineer - CREQ184231 Description Profile : 5-8 years IT experience with minimum 2 years in Spark and Big Data platform Experience in Big Data technologies Hadoop, HDFS, Hive, Yarn, Jenkins, Ansible Job DescriptionResponsibilitiesSkill: Spark, ScalaLevel: Senior Software EngineerCategory: Special/NicheTenure: 1 YearMission :He (or she) is in charge...

  • Big Data Engineer

    1 month ago


    bangalore, India Hexaware Technologies Full time

    Description Data Engineer: Primary : ADF, SQL Seconday : Databricks, Pyspark / Scala, CICD, Data Security, encryption Responsibilities: •Work closely with scrum master, technical leads and provide daily progress updates on the tasks assigned. •Provide support in defining the scope and sizing of the work. •Working closely with...

  • Big Data Engineer

    1 month ago


    bangalore, India TalentPulse Consultant Full time

    Role : Big Data Engineer Job Description : We are currently seeking a highly skilled Big Data Engineer to join our team on a remote basis. The ideal candidate will have at least 5 years of experience in Big Data technologies and a strong proficiency in Python, Spark, and Scala. This is an active position with a high priority, offering a competitive rate...

  • Big Data Engineer

    2 weeks ago


    Bangalore, India TalentPulse Consultant Full time

    Role : Big Data Engineer Job Description : We are currently seeking a highly skilled Big Data Engineer to join our team on a remote basis. The ideal candidate will have at least 5 years of experience in Big Data technologies and a strong proficiency in Python, Spark, and Scala. This is an active position with a high priority, offering a competitive rate...

  • Big Data Engineer

    4 weeks ago


    Bangalore, Karnataka, India TalentPulse Consultant Full time

    Role : Big Data EngineerJob Description :We are currently seeking a highly skilled Big Data Engineer to join our team on a remote basis.The ideal candidate will have at least 5 years of experience in Big Data technologies and a strong proficiency in Python, Spark, and Scala.This is an active position with a high priority, offering a competitive rate and the...

  • Big Data Engineer

    4 days ago


    bangalore, India Infogain Full time

    Big Data Engineer (Lead) with skills Big Data, Data Warehouse, Azure Datalake for location Any Infogain Base Location (Noida, Gurugram, Bangalore, Mumbai, Pune) Posted on: May 14, Share on Linkedin Share on Twitter Share on Facebook ROLES & RESPONSIBILITIES Responsibilities: · Strong experience of minimum 5+ years in Snowflake &...

  • Big Data Engineer

    2 days ago


    bangalore, India Infogain Full time

    Big Data Engineer (Lead) with skills Big Data, Data Warehouse, Azure Datalake for location Any Infogain Base Location (Noida, Gurugram, Bangalore, Mumbai, Pune) Posted on: May 16, Share on Linkedin Share on Twitter Share on Facebook ROLES & RESPONSIBILITIES Responsibilities: · Strong experience of minimum 5+ years in Snowflake &...

  • Aws big data engineer

    1 month ago


    bangalore, India Global Pharma Tek Full time

    ob Description Technical skills . Having ETL development experience with overall to years in AWS , Big data engineering related application Minimum end to end ETL project implementation Proficient in PySpark, Python. Proficient in Big Data technologies - Hadoop, Spark, Hive, Databricks. Expert in AWS to develop, deploy and maintain ETL applications...

  • Senior Data Engineer

    4 weeks ago


    Bangalore, Karnataka, India Muoro Full time

    Job Description :About the role :We are looking for a Senior Data Engineer to join our growing engineering team. As a member of the team:- You will work on enterprise data platforms, architect and implement data lakes both on-prem and in the cloud.- You will be responsible for evolving technical architecture, design and implementation of data solutions using...

  • Senior Data Engineer

    1 month ago


    Bangalore, India Muoro Full time

    Job Description : About the role :We are looking for a Senior Data Engineer to join our growing engineering team. As a member of the team:- You will work on enterprise data platforms, architect and implement data lakes both on-prem and in the cloud.- You will be responsible for evolving technical architecture, design and implementation of data solutions...

  • Senior Data Engineer

    1 month ago


    bangalore, India Squareroot Consulting Pvt Ltd Full time

    We are seeking a highly skilled and experienced Senior Data Engineer to join our team.Location : Bangalore (Work from Office all 5 days)Experience : 6+ YearsRole : Senior Data EngineerThe ideal candidate will have a strong background in big data technologies and extensive hands-on experience with Cloudera, Spark, Hive, HDFS, Flink, Data Lake, and...

  • Senior Data Engineer

    4 weeks ago


    Bangalore, Karnataka, India Squareroot Consulting Pvt Ltd Full time

    We are seeking a highly skilled and experienced Senior Data Engineer to join our team.Location : Bangalore (Work from Office all 5 days)Experience : 6+ YearsRole : Senior Data EngineerThe ideal candidate will have a strong background in big data technologies and extensive hands-on experience with Cloudera, Spark, Hive, HDFS, Flink, Data Lake, and...

  • Big Data Architect

    1 month ago


    bangalore, India RiskInsight Consulting Pvt Ltd Full time

    Responsibilities Design and architect Big Data solutions to meet business requirements. Collaborate with stakeholders to understand their data and analytics needs. Create and maintain data models, data flow diagrams, and data integration strategies. Develop and implement data ingestion, processing, and storage solutions using Big Data technologies. ...

  • Senior Data Engineer

    2 weeks ago


    Bangalore, India Squareroot Consulting Pvt Ltd Full time

    We are seeking a highly skilled and experienced Senior Data Engineer to join our team.Location : Bangalore (Work from Office all 5 days)Experience : 6+ YearsRole : Senior Data EngineerThe ideal candidate will have a strong background in big data technologies and extensive hands-on experience with Cloudera, Spark, Hive, HDFS, Flink, Data Lake, and...

  • Big Data Developer

    3 weeks ago


    bangalore, India BigShyft Full time

    JobDescription: We are seeking askilled and experienced Big Data Developer/Data Engineer to joinour clients team in Bengaluru. The ideal candidate should have astrong background in Java Spark Hadoop and Scala with 47 years ofrelevant workexperience.Responsibilities: Designdevelop and maintain largescale data processing systems leveragingHadoop Spark and...

  • Big Data Engineer

    7 days ago


    bangalore, India TekWissen India Full time

    Role: Big Data EngineerLocation: Bangalore-OnsiteWorktype: Contract for 6 monthsPRINCIPAL RESPONSIBILITIES:- Gain core business understanding of Textron Aviation and aircraft design, operation, and support- Query, clean, transform, and stage data (ETL/ELT) across on-prem and cloud environments- Support data analytics and data science activities by...


  • bangalore, India Recrud Full time

    Senior Big Data Engineer Position. Experience Required: 6 -10 Years. Employment Type: Full-Time. About Our Client:. Our client is a distinguished product engineering consulting firm based in Bangalore, dedicated to providing innovative offshore development solutions. With over half a century of combined expertise in product engineering and consulting...

  • Big Data Developer

    1 month ago


    bangalore, India Workassist.in Full time

    Our hiring partner is looking for :- This is a full-time on-site role for a Big Data Developer at Big Four Consulting Firm Gurugram.- The Big Data Developer will be responsible for tasks related to Data Engineering, Extract Transform Load (ETL), Big Data, Data Warehousing, and Software Data Engineering, Big Data, Extract Transform Load (ETL), and Data...

  • Big data architect

    1 month ago


    bangalore, India Virtusa Full time

    Big data architect - CREQ185795 Description Position: Big data architect Location: BLR Primary skills: Big data hadoop spark & aws Experience in building SparkStreaming process. Proficient in understanding distributed computing principles. Experience in managing Hadoop cluster, BigData with all services.Proficiency with Hadoop MapReduce HDFS Pig...