Big Data Engineer

2 months ago


Bangalore, Karnataka, India Zelus India Full time

Job Description :


- Design, develop, and maintain scalable and high-performance data processing applications using Apache Spark and Scala, leveraging distributed computing frameworks for processing large volumes of data.

- Data Ingestion and Transformation : Implement data ingestion pipelines to ingest data from various sources into Azure Data Lake Storage or Azure Blob Storage, and transform the data using Spark RDDs, DataFrames, and Datasets.

- Spark Development : Develop Spark applications using Scala programming language, utilizing Spark APIs for data manipulation, transformation, aggregation, and analysis

- Integrate Spark applications with other Azure services such as Azure Databricks, Azure Synapse Analytics, Azure HDInsight, and Azure SQL Database for end-to-end data processing and analytics workflows.

- Streaming Data Processing : Implement real-time data processing solutions using Spark Streaming or Azure Stream Analytics, enabling real-time insights and analytics on streaming data sources.

- Data Orchestration : Design and implement data orchestration workflows using Apache Airflow or Azure Data Factory to schedule and manage Spark jobs and data pipelines.

- Experience with cloud platforms, particularly Microsoft Azure, and familiarity with Azure services such as Azure Data Lake Storage, Azure Blob Storage, Azure Databricks, Azure Synapse Analytics, and Azure Data Factory.

- Strong understanding of distributed computing principles and big data processing concepts, including data partitioning, parallel processing, and fault tolerance.

(ref:hirist.tech)

  • Bangalore, Karnataka, India Squareroot Consulting Pvt Ltd Full time

    We are seeking a highly skilled and experienced Senior Data Engineer to join our team.Location : Bangalore (Work from Office all 5 days)Experience : 6+ YearsRole : Senior Data EngineerThe ideal candidate will have a strong background in big data technologies and extensive hands-on experience with Cloudera, Spark, Hive, HDFS, Flink, Data Lake, and...

  • Senior Data Engineer

    2 months ago


    Bangalore, Karnataka, India Squareroot Consulting Pvt Ltd Full time

    We are seeking a highly skilled and experienced Senior Data Engineer to join our team.Location : Bangalore (Work from Office all 5 days)Experience : 6+ YearsRole : Senior Data EngineerThe ideal candidate will have a strong background in big data technologies and extensive hands-on experience with Cloudera, Spark, Hive, HDFS, Flink, Data Lake, and...


  • Bangalore, Karnataka, India Marktine Technology Solutions Full time

    Must Have Skills :- Solid Knowledge on DWH, ETL and Big Data Concepts- Excellent SQL Skills (With knowledge of SQL Analytics Functions)- Working Experience on any ETL tool i.e. SSIS / Informatica- Working Experience on any Azure or AWS Big Data Tools.- Experience on Implementing Data Jobs (Batch / Real time Streaming)- Excellent written and verbal...

  • Data Architect

    2 weeks ago


    Bangalore, Karnataka, India Indium Software Full time

    Job Description :We are looking for a highly experienced Data Architect to join our team. The ideal candidate will have over a decade of experience with Big Data technologies, including Apache Spark, Kafka, Cloud Native solutions, and Data Lakes. This role involves working in a customer-facing, post-sales capacity, focusing on technical architecture and...

  • Data Architect

    1 week ago


    Bangalore, Karnataka, India Indium Software Full time

    Job Description :We are looking for a highly experienced Data Architect to join our team. The ideal candidate will have over a decade of experience with Big Data technologies, including Apache Spark, Kafka, Cloud Native solutions, and Data Lakes. This role involves working in a customer-facing, post-sales capacity, focusing on technical architecture and...

  • Product Architect

    1 week ago


    Bangalore, Karnataka, India Experis IT Private Limited Full time

    REQUIREMENTS :Qualification : Bachelors / MastersExperience : 6.5-9 Years Job Location : BangaloreJob Type : Full timeROLES and RESPONSIBILITIES :- Working closely with the client to understand the business use case, build the data architecture, build the data pipeline, by using scalable, highly available and fault tolerant services in big data. - Provide...

  • Product Architect

    2 weeks ago


    Bangalore, Karnataka, India Experis IT Private Limited Full time

    REQUIREMENTS :Qualification : Bachelors / MastersExperience : 6.5-9 Years Job Location : BangaloreJob Type : Full timeROLES and RESPONSIBILITIES :- Working closely with the client to understand the business use case, build the data architecture, build the data pipeline, by using scalable, highly available and fault tolerant services in big data. - Provide...

  • Software Engineer

    1 week ago


    Bangalore, Karnataka, India symphoni HR Full time

    Skill: Java (Data structures/Algorithims) & Spark - Design, develop and implement, scalable, maintainable, and efficient software solutions using best practices and modern technologies. - Write clean, well designed, and testable code, following coding standards and guidelines. - Conduct code reviews and provide constructive feedback to team members to ensure...

  • Software Engineer

    2 months ago


    Bangalore, Karnataka, India symphoni HR Full time

    Skill: Java (Data structures/Algorithims) & Spark - Design, develop and implement, scalable, maintainable, and efficient software solutions using best practices and modern technologies. - Write clean, well designed, and testable code, following coding standards and guidelines. - Conduct code reviews and provide constructive feedback to team members to ensure...

  • Data Engineer

    2 months ago


    Bangalore, Karnataka, India LOGIC PLANET IT SERVICES Full time

    About the job :We are hiring for a global technology company, home to more than 224,000 people across 60 countries.Role : Data EngineerExp : 5+ years Mode of Work : HybridMode of Hire : FTEAvailability : Immediate joinee or serving maximum of 30 days notice period.Job Description :- 5+ years of experience operating and deploying solutions using AWS Service...


  • Bangalore, Karnataka, India Thoucentric Full time

    Job Description :As a Big Data Engineer, you will be responsible for designing, developing, and maintaining our big data infrastructure. You will work with large datasets, perform data processing, and support various business functions by creating data pipelines, data processing jobs, and data integration solutions. You will be working in a dynamic and...


  • Bangalore, Karnataka, India Thoucentric Full time

    Job Description :As a Big Data Engineer, you will be responsible for designing, developing, and maintaining our big data infrastructure. You will work with large datasets, perform data processing, and support various business functions by creating data pipelines, data processing jobs, and data integration solutions. You will be working in a dynamic and...


  • Bangalore, Karnataka, India TecHHire Global Full time

    Job Description :Function : Technical Management - Engineering Management, Software Engineering - Backend Development, Big Data / DWH / ETLResponsibilities :- Design and implement large-scale data processing systems using the Databricks platform.- Work closely with clients to understand their requirements and design solutions that meet their needs.-...

  • Databricks Architect

    1 month ago


    Bangalore, Karnataka, India TecHHire Global Full time

    Job Description :Function : Technical Management - Engineering Management, Software Engineering - Backend Development, Big Data / DWH / ETLResponsibilities :- Design and implement large-scale data processing systems using the Databricks platform.- Work closely with clients to understand their requirements and design solutions that meet their needs.-...

  • Big Data Consultant

    1 week ago


    Bangalore, Karnataka, India Nilasu consulting Full time

    Summary :We are seeking a highly skilled consultant with expertise in Spark/Scala and Python to join our team. You will play a key role in developing and maintaining big data applications using Spark and Scala with a focus on functional programming principles. Additionally, your experience in Python and microservices architecture will be valuable in building...

  • Big Data Consultant

    2 months ago


    Bangalore, Karnataka, India Nilasu consulting Full time

    Summary :We are seeking a highly skilled consultant with expertise in Spark/Scala and Python to join our team. You will play a key role in developing and maintaining big data applications using Spark and Scala with a focus on functional programming principles. Additionally, your experience in Python and microservices architecture will be valuable in building...

  • Big Data Engineer

    1 week ago


    Bangalore, Karnataka, India Randstad Full time

    Primary Responsibilities :- Design, develop, and implement big data solutions using Java, Python, or Scala.- Utilize SQL and Spark Scala for data processing and analysis.- Implement and manage data storage solutions using AWS services like S3, EC2, and Lambda.- Work with various databases such as Postgres, MySQL, Oracle, etc., for data storage and...

  • Big Data Engineer

    2 months ago


    Bangalore, Karnataka, India randstad Full time

    Primary Responsibilities :- Design, develop, and implement big data solutions using Java, Python, or Scala.- Utilize SQL and Spark Scala for data processing and analysis.- Implement and manage data storage solutions using AWS services like S3, EC2, and Lambda.- Work with various databases such as Postgres, MySQL, Oracle, etc., for data storage and...

  • Data Engineer

    2 months ago


    Bangalore, Karnataka, India ARR Recruitment Solutions Full time

    Job Description : We are looking for a highly skilled Senior Data Engineer with extensive experience in data integration, processing, and application development for handling large-scale data sets measured in Terabytes from multiple sources.- The ideal candidate will have a strong background in AWS services, particularly Lambda, EMR, and Glue, along with...


  • Bangalore, Karnataka, India Zyoin group Full time

    Job Description :We are looking for Principal Data Engineers to help us build and enhance big data platforms to achieve availability, scalability, and operational effectiveness. The right individual will embrace the opportunity to tackle challenging problems and use their influence to drive continual improvement.You will also work on the cutting edge of...