Big Data Engineer

2 months ago


Bangalore, Karnataka, India Zelus India Full time

Job Description :


- Design, develop, and maintain scalable and high-performance data processing applications using Apache Spark and Scala, leveraging distributed computing frameworks for processing large volumes of data.

- Data Ingestion and Transformation : Implement data ingestion pipelines to ingest data from various sources into Azure Data Lake Storage or Azure Blob Storage, and transform the data using Spark RDDs, DataFrames, and Datasets.

- Spark Development : Develop Spark applications using Scala programming language, utilizing Spark APIs for data manipulation, transformation, aggregation, and analysis

- Integrate Spark applications with other Azure services such as Azure Databricks, Azure Synapse Analytics, Azure HDInsight, and Azure SQL Database for end-to-end data processing and analytics workflows.

- Streaming Data Processing : Implement real-time data processing solutions using Spark Streaming or Azure Stream Analytics, enabling real-time insights and analytics on streaming data sources.

- Data Orchestration : Design and implement data orchestration workflows using Apache Airflow or Azure Data Factory to schedule and manage Spark jobs and data pipelines.

- Experience with cloud platforms, particularly Microsoft Azure, and familiarity with Azure services such as Azure Data Lake Storage, Azure Blob Storage, Azure Databricks, Azure Synapse Analytics, and Azure Data Factory.

- Strong understanding of distributed computing principles and big data processing concepts, including data partitioning, parallel processing, and fault tolerance.

(ref:hirist.tech)
  • Senior Data Engineer

    2 months ago


    Bangalore, Karnataka, India Squareroot Consulting Pvt Ltd Full time

    We are seeking a highly skilled and experienced Senior Data Engineer to join our team.Location : Bangalore (Work from Office all 5 days)Experience : 6+ YearsRole : Senior Data EngineerThe ideal candidate will have a strong background in big data technologies and extensive hands-on experience with Cloudera, Spark, Hive, HDFS, Flink, Data Lake, and...

  • Senior Data Engineer

    3 weeks ago


    Bangalore, Karnataka, India Squareroot Consulting Pvt Ltd Full time

    We are seeking a highly skilled and experienced Senior Data Engineer to join our team.Location : Bangalore (Work from Office all 5 days)Experience : 6+ YearsRole : Senior Data EngineerThe ideal candidate will have a strong background in big data technologies and extensive hands-on experience with Cloudera, Spark, Hive, HDFS, Flink, Data Lake, and...

  • Big Data Engineer

    4 days ago


    Bangalore, Karnataka, India WEN Full time

    Role : Big Data Engineer (Scala)Responsibilities :- Develop and maintain scalable data pipelines using Spark and Hive within the Hadoop ecosystem- Utilize Scala programming language to implement efficient data processing solutions- Collaborate with cross-functional teams to understand data requirements and deliver optimal solutions- Implement best practices...

  • Data Engineer

    4 days ago


    Bangalore/Hyderabad, Karnataka, India Pan Asia Man power Consultants Recruiting agency Full time

    Data EngineerExperience Range : 6 to 14 YearsLocation : Bangalore (Hybrid: Work From Office - 3 Days in a Week)Full Time Skill Sets :- Experience designing and implementing data pipelines- Experience with SQL, RDBMS, or other data management, reporting and query tools.- Big Data Technologies - Hadoop, Hive, Spark, KafkaSecondary Skill Sets :- Added advantage...


  • Bangalore, Karnataka, India Marktine Technology Solutions Full time

    Must Have Skills :- Solid Knowledge on DWH, ETL and Big Data Concepts- Excellent SQL Skills (With knowledge of SQL Analytics Functions)- Working Experience on any ETL tool i.e. SSIS / Informatica- Working Experience on any Azure or AWS Big Data Tools.- Experience on Implementing Data Jobs (Batch / Real time Streaming)- Excellent written and verbal...

  • Lead Data Engineer

    4 days ago


    Bangalore, Karnataka, India NINETY1 GLOBAL CONSULTING PRIVATE LIMITED Full time

    Job Description:Experience: 8-12 yearsLocation: Bangalore (WFO)- 8 years' experience in developing scalable Big Data applications or solutions on distributed platforms. - Able to partner with others in solving complex problems by taking a broad perspective to identify innovative solutions.- Strong skills building positive relationships across Product and...

  • Data Architect

    3 weeks ago


    Bangalore, Karnataka, India Indium Software Full time

    Job Description :We are looking for a highly experienced Data Architect to join our team. The ideal candidate will have over a decade of experience with Big Data technologies, including Apache Spark, Kafka, Cloud Native solutions, and Data Lakes. This role involves working in a customer-facing, post-sales capacity, focusing on technical architecture and...

  • Data Architect

    3 weeks ago


    Bangalore, Karnataka, India Indium Software Full time

    Job Description :We are looking for a highly experienced Data Architect to join our team. The ideal candidate will have over a decade of experience with Big Data technologies, including Apache Spark, Kafka, Cloud Native solutions, and Data Lakes. This role involves working in a customer-facing, post-sales capacity, focusing on technical architecture and...

  • Product Architect

    3 weeks ago


    Bangalore, Karnataka, India Experis IT Private Limited Full time

    REQUIREMENTS :Qualification : Bachelors / MastersExperience : 6.5-9 Years Job Location : BangaloreJob Type : Full timeROLES and RESPONSIBILITIES :- Working closely with the client to understand the business use case, build the data architecture, build the data pipeline, by using scalable, highly available and fault tolerant services in big data. - Provide...

  • Product Architect

    3 weeks ago


    Bangalore, Karnataka, India Experis IT Private Limited Full time

    REQUIREMENTS :Qualification : Bachelors / MastersExperience : 6.5-9 Years Job Location : BangaloreJob Type : Full timeROLES and RESPONSIBILITIES :- Working closely with the client to understand the business use case, build the data architecture, build the data pipeline, by using scalable, highly available and fault tolerant services in big data. - Provide...

  • Software Engineer

    2 months ago


    Bangalore, Karnataka, India symphoni HR Full time

    Skill: Java (Data structures/Algorithims) & Spark - Design, develop and implement, scalable, maintainable, and efficient software solutions using best practices and modern technologies. - Write clean, well designed, and testable code, following coding standards and guidelines. - Conduct code reviews and provide constructive feedback to team members to ensure...

  • Software Engineer

    3 weeks ago


    Bangalore, Karnataka, India symphoni HR Full time

    Skill: Java (Data structures/Algorithims) & Spark - Design, develop and implement, scalable, maintainable, and efficient software solutions using best practices and modern technologies. - Write clean, well designed, and testable code, following coding standards and guidelines. - Conduct code reviews and provide constructive feedback to team members to ensure...

  • Data Engineer

    2 months ago


    Bangalore, Karnataka, India LOGIC PLANET IT SERVICES Full time

    About the job :We are hiring for a global technology company, home to more than 224,000 people across 60 countries.Role : Data EngineerExp : 5+ years Mode of Work : HybridMode of Hire : FTEAvailability : Immediate joinee or serving maximum of 30 days notice period.Job Description :- 5+ years of experience operating and deploying solutions using AWS Service...


  • Bangalore, Karnataka, India Thoucentric Full time

    Job Description :As a Big Data Engineer, you will be responsible for designing, developing, and maintaining our big data infrastructure. You will work with large datasets, perform data processing, and support various business functions by creating data pipelines, data processing jobs, and data integration solutions. You will be working in a dynamic and...


  • Bangalore, Karnataka, India Thoucentric Full time

    Job Description :As a Big Data Engineer, you will be responsible for designing, developing, and maintaining our big data infrastructure. You will work with large datasets, perform data processing, and support various business functions by creating data pipelines, data processing jobs, and data integration solutions. You will be working in a dynamic and...

  • Databricks Architect

    2 months ago


    Bangalore, Karnataka, India TecHHire Global Full time

    Job Description :Function : Technical Management - Engineering Management, Software Engineering - Backend Development, Big Data / DWH / ETLResponsibilities :- Design and implement large-scale data processing systems using the Databricks platform.- Work closely with clients to understand their requirements and design solutions that meet their needs.-...

  • Databricks Architect

    3 weeks ago


    Bangalore, Karnataka, India TecHHire Global Full time

    Job Description :Function : Technical Management - Engineering Management, Software Engineering - Backend Development, Big Data / DWH / ETLResponsibilities :- Design and implement large-scale data processing systems using the Databricks platform.- Work closely with clients to understand their requirements and design solutions that meet their needs.-...

  • Technical Lead

    4 days ago


    Bangalore, Karnataka, India Strategic HR Solutions Full time

    Job DescriptionAs a Technical Lead, you will be responsible for building a highly-scalable and extensible big data platform that provides the foundation for collecting, storing, modeling, and analyzing massive data sets from multiple channels. This position reports to the Engineering manager. Responsibilities: - Align our client with key Client initiatives -...

  • Big Data Consultant

    2 months ago


    Bangalore, Karnataka, India Nilasu consulting Full time

    Summary :We are seeking a highly skilled consultant with expertise in Spark/Scala and Python to join our team. You will play a key role in developing and maintaining big data applications using Spark and Scala with a focus on functional programming principles. Additionally, your experience in Python and microservices architecture will be valuable in building...

  • Big Data Consultant

    3 weeks ago


    Bangalore, Karnataka, India Nilasu consulting Full time

    Summary :We are seeking a highly skilled consultant with expertise in Spark/Scala and Python to join our team. You will play a key role in developing and maintaining big data applications using Spark and Scala with a focus on functional programming principles. Additionally, your experience in Python and microservices architecture will be valuable in building...