Apache Spark Engineer

4 weeks ago


Anywhere in IndiaMultiple Locations IT Source Global Full time
Job Title: Apache Spark Developer

We are seeking a highly skilled Apache Spark Developer to join our team at IT Source Global. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining large-scale data processing pipelines using Apache Spark and Scala.

Key Responsibilities:
  • Design and develop efficient data processing pipelines using Apache Spark and Scala
  • Implement complex data transformations, aggregations, and analytics using Spark
  • Integrate various data sources, including structured and unstructured data, into the data processing environment
  • Optimize Spark jobs and Scala code for performance, memory management, and resource utilization
  • Collaborate with data scientists, data engineers, and business analysts to understand data requirements and provide technical solutions
  • Develop and maintain ETL processes to extract, transform, and load data into data lakes, data warehouses, and other storage solutions
  • Implement testing frameworks for data pipelines, ensuring data quality and accuracy through rigorous validation processes
  • Utilize version control systems such as Git for managing code and collaborating with team members
  • Create and maintain comprehensive documentation for data pipelines, processes, and best practices
Requirements:
  • Bachelor's degree in Computer Science, Data Engineering, or a related field
  • 3+ years of experience in big data development with Apache Spark
  • Strong proficiency in Scala programming
  • Experience with distributed computing and parallel processing
  • Familiarity with Hadoop ecosystem components (HDFS, Hive, HBase, etc.)
Preferred Qualifications:
  • Experience with real-time data processing frameworks (e.g., Apache Kafka, Apache Flink)
  • Knowledge of machine learning libraries and tools (e.g., MLlib)
  • Familiarity with data visualization tools and techniques
  • Experience with CI/CD pipelines and automation tools
  • Understanding of data security and compliance standards
Benefits:
  • Competitive salary
  • Health, dental, and vision insurance
  • 401(k) with company match
  • Professional development opportunities
  • Flexible work hours and remote work options

To apply, please submit your resume and a cover letter detailing your experience and qualifications.



  • Anywhere in India/Multiple Locations IT Source Global Full time

    Job Title: Apache Spark DeveloperWe are seeking a highly skilled Apache Spark Developer to join our team at IT Source Global. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining data processing pipelines using Apache Spark and Scala.Key Responsibilities:Design and develop data processing pipelines...

  • Data Engineer

    3 weeks ago


    Anywhere in India/Multiple Locations Huntsmen & Barons Full time

    About the Role:Data Engineer - Apache Spark ExpertHuntsmen & Barons is seeking an experienced Data Engineer with a strong background in Apache Spark to join our team. As a Data Engineer, you will play a key role in designing and developing big data platforms, leveraging expertise in data engineering, Spark, HBase, Hive, and other Hadoop ecosystem...

  • Apache Spark Developer

    2 months ago


    Anywhere in India/Multiple Locations, IN IT Source Global Full time

    We have Immediate openings on Apache developerJob description : - Data Pipeline Development : Design, develop, and maintain data processing pipelines using Apache Spark and Scala, ensuring high performance and scalability.- Data Transformation and Analysis : Implement complex data transformations, aggregations, and analytics using Spark, optimizing for...


  • Anywhere in India/Multiple Locations IT Source Global Full time

    We have Immediate openings on Apache developerJob description : - Data Pipeline Development : Design, develop, and maintain data processing pipelines using Apache Spark and Scala, ensuring high performance and scalability.- Data Transformation and Analysis : Implement complex data transformations, aggregations, and analytics using Spark, optimizing for...

  • Data Architect

    2 months ago


    Anywhere in India/Multiple Locations, IN RAPINNO TECH SOLUTIONS PRIVATE LIMITED Full time

    Position Overview :We are looking for an experienced Data Architect with expertise in Databricks, Apache Spark, and ETL processes. The ideal candidate will have a proven track record of designing and building robust data applications from the ground up. You will play a key role in shaping our data strategy and ensuring our data solutions meet business needs...

  • Data Engineer

    2 months ago


    Anywhere in India/Multiple Locations, IN Huntsmen & Barons Full time

    Role : Data Engineer - Apache spark,Spark, Scala, PYspark, Hadoop, Hbase, HiveLocation : Pan IndiaWork Mode- HybridYear of Experience :Minimum 5 Years tp 12 YearsJob Description :- Relevant Experience 5 years.- The indent is created for Apache spark. But the actual demand is for Spark/ Scala.- Expertise in Data Engineering.- Hands-on experience in design and...

  • Data Engineer

    4 weeks ago


    Anywhere in India/Multiple Locations Huntsmen & Barons Full time

    Role : Data Engineer - Apache spark,Spark, Scala, PYspark, Hadoop, Hbase, HiveLocation : Pan IndiaWork Mode- HybridYear of Experience :Minimum 5 Years tp 12 YearsJob Description :- Relevant Experience 5 years.- The indent is created for Apache spark. But the actual demand is for Spark/ Scala.- Expertise in Data Engineering.- Hands-on experience in design and...

  • Data Engineer

    1 month ago


    Anywhere in India/Multiple Locations Huntsmen & Barons Full time

    Role : Data Engineer - Spark/Scala. Job Description : - Relevant Experience 5 years.- The indent is created for Apache spark. But the actual demand is for Spark/ Scala.- Expertise in Data Engineering. - Hands-on experience in design and development of big data platform.- Deep understanding of modern data processing technology stacks: Spark, HBase, Hive...

  • Senior Data Engineer

    4 weeks ago


    Anywhere in India/Multiple Locations/Remote Hunarstreet Technologies Pvt Ltd Full time

    Job Title : Senior Data Engineer (Remote) Location : RemoteWe are hiring a Senior Data Engineer for one of our leading software development clients. The Senior Data Engineer will play a key role in building and extending data pipeline architecture, optimizing data flow, and supporting cross-functional teams. The ideal candidate has extensive experience in...

  • Senior Data Engineer

    2 months ago


    Anywhere in India/Multiple Locations/Remote, IN Hunarstreet Technologies Pvt Ltd Full time

    Job Title : Senior Data Engineer (Remote)Location : RemoteWe are hiring a Senior Data Engineer for one of our leading software development clients. The Senior Data Engineer will play a key role in building and extending data pipeline architecture, optimizing data flow, and supporting cross-functional teams. The ideal candidate has extensive experience in...

  • Big Data Engineer

    4 weeks ago


    Anywhere in India/Multiple Locations/Remote Hum Technologies Full time

    Big Data Engineer with strong exp in Hadoop, Spark, SQL & AWS for one of the Fortune 500 Clients Company Name : Hum TechnologiesClient : One of the Fortune 500 CompaniesLocation : : Apache NiFi, HDFS, Hive, Impala, Spark, Spark Streaming, Kafka, Java/Scala, SQL, Shell scripting Roles & Responsibilities : - Responsible for designing and implementing...

  • Big Data Engineer

    2 months ago


    Anywhere in India/Multiple Locations/Remote, IN Hum Technologies Full time

    Big Data Engineer with strong exp in Hadoop, Spark, SQL & AWS for one of the Fortune 500 Clients Company Name : Hum TechnologiesClient : One of the Fortune 500 CompaniesLocation : : Apache NiFi, HDFS, Hive, Impala, Spark, Spark Streaming, Kafka, Java/Scala, SQL, Shell scriptingRoles & Responsibilities :- Responsible for designing and implementing data...

  • Spark Developer

    1 month ago


    Anywhere in India/Multiple Locations/Metros/Pune Talent Pro Full time

    Job DescriptionAs a Spark Developer at Talent Pro, you will play a key role in building scalable and efficient data processing pipelines using Apache Spark. Your primary responsibility will be to design, develop, and manage large-scale data processing systems, ensuring high performance and reliability.Key Responsibilities:Collaborate with development teams...

  • Spark Developer

    4 weeks ago


    Anywhere in India/Multiple Locations/Metros/Pune Talent Pro Full time

    Job Title: Spark DeveloperAt Talent Pro, we are seeking a highly skilled Spark Developer to join our team. As a Spark Developer, you will be responsible for designing and developing scalable distributed architecture using Apache Spark.Responsibilities:Work with development teams and product managers to build new product features, enhancements, and data...

  • Data Engineer

    4 weeks ago


    Anywhere in India/Multiple Locations Praxis Consultants Full time

    Job Summary We are looking for a highly skilled and experienced Technical Lead (Sr. Data Engineer/Data Architect / Data Modeler) to join our team. will play a crucial role in designing data architecture and models based on business requirements, ensuring data integrity, security, and performance. Oversee technical aspects of data integration and management...


  • India Marlabs Full time

    **Date**:Oct 11, 2024**Location**:Any Marlabs Office Location, IN**Company**:Marlabs Innovations Pvt Ltd**Description**: - Spark Administrator - 6 Weeks (Starting November 1st 2024)- Key Responsibilities- Install and configure the operating system (Linux distributions such as Ubuntu, CentOS, or RHEL) on cluster nodes, ensuring consistency across the...

  • Spark Developer

    2 months ago


    Anywhere in India/Multiple Locations/Metros/Pune, IN Talent Pro Full time

    Role : Spark DeveloperJob Description :- Work with development teams and product managers to build new product features, enhancements etc.- Build the front-end of applications through appealing visual design- Develop and manage well-functioning databases and applications- Design & develop data and event processing pipelines- Test software to ensure...

  • GCP Data Engineer

    6 months ago


    Pune/Hyderabad/Anywhere in India/Multiple Locations Huquo Full time

    Job Description : Must-Have :- 5+ Years of Experience in Data Engineering and building and maintaining large-scale data pipelines.- Experience with designing and implementing a large-scale Data-Lake on Cloud Infrastructure - Strong technical expertise in Python and SQL- Extremely well-versed in Google Compute Platform including BigQuery, Cloud Storage,...

  • Senior Data Engineer

    4 weeks ago


    Anywhere in India/Multiple Locations Huntsmen & Barons Full time

    Job Description:Data Engineer - Spark/ScalaAt Huntsmen & Barons, we are seeking a highly skilled Data Engineer - Spark/Scala to join our team. As a key member of our data engineering team, you will be responsible for designing and developing our big data platform using Spark and Scala. Your expertise in data engineering will enable us to process and analyze...


  • India Mercor Full time

    Role Overview & Key Responsibilities: We are seeking candidates to assist Mercor in a research project with leading AI labs to benchmark and improve the current AI Models. Qualifications: (Must have) 3+ years of experience with Apache Spark (Must have) SQL + Python experience is mandatory Scala/Java is preferred Analytical Excellence Prior...