Apache Spark Developer

4 weeks ago

Anywhere in IndiaMultiple Locations IT Source Global Full time

We have Immediate openings on Apache developer

Job description :

- Data Pipeline Development : Design, develop, and maintain data processing pipelines using Apache Spark and Scala, ensuring high performance and scalability.

- Data Transformation and Analysis : Implement complex data transformations, aggregations, and analytics using Spark, optimizing for efficiency and speed.

- Data Integration : Integrate various data sources, including structured and unstructured data, into the data processing environment.

- Performance Tuning : Optimize Spark jobs and Scala code for performance, memory management, and resource utilization.

- Collaboration : Work closely with data scientists, data engineers, and business analysts to understand data requirements and provide technical solutions.

- ETL Processes : Develop and maintain ETL processes to extract, transform, and load data into data lakes, data warehouses, and other storage solutions.

- Testing and Validation : Implement testing frameworks for data pipelines, ensuring data quality and accuracy through rigorous validation processes.

- Version Control : Utilize version control systems such as Git for managing code and collaborating with team members.

- Documentation : Create and maintain comprehensive documentation for data pipelines, processes, and best practices.

Required Qualifications :

Education : Bachelor's degree in Computer Science, Data Engineering, or a related field.

Experience :

- 3+ years of experience in big data development with Apache Spark.

- Strong proficiency in Scala programming.

- Experience with distributed computing and parallel processing.

- Familiarity with Hadoop ecosystem components (HDFS, Hive, HBase, etc.).

Skills :

- Proficient understanding of data processing frameworks, particularly Apache Spark.

- Strong knowledge of SQL and experience with databases like MySQL, PostgreSQL, or similar.

- Experience with cloud platforms like AWS, Azure, or Google Cloud, particularly with big data services.

- Knowledge of containerization and orchestration tools (e.g., Docker, Kubernetes) is a plus.

- Strong problem-solving skills and attention to detail.

- Excellent communication and teamwork abilities.

Preferred Qualifications :

- Experience with real-time data processing frameworks (e.g., Apache Kafka, Apache Flink).

- Knowledge of machine learning libraries and tools (e.g., MLlib).

- Familiarity with data visualization tools and techniques.

- Experience with CI/CD pipelines and automation tools.

- Understanding of data security and compliance standards.

Benefits :

- Competitive salary

- Health, dental, and vision insurance

- 401(k) with company match

- Professional development opportunities

- Flexible work hours and remote work options

To Apply : Please submit your resume and a cover letter detailing your experience and why you would be a great fit for this role.

(ref:hirist.tech)

  • Apache Spark Developer

    1 month ago

    Anywhere in India/Multiple Locations IT Source Global Full time

    Job Title: Apache Spark DeveloperWe are seeking a highly skilled Apache Spark Developer to join our team at IT Source Global. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining data processing pipelines using Apache Spark and Scala.Key Responsibilities:Design and develop data processing pipelines...

  • Apache Spark Engineer

    4 weeks ago

    Anywhere in India/Multiple Locations IT Source Global Full time

    Job Title: Apache Spark DeveloperWe are seeking a highly skilled Apache Spark Developer to join our team at IT Source Global. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining large-scale data processing pipelines using Apache Spark and Scala.Key Responsibilities:Design and develop efficient data...

  • Apache Spark Developer

    2 months ago

    Anywhere in India/Multiple Locations, IN IT Source Global Full time

    We have Immediate openings on Apache developerJob description : - Data Pipeline Development : Design, develop, and maintain data processing pipelines using Apache Spark and Scala, ensuring high performance and scalability.- Data Transformation and Analysis : Implement complex data transformations, aggregations, and analytics using Spark, optimizing for...

  • Data Engineer

    3 weeks ago

    Anywhere in India/Multiple Locations Huntsmen & Barons Full time

    About the Role:Data Engineer - Apache Spark ExpertHuntsmen & Barons is seeking an experienced Data Engineer with a strong background in Apache Spark to join our team. As a Data Engineer, you will play a key role in designing and developing big data platforms, leveraging expertise in data engineering, Spark, HBase, Hive, and other Hadoop ecosystem...

  • Data Architect

    2 months ago

    Anywhere in India/Multiple Locations, IN RAPINNO TECH SOLUTIONS PRIVATE LIMITED Full time

    Position Overview :We are looking for an experienced Data Architect with expertise in Databricks, Apache Spark, and ETL processes. The ideal candidate will have a proven track record of designing and building robust data applications from the ground up. You will play a key role in shaping our data strategy and ensuring our data solutions meet business needs...

  • Spark Developer

    4 weeks ago

    Anywhere in India/Multiple Locations/Metros/Pune Talent Pro Full time

    Job Title: Spark DeveloperAt Talent Pro, we are seeking a highly skilled Spark Developer to join our team. As a Spark Developer, you will be responsible for designing and developing scalable distributed architecture using Apache Spark.Responsibilities:Work with development teams and product managers to build new product features, enhancements, and data...

  • Spark Developer

    1 month ago

    Anywhere in India/Multiple Locations/Metros/Pune Talent Pro Full time

    Job DescriptionAs a Spark Developer at Talent Pro, you will play a key role in building scalable and efficient data processing pipelines using Apache Spark. Your primary responsibility will be to design, develop, and manage large-scale data processing systems, ensuring high performance and reliability.Key Responsibilities:Collaborate with development teams...

  • Data Engineer

    4 weeks ago

    Anywhere in India/Multiple Locations Huntsmen & Barons Full time

    Role : Data Engineer - Apache spark,Spark, Scala, PYspark, Hadoop, Hbase, HiveLocation : Pan IndiaWork Mode- HybridYear of Experience :Minimum 5 Years tp 12 YearsJob Description :- Relevant Experience 5 years.- The indent is created for Apache spark. But the actual demand is for Spark/ Scala.- Expertise in Data Engineering.- Hands-on experience in design and...

  • Data Engineer

    2 months ago

    Anywhere in India/Multiple Locations, IN Huntsmen & Barons Full time

    Role : Data Engineer - Apache spark,Spark, Scala, PYspark, Hadoop, Hbase, HiveLocation : Pan IndiaWork Mode- HybridYear of Experience :Minimum 5 Years tp 12 YearsJob Description :- Relevant Experience 5 years.- The indent is created for Apache spark. But the actual demand is for Spark/ Scala.- Expertise in Data Engineering.- Hands-on experience in design and...

  • Spark Developer

    2 months ago

    Anywhere in India/Multiple Locations/Metros/Pune, IN Talent Pro Full time

    Role : Spark DeveloperJob Description :- Work with development teams and product managers to build new product features, enhancements etc.- Build the front-end of applications through appealing visual design- Develop and manage well-functioning databases and applications- Design & develop data and event processing pipelines- Test software to ensure...

  • Data Engineer

    1 month ago

    Anywhere in India/Multiple Locations Huntsmen & Barons Full time

    Role : Data Engineer - Spark/Scala. Job Description : - Relevant Experience 5 years.- The indent is created for Apache spark. But the actual demand is for Spark/ Scala.- Expertise in Data Engineering. - Hands-on experience in design and development of big data platform.- Deep understanding of modern data processing technology stacks: Spark, HBase, Hive...

  • Apache Spark Administrator

    2 months ago

    India Marlabs Full time

    **Date**:Oct 11, 2024**Location**:Any Marlabs Office Location, IN**Company**:Marlabs Innovations Pvt Ltd**Description**: - Spark Administrator - 6 Weeks (Starting November 1st 2024)- Key Responsibilities- Install and configure the operating system (Linux distributions such as Ubuntu, CentOS, or RHEL) on cluster nodes, ensuring consistency across the...

  • Senior Data Engineer

    4 weeks ago

    Anywhere in India/Multiple Locations/Remote Hunarstreet Technologies Pvt Ltd Full time

    Job Title : Senior Data Engineer (Remote) Location : RemoteWe are hiring a Senior Data Engineer for one of our leading software development clients. The Senior Data Engineer will play a key role in building and extending data pipeline architecture, optimizing data flow, and supporting cross-functional teams. The ideal candidate has extensive experience in...

  • Senior Data Engineer

    2 months ago

    Anywhere in India/Multiple Locations/Remote, IN Hunarstreet Technologies Pvt Ltd Full time

    Job Title : Senior Data Engineer (Remote)Location : RemoteWe are hiring a Senior Data Engineer for one of our leading software development clients. The Senior Data Engineer will play a key role in building and extending data pipeline architecture, optimizing data flow, and supporting cross-functional teams. The ideal candidate has extensive experience in...

  • Big Data Engineer

    4 weeks ago

    Anywhere in India/Multiple Locations/Remote Hum Technologies Full time

    Big Data Engineer with strong exp in Hadoop, Spark, SQL & AWS for one of the Fortune 500 Clients Company Name : Hum TechnologiesClient : One of the Fortune 500 CompaniesLocation : : Apache NiFi, HDFS, Hive, Impala, Spark, Spark Streaming, Kafka, Java/Scala, SQL, Shell scripting Roles & Responsibilities : - Responsible for designing and implementing...

  • Big Data Engineer

    2 months ago

    Anywhere in India/Multiple Locations/Remote, IN Hum Technologies Full time

    Big Data Engineer with strong exp in Hadoop, Spark, SQL & AWS for one of the Fortune 500 Clients Company Name : Hum TechnologiesClient : One of the Fortune 500 CompaniesLocation : : Apache NiFi, HDFS, Hive, Impala, Spark, Spark Streaming, Kafka, Java/Scala, SQL, Shell scriptingRoles & Responsibilities :- Responsible for designing and implementing data...

  • Data Engineer

    4 weeks ago

    Anywhere in India/Multiple Locations Praxis Consultants Full time

    Job Summary We are looking for a highly skilled and experienced Technical Lead (Sr. Data Engineer/Data Architect / Data Modeler) to join our team. will play a crucial role in designing data architecture and models based on business requirements, ensuring data integrity, security, and performance. Oversee technical aspects of data integration and management...

  • Data Engineer

    2 months ago

    Anywhere in India/Multiple Locations, IN Praxis Consultants Full time

    Job SummaryWe are looking for a highly skilled and experienced Technical Lead (Sr. Data Engineer/Data Architect / Data Modeler) to join our team. will play a crucial role in designing data architecture and models based on business requirements, ensuring data integrity, security, and performance. Oversee technical aspects of data integration and management...

  • Apache Spark Developer

    3 weeks ago

    India Wipro Full time

    Responsibilities Excellent Knowledge on Spark; The professional must have a thorough understanding Spark framework, Performance Tuning etc Excellent Knowledge and hands-on experience of at least 4+ years in Scala or PySpark Excellent Knowledge of the Hadoop eco System- Knowledge of Hive mandatory Strong Unix and Shell Scripting Skills Excellent...

  • GCP Data Engineer

    6 months ago

    Pune/Hyderabad/Anywhere in India/Multiple Locations Huquo Full time

    Job Description : Must-Have :- 5+ Years of Experience in Data Engineering and building and maintaining large-scale data pipelines.- Experience with designing and implementing a large-scale Data-Lake on Cloud Infrastructure - Strong technical expertise in Python and SQL- Extremely well-versed in Google Compute Platform including BigQuery, Cloud Storage,...