Apache Spark Engineer
4 weeks ago
We are seeking a highly skilled Apache Spark Developer to join our team at IT Source Global. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining large-scale data processing pipelines using Apache Spark and Scala.
Key Responsibilities:- Design and develop efficient data processing pipelines using Apache Spark and Scala
- Implement complex data transformations, aggregations, and analytics using Spark
- Integrate various data sources, including structured and unstructured data, into the data processing environment
- Optimize Spark jobs and Scala code for performance, memory management, and resource utilization
- Collaborate with data scientists, data engineers, and business analysts to understand data requirements and provide technical solutions
- Develop and maintain ETL processes to extract, transform, and load data into data lakes, data warehouses, and other storage solutions
- Implement testing frameworks for data pipelines, ensuring data quality and accuracy through rigorous validation processes
- Utilize version control systems such as Git for managing code and collaborating with team members
- Create and maintain comprehensive documentation for data pipelines, processes, and best practices
- Bachelor's degree in Computer Science, Data Engineering, or a related field
- 3+ years of experience in big data development with Apache Spark
- Strong proficiency in Scala programming
- Experience with distributed computing and parallel processing
- Familiarity with Hadoop ecosystem components (HDFS, Hive, HBase, etc.)
- Experience with real-time data processing frameworks (e.g., Apache Kafka, Apache Flink)
- Knowledge of machine learning libraries and tools (e.g., MLlib)
- Familiarity with data visualization tools and techniques
- Experience with CI/CD pipelines and automation tools
- Understanding of data security and compliance standards
- Competitive salary
- Health, dental, and vision insurance
- 401(k) with company match
- Professional development opportunities
- Flexible work hours and remote work options
To apply, please submit your resume and a cover letter detailing your experience and qualifications.
-
Apache Spark Developer
1 month ago
Anywhere in India/Multiple Locations IT Source Global Full timeJob Title: Apache Spark DeveloperWe are seeking a highly skilled Apache Spark Developer to join our team at IT Source Global. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining data processing pipelines using Apache Spark and Scala.Key Responsibilities:Design and develop data processing pipelines...
-
Data Engineer
3 weeks ago
Anywhere in India/Multiple Locations Huntsmen & Barons Full timeAbout the Role:Data Engineer - Apache Spark ExpertHuntsmen & Barons is seeking an experienced Data Engineer with a strong background in Apache Spark to join our team. As a Data Engineer, you will play a key role in designing and developing big data platforms, leveraging expertise in data engineering, Spark, HBase, Hive, and other Hadoop ecosystem...
-
Apache Spark Developer
2 months ago
Anywhere in India/Multiple Locations, IN IT Source Global Full timeWe have Immediate openings on Apache developerJob description : - Data Pipeline Development : Design, develop, and maintain data processing pipelines using Apache Spark and Scala, ensuring high performance and scalability.- Data Transformation and Analysis : Implement complex data transformations, aggregations, and analytics using Spark, optimizing for...
-
Apache Spark Developer
4 weeks ago
Anywhere in India/Multiple Locations IT Source Global Full timeWe have Immediate openings on Apache developerJob description : - Data Pipeline Development : Design, develop, and maintain data processing pipelines using Apache Spark and Scala, ensuring high performance and scalability.- Data Transformation and Analysis : Implement complex data transformations, aggregations, and analytics using Spark, optimizing for...
-
Data Architect
2 months ago
Anywhere in India/Multiple Locations, IN RAPINNO TECH SOLUTIONS PRIVATE LIMITED Full timePosition Overview :We are looking for an experienced Data Architect with expertise in Databricks, Apache Spark, and ETL processes. The ideal candidate will have a proven track record of designing and building robust data applications from the ground up. You will play a key role in shaping our data strategy and ensuring our data solutions meet business needs...
-
Data Engineer
2 months ago
Anywhere in India/Multiple Locations, IN Huntsmen & Barons Full timeRole : Data Engineer - Apache spark,Spark, Scala, PYspark, Hadoop, Hbase, HiveLocation : Pan IndiaWork Mode- HybridYear of Experience :Minimum 5 Years tp 12 YearsJob Description :- Relevant Experience 5 years.- The indent is created for Apache spark. But the actual demand is for Spark/ Scala.- Expertise in Data Engineering.- Hands-on experience in design and...
-
Data Engineer
4 weeks ago
Anywhere in India/Multiple Locations Huntsmen & Barons Full timeRole : Data Engineer - Apache spark,Spark, Scala, PYspark, Hadoop, Hbase, HiveLocation : Pan IndiaWork Mode- HybridYear of Experience :Minimum 5 Years tp 12 YearsJob Description :- Relevant Experience 5 years.- The indent is created for Apache spark. But the actual demand is for Spark/ Scala.- Expertise in Data Engineering.- Hands-on experience in design and...
-
Data Engineer
1 month ago
Anywhere in India/Multiple Locations Huntsmen & Barons Full timeRole : Data Engineer - Spark/Scala. Job Description : - Relevant Experience 5 years.- The indent is created for Apache spark. But the actual demand is for Spark/ Scala.- Expertise in Data Engineering. - Hands-on experience in design and development of big data platform.- Deep understanding of modern data processing technology stacks: Spark, HBase, Hive...
-
Senior Data Engineer
4 weeks ago
Anywhere in India/Multiple Locations/Remote Hunarstreet Technologies Pvt Ltd Full timeJob Title : Senior Data Engineer (Remote) Location : RemoteWe are hiring a Senior Data Engineer for one of our leading software development clients. The Senior Data Engineer will play a key role in building and extending data pipeline architecture, optimizing data flow, and supporting cross-functional teams. The ideal candidate has extensive experience in...
-
Senior Data Engineer
2 months ago
Anywhere in India/Multiple Locations/Remote, IN Hunarstreet Technologies Pvt Ltd Full timeJob Title : Senior Data Engineer (Remote)Location : RemoteWe are hiring a Senior Data Engineer for one of our leading software development clients. The Senior Data Engineer will play a key role in building and extending data pipeline architecture, optimizing data flow, and supporting cross-functional teams. The ideal candidate has extensive experience in...
-
Big Data Engineer
4 weeks ago
Anywhere in India/Multiple Locations/Remote Hum Technologies Full timeBig Data Engineer with strong exp in Hadoop, Spark, SQL & AWS for one of the Fortune 500 Clients Company Name : Hum TechnologiesClient : One of the Fortune 500 CompaniesLocation : : Apache NiFi, HDFS, Hive, Impala, Spark, Spark Streaming, Kafka, Java/Scala, SQL, Shell scripting Roles & Responsibilities : - Responsible for designing and implementing...
-
Big Data Engineer
2 months ago
Anywhere in India/Multiple Locations/Remote, IN Hum Technologies Full timeBig Data Engineer with strong exp in Hadoop, Spark, SQL & AWS for one of the Fortune 500 Clients Company Name : Hum TechnologiesClient : One of the Fortune 500 CompaniesLocation : : Apache NiFi, HDFS, Hive, Impala, Spark, Spark Streaming, Kafka, Java/Scala, SQL, Shell scriptingRoles & Responsibilities :- Responsible for designing and implementing data...
-
Spark Developer
1 month ago
Anywhere in India/Multiple Locations/Metros/Pune Talent Pro Full timeJob DescriptionAs a Spark Developer at Talent Pro, you will play a key role in building scalable and efficient data processing pipelines using Apache Spark. Your primary responsibility will be to design, develop, and manage large-scale data processing systems, ensuring high performance and reliability.Key Responsibilities:Collaborate with development teams...
-
Spark Developer
4 weeks ago
Anywhere in India/Multiple Locations/Metros/Pune Talent Pro Full timeJob Title: Spark DeveloperAt Talent Pro, we are seeking a highly skilled Spark Developer to join our team. As a Spark Developer, you will be responsible for designing and developing scalable distributed architecture using Apache Spark.Responsibilities:Work with development teams and product managers to build new product features, enhancements, and data...
-
Data Engineer
4 weeks ago
Anywhere in India/Multiple Locations Praxis Consultants Full timeJob Summary We are looking for a highly skilled and experienced Technical Lead (Sr. Data Engineer/Data Architect / Data Modeler) to join our team. will play a crucial role in designing data architecture and models based on business requirements, ensuring data integrity, security, and performance. Oversee technical aspects of data integration and management...
-
Apache Spark Administrator
2 months ago
India Marlabs Full time**Date**:Oct 11, 2024**Location**:Any Marlabs Office Location, IN**Company**:Marlabs Innovations Pvt Ltd**Description**: - Spark Administrator - 6 Weeks (Starting November 1st 2024)- Key Responsibilities- Install and configure the operating system (Linux distributions such as Ubuntu, CentOS, or RHEL) on cluster nodes, ensuring consistency across the...
-
Spark Developer
2 months ago
Anywhere in India/Multiple Locations/Metros/Pune, IN Talent Pro Full timeRole : Spark DeveloperJob Description :- Work with development teams and product managers to build new product features, enhancements etc.- Build the front-end of applications through appealing visual design- Develop and manage well-functioning databases and applications- Design & develop data and event processing pipelines- Test software to ensure...
-
GCP Data Engineer
6 months ago
Pune/Hyderabad/Anywhere in India/Multiple Locations Huquo Full timeJob Description : Must-Have :- 5+ Years of Experience in Data Engineering and building and maintaining large-scale data pipelines.- Experience with designing and implementing a large-scale Data-Lake on Cloud Infrastructure - Strong technical expertise in Python and SQL- Extremely well-versed in Google Compute Platform including BigQuery, Cloud Storage,...
-
Senior Data Engineer
4 weeks ago
Anywhere in India/Multiple Locations Huntsmen & Barons Full timeJob Description:Data Engineer - Spark/ScalaAt Huntsmen & Barons, we are seeking a highly skilled Data Engineer - Spark/Scala to join our team. As a key member of our data engineering team, you will be responsible for designing and developing our big data platform using Spark and Scala. Your expertise in data engineering will enable us to process and analyze...
-
Apache Spark Engineer
6 days ago
India Mercor Full timeRole Overview & Key Responsibilities: We are seeking candidates to assist Mercor in a research project with leading AI labs to benchmark and improve the current AI Models. Qualifications: (Must have) 3+ years of experience with Apache Spark (Must have) SQL + Python experience is mandatory Scala/Java is preferred Analytical Excellence Prior...