Apache Spark Developer

2 months ago

Anywhere in IndiaMultiple Locations, IN IT Source Global Full time

We have Immediate openings on Apache developer

Job description :

- Data Pipeline Development : Design, develop, and maintain data processing pipelines using Apache Spark and Scala, ensuring high performance and scalability.

- Data Transformation and Analysis : Implement complex data transformations, aggregations, and analytics using Spark, optimizing for efficiency and speed.

- Data Integration : Integrate various data sources, including structured and unstructured data, into the data processing environment.

- Performance Tuning : Optimize Spark jobs and Scala code for performance, memory management, and resource utilization.

- Collaboration : Work closely with data scientists, data engineers, and business analysts to understand data requirements and provide technical solutions.

- ETL Processes : Develop and maintain ETL processes to extract, transform, and load data into data lakes, data warehouses, and other storage solutions.

- Testing and Validation : Implement testing frameworks for data pipelines, ensuring data quality and accuracy through rigorous validation processes.

- Version Control : Utilize version control systems such as Git for managing code and collaborating with team members.

- Documentation : Create and maintain comprehensive documentation for data pipelines, processes, and best practices.

Required Qualifications :

Education : Bachelor's degree in Computer Science, Data Engineering, or a related field.

Experience :

- 3+ years of experience in big data development with Apache Spark.

- Strong proficiency in Scala programming.

- Experience with distributed computing and parallel processing.

- Familiarity with Hadoop ecosystem components (HDFS, Hive, HBase, etc.).

Skills :

- Proficient understanding of data processing frameworks, particularly Apache Spark.

- Strong knowledge of SQL and experience with databases like MySQL, PostgreSQL, or similar.

- Experience with cloud platforms like AWS, Azure, or Google Cloud, particularly with big data services.

- Knowledge of containerization and orchestration tools (e.g., Docker, Kubernetes) is a plus.

- Strong problem-solving skills and attention to detail.

- Excellent communication and teamwork abilities.

Preferred Qualifications :

- Experience with real-time data processing frameworks (e.g., Apache Kafka, Apache Flink).

- Knowledge of machine learning libraries and tools (e.g., MLlib).

- Familiarity with data visualization tools and techniques.

- Experience with CI/CD pipelines and automation tools.

- Understanding of data security and compliance standards.

Benefits :

- Competitive salary

- Health, dental, and vision insurance

- 401(k) with company match

- Professional development opportunities

- Flexible work hours and remote work options

To Apply : Please submit your resume and a cover letter detailing your experience and why you would be a great fit for this role.

(ref:hirist.tech)

  • Data Architect

    2 months ago

    Anywhere in India/Multiple Locations, IN RAPINNO TECH SOLUTIONS PRIVATE LIMITED Full time

    Position Overview :We are looking for an experienced Data Architect with expertise in Databricks, Apache Spark, and ETL processes. The ideal candidate will have a proven track record of designing and building robust data applications from the ground up. You will play a key role in shaping our data strategy and ensuring our data solutions meet business needs...

  • Data Engineer

    2 months ago

    Anywhere in India/Multiple Locations, IN Huntsmen & Barons Full time

    Role : Data Engineer - Apache spark,Spark, Scala, PYspark, Hadoop, Hbase, HiveLocation : Pan IndiaWork Mode- HybridYear of Experience :Minimum 5 Years tp 12 YearsJob Description :- Relevant Experience 5 years.- The indent is created for Apache spark. But the actual demand is for Spark/ Scala.- Expertise in Data Engineering.- Hands-on experience in design and...

  • Spark Developer

    2 months ago

    Anywhere in India/Multiple Locations/Metros/Pune, IN Talent Pro Full time

    Role : Spark DeveloperJob Description :- Work with development teams and product managers to build new product features, enhancements etc.- Build the front-end of applications through appealing visual design- Develop and manage well-functioning databases and applications- Design & develop data and event processing pipelines- Test software to ensure...

  • Data Engineer

    2 months ago

    Anywhere in India/Multiple Locations, IN Huntsmen & Barons Full time

    Role : Data Engineer - Spark/Scala. Job Description :- Relevant Experience 5 years.- The indent is created for Apache spark. But the actual demand is for Spark/ Scala.- Expertise in Data Engineering. - Hands-on experience in design and development of big data platform.- Deep understanding of modern data processing technology stacks: Spark, HBase, Hive and...

  • Senior Data Engineer

    2 months ago

    Anywhere in India/Multiple Locations/Remote, IN Hunarstreet Technologies Pvt Ltd Full time

    Job Title : Senior Data Engineer (Remote)Location : RemoteWe are hiring a Senior Data Engineer for one of our leading software development clients. The Senior Data Engineer will play a key role in building and extending data pipeline architecture, optimizing data flow, and supporting cross-functional teams. The ideal candidate has extensive experience in...

  • Big Data Engineer

    2 months ago

    Anywhere in India/Multiple Locations/Remote, IN Hum Technologies Full time

    Big Data Engineer with strong exp in Hadoop, Spark, SQL & AWS for one of the Fortune 500 Clients Company Name : Hum TechnologiesClient : One of the Fortune 500 CompaniesLocation : : Apache NiFi, HDFS, Hive, Impala, Spark, Spark Streaming, Kafka, Java/Scala, SQL, Shell scriptingRoles & Responsibilities :- Responsible for designing and implementing data...

  • Data Engineer

    2 months ago

    Anywhere in India/Multiple Locations, IN Praxis Consultants Full time

    Job SummaryWe are looking for a highly skilled and experienced Technical Lead (Sr. Data Engineer/Data Architect / Data Modeler) to join our team. will play a crucial role in designing data architecture and models based on business requirements, ensuring data integrity, security, and performance. Oversee technical aspects of data integration and management...

  • Capgemini - AWS EMR Engineer

    2 months ago

    Anywhere in India/Multiple Locations, IN Capgemini Full time

    Roles & Responsibilities - AWS+EMR- Minimum 2-6 years of experience in AWS technologies.- Setting up and managing EMR clusters for processing large-scale data using frameworks like Apache Hadoop, Apache Spark, Apache Hive, etc.- Configuring EMR clusters based on specific requirements, including choosing the appropriate instance types, storage configurations,...

  • Python Developer

    1 month ago

    Anywhere in India/Multiple Locations/Pune, IN Intraedge Technologies Ltd. Full time

    Minimum 4+ experience, 8+ for SeniorRequired Technical Qualifications for both levels :- Strong Object oriented programming fundamentals (Abstraction, Inheritance, Polymorphism, Encapsulation)- Strong fundamental knowledge of data structures and algorithms- At least 4 (8 for senior) years of Experience with a recent object oriented language...

  • Big Data Engineer

    3 weeks ago

    Anywhere in India/Multiple Locations, IN Changeleaders Full time

    Key Responsibilities :Data Pipeline Development :- Design, develop, and maintain scalable data pipelines using Big Data technologies (e.g., Apache Hadoop, Apache Spark, Kafka).- Implement data ingestion strategies to efficiently load and transform large datasets from various sources.Data Architecture and Modeling :- Collaborate with data architects to design...

  • NucleusTeq - Big Data Engineer - Spark/Scala

    4 weeks ago

    Anywhere in India/Multiple Locations/Remote, IN NucleusTeq Consulting Pvt. Ltd. Full time

    Role : Big Data Engineer Responsibilities :- Big Data Engineer with 5-6 years of experience- Experience in leading a team of engineers and a good attitude toward learning the domain and implementation- Strong exposure and expertise in Spark (Primary), Scala/Java (Scala Primary), Airflow Orchestration and AWS.- Finalizing the scope of the system and...

  • Senior Data Engineer

    2 months ago

    Anywhere in India/Multiple Locations, IN iKonsult Full time

    Job Description :Essential Technical Skills :- Technical expertise with emerging Big Data technologies, such as : Python, Spark, Hadoop, Clojure, Git, SQL and Databricks; and visualization tools : Tableau and PowerBI- Experience with cloud, container and micro service infrastructures- Experience working with divergent data sets that meet the requirements of...

  • Technical Lead

    2 months ago

    Anywhere in India/Multiple Locations, IN Praxis Consultants Full time

    Job Summary :We are looking for a highly skilled and experienced Technical Lead (Sr. Data Engineer/Data Architect / Data Modeler) to join our team. will play a crucial role in designing data architecture and models based on business requirements, and ensuring data integrity, security, and performance. Oversee technical aspects of data integration and...

  • Excelra - Data Engineer - Python/AWS/Azure

    2 months ago

    Anywhere in India/Multiple Locations, IN Excelra Knowledge Solutions Full time

    Data Engineer (Cloud & Ontologies)Job Description : Excelra is hiring for experienced Data Engineer with expertise in ontologies, AWS, Azure, and various data technologies. The role involves building and maintaining scalable data pipelines, working with cloud platforms, and supporting server and application deployment.Roles and Responsibilities : - Cloud...

  • Data Engineer

    2 months ago

    Hyderabad/Anywhere in India/Multiple Locations, IN Circuit Compilers Full time

    Job Description :- 5 to 8 years in the build release management, software configuration, design, development, and cloud implementation.- Data Engineer with expertise in Python, Scala, PySpark, AWS S3, Glue, Redshift, Lambda, DBT, Serverless, Snowflake, SQL, Datalake, and Data Pipeline.- Leveraged AWS Glue for data cataloging, ETL job orchestration, and...

  • Data Engineer

    2 months ago

    Anywhere in India/Multiple Locations, IN IT Source Global Full time

    Role : Data EngineerJob Description :- Design, build, and maintain large-scale data pipelines and ETL/ELT workflows using PySpark, Python, and SQL for data extraction, transformation, and loading.- Work extensively with AWS services such as Amazon S3 for data storage and Athena for querying and analyzing structured and unstructured data.- Develop and manage...

  • Azure Databricks Engineer

    2 months ago

    Anywhere in India/Multiple Locations, IN Hum Technologies Full time

    We are hiring for Azure Databricks Engineer Data Engineering experience on AWS/Azure and Databricks Strong Experience in Databricks, AWS/Azure, and SQL , creation of jobs using Pyspark. good exposure to Python.For one of the Fortune 500 Clients.Company Name : Hum TechnologiesClient : One of the Fortune 500 CompaniesLocation : Remote/ HybridRole : Azure...

  • Data Engineer

    2 months ago

    Anywhere in India/Multiple Locations, IN Risk Resources India Full time

    Job Description :- Strong experience in programming languages such as Python, Java, or Scala for data manipulation and engineering tasks. - Expertise in SQL and NoSQL databases - Hands-on experience with big data technologies like Hadoop, Spark, Kafka, and Hive to handle large-scale data processing and real-time data streams. - In-depth knowledge of data...

  • Java Full Stack Developer

    6 months ago

    Anywhere in India/Multiple Locations, IN Anlage Infotech Pvt. Ltd. Full time

    One of our client which is a leading IT Services company is looking for:Java Full stack DeveloperJob Location - Mumbai / Bangalore / Pune / Noida / Chennai / HyderabadExp - 5 - 10 yrsPlease find below the JD :Experience in Java Core JSP Struts Framework Web app Deployment in Apache Tomcat ServerExperience in JavaScript html CSS and Oracle DBGood knowledge in...

  • Lead Data Engineer

    4 weeks ago

    Anywhere in India/Multiple Locations, IN Talentoj Full time

    Requirements : - 6+ years' experience with Big Data Technologies such as Apache Spark, Cloud Native, and Data Lakes, and Databricks. - Delivered ETL/ELT solutions including data extraction, transformation, cleansing, data integration, and data management. - Implemented batch ingestion pipelines. - Strong Python skills - automate tasks, write unit tests...