Big Data Developer

4 weeks ago

Chennai, India Hirexa Solutions Full time

Job Description :


We are looking for an experienced Big Data Developer with strong expertise in Hadoop, Spark, Scala, and data pipeline development. The ideal candidate will have a solid background in big data technologies, scalable data processing, and API development, combined with hands-on skills in distributed computing.

This role involves designing and implementing efficient data pipelines, developing APIs, and maintaining large-scale data infrastructure to support data-driven decision-making.

Experience : 6+ years

Education : Bachelor's or Master's degree in Computer Science, Engineering, Mathematics, or a related field.

Roles and Responsibilities :

1. Data Pipeline Development :

- Design and develop robust, scalable data pipelines using Scala and Spark for efficient data processing.

- Collaborate with data analysts and scientists to understand and translate data requirements into technical solutions.

2. Spark and API Development :

- Implement and optimize Spark applications for data transformation and manipulation.

- Develop RESTful APIs to expose processed data to various applications and services.

3. Big Data Infrastructure Management :

- Optimize and maintain the big data infrastructure, including Hadoop, Spark, and other related components.

- Troubleshoot and debug issues related to big data systems, ensuring smooth data processing workflows.

4. Continuous Learning and Innovation : Stay updated with advancements in big data technologies and adopt best practices in big data development.

Skills and Qualifications Required :

1. Big Data Technologies (6+ years) :

- Extensive experience with Big Data frameworks, particularly Hadoop and Spark.

- Strong understanding of the Hadoop ecosystem, including HDFS, YARN, and MapReduce.

- Familiarity with big data workflow schedulers like Oozie and Airflow.

2. Scala and Spark Expertise (4+ years each) :

- Proficiency in Scala programming for big data applications.

- In-depth knowledge of Spark APIs and experience developing efficient Spark applications.

3. Data Pipeline & API Development :

- Proven experience designing and building data pipelines using Scala.

- Skilled in developing RESTful APIs to expose data to external applications.

Additional Skills :

- Experience with Apache Kafka for real-time data streaming (2+ years).

- Strong SQL proficiency for data querying and manipulation (4+ years).

- Knowledge of Unix/Linux operating systems and shell scripting (3-5 years).

- Familiarity with Citi VDI (Virtual Development Infrastructure) is a plus.

Hands-On Development Experience :

- Experience with PySpark, ScalaSpark, and distributed computing.

- 4-6 years of experience in developing and implementing big data applications.

- Knowledge of micro-services architecture and cloud is a plus.

- Java and Scala knowledge is an added advantage.

Preferred Qualifications :

- Experience with Python for big data processing (4-6 years).

- Familiarity with the Hadoop platform, including Hive, HDFS, and Spark

(ref:hirist.tech)

  • Big Data Developer

    1 month ago

    Chennai, Tamil Nadu, India RiskInsight Consulting Pvt Ltd Full time

    Job Title: Big Data DeveloperJob Summary:RiskInsight Consulting Pvt Ltd is seeking a skilled Big Data Developer to join our team. As a Big Data Developer, you will design, develop, and deploy scalable Big Data applications using Apache Spark.Responsibilities:Data Ingestion: Implement data ingestion processes from various data sources to the Big Data...

  • Big Data Developer

    1 month ago

    Chennai, Tamil Nadu, India RiskInsight Consulting Pvt Ltd Full time

    Job Title: Big Data DeveloperAbout the Role:RiskInsight Consulting Pvt Ltd is seeking a skilled Big Data Developer to join our team. As a Big Data Developer, you will be responsible for designing, developing, and deploying scalable Big Data applications using Hadoop ecosystem technologies.Key Responsibilities:Design and develop scalable Big Data applications...

  • Big Data Developer

    2 months ago

    Chennai, Tamil Nadu, India Mavinsys Full time

    Minimum Required Skills & Experience :1. Big Data Technologies (6+ years) :- Extensive experience with Big Data frameworks like Hadoop and Spark.- Deep understanding of Hadoop ecosystem (HDFS, YARN, MapReduce).- Familiarity with big data schedulers (e.g., Oozie, Airflow).2. Scala & Spark Expertise (4+ years each) :- Strong proficiency in Scala programming...

  • Big Data Developer

    2 months ago

    Chennai, India Mavinsys Full time

    Minimum Required Skills & Experience :1. Big Data Technologies (6+ years) :- Extensive experience with Big Data frameworks like Hadoop and Spark.- Deep understanding of Hadoop ecosystem (HDFS, YARN, MapReduce).- Familiarity with big data schedulers (e.g., Oozie, Airflow).2. Scala & Spark Expertise (4+ years each) :- Strong proficiency in Scala programming...

  • Big Data Developer

    1 month ago

    Chennai, India ZF Group Full time

    Job Description:We are currently seeking dynamic individuals to join our team and contribute to the success for our organization.Experience - 5 to 10 yrsRequirementBachelor’s degree in Computer Science, Engineering, Information Technology, or a related field.Demonstrable experience as a Big Data Developer, Data Engineer, or similar role with a minimum of 5...

  • Big Data Developer

    3 weeks ago

    Chennai, India ZF Group Full time

    Job Description:We are currently seeking dynamic individuals to join our team and contribute to the success for our organization.Experience - 5 to 10 yrsRequirement- Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field.- Demonstrable experience as a Big Data Developer, Data Engineer, or similar role with a minimum...

  • Big Data Developer

    3 weeks ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Job DescriptionAs a Data Architecture Specialist at Virtusa, you will have the opportunity to design and build high-performance big data applications using Spark, Sqoop, and Hive.We are seeking a highly experienced and skilled professional with a strong background in big data processing and a bachelor's or master's degree in Computer Science or a related...

  • Big Data Developer

    1 month ago

    chennai, India ZF Group Full time

    Job Description: We are currently seeking dynamic individuals to join our team and contribute to the success for our organization. Experience - 5 to 10 yrs Requirement Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field. Demonstrable experience as a Big Data Developer, Data Engineer, or similar role with a...

  • Big Data Developer

    4 weeks ago

    Chennai, India ZF Group Full time

    Job Description:We are currently seeking dynamic individuals to join our team and contribute to the success for our organization.Experience - 5 to 10 yrsRequirementBachelor’s degree in Computer Science, Engineering, Information Technology, or a related field.Demonstrable experience as a Big Data Developer, Data Engineer, or similar role with a minimum of 5...

  • Big Data Developer

    1 month ago

    Chennai, India ZF Group Full time

    Job Description:We are currently seeking dynamic individuals to join our team and contribute to the success for our organization.Experience - 5 to 10 yrsRequirementBachelor’s degree in Computer Science, Engineering, Information Technology, or a related field.Demonstrable experience as a Big Data Developer, Data Engineer, or similar role with a minimum of 5...

  • Big Data Developer

    1 month ago

    chennai, India ZF Group Full time

    Job Description:We are currently seeking dynamic individuals to join our team and contribute to the success for our organization.Experience - 5 to 10 yrsRequirementBachelor’s degree in Computer Science, Engineering, Information Technology, or a related field.Demonstrable experience as a Big Data Developer, Data Engineer, or similar role with a minimum of 5...

  • Big Data Developer

    1 month ago

    Chennai, India ZF Group Full time

    Job Description: We are currently seeking dynamic individuals to join our team and contribute to the success for our organization. Experience - 5 to 10 yrs Requirement Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field. Demonstrable experience as a Big Data Developer, Data Engineer, or similar role with a...

  • Big Data Developer

    3 weeks ago

    chennai, India ZF Group Full time

    Job Description: We are currently seeking dynamic individuals to join our team and contribute to the success for our organization. Experience - 5 to 10 yrs Requirement Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field. Demonstrable experience as a Big Data Developer, Data Engineer, or similar role with a...

  • Big Data Developer

    3 weeks ago

    Chennai, Tamil Nadu, India Tata Consultancy Services Full time

    Job Title: Big Data DeveloperDescription:TCS is hiring a Big Data Developer to work on our exciting projects. As a Big Data Developer, you will be responsible for transferring and transforming data with Azure Synapse Analytics pipelines.Responsibilities:Transfer and transform data with Azure Synapse Analytics pipelines.Build a data pipeline in Azure Synapse...

  • Big Data Developer

    4 weeks ago

    Chennai, India Sadup Softech Full time

    Job Position: BIG DATA DEVELOPERLocation: Chennai Exp: 6+ YearsMust Have SkillsSpark-SQL, Scala, Spark, SQL, Hive Bachelors/Masters in Engineering, Computer Science, or equivalent experience 7+ years of experience in the IT industry, experience in Data space is preferred. Advanced scripting experience Expertise in Hadoop, Spark, Spark-sql ,sql, Scala...

  • Big Data Developer

    3 weeks ago

    Chennai, Tamil Nadu, India Snaphunt Full time

    The OpportunitySnaphunt is seeking a talented Big Data Developer to join their team. As a Big Data Developer, you will be responsible for designing, developing, and maintaining scalable data processing pipelines using Kafka, PySpark, Python/Scala, and Spark.The RequirementsDesign and implement robust data processing solutions using Kafka, PySpark,...

  • Big Data Developer

    2 weeks ago

    Chennai, Tamil Nadu, India Mavinsys Full time

    Data Infrastructure ExpertKey Responsibilities• Design and develop scalable data pipelines using Scala and Spark.• Implement robust and efficient Spark applications for data processing.• Develop RESTful APIs to provide access to processed data for various applications.• Optimize and maintain existing big data infrastructure and pipelines.•...

  • Big Data Developer

    5 months ago

    Chennai, India Virtusa Full time

    Big Data Developer - CREQ188135 Description Job Description: 5+ years of hands-on experience in designing, building and supporting Data Applications using Spark, Sqoop and Hive Bachelors or masters degree in Computer Science or related field Strong knowledge of working with large data sets and high-capacity big data processing platform Strong experience...

  • Senior Big Data Developer

    4 weeks ago

    Chennai, Tamil Nadu, India Risk Resources LLP Full time

    Job Title: Senior Big Data DeveloperRole OverviewThe role of a Senior Big Data Developer is crucial in our organization as they are responsible for developing and maintaining efficient, scalable data processing applications using Pyspark.Key Responsibilities* Experience working in big data platforms for implementing projects with high SLA of data...

  • Big Data Developer

    1 month ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Job Description:As a Big Data Developer at Virtusa, you will be responsible for designing, building, and supporting data applications using Spark, Sqoop, and Hive. You will work with large data sets and high-capacity big data processing platforms, leveraging your expertise in Unix and Shell scripting, as well as your knowledge of the Hadoop ecosystem and its...