Big Data Developer

2 months ago

HyderabadBangaloreChennaiBhubaneshwar, IN Rigel networks Full time

Job Title : Big Data/Spark Scala Developer

Location : Type : Full-Time

Job Overview :

We are looking for an experienced Big Data/Spark Scala Developer to join our team. The ideal candidate will have a deep understanding of big data technologies and strong programming skills in Scala. In this role, you will be responsible for designing, developing, and optimizing data processing pipelines using Apache Spark, as well as working with large datasets to derive actionable insights.

Key Responsibilities :

- Data Pipeline Development : Design, develop, and maintain scalable and high-performance data pipelines using Apache Spark and Scala to process and analyze large datasets.

- Data Integration : Integrate data from various sources, including databases, data lakes, and real-time data streams. Ensure data consistency and quality across all data sources.

- Optimization : Optimize Spark jobs for performance, including tuning job configurations, optimizing query performance, and managing resources effectively.

- Collaboration : Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs.

- Code Quality : Write clean, maintainable, and efficient code. Conduct code reviews and ensure adherence to best practices and coding standards.

- Testing and Debugging : Implement robust testing strategies and debug issues in data processing pipelines to ensure reliability and accuracy.

- Documentation : Create and maintain comprehensive documentation for data processing workflows, system configurations, and development procedures.

- Continuous Improvement : Stay up-to-date with the latest industry trends, tools, and technologies related to big data and Spark. Propose and implement improvements to enhance data processing capabilities.

Qualifications :

- Education : Bachelor's or Master's degree in Computer Science, Data Science, Engineering, or a related field.

- Experience : Minimum of 7 years of experience in big data technologies, with a focus on Apache Spark and Scala development.

- Technical Skills :

1. Apache Spark : Extensive experience with Apache Spark, including Spark SQL, DataFrames, and RDDs.

2. Programming : Proficiency in Scala for big data processing. Knowledge of other programming languages such as Python or Java is a plus.

3. Big Data Ecosystem : Experience with Hadoop, Hive, HBase, or other big data technologies.

4. Data Integration Tools : Familiarity with data integration tools and technologies (e.g., Kafka, Flume).

5. Databases : Experience with SQL and NoSQL databases.

- Analytical Skills : Strong analytical and problem-solving skills with the ability to work with complex data structures and derive meaningful insights.

- Communication : Excellent verbal and written communication skills, with the ability to convey technical information to non-technical stakeholders.

- Project Management : Experience with Agile methodologies and tools is a plus.

Desired Attributes :

- Attention to Detail : High level of accuracy and attention to detail in data processing and analysis.

- Innovation : Ability to think creatively and propose innovative solutions to complex data challenges.

- Team Player : Collaborative approach with a strong ability to work effectively in a team environment.

(ref:hirist.tech)

  • Big Data Engineer

    1 month ago

    Chennai/Hyderabad/Bangalore/Coimbatore/Mumbai/Delhi/Pune, IN GAIUS HYPERLOCAL PRIVATE LIMITED Full time

    Job Description : Big Data Engineer - Scala, SparkYears of exp: 6 to 13 yearsJob location: Chennai / Hyderabad / Bangalore / Pune / Mumbai / Delhi / CoimbatoreAbout the Role :We are seeking a highly skilled Big Data Engineer to join our team and help us leverage the power of big data to drive innovation and business value. As a Big Data Engineer, you will be...

  • Data Engineer

    1 month ago

    Bangalore/Pune/Trivandrum/Thiruvananthapuram, IN Rao Career Solutions Full time

    Job Summary :- We are looking for a skilled Data Engineer with hands-on experience in PySpark and Big Data platforms to join our growing team. The ideal candidate will have strong expertise in building data pipelines, working with big data technologies like Apache Hive and Parquet, and handling diverse data use cases. You will work closely with data...

  • PySpark Developer

    2 months ago

    Pune/Bangalore/Chennai/Hyderabad/Mysore, IN Vtricks Technologies Full time

    Job Title : PySpark Developer Location : Pune, Bangalore, Chennai, Hyderabad, MysoreMode of Work : Onsite/HybridNotice Period : Immediate to 10 daysJob Description :We are looking for an experienced PySpark Developer with strong hands-on experience in working with Data Frames and the PySpark development process using Azure services. The ideal candidate...

  • Big Data Engineer

    1 month ago

    Bangalore/Pune/Mumbai/Chennai, IN a client of change leaders Full time

    Job Description : - 5+ years of experience developing scalable Big Data applications or solutions on distributed platforms- Design and implement data pipelines using Snowflake, PostgreSQL, and Python.- Experience in AWS/ Azure/ GCP and good to have other cloud platform tools- Experience working with Big Data platforms, including Hadoop- Hive, Data Bricks...

  • Hadoop Administrator

    5 months ago

    Mumbai/Bangalore/Bhubaneshwar/Anywhere in India/Multiple Locations, IN Tehno Right Full time

    Job Role : Hadoop Administrator (Role open for multiple locations) - WFH and WFOJob description :What is your Role ?- You will manage Hadoop clusters, data storage, server resources, and other virtual computing platforms. - You perform a variety of functions, including data migration, virtual machine set-up and training, troubleshooting end-user problems,...

  • Program Lead/Technology Consultant

    1 month ago

    Chennai/Bangalore/Hyderabad, IN Resourcetree Full time

    About the role :As Program Lead - Technology Consulting, you will be key member of the teams working on end-to-end AI and analytics solutions being built for our clients. While your primary focus will be on the data engineering and application engineering components of the overall solution, you will collaborate with internal data science teams for the ML...

  • Big Data Engineer

    2 months ago

    Hyderabad/Pune/Delhi NCR, IN Ace Recruit Full time

    Job Description :As a Big Data Engineer (Azure), you will build and learn about a variety of analytics solutions & platforms, data lakes, modern data platforms, data fabric solutions, etc. using different Open Source, Big Data, and Cloud technologies on Microsoft Azure. On a typical day, you might :- Design and build scalable & metadata-driven data ingestion...

  • Big Data Team Lead

    1 month ago

    Bangalore/Chennai/Pune, IN Rigel networks Full time

    Job Title : Senior Software Developer / Applications Development ManagerLocation : Pune, Chennai, Bangalore Shift Timings : 10 AM - 7 PMPosition Overview : We are seeking an experienced Senior Software Developer / Applications Development Manager with a strong background in software development and project management. The ideal candidate will possess...

  • Big Data Engineer

    1 month ago

    Chennai/Pune, IN Pan Asia Man power Consultants Recruiting agency Full time

    Job Title : Big Data EngineerExperience Range : 6+ YearsLocation : Chennai/Pune (Hybrid - 2 Days Work From Office in a Week)Job Description :We are seeking a skilled Big Data Engineer to join our Agile team and contribute to building backend data applications and services that support our analytics platforms. This role requires a hands-on developer with...

  • Big Data Engineer

    1 month ago

    Chennai/Noida/Gurgaon/Gurugram/Mumbai, IN ANRI Solutions HR Services Full time

    Role : Big Data Engineer Looking for an experienced Spark and Scala developer able to create an end-to-end data pipeline.Associate should be self-driven, can work with minimal guidance and guide the team technically. Job Details : 1. Expertise in Spark framework with scala language and Pyspark 2. Expertise in ETL and Data Warehousing and cloud concepts 3....

  • Data Engineer

    2 months ago

    Hyderabad/Bhubaneshwar, IN TALENTOV Full time

    Job Title : Senior Data EngineerLocation : Hyderabad/BhubhaneshwarJob Summary :We are seeking a skilled and experienced Senior Data Engineer to join our dynamic data team. The ideal candidate will have a strong background in data engineering, data warehousing, and a proven track record of implementing scalable data solutions. You will work closely with data...

  • Big Data Administrator

    1 month ago

    Bangalore/Gurgaon/Gurugram/Nagpur, IN Intuitive Apps Full time

    As a Big Data Admin, you will play a crucial role in ensuring the smooth operation and optimization of our Hadoop and Spark infrastructure. Your responsibilities will include :Hadoop Installation and Deployment :- Install Hadoop in Linux environments.- Deploy and maintain Hadoop clusters.- Monitor cluster health to ensure continuous uptime.- Analyze storage...

  • Data Modeler

    1 month ago

    Bangalore/Hyderabad/Chennai/Pune, IN Hashone Careers Private Limited Full time

    Role Overview :We are seeking an experienced Data Modeller with a strong foundation in data architecture, modelling, and analysis. The ideal candidate will be proficient in Erwin Data Modelling and have a deep understanding of data quality, data lineage, and data cataloging. You will play a crucial role in designing and optimizing data models for both OLTP...

  • Big Data Engineer

    6 months ago

    Rajasthan/Karnataka/Metros/Jaipur/Udaipur/Bangalore/Raipur, IN ATech Full time

    Hiring For Big Data Engineer (Spark, Scala, AWS)Job description :- Spark, SCALA, AWS OR Spark, python with AWS lambda exposure.- Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements.- Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing...

  • Data Engineer

    4 weeks ago

    Noida/Bangalore/Chennai/Hyderabad/Coimbatore, IN Huntsmen & Barons Full time

    Role : Data Engineer - Apache spark/Spark/Scala/pysparkJob Description :- Relevant Experience 5 years.- The indent is created for Apache spark. But the actual demand is for Spark/ Scala.- Expertise in Data Engineering.- Hands-on experience in design and development of big data platform.- Deep understanding of modern data processing technology stacks: Spark,...

  • Informatica PowerCenter/BDM Developer

    2 months ago

    Mumbai/Pune/Chennai/Hyderabad/Gurgaon/Gurugram, IN Risk Resources Full time

    Job Description : We are looking for a highly skilled Informatica PowerCenter / BDM Developer with 7-12 years of experience in designing, developing, and managing data integration solutions. The candidate should have strong expertise in Informatica PowerCenter or Big Data Management (BDM), and extensive experience in building efficient ETL/ELT workflows and...

  • Kafka Developer

    1 month ago

    Bangalore/Pune/Bhubaneshwar/Cochin/Kochi, IN Risk Resources LLP Full time

    Overview :As a Kafka Developer, you will play a critical role in the design, development, and implementation of Kafka-based event streaming solutions. You will work closely with cross-functional teams to build scalable, high-performance data pipelines and real-time streaming applications to meet the organization's evolving data needs. This role is...

  • Wipro - Senior Data Engineer - Python/Scala

    2 months ago

    Bangalore/Hyderabad/Chennai/Pune, IN Wipro Ltd Full time

    Job Description : In this role, you will :- Design & Develop batch and streaming solutions using various Big Data technologies- Work closely with architect to design secure and scalable- Be responsible for assessing technologies and approaches for ingestion, transformation and storage- Develop software utilizing open-source technologies to interface with...

  • GCP Data Engineer

    1 month ago

    Bangalore/Hyderabad/Chennai, IN TechnoGen India Private Limited Full time

    What you'll be doing :As a Data Engineer with ETL/ELT expertise for our growing data platform and analytics teams, you will understand and enable the required data sets from different sources. You will take both structured and unstructured data into our data warehouse and data lake with real-time streaming and/or batch processing to generate insights and...

  • Azure Data Engineer

    1 month ago

    Pune/Hyderabad/Bangalore/Chennai, IN BSRI Solutions Full time

    Job Description :We are seeking a talented and experienced Azure Data Engineer to join our team. The ideal candidate will have a strong background in data engineering with a focus on Microsoft Azure services. The candidate should have a minimum of 5-10 years of hands-on experience with Azure Data Factory (ADF), Azure Synapse Analytics, Synapse Pipelines,...