Big Data Developer

3 weeks ago

Greater Bengaluru Area, India Pixomania Full time
As a Big Data Developer, you will design, develop, and optimize distributed data processing systems using Apache Spark and Hive. You will work on large-scale data platforms and ensure the efficient processing of massive datasets in real-time and batch environments. Your experience with Kafka will be an added advantage in building robust and scalable streaming data systems.

Responsibilities:

Develop, test, and deploy distributed data processing systems using Apache Spark and Hive.
Design and implement scalable and efficient ETL pipelines for large datasets.
Optimize performance of Spark jobs for high throughput and low-latency processing.
Write clean, efficient, and maintainable code in Scala or Java.
Work with data engineers and other teams to integrate data solutions and ensure smooth data flow.
Implement and maintain Hive queries, partitions, and tables for batch processing.
Design and maintain data processing architectures in cloud environments (AWS, Azure, GCP).
Collaborate with business and technical stakeholders to deliver high-quality data solutions.
Work on Kafka (if applicable) to design and implement real-time data streaming applications.
Troubleshoot and optimize performance issues related to Spark and Hive-based systems.
Mentor junior developers and ensure high-quality standards are followed throughout the development lifecycle.
Required Skills and Experience:

5+ years of experience in Big Data development, with extensive hands-on experience in Apache Spark and Hive.
Strong experience with Spark for batch and real-time data processing.
Proven ability to design and implement Hive queries, partitions, and data models.
Good knowledge of Hadoop ecosystem tools and how they integrate with Spark.
Proficiency in SQL for querying and data manipulation.
Familiarity with cloud technologies and platforms (AWS, Azure, GCP).
Experience working with large-scale distributed systems.
Strong understanding of ETL pipelines, data integration, and data processing workflows.
Knowledge of Kafka for stream processing is a plus.
Experience in developing and optimizing high-performance data processing jobs.
Ability to work in a hybrid environment, with occasional on-site presence in Bengaluru/Hyderabad.
Preferred Skills:

Location - Bengaluru/ Hyderabad

  • Big Data Developer

    3 weeks ago

    Greater Bengaluru Area, India Pixomania Full time

    Job Title: Big Data Developer (Scala + Spark)Location: RemoteExperience: 5+ yearsJob Type: Contractual RemoteAbout Us:We are a forward-thinking technology company committed to delivering innovative solutions in Big Data analytics. Join our talented team as a Big Data Developer with expertise in Scala and Apache Spark to drive impactful projects and...

  • Big Data Developer

    3 weeks ago

    Greater Bengaluru Area, India Pixomania Full time

    Job Title: Big Data Developer (Scala + Spark)Location: RemoteExperience: 5+ yearsJob Type: Contractual RemoteAbout Us:We are a forward-thinking technology company committed to delivering innovative solutions in Big Data analytics. Join our talented team as a Big Data Developer with expertise in Scala and Apache Spark to drive impactful projects and...

  • Big Data Developer

    3 weeks ago

    Greater Bengaluru Area, India Pixomania Full time

    As a Big Data Developer, you will design, develop, and optimize distributed data processing systems using Apache Spark and Hive. You will work on large-scale data platforms and ensure the efficient processing of massive datasets in real-time and batch environments. Your experience with Kafka will be an added advantage in building robust and scalable...

  • Big Data Developer

    3 days ago

    Greater Bengaluru Area, India Tata Consultancy Services Full time

    Dear AssociatesGreetings from TATA Consultancy Services!!Thank you for expressing your interest in exploring a career possibility with the TCS FamilyHiring For : Big data DeveloperMust Have :SparkPysparkHiveHBaseExperience: 3 to 8 yrs Location : Mumbai/Pune/ChennaiIf interested kindly fill the details and send your resume .Note: only Eligible candidates with...

  • Big Data Developer

    8 months ago

    Bengaluru, India Unison Consulting Pte Ltd Full time

    Job Description:Unison Consulting Pte Ltd is seeking an experienced Big Data Developer to join our team. As a Big Data Developer, you will be responsible for designing, developing, and maintaining big data solutions to enable data-driven decision making and analytics capabilities for our clients. You will work with large volumes of structured and...

  • Big Data Developer

    14 hours ago

    Greater Bengaluru Area, India Impetus Full time

    Location : Bengaluru, Karnataka, IndiaYears of Experience : 5 to 9 yrsMandatory Skills - spark, Pyspark, oops, Data framesJob DescriptionExperience in working on Spark framework, good understanding of core concepts, optimizations, and best practicesGood hands-on experience in writing code in PySpark, should understand design principles and OOPS Good...

  • Big Data Developer

    1 day ago

    Greater Bengaluru Area, India Impetus Full time

    Location : Bengaluru, Karnataka, India Years of Experience : 5 to 9 yrs Mandatory Skills - spark, Pyspark, oops, Data frames Job Description Experience in working on Spark framework, good understanding of core concepts, optimizations, and best practices Good hands-on experience in writing code in PySpark, should understand design principles and OOPS ...

  • Big Data Developer

    4 weeks ago

    Bengaluru, India Advent Global Solutions Full time

    Position : Big Data Developer/Big Data Senior DeveloperLocation : Chennai, Bangalore, Pune, Gurgaon, and MumbaiExperience : 5 to 14 yearsRelevant : should be 4+ yrs for Junior and 7+ yrs for SeniorPrimary skills : Spark, SQL,HiveDevelop and maintain architectures such as databases and large-scale processing systems using Python, Scala, SQL, and Unix...

  • Big Data Developer

    3 weeks ago

    Bangalore Metropolitan Area, India Pixomania Full time

    As a Big Data Developer, you will design, develop, and optimize distributed data processing systems using Apache Spark and Hive. You will work on large-scale data platforms and ensure the efficient processing of massive datasets in real-time and batch environments. Your experience with Kafka will be an added advantage in building robust and scalable...

  • Big Data Developer

    4 weeks ago

    Bengaluru, India Tata Consultancy Services Full time

    TCS has always been in the spotlight for being adept in "the next big technologies" What we can offer you is a space to explore varied technologies and quench your techie soul.Role: Big Data DeveloperLocation: Bangalore/Chennai/Hyderabad/PuneExperience:5-10 Years- Required Technical skills: Bigdata, Hadoop Technologies- 4+ years of experience in Big Data,...

  • Big data developer

    4 weeks ago

    Bengaluru, India Tata Consultancy Services Full time

    TCS has always been in the spotlight for being adept in "the next big technologies" What we can offer you is a space to explore varied technologies and quench your techie soul.Role: Big Data DeveloperLocation: Bangalore/Chennai/Hyderabad/PuneExperience:5-10 YearsRequired Technical skills: Bigdata, Hadoop Technologies4+ years of experience in Big Data,...

  • Big Data Developer

    3 weeks ago

    Bengaluru, India LTIMindtree Full time

    Please fill this form for quick responses ;-If it is not able to access please past it on Google and try to fill it...Link:- Big Data Developer/ Big Data EngineerSkills:- Pyspark, Sql, Hive,Exp:- 5 + YearsLoc:- Bengaluru, NCR Noida, Gurugram, DelhiMode;- HybridRegards,Dhamodhar. GTA

  • Big data developer

    3 weeks ago

    Bengaluru, India LTIMindtree Full time

    Please fill this form for quick responses ;-If it is not able to access please past it on Google and try to fill it...Link:- Lx Fw Yq9Role:- Big Data Developer/ Big Data EngineerSkills:- Pyspark, Sql, Hive,Exp:- 5 + YearsLoc:- Bengaluru, NCR Noida, Gurugram, DelhiMode;- HybridRegards,Dhamodhar. GTA

  • Big Data Developer

    1 month ago

    Bengaluru, India Tata Consultancy Services Full time

    TCS has always been in the spotlight for being adept in "the next big technologies" What we can offer you is a space to explore varied technologies and quench your techie soul.Role: Big Data DeveloperLocation: Bangalore/Chennai/Hyderabad/PuneExperience:5-10 YearsRequired Technical skills: Bigdata, Hadoop Technologies4+ years of experience in Big Data,...

  • Big Data Developer

    3 months ago

    Bengaluru, India Tata Consultancy Services Full time

    Greetings from TCS!TCS is hiring forBig Data DeveloperDesired Experience:5+ YearsJob Location:Chennai/ Bangalore / Pune / MumbaiRequire skillset:Big Data, Hadoop, Spark, Scala, HiveJob Description:-Must Have Technical Qualifications:SparkHadoopHiveScalaNice to Have Qualifications:Data Engineer CertificationExperience with Apache Beam.Technical...

  • Big Data Developer

    4 months ago

    Bengaluru, India IBCScorp Full time

    Hi,We are hiring for Bigdata Developer for one of our Prestigious Client @ Bangalore Location. Job Location: BangalorePosition: Big Data DeveloperExperience: 5+yrsTotal YOE: 5+ Years Technical Experience 2+ Years of experience with Big Query SQL. 4+ years of experience with Scala and SPARK. Very strong in SQL 4+ yrs Experience with GCP Cloud. Candidate...

  • Big Data Developer

    4 weeks ago

    Bengaluru, India Tata Consultancy Services Full time

    TCS has always been in the spotlight for being adept in "the next big technologies" What we can offer you is a space to explore varied technologies and quench your techie soul.Role: Big Data DeveloperLocation: Bangalore/Chennai/Hyderabad/PuneExperience:5-10 YearsRequired Technical skills: Bigdata, Hadoop Technologies4+ years of experience in Big Data,...

  • BIG DATA Developer

    2 months ago

    Bengaluru, India IBCScorp Full time

    Please see below the JD for Bigdata developer:Candidate should be available to interview in person in BangaloreWorking within a cross functional, agile development team, this Big Data Developer will be responsible for hands on development and technical delivery of innovative components used in data driven solutions.The candidate will have a sound working...

  • Big Data Developer

    3 weeks ago

    Bengaluru, India LTIMindtree Full time

    Please fill this form for quick responses ;- If it is not able to access please past it on Google and try to fill it... Link:- Role:- Big Data Developer/ Big Data Engineer Skills:- Pyspark, Sql, Hive, Exp:- 5 + Years Loc:- Bengaluru, NCR Noida, Gurugram, Delhi Mode;- Hybrid Regards, Dhamodhar. G TA

  • Big Data- AWS

    6 months ago

    Greater Noida, India Coforge Full time

    Job Description Job description Role and responsibilities : - Creating Project Technical Documentation - Designing Solution architecture, and work on Data Ingestion, Preparation and Transformation. Debugging the production failures and identifying the solution. - Developing efficient frameworks for development and testing using (AWS Dynamo DB, EKS,...