Big Data Developer

3 weeks ago

Bangalore, India Pixomania Full time
As a Big Data Developer, you will design, develop, and optimize distributed data processing systems using Apache Spark and Hive. You will work on large-scale data platforms and ensure the efficient processing of massive datasets in real-time and batch environments. Your experience with Kafka will be an added advantage in building robust and scalable streaming data systems.

Responsibilities:

Develop, test, and deploy distributed data processing systems using Apache Spark and Hive.
Design and implement scalable and efficient ETL pipelines for large datasets.
Optimize performance of Spark jobs for high throughput and low-latency processing.
Write clean, efficient, and maintainable code in Scala or Java.
Work with data engineers and other teams to integrate data solutions and ensure smooth data flow.
Implement and maintain Hive queries, partitions, and tables for batch processing.
Design and maintain data processing architectures in cloud environments (AWS, Azure, GCP).
Collaborate with business and technical stakeholders to deliver high-quality data solutions.
Work on Kafka (if applicable) to design and implement real-time data streaming applications.
Troubleshoot and optimize performance issues related to Spark and Hive-based systems.
Mentor junior developers and ensure high-quality standards are followed throughout the development lifecycle.
Required Skills and Experience:

5+ years of experience in Big Data development, with extensive hands-on experience in Apache Spark and Hive.
Strong experience with Spark for batch and real-time data processing.
Proven ability to design and implement Hive queries, partitions, and data models.
Good knowledge of Hadoop ecosystem tools and how they integrate with Spark.
Proficiency in SQL for querying and data manipulation.
Familiarity with cloud technologies and platforms (AWS, Azure, GCP).
Experience working with large-scale distributed systems.
Strong understanding of ETL pipelines, data integration, and data processing workflows.
Knowledge of Kafka for stream processing is a plus.
Experience in developing and optimizing high-performance data processing jobs.
Ability to work in a hybrid environment, with occasional on-site presence in Bengaluru/Hyderabad.
Preferred Skills:

Location - Bengaluru/ Hyderabad

  • Big Data Developer

    3 weeks ago

    Bangalore, India Pixomania Full time

    Job Title: Big Data Developer (Scala + Spark)Location: Remote Experience: 5+ years Job Type: Contractual Remote About Us: We are a forward-thinking technology company committed to delivering innovative solutions in Big Data analytics. Join our talented team as a Big Data Developer with expertise in Scala and Apache Spark to drive impactful projects and...

  • Big Data Developer

    3 weeks ago

    bangalore, India Advent Global Solutions Full time

    Position : Big Data Developer/Big Data Senior Developer Location : Chennai, Bangalore, Pune, Gurgaon, and Mumbai Experience : 5 to 14 years Relevant : should be 4+ yrs for Junior and 7+ yrs for Senior Primary skills : Spark, SQL,Hive Develop and maintain architectures such as databases and large-scale processing systems using Python, Scala, SQL, and Unix...

  • Big Data Developer

    3 weeks ago

    bangalore, India Advent Global Solutions Full time

    Position : Big Data Developer/Big Data Senior DeveloperLocation : Chennai, Bangalore, Pune, Gurgaon, and MumbaiExperience : 5 to 14 yearsRelevant : should be 4+ yrs for Junior and 7+ yrs for SeniorPrimary skills : Spark, SQL,HiveDevelop and maintain architectures such as databases and large-scale processing systems using Python, Scala, SQL, and Unix...

  • Big Data Developer

    3 weeks ago

    bangalore, India Advent Global Solutions Full time

    Position : Big Data Developer/Big Data Senior DeveloperLocation : Chennai, Bangalore, Pune, Gurgaon, and MumbaiExperience : 5 to 14 yearsRelevant : should be 4+ yrs for Junior and 7+ yrs for SeniorPrimary skills : Spark, SQL,HiveDevelop and maintain architectures such as databases and large-scale processing systems using Python, Scala, SQL, and Unix...

  • Big data developer

    4 weeks ago

    Bangalore, India Tata Consultancy Services Full time

    TCS has always been in the spotlight for being adept in "the next big technologies" What we can offer you is a space to explore varied technologies and quench your techie soul. Role: Big Data Developer Location: Bangalore/Chennai/Hyderabad/Pune Experience:5-10 Years Required Technical skills: Bigdata, Hadoop Technologies 4+ years of experience in...

  • Big Data Developer

    4 weeks ago

    bangalore, India Tata Consultancy Services Full time

    TCS has always been in the spotlight for being adept in "the next big technologies" What we can offer you is a space to explore varied technologies and quench your techie soul. Role: Big Data Developer Location: Bangalore/Chennai/Hyderabad/Pune Experience:5-10 Years Required Technical skills: Bigdata, Hadoop Technologies 4+ years of experience in Big Data,...

  • Big Data Developer

    4 weeks ago

    bangalore, India Tata Consultancy Services Full time

    TCS has always been in the spotlight for being adept in "the next big technologies" What we can offer you is a space to explore varied technologies and quench your techie soul.Role: Big Data DeveloperLocation: Bangalore/Chennai/Hyderabad/PuneExperience:5-10 YearsRequired Technical skills: Bigdata, Hadoop Technologies4+ years of experience in Big Data,...

  • Big Data Developer

    3 weeks ago

    Bangalore, India LTIMindtree Full time

    Please fill this form for quick responses ;- If it is not able to access please past it on Google and try to fill it... Link:- Role:- Big Data Developer/ Big Data Engineer Skills:- Pyspark, Sql, Hive, Exp:- 5 + Years Loc:- Bengaluru, NCR Noida, Gurugram, Delhi Mode;- Hybrid Regards, Dhamodhar. G TA

  • Senior big data developer

    2 months ago

    Bangalore, India IntraEdge Full time

    We are looking for a highly skilled Senior Big Data Developer to join our engineering team. The successful candidate will be responsible for designing, developing, and maintaining large-scale data processing systems. You will work closely with cross-functional teams to architect scalable data pipelines and provide valuable insights through advanced data...

  • Big Data Developer

    2 months ago

    bangalore, India Tata Consultancy Services Full time

    Greetings from TCS!TCS is hiring for Big Data DeveloperDesired Experience: 5+ YearsJob Location: Chennai/ Bangalore / Pune / MumbaiRequire skillset: Big Data, Hadoop, Spark, Scala, HiveJob Description:-Must Have Technical Qualifications:SparkHadoopHiveScalaNice to Have Qualifications: Data Engineer CertificationExperience with Apache Beam.Technical...

  • Senior Big Data Developer

    2 months ago

    bangalore, India IntraEdge Full time

    We are looking for a highly skilled Senior Big Data Developer to join our engineering team. The successful candidate will be responsible for designing, developing, and maintaining large-scale data processing systems. You will work closely with cross-functional teams to architect scalable data pipelines and provide valuable insights through advanced data...

  • Senior Big Data Developer

    2 months ago

    bangalore, India IntraEdge Full time

    We are looking for a highly skilled Senior Big Data Developer to join our engineering team. The successful candidate will be responsible for designing, developing, and maintaining large-scale data processing systems. You will work closely with cross-functional teams to architect scalable data pipelines and provide valuable insights through advanced data...

  • Big data developer

    4 weeks ago

    Bangalore, India Luxoft Full time

    Project Description: The Cirrus project is developing cloud-based compliance state-of-the-art archive products to archive and retain real-time communications data in line with internal and external regulatory requirements. The product is developed in-house in an agile development setup, based on business requirements from multiple stakeholder parties. By...

  • Big Data Developer

    4 weeks ago

    Bangalore Urban, India Tata Consultancy Services Full time

    TCS has always been in the spotlight for being adept in "the next big technologies" What we can offer you is a space to explore varied technologies and quench your techie soul. Role: Big Data Developer Location: Bangalore/Chennai/Hyderabad/Pune Experience:5-10 Years Required Technical skills: Bigdata, Hadoop Technologies 4+ years of experience in Big...

  • Big Data Developer

    4 weeks ago

    Bangalore Urban, India Tata Consultancy Services Full time

    TCS has always been in the spotlight for being adept in "the next big technologies" What we can offer you is a space to explore varied technologies and quench your techie soul.Role: Big Data DeveloperLocation: Bangalore/Chennai/Hyderabad/PuneExperience:5-10 YearsRequired Technical skills: Bigdata, Hadoop Technologies4+ years of experience in Big Data,...

  • Big Data Developer

    4 weeks ago

    Bangalore, India Luxoft Full time

    Project Description: The Cirrus project is developing cloud-based compliance state-of-the-art archive products to archive and retain real-time communications data in line with internal and external regulatory requirements. The product is developed in-house in an agile development setup, based on business requirements from multiple stakeholder parties. By...

  • Big Data Developer

    4 weeks ago

    Bangalore, India Luxoft Full time

    Project Description: The Cirrus project is developing cloud-based compliance state-of-the-art archive products to archive and retain real-time communications data in line with internal and external regulatory requirements. The product is developed in-house in an agile development setup, based on business requirements from multiple stakeholder parties. By...

  • Senior big data developer

    1 month ago

    Bangalore City, India IntraEdge Full time

    We are looking for a highly skilled Senior Big Data Developer to join our engineering team. The successful candidate will be responsible for designing, developing, and maintaining large-scale data processing systems. You will work closely with cross-functional teams to architect scalable data pipelines and provide valuable insights through advanced data...

  • Big Data Developer

    4 weeks ago

    Bangalore, India Grid Dynamics Full time

    Note - Candidate should be an Immediate joiner or within 30 days. This is a Hybrid position in Bangalore (Only Local candidates will be considered for this role or some who can relocate in a day or two) Details required: (Share your resume to with below details for faster response) Total Years of experience - Notice period - CTC - Expected CTC - ...

  • Big Data Engineer

    2 months ago

    Bangalore, India Draup Full time

    Job Title: Big Data EngineerAbout the Role:The Big Data Engineer at Draup is responsible for designing and implementing scalable techniques and processes for data storage, transformations, and analysis. This role involves building generic and reusable data platforms and pipelines. You will work with a team of experienced developers, engineers, and product...