Apache Spark and Hadoop Developer

5 days ago

Bengaluru India Tata Consultancy Services Full time

Job Description Greetings from TCS Skillset : Apache Spark and Hadoop Location: Bangalore Experience Range: 4 - 10 years Must-have skills - Design, build, and maintain scalable big data pipelines for processing and analyzing large datasets using Spark, Airflow, Bodo, Flume, Flink, etc. - Utilize technologies like Hadoop, Spark, Kafka, and NoSQL databases. - Develop data ingestion, transformation, and aggregation strategies. - Design and implement data warehouses and data marts using Presto, Snowflake, or similar technologies. - Create efficient data models and schemas for optimal query performance. - Great to have an understanding of graph query (GQL), Gremlin, Cypher, etc. - Know to build business system-engineering architectures using graphs, how to scale the workload on distributed systems. - Hands on experience applying graphs for business impact, including experience on data management, infrastructure, budgeting, trade-offs, project workflow management, and business processes engineering. - Strong programming skills: Python/Scala/Rust for Supercomputing on TB+ data. - Experience with creating quantitative analytics and business operation dashboards using Tableau, SuperSet or other visualisation tools. - Experience with data mining, relational and noSQL databases, and data warehouse for data automation. - Experience working with large language model and natural language processing is a plus. - Excellent problem solving, critical thinking backed by expert understanding of probability, statistics, algorithms and mathematics. - Be able to strategize and innovate business solutions for high impact projects. - Be able to evaluate and transform latest, untested academic methods into production workflows. - Self-driven, highly motivated and ability to work both independently and within a team. - Operate optimally in fast pace development environment with dynamic changes, tight deadlines and limited resources.

  • Apache Hadoop Developer

    4 days ago

    Bengaluru, Karnataka, India BairesDev Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide.When you apply for this position, you're...

  • Apache Spark

    4 weeks ago

    Bengaluru, India Teamware Solutions Full time

    Job Description Responsibilities: - Develop and maintain data processing workflows using Apache Spark and Scala - Implement batch and streaming data pipelines - Optimize Spark jobs for better performance and scalability - Collaborate with data engineers and analysts to deliver data solutions - Debug and resolve issues in production big data environments -...

  • Apache Spark

    1 week ago

    Bengaluru/bangalore Chennai Gurgaon Hyderabad/secunderabad Pune, India ICS Consultancy Services Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job Description:Project Role : Application Developer  Project Role Description : Design, build and configure applications to meet business process and application requirements Must have Skills : Apache Spark  Good to Have Skills :  Microsoft SQL Server,Unix to Linux Migration,Data Warehouse ETL Testing,Amazon Web Service  Job Requirements :    Key...

  • Apache Hadoop Developer

    3 weeks ago

    Bengaluru, India BairesDev Full time

    Job Description At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide. When you apply for this...

  • Apache Spark

    1 week ago

    Bengaluru/bangalore Chennai Hyderabad/secunderabad Kolkata Noida, India ICS Consultancy Services Full time ₹ 1,00,00,000 - ₹ 2,00,00,000 per year

    Job Description:Project Role : Application Developer  Project Role Description : Design, build and configure applications to meet business process and application requirements Must Have Skills : Apache Spark  Good To Have Skills :   Job Requirements : Key Responsibilities: AStrong  experience in creating ScalaSpark jobs for data transformation and...

  • Apache Spark

    4 days ago

    Bengaluru/bangalore Gurgaon Kolkata Noida Pune, India ICS Consultancy Services Full time ₹ 5,00,000 - ₹ 25,00,000 per year

    Job Description:Project Role : Application Developer  Project Role Description : Design, build and configure applications to meet business process and application requirements.  Must have Skills : Apache Spark  Good to Have Skills :  Hadoop,  Job Requirements :    Key Responsibilities :   A: Create scala/spark jobs for data transformation and...

  • Hadoop Developer

    7 days ago

    Bengaluru, Karnataka, India Coders Brain Technology Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Position : Hadoop Developer  Experience: 6+ Years Location : Chennai/Bangalore(Work from Office)  Mandatory Skill- pySpark,Hadoop,GCP, Python • Professional experience with a cloud platform • Developer must have sound knowledge in Apache Spark and Python programming. • Deep experience in developing data processing tasks using pySpark such as...

  • Sr Backline Engineer

    2 days ago

    Bengaluru, India Databricks Full time

    Mission Databricks helps data teams solve the world’s toughest problems. As the leader in Unified Data Analytics, Databricks helps organizations make all their data ready for analytics, empower data-driven decisions across the organization, and rapidly adopt machine learning to outpace the competition Databricks is venture-backed and founded by the...

  • Scala/Spark Developer

    1 week ago

    India Photon Group Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    DescriptionDevelop, test, and deploy data processing applications using Apache Spark and Scala. Optimize and tune Spark applications for better performance on large-scale data sets.  Work with the Cloudera Hadoop ecosystem (e.g., HDFS, Hive, Impala, HBase, Kafka) to build data pipelines and storage solutions.  Collaborate with data scientists, business...

  • Apache spark with cloud Professional

    7 days ago

    Bengaluru, Chennai, Coimbatore, India Krazy Mantra HR Solutions Pvt. Ltd Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    We are looking for a skilled professional with expertise in Apache Spark and cloud technologies to join our team. The ideal candidate will have 5-7 years of experience.Roles and ResponsibilityDesign, develop, and implement scalable data processing pipelines using Apache Spark.Collaborate with cross-functional teams to integrate Spark applications with cloud...