Big Data Lead

1 week ago

Bengaluru, Karnataka, India IQVIA Full time US$ 90,000 - US$ 1,20,000 per year

Job Title / Primary Skill: Big Data Developer (Lead/Associate Manager)

Management Level: G150

Years of Experience: 8 to 13 years

Job Location: Bangalore (Hybrid)

Must Have Skills: Big data, Spark, Scala, SQL, Hadoop Ecosystem.

Educational Qualification: BE/BTech/ MTech/ MCA, Bachelor or masters degree in Computer Science,

Job Overview

Overall Experience 8+ years in IT, Software Engineering or relevant discipline.

Designs, develops, implements, and updates software systems in accordance with the needs of the organization.

Evaluates, schedules, and resources development projects; investigates user needs; and documents, tests, and maintains computer programs.

Job Description:

We look for developers to have good knowledge of Scala programming skills and Knowledge of SQL

Technical Skills:

  • Scala, Python -> Scala is often used for Hadoop-based projects, while Python and Scala are choices for Apache Spark-based projects.
  • SQL -> Knowledge of SQL (Structured Query Language) is important for querying and manipulating data
  • Shell Script -> Shell scripts are used for batch processing of data, it can be used for scheduling the jobs and shell scripts are often used for deploying applications
  • Spark Scala -> Spark Scala allows you to write Spark applications using the Spark API in Scala
  • Spark SQL -> It allows to work with structured data using SQL-like queries and Data Frame APIs.
  • We can execute SQL queries against Data Frames, enabling easy data exploration, transformation, and analysis.

The typical tasks and responsibilities of a Big Data Developer include:

  1. Data Ingestion: Collecting and importing data from various sources, such as databases, logs, APIs into the Big Data infrastructure.

  2. Data Processing: Designing data pipelines to clean, transform, and prepare raw data for analysis. This often involves using technologies like Apache Hadoop, Apache Spark.

  3. Data Storage: Selecting appropriate data storage technologies like Hadoop Distributed File System (HDFS), HIVE, IMPALA, or cloud-based storage solutions (Snowflake, Databricks).

  • Big Data Architect Lead

    2 weeks ago

    Bengaluru, Karnataka, India beBeeBigData Full time ₹ 1,04,000 - ₹ 1,30,878

    Lead Big Data EngineerYou will be responsible for translating client requirements into design, architecting and implementing big data solutions on Azure Cloud.Provide technical leadership in the areas of data engineering including data ingestion, data access, modeling, data processing, visualization, design and implementation.Lead a team to deliver high...

  • AWS Big Data Lead

    1 week ago

    Bengaluru, Karnataka, India CODERS BRAIN TECHNOLOGY PRIVATE LIMITED Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    About the RoleWe are seeking a highly skilled AWS Big Data Lead & Technical Architect with over 10 years of experience in designing, implementing, and leading enterprise-grade Big Data solutions. The ideal candidate will combine deep technical expertise in AWS and Big Data technologies with strong leadership and architectural skills to drive large-scale data...

  • Leading Big Data Engineer

    1 week ago

    Bengaluru, Karnataka, India beBeeEngineering Full time ₹ 2,00,00,000 - ₹ 3,50,00,000

    Job Opportunity:Big Data Engineering LeadershipWe are seeking a seasoned big data engineer to lead our distributed engineering teams across the globe.Collaborate with cross-functional teams to drive strategic initiatives and foster a culture of innovation.Develop expertise in scalable technologies such as Scala, Python, Apache Spark, HDFS, and Iceberg table...

  • Lead Big Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India beBeeBigData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    We are seeking a seasoned big data professional with expertise in Scala and cloud technologies to lead our data engineering initiatives.">Key Responsibilities:

  • AWS Big Lead Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Coders Brain Technology Private Limited Full time

    About the Role :We are looking for an experienced AWS Big Data Lead Engineer to design, build, and manage scalable data solutions on AWS. The ideal candidate will have extensive hands-on experience in Big Data technologies, cloud-native architecture, and leadership to guide engineering teams in delivering enterprise-grade data pipelines and analytics...

  • Big Basket

    2 weeks ago

    Bengaluru, Karnataka, India big basket Full time

    What would you be doing/ Expected from this role ?- Collaborate with cross-functional teams including data scientists, engineers, and product managers to deliver AI-driven solutions.- Drive the architecture of machine learning pipelines from data collection and preprocessing to model training, evaluation, and deployment.- Implement scalable machine learning...

  • Big Basket

    2 weeks ago

    Bengaluru, Karnataka, India big basket Full time

    As a Software Engineer we are looking for developers who share our passion for building world class software. If you are excited about the prospect of changing the way India shops for groceries and being a pioneer then this a great home for you. Not to mention the thrill of building products that millions use daily and using cutting edge technology to make...

  • Lead Big Data Solutions Architect

    2 weeks ago

    Bengaluru, Karnataka, India beBeeSolutions Full time ₹ 20,00,000 - ₹ 30,00,000

    Big Data Solutions ArchitectWe are seeking a seasoned Big Data Solutions Architect to design, develop, and optimize data pipelines and ETL workflows.About the Role:The successful candidate will have strong hands-on experience with big data technologies such as Apache Hadoop, Spark, and PySpark.They will be proficient in Scala, Python, and SQL, and have...

  • Lead Big Data Analyst

    2 weeks ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    As an Associate Data Scientist, you will be working to solve business problems using leading-edge and open-source tools. You will collaborate with other scientists, engineers, consultants, and database administrators of all backgrounds and disciplines.Key ResponsibilitiesImplementing and validating predictive models, creating and maintaining statistical...

  • Big Data Developer

    6 days ago

    Bengaluru, Karnataka, India Tech Consulting Full time

    Position Overview:We are seeking a skilled Big Data Developer to join our growing delivery team, with a dual focus on hands-on project support and mentoring junior engineers. This role is ideal for a developer who not only thrives in a technical, fast-paced environment but is also passionate about coaching and developing the next generation of talent.You...