Big Data Engineer

2 weeks ago

Bengaluru, India VidPro Consultancy Services Full time

Role : Big Data Engineer

Location : Bangalore

Job Type : Fulltime, Hybrid

Skills : Python, Spark, SQL, Databricks, Hadoop, Hive, Spark, AWS, DSA


Job Overview

We are seeking a Junior Big Data Engineer specializing in AWS for a full-time role located in Bangalore Urban, Bangalore Rural, Bangalore. As a key contributor in this role, you will leverage your expertise in big data technologies to design, implement, and optimize complex data processing solutions. Your contribution will be pivotal in enhancing our clients' data architectures and ensuring robust data pipelines.


Qualifications and Skills

  • Proficiency in Python (Mandatory skill) is required to efficiently handle data processing tasks and pipeline creation.
  • Experience with PySpark (Mandatory skill) for scalable data analytics and large data set processing is essential.
  • Expertise in Databricks (Mandatory skill) to collaborate and transform data in a cloud-based environment is necessary.
  • Strong knowledge of Amazon Redshift for creating and managing data warehouses and ensuring data storage efficiency.
  • Experience with AWS Glue for developing and managing ETL jobs to prepare data for analytics.
  • Familiarity with AWS Lambda for executing code in response to triggers, ensuring a flexible data processing environment.
  • Experience with Hadoop frameworks for processing large volumes of data and managing data storage effectively.
  • Advanced SQL skills for querying, managing, and structuring data stored in relational databases.


Roles and Responsibilities

  • Design and implement scalable data processing pipelines to handle large data sets and support business goals.
  • Collaborate with clients and team members to gather data requirements and ensure accurate data structuring.
  • Optimize data architectures for performance, reliability, and scalability within cloud environments.
  • Utilize AWS services to develop innovative data solutions tailored to client needs.
  • Conduct regular data monitoring and maintenance to ensure data integrity and security.
  • Analyze and interpret complex data sets, providing actionable insights to stakeholders.
  • Stay updated with emerging trends and technologies in big data engineering to drive continuous improvement.
  • Participate in code reviews and contribute to maintaining coding standards and practices.

  • Big Data Engineer

    4 weeks ago

    Bengaluru, India Coders Brain Full time

    Job Description Roles and Responsibilities: - Develop scalable infrastructure and platform to collect, analyze and process large amounts of structured and unstructured data with real-time data interpretation. - Work closely across an array of various teams and organizations in the company (including partners, customers) Basic Qualifications: - Bachelor's...

  • Big Data Engineer

    3 weeks ago

    Bengaluru, India Reveille Technologies,Inc Full time

    We're Hiring: Big Data Engineer – Full-Time Looking for a skilledBig Data Engineerto join our team inBangaloreRole: Big Data Engineer Grade: G4 |TSR Location: Bangalore (Onsite/Hybrid – specify if needed) Budget: ₹13,39,200 Experience: 4–6 years Skills: Azure Databricks (ADB) PySpark SQL Spark (Scala/Python) Terraform Airflow & Kubernetes Data...

  • Big Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Reveille Technologies,Inc Full time ₹ 16,00,000 - ₹ 24,00,000 per year

    We're Hiring: Big Data Engineer – Full-TimeLooking for a skilledBig Data Engineerto join our team inBangaloreRole: Big Data EngineerGrade: G4 |TSRLocation: Bangalore (Onsite/Hybrid – specify if needed)Budget: ₹13,39,200Experience: 4–6 yearsSkills:Azure Databricks (ADB)PySparkSQLSpark (Scala/Python)TerraformAirflow & KubernetesData Warehousing &...

  • Big Data Engineer

    3 weeks ago

    Bengaluru, India The IT Firm Full time

    Role OverviewAs a Big Data Engineer, you will be responsible for building and maintaining scalable data pipelines and architectures that support data analytics, reporting, and real-time processing. You will collaborate with data scientists, business analysts, and software engineers to design robust solutions that drive data-driven decisions. Mandatory Skill...

  • Big Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India The IT Firm Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Role OverviewAs a Big Data Engineer, you will be responsible for building and maintaining scalable data pipelines and architectures that support data analytics, reporting, and real-time processing. You will collaborate with data scientists, business analysts, and software engineers to design robust solutions that drive data-driven decisions.Mandatory Skill...

  • Big Data Engineer

    2 days ago

    Bengaluru, Karnataka, India Deqode Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Profile: Big Data Engineer (System Design)Experience:5+ yearsLocation:BangaloreWork Mode:HybridAbout The RoleWe're looking for an experiencedBig Data Engineerwith system design expertise to architect and build scalable data pipelines and optimize big data solutions.Key ResponsibilitiesDesign, develop, and maintain data pipelines and ETL processes using...

  • Big Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Fractal Analytics Private Limited Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    BigData Engineer It's fun to work in a company where people truly BELIEVE in what they are doing We're committed to bringing passion and customer focus to the business.About the Role:We are seeking highly skilled Big Data Engineers/Senior Engineers with a strong background in building and supporting big data solutions. The ideal candidate will have...

  • Big Basket

    4 weeks ago

    Bengaluru, India Big Basket Full time

    What would you be doing/ Expected from this role? - Be responsible for Platforms Team roadmap and engineering execution from conceptualization to delivery of current and next-generation tools & technologies.- Manage release planning, execution, deployment of all tools maintained by Platforms Team, for running production and non-production environments...

  • Big Data Engineer

    3 weeks ago

    Bengaluru, India Suzva Software Technologies Full time

    Hiring: Big Data Engineer (PySpark)Client: (Details will be shared upon shortlisting)Location: Bangalore (Onsite)Experience: 6-8 years (Relevant)Joining: Immediate to 15 days Key Skills: Strong hands-on experience in PySpark Expertise in Big Data technologies Solid understanding of SQL, Data Pipelines, and ETL Interested candidates can share their...

  • Big Data Engineer

    3 weeks ago

    Bengaluru, Chennai, Noida, India Talodyn Networks Private Limited Full time

    Job Description Data Architecture - Big Data Engineer What does a Data Engineer do As an experienced member of our Data Engineering Platform Group you will be responsible for building and taking ownership over the successful design and development of data engineering projects within Fiserv's Enterprise Data Analytics division. You will be required to...