Big Data Engineer

3 weeks ago

Bengaluru, Karnataka, India Scoop Technologies Full time

Job Role : Big Data Engineer

Work Location : Bangalore (CV Ramen Nagar location)

Experience : 7+ Years

Notice Period : Immediate - 30 days

Mandatory Skills : Big Data, Python, SQL, Spark/Pyspark, AWS Cloud

JD and required Skills & Responsibilities :

- Actively participate in all phases of the software development lifecycle, including requirements gathering, functional and technical design, development, testing, roll-out, and support.

- Solve complex business problems by utilizing a disciplined development methodology.

- Produce scalable, flexible, efficient, and supportable solutions using appropriate technologies.

- Analyse the source and target system data. Map the transformation that meets the requirements.

- Interact with the client and onsite coordinators during different phases of a project.

- Design and implement product features in collaboration with business and Technology stakeholders.

- Anticipate, identify, and solve issues concerning data management to improve data quality.

- Clean, prepare, and optimize data at scale for ingestion and consumption.

- Support the implementation of new data management projects and re-structure the current data architecture.

- Implement automated workflows and routines using workflow scheduling tools.

- Understand and use continuous integration, test-driven development, and production deployment frameworks.

- Participate in design, code, test plans, and dataset implementation performed by other data engineers in support of maintaining data engineering standards.

- Analyze and profile data for the purpose of designing scalable solutions.

- Troubleshoot straightforward data issues and perform root cause analysis to proactively resolve product issues.

Required Skills :

- 5+ years of relevant experience developing Data and analytic solutions.

- Experience building data lake solutions leveraging one or more of the following AWS, EMR, S3, Hive & PySpark

- Experience with relational SQL.

- Experience with scripting languages such as Python.

- Experience with source control tools such as GitHub and related dev process.

- Experience with workflow scheduling tools such as Airflow.

- In-depth knowledge of AWS Cloud (S3, EMR, Databricks)

- Has a passion for data solutions.

- Has a strong problem-solving and analytical mindset

- Working experience in the design, Development, and test of data pipelines.

- Experience working with Agile Teams.

- Able to influence and communicate effectively, both verbally and in writing, with team members and business stakeholders

- Able to quickly pick up new programming languages, technologies, and frameworks.

- Bachelor's degree in computer science

(ref:hirist.tech)

  • Big Data Engineer

    7 days ago

    Bengaluru, Karnataka, India Scoop Technologies Pvt Ltd Full time

    Job Title : Big Data Engineer Location : Bangalore Employment Type : Full-time Work Mode : 4 Days Work from Office Notice Period : Immediate to 30 days Job Description : We are looking for a Big Data Engineer with expertise in Hadoop, Hive, SQL, Python, PySpark, and AWS to join our team in Bangalore. The ideal candidate will have hands-on experience in...

  • Big Data Engineer

    3 days ago

    Bengaluru, Karnataka, India Human Horizon Full time

    Responsibilities : - Design, develop, and execute test cases for large-scale big data platforms. - Perform testing on various Big Data technologies (e. g., Hadoop, Hive, Spark, NoSQL databases). - Ensure data accuracy, integrity, and quality in data processing and transformations. - Work closely with developers, data engineers, and business analysts to...

  • Big Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Scoop Technologies Pvt Ltd Full time

    Job Title : Big Data EngineerLocation : BangaloreEmployment Type : Full-timeWork Mode : 4 Days Work from OfficeNotice Period : Immediate to 30 daysJob Description :We are looking for a Big Data Engineer with expertise in Hadoop, Hive, SQL, Python, PySpark, and AWS to join our team in Bangalore. The ideal candidate will have hands-on experience in designing,...

  • Big Data Engineer

    6 days ago

    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Welcome to Tata Consultancy Services, where we are driven by innovation and excellence. We are seeking an experienced Big Data Engineer to join our team.As a Big Data Engineer, you will work on designing and implementing scalable ETL/ELT processes and modeling data for low latency reporting. You will also work with large cloud data lakes and have experience...

  • Big Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India Human Horizon Full time

    Responsibilities :- Design, develop, and execute test cases for large-scale big data platforms.- Perform testing on various Big Data technologies (e. g., Hadoop, Hive, Spark, NoSQL databases).- Ensure data accuracy, integrity, and quality in data processing and transformations.- Work closely with developers, data engineers, and business analysts to...

  • Big Data Engineer

    4 weeks ago

    Bengaluru, Karnataka, India Human Horizon Full time

    Responsibilities :- Design, develop, and execute test cases for large-scale big data platforms.- Perform testing on various Big Data technologies (e. g., Hadoop, Hive, Spark, NoSQL databases).- Ensure data accuracy, integrity, and quality in data processing and transformations.- Work closely with developers, data engineers, and business analysts to...

  • Big Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India Telescopp Full time

    Job DescriptionJob OverviewWe are looking for a talented Big Data Engineer to join our team in Bengaluru. This is a mid-level, full-time position requiring a minimum of 4 years to a maximum of 6 years of relevant work experience. As a Big Data Engineer at Telescopp, you will be responsible for developing, testing, and maintaining big data solutions to...

  • Big Data Engineer

    7 days ago

    Bengaluru, Karnataka, India Vishanz Business Services Private Limited Full time

    Vishanz Business Services Private Limited is seeking an experienced Big Data Engineer to design and develop scalable data solutions. As a Data Engineer, you will work on creating data pipelines and ETL processes that handle large volumes of data.Key TasksDesign and develop big data solutions using Microsoft Azure services.Collaborate with data scientists and...

  • Big Data Engineer

    4 weeks ago

    Bengaluru, Karnataka, India Telescopp Full time

    Job OverviewWe are looking for a talented Big Data Engineer to join our team in Bengaluru. This is a mid-level, full-time position requiring a minimum of 4 years to a maximum of 6 years of relevant work experience. As a Big Data Engineer at Telescopp, you will be responsible for developing, testing, and maintaining big data solutions to support our client's...

  • Big Data Engineer

    7 days ago

    Bengaluru, Karnataka, India TEKDOORS Full time

    We are looking for a Big Data Engineer to build scalable data pipelines and support enterprise analytics applications by integrating and processing large datasets. Roles and Responsibilities : - Develop and manage data pipelines for integrating, consolidating, and cleansing data. - Utilize ETL tools and REST APIs for data integration and management. - Plan,...

  • Big Data Engineer

    4 weeks ago

    Bengaluru, Karnataka, India Infogain Full time

    Big Data Engineer (Standard) with skills Data Engineering, Data Warehouse, Big Data, databricks, Apache Spark, Azure Datalake, Pyspark for location Bangalore, India Posted on: July 30, Share on Linkedin Share on Twitter Share on Facebook ROLES & RESPONSIBILITIES Advanced knowledge of Azure data services such as Azure SQL Database, Azure...

  • Big Data Engineer

    7 days ago

    Bengaluru, Karnataka, India Synechron Technologies Private Limited Full time

    Job DescriptionAbout UsSynechron Technologies Private Limited is a global digital transformation consulting and technology services company.Key ResponsibilitiesDesign and develop big data solutions using Hadoop, Spark, Hive, and Python.Develop and implement data processing pipelines for large-scale data sets.Analyze and optimize system performance for...

  • Big Data Engineer

    4 weeks ago

    Bengaluru, Karnataka, India Infogain Full time

    Big Data Engineer (Lead) with skills Data Engineering, Data Warehouse, Big Data, snowflake, Azure Datalake for location Any Infogain Base Location (Noida, Gurugram, Bangalore, Mumbai, Pune) Posted on: July 30, Share on Linkedin Share on Twitter Share on Facebook ROLES & RESPONSIBILITIES Responsibilities: · Strong experience of minimum 5+...

  • Big Data Infrastructure Engineer

    4 days ago

    Bengaluru, Karnataka, India Licious Full time

    Big Data Infrastructure EngineerLicious is seeking an experienced Big Data Infrastructure Engineer to join our team. As a Big Data Infrastructure Engineer, you will be responsible for designing and implementing scalable big data infrastructure that supports our core business processes. **Responsibilities**:* Design and implement scalable big data...

  • Big Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India TEKDOORS Full time

    We are looking for a Big Data Engineer to build scalable data pipelines and support enterprise analytics applications by integrating and processing large datasets.Roles and Responsibilities :- Develop and manage data pipelines for integrating, consolidating, and cleansing data.- Utilize ETL tools and REST APIs for data integration and management.- Plan,...

  • Big Data Engineer

    4 weeks ago

    Bengaluru, Karnataka, India TEKDOORS Full time

    We are looking for a Big Data Engineer to build scalable data pipelines and support enterprise analytics applications by integrating and processing large datasets.Roles and Responsibilities :- Develop and manage data pipelines for integrating, consolidating, and cleansing data.- Utilize ETL tools and REST APIs for data integration and management.- Plan,...

  • Big Data Engineering Expert

    6 days ago

    Bengaluru, Karnataka, India Tredence Inc. Full time

    About the PositionWe are looking for a highly skilled Big Data Engineering Expert to join our team. As a key member of our engineering group, you will be responsible for designing, developing, and implementing big data engineering solutions using cloud-based technologies such as Azure, GCP, and Snowflake.Key Responsibilities:Design and develop big data...

  • Big Data Engineer

    7 days ago

    Bengaluru, Karnataka, India Freight Tiger Full time

    About the JobWe are seeking an experienced Data Engineer to join our dynamic team in Bangalore. This role is crucial in building and maintaining our data infrastructure, enabling us to derive valuable insights and drive data-driven decisions.ResponsibilitiesDesign and Develop Data Pipelines: Build and optimize robust data pipelines to extract, transform, and...

  • Big Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India VidPro Consultancy Services Full time

    Job DescriptionRole : Big Data EngineerLocation : BangaloreJob Type : Fulltime, HybridSkills : Python, Spark, SQL, Databricks, Hadoop, Hive, Spark, AWS, DSAJob OverviewWe are seeking a Junior Big Data Engineer specializing in AWS for a full-time role located in Bangalore Urban, Bangalore Rural, Bangalore. As a key contributor in this role, you will leverage...

  • Big Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India VidPro Consultancy Services Full time

    Job DescriptionRole : Big Data EngineerLocation : BangaloreJob Type : Fulltime, HybridSkills : Python, Spark, SQL, Databricks, Hadoop, Hive, Spark, AWS, DSAJob OverviewWe are seeking a Junior Big Data Engineer specializing in AWS for a full-time role located in Bangalore Urban, Bangalore Rural, Bangalore. As a key contributor in this role, you will leverage...