Big Data Engineer

4 weeks ago

Bengaluru, Karnataka, India Scoop Technologies Full time

Job Role : Big Data Engineer

Work Location : Bangalore (CV Ramen Nagar location)

Experience : 7+ Years

Notice Period : Immediate - 30 days

Mandatory Skills : Big Data, Python, SQL, Spark/Pyspark, AWS Cloud

JD and required Skills & Responsibilities :

- Actively participate in all phases of the software development lifecycle, including requirements gathering, functional and technical design, development, testing, roll-out, and support.

- Solve complex business problems by utilizing a disciplined development methodology.

- Produce scalable, flexible, efficient, and supportable solutions using appropriate technologies.

- Analyse the source and target system data. Map the transformation that meets the requirements.

- Interact with the client and onsite coordinators during different phases of a project.

- Design and implement product features in collaboration with business and Technology stakeholders.

- Anticipate, identify, and solve issues concerning data management to improve data quality.

- Clean, prepare, and optimize data at scale for ingestion and consumption.

- Support the implementation of new data management projects and re-structure the current data architecture.

- Implement automated workflows and routines using workflow scheduling tools.

- Understand and use continuous integration, test-driven development, and production deployment frameworks.

- Participate in design, code, test plans, and dataset implementation performed by other data engineers in support of maintaining data engineering standards.

- Analyze and profile data for the purpose of designing scalable solutions.

- Troubleshoot straightforward data issues and perform root cause analysis to proactively resolve product issues.

Required Skills :

- 5+ years of relevant experience developing Data and analytic solutions.

- Experience building data lake solutions leveraging one or more of the following AWS, EMR, S3, Hive & PySpark

- Experience with relational SQL.

- Experience with scripting languages such as Python.

- Experience with source control tools such as GitHub and related dev process.

- Experience with workflow scheduling tools such as Airflow.

- In-depth knowledge of AWS Cloud (S3, EMR, Databricks)

- Has a passion for data solutions.

- Has a strong problem-solving and analytical mindset

- Working experience in the design, Development, and test of data pipelines.

- Experience working with Agile Teams.

- Able to influence and communicate effectively, both verbally and in writing, with team members and business stakeholders

- Able to quickly pick up new programming languages, technologies, and frameworks.

- Bachelor's degree in computer science

(ref:hirist.tech)

  • Big Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India beBee Careers Full time

    Big Data Engineer Position:We are looking for a skilled big data engineer to join our team. As a big data engineer, you will design, develop, and maintain the data infrastructure to support business intelligence, data analytics, and data science workloads.Key Responsibilities:Design and implement big data infrastructure to support business intelligence and...

  • Big Data Engineer

    2 days ago

    Bengaluru, Karnataka, India Highbrow Technology Inc Full time

    Big Data Engineer RoleWe are seeking an experienced Big Data professional to join our team at Highbrow Technology Inc.About the Role4-8 years of experience in big data engineeringStrong expertise in Hadoop ecosystems, PySpark, and shell scriptingThe ideal candidate will have hands-on experience in working with big data tools and technologies, and a passion...

  • Big Data Engineer

    4 days ago

    Bengaluru, Karnataka, India beBee Careers Full time

    Job Title: Big Data EngineerAs a Big Data Engineer, you will be responsible for designing, developing, and maintaining scalable and efficient data pipelines, data warehouses, and data integration processes. The successful candidate will have experience working with big data technologies such as Hadoop, Spark, or NoSQL databases, as well as hands-on...

  • Big Data Engineer

    3 days ago

    Bengaluru, Karnataka, India beBee Careers Full time

    Big Data Engineer">We are seeking a skilled professional to design, build, and maintain scalable big data pipelines for processing and analyzing large datasets using Spark, Airflow, Bodo, Flume, Flink, etc.">As a Big Data Engineer, you will be responsible for designing scalable big data pipelines for processing and analyzing large datasets. This involves...

  • Data Engineer for Big Data

    3 days ago

    Bengaluru, Karnataka, India beBee Careers Full time

    Data Engineer for Big DataJob Description:We are seeking a highly skilled Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing and implementing data pipelines, ETL processes, and data warehousing solutions using cloud-based big data platforms such as AWS and Azure. You will work closely with engineers and...

  • Big Data Engineer

    1 day ago

    Bengaluru, Karnataka, India beBee Careers Full time

    Big Data Engineer PositionWe are seeking an experienced Data Engineer to design and develop cutting-edge data platforms, leveraging the power of data to drive business growth.About the RoleYou will have a significant impact on the business by solving complex problems and optimizing data delivery.Create scalable solutions for big data processing.Develop and...

  • Big Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India beBee Careers Full time

    Job SummaryWe are seeking a skilled Big Data Engineer to ensure the accuracy, reliability, and performance of large-scale data processing systems.The ideal candidate will have expertise in Big Data testing, SQL, and data validation techniques to support end-to-end data pipelines and analytics solutions.Main Responsibilities:Design and execute test strategies...

  • Big Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India beBee Careers Full time

    Job Summary:Big Data EngineerWe are looking for an experienced Big Data Engineer to design, develop, and maintain large-scale data warehouses on Snowflake.The ideal candidate will have expertise in SQL, data modeling, ETL processes, and cloud computing, particularly with AWS or Azure.This role requires excellent problem-solving skills, attention to detail,...

  • Big Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India beBee Careers Full time

    Big Data EngineerWe are looking for an experienced Big Data Engineer to build and manage scalable big data solutions using Azure Databricks and Python. The ideal candidate should have over 5 years of experience in building and managing large-scale data pipelines and workflows. As a key member of our team, you will work closely with data engineers and...

  • Big Data Engineer

    6 days ago

    Bengaluru, Karnataka, India beBee Careers Full time

    Job Title: Big Data Engineer">">About the Role:">">We are seeking an experienced Big Data Engineer to join our team in India. This is an exciting opportunity to work on real-world data using big data technologies.">">The successful candidate will have skills in Python, Spark, and experience in building big data platforms. They will also possess good...