Big Data Engineer

3 days ago

Bengaluru, Karnataka, India Scoop Technologies Full time

Job Role : Big Data Engineer

Work Location : Bangalore (CV Ramen Nagar location)

Experience : 7+ Years

Notice Period : Immediate - 30 days

Mandatory Skills : Big Data, Python, SQL, Spark/Pyspark, AWS Cloud

JD and required Skills & Responsibilities :

- Actively participate in all phases of the software development lifecycle, including requirements gathering, functional and technical design, development, testing, roll-out, and support.

- Solve complex business problems by utilizing a disciplined development methodology.

- Produce scalable, flexible, efficient, and supportable solutions using appropriate technologies.

- Analyse the source and target system data. Map the transformation that meets the requirements.

- Interact with the client and onsite coordinators during different phases of a project.

- Design and implement product features in collaboration with business and Technology stakeholders.

- Anticipate, identify, and solve issues concerning data management to improve data quality.

- Clean, prepare, and optimize data at scale for ingestion and consumption.

- Support the implementation of new data management projects and re-structure the current data architecture.

- Implement automated workflows and routines using workflow scheduling tools.

- Understand and use continuous integration, test-driven development, and production deployment frameworks.

- Participate in design, code, test plans, and dataset implementation performed by other data engineers in support of maintaining data engineering standards.

- Analyze and profile data for the purpose of designing scalable solutions.

- Troubleshoot straightforward data issues and perform root cause analysis to proactively resolve product issues.

Required Skills :

- 5+ years of relevant experience developing Data and analytic solutions.

- Experience building data lake solutions leveraging one or more of the following AWS, EMR, S3, Hive & PySpark

- Experience with relational SQL.

- Experience with scripting languages such as Python.

- Experience with source control tools such as GitHub and related dev process.

- Experience with workflow scheduling tools such as Airflow.

- In-depth knowledge of AWS Cloud (S3, EMR, Databricks)

- Has a passion for data solutions.

- Has a strong problem-solving and analytical mindset

- Working experience in the design, Development, and test of data pipelines.

- Experience working with Agile Teams.

- Able to influence and communicate effectively, both verbally and in writing, with team members and business stakeholders

- Able to quickly pick up new programming languages, technologies, and frameworks.

- Bachelor's degree in computer science

(ref:hirist.tech)

  • Big Data Engineer

    2 days ago

    Bengaluru, Karnataka, India beBee Careers Full time

    Big Data EngineerWe are seeking a highly skilled Big Data Engineer to join our team. As a Big Data Engineer, you will be responsible for designing, building, and maintaining scalable big data pipelines for processing and analyzing large datasets using Spark, Airflow, Bodo, Flume, Flink, etc.

  • Big Data Engineer

    2 days ago

    Bengaluru, Karnataka, India beBee Careers Full time

    Big Data EngineerWe are seeking an experienced Big Data Engineer to join our team. As a key member of the data platform engineering team, you will be responsible for designing, implementing, and operating large-scale big data systems.Job DescriptionDrive the design and implementation of big data architectures, big data modeling, and big data applications...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Data caliper Full time

    Job DescriptionWe are seeking a highly experienced Data Engineer with strong expertise in ETL development using Informatica Big Data Management (BDM) to join our team. The ideal candidate will bring over 8 years of hands-on experience in managing complex data pipelines, working with unstructured and structured data sources, and building scalable data models...

  • Big Data Engineer

    3 days ago

    Bengaluru, Karnataka, India beBee Careers Full time

    Big Data Engineer RoleAbout the Job:We are looking for a Big Data Engineer to design and develop scalable big data systems.Job Description:Design, build, and maintain big data pipelines using Spark, Airflow, Bodo, Flume, Flink, etc.Implement data warehousing and data mart solutions using Presto, Snowflake, or similar technologies.Key Responsibilities:Develop...

  • Data Engineer

    14 hours ago

    Bengaluru, Karnataka, India beBee Careers Full time

    Job Title: Data Engineer - Big DataJob Description:We are seeking a highly skilled Data Engineer to join our team. As a key member of our organization, you will be responsible for designing, developing, and maintaining large-scale data systems. Your expertise in big data technologies such as Hadoop, Spark, and NoSQL databases will play a crucial role in...

  • Big Data Engineer

    4 days ago

    Bengaluru, Karnataka, India beBee Careers Full time

    Big Data EngineeringWe are seeking an experienced Big Data Engineer to join our team. The ideal candidate will have a strong background in big data, Hadoop, SPARK, and Impala.Mandatory Skills Description:• 5+ years of experience with big data technologies.• Strong Python skills, with at least 4 years of hands-on experience.• Database experience,...

  • Big Data Engineer

    3 days ago

    Bengaluru, Karnataka, India beBee Careers Full time

    Big Data Engineer RoleKey ResponsibilitiesDesign and develop scalable data pipelines using Apache Spark (PySpark)Perform complex data transformations and aggregations using SQL and PySparkDevelop and maintain robust data solutions on big data platforms like Hadoop, Databricks, or AWS EMRWork with cross-functional teams to understand data requirements and...

  • Big Data Engineer

    3 days ago

    Bengaluru, Karnataka, India beBee Careers Full time

    Job Title: Data Scientist, Big Data EngineerOur team is looking for an experienced Data Engineer with a strong background in designing and developing large-scale data processing systems. The ideal candidate will have a solid understanding of distributed computing principles, data engineering implementation experience with Big Data technologies in cloud and...

  • Big Data Engineer

    1 week ago

    Bengaluru, Karnataka, India beBee Careers Full time

    Job Title: Big Data EngineerWe are seeking a highly skilled Big Data Engineer to design, build, and maintain scalable big data pipelines for processing and analyzing large datasets using Spark, Airflow, Bodo, Flume, Flink, etc.The ideal candidate will have experience with technologies like Hadoop, Spark, Kafka, and NoSQL databases, and be able to develop...

  • Data Engineer

    2 days ago

    Bengaluru, Karnataka, India beBee Careers Full time

    Data Engineer - Big DataJob Summary:We are seeking an experienced Data Engineer - Big Data to join our team. As a Data Engineer - Big Data, you will be responsible for designing, building, and maintaining large-scale data processing systems using Apache Spark and Scala. You will also be expected to troubleshoot complex issues in distributed systems and...