Senior Big Data Engineer

1 month ago

Bangalore, India Looper Development Services Private Limited Full time

Role Objective :

Data Engineer will be responsible for expanding and optimizing our data and database architecture, as well as optimizing data flow and collection for cross functional teams. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building.


The Data Engineer will support our software developers, database architects, data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. They must be self-directed and comfortable supporting the data needs of multiple teams, systems, and products.

Roles & Responsibilities :

- Sound knowledge in Spark architecture and distributed computing.

- Proficient in Spark - including RDD and Dataframes core functions, troubleshooting and performance tuning.

- Good understanding in object-oriented concepts and hands on experience on Scala or on java with excellent programming logic and technique.

- Good in functional programming and OOPS concept on Scala

- Good experience in SQL - should be able to write complex queries.

- Managing the team of Associates and Senior Associates and ensuring the utilization is maintained across the project.

- Able to mentor new members for onboarding to the project.

- Understand the client requirement and able to design, develop from scratch and deliver.

- AWS cloud experience would be preferable.

- Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services - DynamoDB, RedShift, Kinesis, Lambda, S3, etc. (prefered)

- Hands on experience utilizing AWS Management Tools (CloudWatch, CloudTrail) to proactively monitor large and complex deployments (prefered)

- Experience in analyzing, re-architecting, and re-platforming on-premise data warehouses to data platforms on AWS (prefered)

- Leading the client calls to flag off any delays, blockers, escalations and collate all the requirements.

- Managing project timing, client expectations and meeting deadlines.

- Should have played project and team management roles.

- Facilitate meetings within the team on regular basis.

- Understand business requirement and analyze different approaches and plan deliverables and milestones for the project.

- Optimization, maintenance, and support of pipelines.

- Strong analytical and logical skills.

- Ability to comfortably tackling new challenges and learn

Expertise And Qualifications :

Must have Skills :


- Scala

- Spark

- AWS preferable/Any cloud

- SQL (Intermediate to advanced level)

- Object-Oriented Programming

- Good knowledge on ETL/ELT processes and tools

Good to Have Skills :

- AWS Certification

- Basic knowledge of streaming

- Knowledge on hive, Hadoop, Kafka, Streaming

- Git/similar version control tool

- Knowledge in CI/CD, Microservices (ref:hirist.tech)

  • Big Data Engineer

    3 weeks ago

    Bangalore, India Madhees Techno Consulting Pvt. Ltd Full time

    Job Title: Senior Data Engineer at Madhees Techno Consulting Pvt. LtdAbout the Role:We are seeking a highly skilled Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing, developing, and implementing scalable and efficient big data solutions using Spark, Scala, and other relevant...

  • Big Data Engineer

    3 weeks ago

    Bangalore, India Publicis Sapient Full time

    Senior Associate Data Engineering Level 2This role involves translating client requirements into technical design and implementing data engineering solutions. You will utilize deep understanding of data integration and big data design principles to create custom solutions or implement package solutions. Your key responsibilities will include data ingestion,...

  • Big Data Engineer

    4 weeks ago

    Bangalore, India Draup Full time

    Job Title: Big Data EngineerJob Summary:The Big Data Engineer at Draup is responsible for designing and implementing scalable data storage, transformation, and analysis techniques. This role involves working with a proficient team of developers, engineers, and product team to build generic and reusable data platforms and pipelines.Key Responsibilities:Design...

  • Senior Data Engineer

    1 month ago

    Bangalore, Karnataka, India Visionyle Solutions Full time

    Title : Senior Data EngineerExperience : 6+ YearsNotice Period : Immediate to 15 Days (Serving NP candidates)Work Location - BangaloreMandatory skills : Big data applications, Spark, python, scala , AWS S3,Airflow,Data bricks, data platforms, Jenkins, CICD PipelinesRequirements :- 6+ years of experience in developing scalable Big Data applications or...

  • Big Data Engineer

    3 weeks ago

    Bangalore, India Draup Full time

    Job Title: Big Data EngineerAbout the Role:The Big Data Engineer at Draup is responsible for designing and implementing scalable techniques and processes for data storage, transformations, and analysis. This role involves building generic and reusable data platforms and pipelines. You will work with a team of experienced developers, engineers, and product...

  • Big Data Engineer

    4 weeks ago

    Bangalore, India Draup Full time

    Job Title: Big Data EngineerAt Draup, we are seeking a highly skilled Big Data Engineer to join our team. As a Big Data Engineer, you will be responsible for designing and implementing scalable techniques and processes for data storage, transformations, and analysis.Key Responsibilities:Design and develop scalable architectures for data storage,...

  • Big Data Engineer

    4 weeks ago

    Bangalore, India Talpro Full time

    Job Title: Big Data EngineerJob Summary:Talpro is seeking a highly skilled Big Data Engineer to join our team. As a Big Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines using Python, PySpark, and Scala.Key Responsibilities:Develop and maintain scalable data pipelines using Python, PySpark, and...

  • Big Data Engineer

    3 weeks ago

    Bangalore, India Rigel networks Full time

    Job Title: Big Data EngineerJob Description:At Rigel Networks, we are seeking a highly skilled Big Data Engineer to lead our data engineering efforts. The ideal candidate will have extensive experience in designing and implementing big data systems using PySpark, database migration, transformation, and integration solutions for data warehousing projects.Key...

  • Senior Big Data Developer

    6 days ago

    bangalore, India IntraEdge Full time

    We are looking for a highly skilled Senior Big Data Developer to join our engineering team. The successful candidate will be responsible for designing, developing, and maintaining large-scale data processing systems. You will work closely with cross-functional teams to architect scalable data pipelines and provide valuable insights through advanced data...

  • Senior big data developer

    3 weeks ago

    Bangalore, India IntraEdge Full time

    We are looking for a highly skilled Senior Big Data Developer to join our engineering team. The successful candidate will be responsible for designing, developing, and maintaining large-scale data processing systems. You will work closely with cross-functional teams to architect scalable data pipelines and provide valuable insights through advanced data...

  • Big Data Engineer

    4 weeks ago

    Bangalore, India Talpro Full time

    Job Title: Big Data EngineerAt Talpro, we are seeking a highly skilled Big Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining large-scale data processing systems.Key Responsibilities:Design and implement scalable data pipelines using Python, PySpark, and...

  • Big Data Engineer

    3 weeks ago

    Bangalore, India Draup Full time

    Role : Big Data EngineerJob Description :The Big Data Engineer at Draup is responsible for designing, implementing, and maintaining scalable techniques and processes for data storage, transformations, and analysis. This role involves working with a proficient team of developers, engineers, and product team to shape up the data products. The ideal candidate...

  • Big Data Engineer

    3 weeks ago

    Hyderabad/Bangalore/Chennai/Bhubaneshwar, India Rigel networks Full time

    Job Title: Senior Scala Developer for Big DataLocation: Type: Full-TimeJob Overview:We are seeking a highly skilled Senior Scala Developer for Big Data to lead our team in designing, developing, and optimizing data processing pipelines using Apache Spark. The ideal candidate will have a deep understanding of big data technologies and strong programming...

  • Senior Big Data Developer

    1 week ago

    bangalore, India IntraEdge Full time

    We are looking for a highly skilled Senior Big Data Developer to join our engineering team. The successful candidate will be responsible for designing, developing, and maintaining large-scale data processing systems. You will work closely with cross-functional teams to architect scalable data pipelines and provide valuable insights through advanced data...

  • Senior Big Data Architect

    3 weeks ago

    Bangalore, India Yuma Full time

    Job Title: Senior Data EngineerJob Description:In this role, you will be responsible for designing, developing, and implementing big data solutions using various technologies. You will work closely with the architect to design secure and scalable solutions.Key Responsibilities:Design and develop batch and streaming solutions using big data technologies.Work...

  • Big Data Engineer

    3 weeks ago

    Bangalore, India MaimsD Technology Full time

    About the Role:At MaimsD Technology, we are seeking a highly skilled Big Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining applications that process and analyze large datasets. You will work closely with data analysts, data scientists, and other team members to...

  • Big Data Engineer

    3 weeks ago

    Bangalore, India Talpro Full time

    Job Title: Big Data EngineerJob Summary:Talpro is seeking an experienced Big Data Engineer to join our team. The successful candidate will be responsible for designing, developing, and maintaining scalable data pipelines using Python, PySpark, and Scala.Key Responsibilities:Develop and maintain scalable data pipelines using Python, PySpark, and Scala.Work...

  • Big Data Engineer

    4 weeks ago

    Bangalore, India TekPillar Full time

    Job SummaryWe are seeking a highly skilled Big Data Engineer to join our team at TekPillar. The ideal candidate will have a strong background in big data tools, programming languages, and cloud experience.Key ResponsibilitiesDesign and implement large-scale data engineering initiativesLead a team of engineers in the development and deployment of data...

  • Big Data Engineer

    4 weeks ago

    Bangalore, India Tavant Full time

    Job Title: Big Data EngineerJob Summary:We are seeking a highly skilled Big Data Engineer to join our team at Tavant. As a key member of our data engineering team, you will be responsible for designing and implementing scalable and efficient data processing pipelines using Apache Spark and Java.Key Responsibilities:Design and implement scalable data...

  • Big Data Engineer

    4 weeks ago

    Bangalore, India MaimsD Technology Full time

    About the RoleWe are seeking a highly skilled Big Data Engineer to join our team at MaimsD Technology. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining applications that process and analyze large datasets.Key ResponsibilitiesDesign and implement data pipelines for data ingestion, transformation,...