Lead Big Data Engineer

1 month ago


Bangalore, Karnataka, India TekPillar Full time

Job Title : Senior Data Engineer (Big Data)

Experience : 4-9 Years

Location : Bangalore

Job Overview :

We are seeking a skilled Senior Data Engineer to coordinate with cross-functional teams, manage and prioritize projects, and drive key data engineering initiatives. The successful candidate will have a deep understanding of big data tools and the ability to translate business needs into data models. This role requires proficiency in creating robust data pipelines, developing and maintaining data lakes, and implementing CI/CD pipelines for data solutions.


Key Responsibilities :


- Requirement Analysis : Understand and translate business needs into scalable data models to support long-term solutions.


- Data Modeling : Collaborate with the Business team to implement data strategies, design data flows, and develop conceptual data models.


- Data Pipeline Design : Create robust and scalable data pipelines across multiple domains.


- Data Integration : Develop and maintain data lakes by acquiring data from primary and secondary sources, ensuring flexibility and scalability in data evaluation.


- Testing and Quality Assurance : Define and manage data load procedures to ensure data quality, adhering to business rules and quality thresholds.


- Deployment : Implement data strategies and develop physical data models in collaboration with development teams, data analysts, and information system teams to ensure operational excellence.


- Team Leadership : Coordinate with other teams to manage and prioritize data-related projects.


- Stakeholder Management : Collaborate with internal stakeholders to understand and deliver solutions based on their data requirements.


- CI/CD Implementation : Build and maintain CI/CD pipelines for data solutions, ensuring smooth updates to the data Skills :


- Big Data Tools : Expertise in Hadoop, Hive, and Spark.


- Programming Languages : Proficient in Scala, Python, SQL, and PySpark.


- Cloud Services : Experience with AWS or Azure.


- Stream Processing : Experience with Kafka, Spark-Streaming, etc.


- Database Management : Hands-on experience with relational (Postgres) and NoSQL (Cassandra) databases.


- Unix/Linux : Experience with Unix/Linux systems and scripting (Bash).


- Workflow Management : Familiarity with tools like Airflow for managing data pipelines.


- Data Transformation : Deep understanding of data transformation, metadata, and workload management Qualifications :


- Educational Qualification : Bachelor's degree in a relevant field.


- Industry Experience : Experience in the Satellite/Telecom industry will be an added advantage.


- Leadership Skills : Proven experience in leading large data engineering projects and cross-functional teams

(ref:hirist.tech)

  • Bangalore, Karnataka, India FxConsulting Full time

    Job Title : Lead Data EngineerWe are seeking a highly skilled Lead Data Engineer with expertise in Big Data technologies to join our dynamic team. As a Lead Data Engineer, you will play a pivotal role in designing, building, and optimizing large-scale data pipelines and systems to drive business insights and innovation.Key Responsibilities : - Design,...

  • Data Engineer

    2 months ago


    Bangalore, Karnataka, India Coders Brain Technology Private Limited Full time

    Qualifications : - Bachelor's degree in Computer Science/Engineering or similar - 5-7 years of data engineering experience with focus on designing and implementing scalable big data transformations and pipelines - Proficiency with Cloud Platforms like Azure(preferred), AWS, and/or Google cloud, including knowledge of core services like Azure: Data...


  • Bangalore, Karnataka, India TekPillar Full time

    Join our dynamic team in Bangalore as a Big Data Engineer contribute to our key data engineering initiatives! You will play a crucial role in managing and prioritizing projects while driving strategies and executing data engineering solutions.Job Title : Big Data Engineer/ LeadExperience : 4 to 9 YearsLocation : BangaloreKey Responsibilities : - Collaborate...

  • Solution Engineer

    2 months ago


    Bangalore, Karnataka, India HEVO DATA Full time

    Overview :. We're seeking a talented Solutions Engineer with strong background in Data Domain to join our growing Sales team. You'll work one-on-one with the sales team to help discover our customers' business and technical needs and demonstrate how Hevo's platform can solve their real-time data replication needs. In this role, you will have...


  • Bangalore, Karnataka, India Draup Full time

    Role : Big Data EngineerJob Description :The Big Data Engineer at Draup is responsible for building scalable techniques and processes for data storage, transformations, and analysis. The role includes designing, and implementation of the optimal, generic, and reusable data-platforms and pipelines. You will work with a very proficient, inquisitive,...

  • Lead Data Engineer

    2 months ago


    Bangalore, Karnataka, India Sentienz Full time

    Job Description : As a Lead Data Engineer, you will play a pivotal role in driving our data-driven initiatives. You will be responsible for designing, developing, and maintaining robust data engineering solutions that support our analytics and decision-making processes.Responsibilities :- Lead and mentor a team of data engineers, providing technical guidance...

  • Big Data Engineer

    2 months ago


    Bangalore, Karnataka, India Tavant Full time

    Key Responsibilities :- Design and implement scalable and efficient data processing pipelines using Apache Spark and Java.- Develop APIs and services for data ingestion, transformation, and aggregation to support analytics and machine learning models.- Collaborate with data scientists, analysts, and cross-functional teams to understand data requirements and...

  • Technical Lead

    1 month ago


    Bangalore, Karnataka, India FxConsulting Full time

    Job Title : Lead Data EngineerWe are seeking a highly skilled Lead Data Engineer with expertise in Big Data technologies to join our dynamic team. As a Lead Data Engineer, you will play a pivotal role in designing, building, and optimizing large-scale data pipelines and systems to drive business insights and innovation.Key Responsibilities :1. Design,...

  • Senior Data Engineer

    1 month ago


    Bangalore, Karnataka, India Visionyle Solutions Full time

    Title : Senior Data EngineerExperience : 6+ YearsNotice Period : Immediate to 15 Days (Serving NP candidates)Work Location - BangaloreMandatory skills : Big data applications, Spark, python, scala , AWS S3,Airflow,Data bricks, data platforms, Jenkins, CICD PipelinesRequirements :- 6+ years of experience in developing scalable Big Data applications or...


  • Bangalore, Karnataka, India Talpro Full time

    Job Description :Key Responsibilities :Design and Development :- Architect, design, and develop scalable and high-performance data platforms.- Implement data ingestion, transformation, and processing pipelines using technologies such as Apache Kafka, Hadoop, Spark, and Flink.- Develop real-time data processing and stream processing applications.Data...

  • Robosoft Technologies

    4 weeks ago


    Bangalore, Karnataka, India Robosoft Technologies Full time

    Experience :- Overall 8+ years of IT experience with 5+ years in Data related technologies, and expertise of 3+ years in data-related AWS Cloud services.- Have 3+ year of experience in ClickHouse or any other OLAP system.- Mandatory to have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end Big Data solutions.- Expert in...

  • Lead Data Engineer

    2 months ago


    Bangalore, Karnataka, India Rigel networks Full time

    Job Title : Lead Data EngineerLocation : BangaloreNotice Period : Immediate - 15 DaysAbout the Role :We are seeking a highly skilled Lead Data Engineer with extensive experience in data engineering and a strong background in Big Data technologies. The ideal candidate will have 7+ years of relevant experience, with expertise in Python, SQL, and AWS, and a...

  • Big Data Engineer

    1 month ago


    Bangalore, Karnataka, India TekPillar Full time

    Position : Big Data EngineerExperience : 4 to 7 YearsLocation : BangaloreMandatory Skills: Hadoop , Hive , Spark , Python , SQL , AWS/Azure , AirflowJob Description :- 4-6 years of relevant experience with a bachelor's degree- Exposure to Satellite/Telecom industries- Proven leadership in large data engineering initiatives- Proficiency in Big Data tools...

  • Platform Architect

    4 months ago


    Bangalore, Karnataka, India Talpro Full time

    Educational Qualification : - BE./ B.Tech., M./M.Tech. or M.Sc. in Computer Science from Tier 1 university Skill Set :- 10+ years of experience in end-to-end software development and system architecture, preferably in a product development organisation- Hands-on experience in Java and Big Data tech stack - Apache Pinot/Druid/Hudi, HDFS, Hive, Presto, Flume,...

  • Big Data Engineer

    2 months ago


    Bangalore, Karnataka, India TEAMLEASE DIGITAL PRIVATE LIMITED Full time

    Job Title : Data EngineerLocation : Bangalore / HyderabadJob Type : Full-timeWork Arrangement : 4 days a week in the office, 1 day remoteExperience Level : 4+ yearsRole Overview : We are seeking an experienced Data Engineer with a strong background in Hadoop, MapReduce, and Java. The ideal candidate will have a proven track record of handling large-scale...


  • Bangalore, Karnataka, India MakeMyTrip Full time

    Our main focus is on Scala , Spark , Spark Streaming .MakeMyTrip - Lead Data EngineerAbout the Role :MakeMyTrip, India's leading online travel company, is searching for a seasoned Lead Data Engineer to join our Data Platform Team. You will play a pivotal role in architecting, developing, and managing our data infrastructure and pipelines, empowering...

  • Big Data Engineer

    2 months ago


    Bangalore, Karnataka, India Talpro Full time

    Job Description :Key Responsibilities :- Develop and maintain scalable data pipelines using Python, PySpark, and Scala.- Work with Big Data streaming services such as Kinesis, Kafka, and others to process and analyze large data sets.- Manage and optimize NoSQL data stores such as HBase and DynamoDB.- Utilize Hadoop and Big Data processing frameworks like...


  • Bangalore, Karnataka, India Altimetrik Full time

    Role : Data engineer - Good understanding of AWS Data Stack including services like S3, Redshift, EMR, Lambda, Athena and Glue. Good understanding of Hadoop/HDFS- Designing and developing ETL jobs across multiple platforms- Designing and developing schema definitions and support data warehouse/mart to enable integration of disparate data sources from within...

  • Lead Data Engineer

    2 months ago


    Bangalore, Karnataka, India Rigel Networks Pvt Ltd Full time

    Lead Data Engineer :Top Skills :- Python- SQL- AWS- Spark- 7+ years relevant work experience in the Data Engineering field- 5+ years of experience working with Hadoop and Big Data processing frameworks (Hadoop, Spark, Hive, Flink, Airflow etc.)- 5+ years of Strong experience with relational SQL and at least one programming language such as Python, Scala, or...

  • Big Data Engineer

    2 months ago


    Bangalore, Karnataka, India MaimsD Technology Full time

    Job Title : Big Data DeveloperExperience : 4-7 yearsLocation : BengaluruJob Type : Full-TimeJob Description : About the Role : As a Big Data Developer, you will be responsible for designing, developing, and maintaining applications that process and analyze large datasets. You will work closely with data analysts, data scientists, and other team members to...