Principal Data Engineer

2 months ago

bangalore, India Taglynk Full time

Job Summary:

The Principal Data Engineer is responsible for leading the design, development, and optimization of scalable data pipelines, data lakes, and large-scale datasets. With over 9 years of experience in data engineering and expertise in tools like ElasticSearch, Redshift, and data lakes, this role ensures the efficient processing and storage of massive data streams, driving business intelligence and data-driven decision-making across the organization.

Key Responsibilities:

Data Architecture & Pipeline Development:

  • Design and implement highly scalable, reliable, and efficient data pipelines to process large datasets using modern data engineering tools and practices.
  • Architect and manage data lakes, enabling storage and querying of unstructured and structured data.
  • Lead initiatives to optimize data retrieval and integration using ElasticSearch, Redshift, and other distributed systems.

Large Dataset Management:

  • Oversee the storage, processing, and analysis of large datasets, ensuring data quality, security, and performance.
  • Develop and maintain ETL processes to manage data flow across multiple sources, transforming and enriching data for analytics and reporting.
  • Collaborate with data scientists and analysts to provide seamless access to data and drive analytical outcomes.

Data Governance & Compliance:

  • Ensure data governance policies are adhered to, maintaining high standards of data quality, privacy, and security.
  • Implement and enforce best practices for data access control, data lineage, and monitoring.
  • Ensure compliance with industry standards and regulations regarding data protection (e.g., GDPR).

Performance Optimization:

  • Continuously monitor and optimize the performance of ElasticSearch, Redshift, and other data infrastructure.
  • Identify and resolve bottlenecks, and work on improving query performance and cost efficiency.
  • Experiment with new technologies and tools to keep data engineering processes at the cutting edge.

Team Leadership & Mentorship:

  • Lead and mentor a team of data engineers, providing guidance on best practices, coding standards, and architecture design.
  • Collaborate with cross-functional teams (e.g., product, engineering, data science) to meet business requirements and drive data innovation.
  • Participate in code reviews and ensure adherence to coding standards and architectural guidelines.

Innovation & Strategic Planning:

  • Stay updated on emerging trends in big data technologies and proactively recommend improvements or changes to the existing data architecture.
  • Contribute to the long-term strategy for data architecture, ensuring scalability and flexibility for future growth.

Key Qualifications:

  • Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
  • 9+ years of experience in data engineering, with at least 5 years of hands-on experience working with large datasets and managing data architectures involving ElasticSearch, Redshift, and data lakes.
  • Proven experience in designing and implementing data pipelines, ETL processes, and distributed data systems.
  • Expertise in cloud-based data platforms (AWS, Azure, etc.), particularly in managing Redshift and data lake infrastructure.
  • Strong programming skills in Python, Java, or Scala, and experience with SQL.
  • Solid understanding of data modeling, database design, and performance tuning for large-scale systems.

  • Principal Data Engineer

    2 months ago

    Bangalore, Karnataka, India Rock interviews Full time

    What We Are Looking For : As a Principal Data Engineer, you will take ownership of the ideation, architecture, design, and development of our enterprise data platform. You will play a key role in collaborating with cloud and security architects to ensure a seamless integration within our broader technology strategy. We are looking for someone with a passion...

  • Principal Data Architect

    4 weeks ago

    Bangalore, India Freshworks Full time

    Job Title: Principal Data EngineerWe are seeking a highly skilled Principal Data Engineer to lead our efforts in building enterprise-scale data platforms. This role demands a deep understanding of the data engineering landscape and expertise in shaping the future of our data ecosystem.Responsibilities:Design and develop new enterprise data platforms with a...

  • Principal Data Architect

    1 month ago

    Bangalore, India Rock interviews Full time

    Job Title: Principal Data EngineerWe are seeking a highly skilled Principal Data Engineer to join our team at Rock Interviews. As a key member of our data engineering team, you will be responsible for designing, building, and maintaining our enterprise data platform.Key Responsibilities:Architect and develop end-to-end data platforms on Google Cloud Platform...

  • Principal Data Architect

    1 month ago

    Bangalore, India Freshworks Full time

    Job Title: Principal Data EngineerWe are seeking a highly skilled and experienced Principal Data Engineer to lead our data engineering efforts and build enterprise-scale data platforms.Key Responsibilities:Design and develop new data platforms, collaborating with cloud and security architects to ensure seamless alignment with our technology...

  • Lead Data Engineer

    6 months ago

    bangalore, India MakeMyTrip Full time

    Position: Lead Data Engineer / Principal Data EngineerExperience: Must have 6+ years of experienceAbout Role:We are looking for experienced Data engineers with excellent problem-solving skills to develop machine-learning powered Data Products design to enhance customer experiences.About us:Nurtured from the seed of a single great idea - to empower the...

  • Principal Software Engineer

    2 months ago

    bangalore, India Oracle Full time

    Principal Software Development Engineer Cloud Engineering Infrastructure Development 8-10+ years of total experience in software development Bangalore location only As a Principal Software Engineer / Principal Member of Technical Staff, you will work with teams of software engineers responsible for the software design, development, and operations for our...

  • Senior/Lead/Principal Engineer

    6 months ago

    Bangalore, India Squareroot Consulting Pvt Ltd Full time

    Hiring Java Engineer - Senior/Lead/Principal Engineer for US based Global Enterprise Product Consulting company for their Bangalore team.Experience : 5+ YearsLocation : BangaloreRole : Senior/ Lead/ PrincipalJob Description :- Experience: 5+ Years for Java Senior/ Lead / Principal Engineer.- Strong knowledge of Object Oriented analysis and design, software...

  • Freshworks - Principal Data Engineer - Synapse Analytics

    2 months ago

    Bangalore, Karnataka, India Freshworks Full time

    Job Description :We are seeking a dynamic and highly skilled Principal Data Engineer who has extensive experience building enterprise scale data platforms and lead these foundational efforts. This role demands someone who not only possesses a profound understanding of the data engineering landscape but is also at the forefront of their game.The ideal...

  • Principal Engineer

    3 weeks ago

    bangalore, India Bizongo Full time

    About Bizongo: At Bizongo, we help brands and enterprises achieve scalable growth through our AI-powered transaction first vendor digitization and embedded financing platform. Bizongo’s platform has an embedded ecosystem of 450+ brands/enterprises across diverse industries, an extensive vendor network of 7500+ businesses, and 30+ financial institutions. We...

  • Principal Engineer

    3 weeks ago

    bangalore, India Bizongo Full time

    About Bizongo:At Bizongo, we help brands and enterprises achieve scalable growth through our AI-powered transaction first vendor digitization and embedded financing platform. Bizongo’s platform has an embedded ecosystem of 450+ brands/enterprises across diverse industries, an extensive vendor network of 7500+ businesses, and 30+ financial institutions. We...

  • Principal Engineer

    2 weeks ago

    bangalore, India Angel One Full time

    About Angel one : Angel One Limited is a Fintech company providing broking services, margin trading facility, research services, depository services, investment education and financial products distribution to its clients, on a mission to become the No. 1 fintech organization in India. With about 23.0 million registered clients, we are onboarding an average...

  • Principal Engineer

    2 weeks ago

    bangalore, India Angel One Full time

    About Angel one :Angel One Limited is a Fintech company providing broking services, margin trading facility, research services, depository services, investment education and financial products distribution to its clients, on a mission to become the No. 1 fintech organization in India. With about 23.0 million registered clients, we are onboarding an average...

  • Principal engineer

    3 weeks ago

    Bangalore, India Angel One Full time

    About Angel one : Angel One Limited is a Fintech company providing broking services, margin trading facility, research services, depository services, investment education and financial products distribution to its clients, on a mission to become the No. 1 fintech organization in India. With about 23.0 million registered clients, we are onboarding an...

  • Principal Engineer

    3 weeks ago

    bangalore, India Angel One Full time

    About Angel one : Angel One Limited is a Fintech company providing broking services, margin trading facility, research services, depository services, investment education and financial products distribution to its clients, on a mission to become the No. 1 fintech organization in India. With about 23.0 million registered clients, we are onboarding an average...

  • Senior/Lead/Principal Engineer

    6 months ago

    Bangalore, India Squareroot Consulting Pvt Ltd Full time

    Hiring Java Engineer - Senior/Lead/Principal Engineer for US based Global Enterprise Product Consulting company for their Bangalore team.Experience : 5+ YearsLocation : BangaloreRole : Senior/ Lead/ PrincipalJob Description :- Experience: 5+ Years for Java Senior/ Lead / Principal Engineer.- Strong knowledge of Object-Oriented analysis and design, software...

  • Principal Engineer

    2 weeks ago

    bangalore, India Angel One Full time

    About Angel one :Angel One Limited is a Fintech company providing broking services, margin trading facility, research services, depository services, investment education and financial products distribution to its clients, on a mission to become the No. 1 fintech organization in India. With about 23.0 million registered clients, we are onboarding an average...

  • Principal Data Engineer

    2 months ago

    Bangalore Urban, India Taglynk Full time

    Job Summary: The Principal Data Engineer is responsible for leading the design, development, and optimization of scalable data pipelines, data lakes, and large-scale datasets. With over 9 years of experience in data engineering and expertise in tools like ElasticSearch, Redshift, and data lakes, this role ensures the efficient processing and storage of...

  • Principal Data Engineer

    2 months ago

    Bangalore Urban, India Taglynk Full time

    Job Summary:The Principal Data Engineer is responsible for leading the design, development, and optimization of scalable data pipelines, data lakes, and large-scale datasets. With over 9 years of experience in data engineering and expertise in tools like ElasticSearch, Redshift, and data lakes, this role ensures the efficient processing and storage of...

  • Principal Engineer

    4 weeks ago

    Bangalore, India Bizongo Full time

    About Bizongo: At Bizongo, we help brands and enterprises achieve scalable growth through our AI-powered transaction first vendor digitization and embedded financing platform. Bizongo’s platform has an embedded ecosystem of 450+ brands/enterprises across diverse industries, an extensive vendor network of 7500+ businesses, and 30+ financial institutions....

  • Principal Engineer

    4 weeks ago

    Bangalore, India Bizongo Full time

    About Bizongo: At Bizongo, we help brands and enterprises achieve scalable growth through our AI-powered transaction first vendor digitization and embedded financing platform. Bizongo’s platform has an embedded ecosystem of 450+ brands/enterprises across diverse industries, an extensive vendor network of 7500+ businesses, and 30+ financial institutions....