Principal Data Engineer

6 hours ago

Bangalore Urban, India Taglynk Full time

Job Summary:

The Principal Data Engineer is responsible for leading the design, development, and optimization of scalable data pipelines, data lakes, and large-scale datasets. With over 9 years of experience in data engineering and expertise in tools like ElasticSearch, Redshift, and data lakes, this role ensures the efficient processing and storage of massive data streams, driving business intelligence and data-driven decision-making across the organization.

Key Responsibilities:

Data Architecture & Pipeline Development:

  • Design and implement highly scalable, reliable, and efficient data pipelines to process large datasets using modern data engineering tools and practices.
  • Architect and manage data lakes, enabling storage and querying of unstructured and structured data.
  • Lead initiatives to optimize data retrieval and integration using ElasticSearch, Redshift, and other distributed systems.

Large Dataset Management:

  • Oversee the storage, processing, and analysis of large datasets, ensuring data quality, security, and performance.
  • Develop and maintain ETL processes to manage data flow across multiple sources, transforming and enriching data for analytics and reporting.
  • Collaborate with data scientists and analysts to provide seamless access to data and drive analytical outcomes.

Data Governance & Compliance:

  • Ensure data governance policies are adhered to, maintaining high standards of data quality, privacy, and security.
  • Implement and enforce best practices for data access control, data lineage, and monitoring.
  • Ensure compliance with industry standards and regulations regarding data protection (e.g., GDPR).

Performance Optimization:

  • Continuously monitor and optimize the performance of ElasticSearch, Redshift, and other data infrastructure.
  • Identify and resolve bottlenecks, and work on improving query performance and cost efficiency.
  • Experiment with new technologies and tools to keep data engineering processes at the cutting edge.

Team Leadership & Mentorship:

  • Lead and mentor a team of data engineers, providing guidance on best practices, coding standards, and architecture design.
  • Collaborate with cross-functional teams (e.g., product, engineering, data science) to meet business requirements and drive data innovation.
  • Participate in code reviews and ensure adherence to coding standards and architectural guidelines.

Innovation & Strategic Planning:

  • Stay updated on emerging trends in big data technologies and proactively recommend improvements or changes to the existing data architecture.
  • Contribute to the long-term strategy for data architecture, ensuring scalability and flexibility for future growth.

Key Qualifications:

  • Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
  • 9+ years of experience in data engineering, with at least 5 years of hands-on experience working with large datasets and managing data architectures involving ElasticSearch, Redshift, and data lakes.
  • Proven experience in designing and implementing data pipelines, ETL processes, and distributed data systems.
  • Expertise in cloud-based data platforms (AWS, Azure, etc.), particularly in managing Redshift and data lake infrastructure.
  • Strong programming skills in Python, Java, or Scala, and experience with SQL.
  • Solid understanding of data modeling, database design, and performance tuning for large-scale systems.

  • Principal Data Engineer

    1 day ago

    Bangalore Urban, India Taglynk Full time

    Job Summary: The Principal Data Engineer is responsible for leading the design, development, and optimization of scalable data pipelines, data lakes, and large-scale datasets. With over 9 years of experience in data engineering and expertise in tools like ElasticSearch, Redshift, and data lakes, this role ensures the efficient processing and storage of...

  • Data & Analytics Principal Consultant

    4 weeks ago

    Bangalore, India NTT DATA Services Full time

    Req ID: 295134  NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data & Analytics Principal Consultant to join our team in Bangalore, Karnātaka (IN-KA), India...

  • Data & Analytics Principal Consultant

    4 weeks ago

    Bangalore, India NTT DATA Services Full time

    Req ID: 295076  NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data & Analytics Principal Consultant to join our team in Bangalore, Karnātaka (IN-KA), India...

  • Data & Analytics Principal Consultant

    2 months ago

    Bangalore, India NTT DATA Services Full time

    Req ID: 287717  NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data & Analytics Principal Consultant to join our team in Bangalore, Karnātaka (IN-KA), India...

  • Principal Engineer

    4 months ago

    bangalore, India Amazon Development Center India Private Limited - Kerala - P86 Full time

    Principal Engineers provide technical leadership at Amazon.com. They help establish technical standards and drive Amazon’s overall technical architecture, engineering practices, and engineering methodologies. They work on our hardest problems, building high quality, architecturally sound systems that are aligned with our business needs. They think globally...

  • Lead Data Engineer

    4 months ago

    bangalore, India MakeMyTrip Full time

    Position: Lead Data Engineer / Principal Data EngineerExperience: Must have 6+ years of experienceAbout Role:We are looking for experienced Data engineers with excellent problem-solving skills to develop machine-learning powered Data Products design to enhance customer experiences.About us:Nurtured from the seed of a single great idea - to empower the...

  • Data & Analytics Principal Consultant

    1 month ago

    Bangalore, India NTT DATA Services Full time

    Req ID: 287719  NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data & Analytics Principal Consultant to join our team in Bangalore, Karnātaka (IN-KA), India...

  • Associate Principal Engineer

    2 weeks ago

    bangalore, India Condé Nast Technology Lab Full time

    Associate Principal Engineer - EngagementCondé Nast has a huge global footprint across 32 countries worldwide with total monthly visitors reaching more than 550 million. We aim to inspire, inform and entertain our audiences through our portfolio of well known brands such as Vogue, Wired, Vanity Fair, Architectural Digest, GQ and many other leading...

  • Principal Engineer

    2 weeks ago

    bangalore, India slice Full time

    About the roleWe are looking for a Principal Engineer to join our technology team. A Principal Engineer is an innovative, quality-focused, product/business thinker. You are someone who has a passion for technology and innovative product development, who thinks deeply about technical context, and who builds software using your proven experience. You’re...

  • Principal Software Engineer

    2 weeks ago

    bangalore, India Harness Full time

    Harness is a high-growth company that is disrupting the software delivery market. Our mission is to enable the 30 million software developers in the world to deliver code to their users reliably, efficiently, securely and quickly, increasing customers’ pace of innovation while improving the developer experience. We offer solutions for every step of the...

  • Associate Principal Engineer

    1 week ago

    Bangalore, India Condé Nast Technology Lab Full time

    Associate Principal Engineer - Engagement Condé Nast has a huge global footprint across 32 countries worldwide with total monthly visitors reaching more than 550 million. We aim to inspire, inform and entertain our audiences through our portfolio of well known brands such as Vogue, Wired, Vanity Fair, Architectural Digest, GQ and many other leading...

  • Principal Engineer

    6 days ago

    bangalore, India Mercedes-Benz Full time

    Tätigkeitsbereich:Forschung & Entwicklung incl. DesignFachabteilung:Body E/E - Systems & ComponentsGesellschaft:Mercedes-Benz Research and Development India Private LimitedStandort:Mercedes-Benz Research and Development India, BangaloreStartdatum:..4Veröffentlichungsdatum:..4Stellennummer:MER4Arbeitszeit:Vollzeit Join usAufgaben As a Principal Engineer for...

  • Principal Analog Design Engineer

    7 days ago

    bangalore, India Onsemi Full time

    A principal analog design engineer is to develop analog design subsystems and blocks through all phases of the development process, including design, simulation, and supervision of the layout/verification processes and post silicon validation support activities.  #LI-RT1 A Principal Analog IC Design Engineer, will be responsible for leading and...

  • Software Principal Engineer

    7 days ago

    bangalore, India Dell International Services India Pvt Ltd (7451) Full time

    Software Principal Engineer The position is for Engineer working on next-generation enterprise infrastructure solutions for AI/ML workloads. The engineer will be responsible for leading a team of 8 to 10 engineers, involving in defining the product behavior and working on the design specifications. You will work with subject matter experts in...

  • Principal Engineer

    3 days ago

    bangalore, India Everbridge Full time

    Everbridge: a fast-growing global provider of SaaS-based critical communications and enterprise safety solutions are currently seeking for a Principal Engineer to be based in our Bangalore office.As a Technical Lead, you will provide a focus for technical decision making, and an identifiable leader for the team and wider product area and ensure the...

  • Associate Principal Engineer

    4 days ago

    Bangalore City, India Condé Nast Technology Lab Full time

    Associate Principal Engineer - EngagementCondé Nast has a huge global footprint across 32 countries worldwide with total monthly visitors reaching more than 550 million. We aim to inspire, inform and entertain our audiences through our portfolio of well known brands such as Vogue, Wired, Vanity Fair, Architectural Digest, GQ and many other leading...

  • Associate Principal Engineer

    4 days ago

    Bangalore City, India Condé Nast Technology Lab Full time

    Associate Principal Engineer - EngagementCondé Nast has a huge global footprint across 32 countries worldwide with total monthly visitors reaching more than 550 million. We aim to inspire, inform and entertain our audiences through our portfolio of well known brands such as Vogue, Wired, Vanity Fair, Architectural Digest, GQ and many other leading...

  • Principal Engineer – Data

    4 months ago

    bangalore, India GSK Full time

    GSK is one of the world’s foremost pharmaceutical and healthcare companies, and we are proud to be part of an industry that improves the lives of others. We embark on a significant transformation journey to support GSK in becoming a top-quartile data-enabled organization. This is an exciting time to join GSK. We are embracing new data technologies to...

  • Principal Software Engineer

    4 months ago

    bangalore, India Oracle Full time

    Principal Software Development EngineerCloud Engineering Infrastructure Development8-10+ years of total experience in software developmentBangalore location onlyAs a Principal Software Engineer / Principal Member of Technical Staff, you will work with teams of software engineers responsible for the software design, development, and operations for our new...

  • Data Architect

    2 months ago

    Bangalore Urban, India Grid Dynamics Full time

    Experience: 12-16Years(only)Location: Bangalore/HyderabadA result- oriented thought-leader to drive the development of the data engineering practiceA trusted advisor and business partner to customers across verticals, and consulting team leader who establishes engineering processes and skill development.Responsibilities:Trusted Advisor: Provide direction to...