Principal Data Engineer

3 months ago

Bangalore Urban, India Taglynk Full time

Job Summary:

The Principal Data Engineer is responsible for leading the design, development, and optimization of scalable data pipelines, data lakes, and large-scale datasets. With over 9 years of experience in data engineering and expertise in tools like ElasticSearch, Redshift, and data lakes, this role ensures the efficient processing and storage of massive data streams, driving business intelligence and data-driven decision-making across the organization.

Key Responsibilities:

Data Architecture & Pipeline Development:

  • Design and implement highly scalable, reliable, and efficient data pipelines to process large datasets using modern data engineering tools and practices.
  • Architect and manage data lakes, enabling storage and querying of unstructured and structured data.
  • Lead initiatives to optimize data retrieval and integration using ElasticSearch, Redshift, and other distributed systems.

Large Dataset Management:

  • Oversee the storage, processing, and analysis of large datasets, ensuring data quality, security, and performance.
  • Develop and maintain ETL processes to manage data flow across multiple sources, transforming and enriching data for analytics and reporting.
  • Collaborate with data scientists and analysts to provide seamless access to data and drive analytical outcomes.

Data Governance & Compliance:

  • Ensure data governance policies are adhered to, maintaining high standards of data quality, privacy, and security.
  • Implement and enforce best practices for data access control, data lineage, and monitoring.
  • Ensure compliance with industry standards and regulations regarding data protection (e.g., GDPR).

Performance Optimization:

  • Continuously monitor and optimize the performance of ElasticSearch, Redshift, and other data infrastructure.
  • Identify and resolve bottlenecks, and work on improving query performance and cost efficiency.
  • Experiment with new technologies and tools to keep data engineering processes at the cutting edge.

Team Leadership & Mentorship:

  • Lead and mentor a team of data engineers, providing guidance on best practices, coding standards, and architecture design.
  • Collaborate with cross-functional teams (e.g., product, engineering, data science) to meet business requirements and drive data innovation.
  • Participate in code reviews and ensure adherence to coding standards and architectural guidelines.

Innovation & Strategic Planning:

  • Stay updated on emerging trends in big data technologies and proactively recommend improvements or changes to the existing data architecture.
  • Contribute to the long-term strategy for data architecture, ensuring scalability and flexibility for future growth.

Key Qualifications:

  • Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
  • 9+ years of experience in data engineering, with at least 5 years of hands-on experience working with large datasets and managing data architectures involving ElasticSearch, Redshift, and data lakes.
  • Proven experience in designing and implementing data pipelines, ETL processes, and distributed data systems.
  • Expertise in cloud-based data platforms (AWS, Azure, etc.), particularly in managing Redshift and data lake infrastructure.
  • Strong programming skills in Python, Java, or Scala, and experience with SQL.
  • Solid understanding of data modeling, database design, and performance tuning for large-scale systems.

  • Principal Data Engineer

    3 months ago

    Bangalore Urban, India Taglynk Full time

    Job Summary: The Principal Data Engineer is responsible for leading the design, development, and optimization of scalable data pipelines, data lakes, and large-scale datasets. With over 9 years of experience in data engineering and expertise in tools like ElasticSearch, Redshift, and data lakes, this role ensures the efficient processing and storage of...

  • Principal Data Scientist

    2 weeks ago

    Bangalore Urban, India Target Full time

    As a Fortune 50 company with more than 400,000 team members worldwide, Target is an iconic brand and one of America's leading retailers. At Target, we have a timeless purpose and a proven strategy. And that hasn’t happened by accident. Some of the best minds from diverse backgrounds come together at Target to redefine retail in an inclusive learning...

  • Principal Data Scientist

    2 weeks ago

    Bangalore Urban, India Target Full time

    As a Fortune 50 company with more than 400,000 team members worldwide, Target is an iconic brand and one of America's leading retailers. At Target, we have a timeless purpose and a proven strategy. And that hasn’t happened by accident. Some of the best minds from diverse backgrounds come together at Target to redefine retail in an inclusive learning...

  • Principal Data Engineer

    1 month ago

    Bangalore, India Rock interviews Full time

    What We Are Looking For : As a Principal Data Engineer, you will take ownership of the ideation, architecture, design, and development of our enterprise data platform. You will play a key role in collaborating with cloud and security architects to ensure a seamless integration within our broader technology strategy. We are looking for someone with a passion...

  • Principal ML Scientist

    3 months ago

    Bangalore Urban, India Nykaa Full time

    Role Description: Principal Machine Learning Scientist - SearchLocation: BangaloreAbout the Team: Join Nykaa's Data Science team as a Principal Machine Learning Scientist, where you'll play a pivotal role in driving advancements in search relevance and ranking across our platforms. In this role, you will analyze data, develop machine learning models, and...

  • Senior Data Quality Strategist

    4 weeks ago

    Remote/Bangalore, India The Principal TAG Full time

    About The Principal TAGWe are seeking an experienced Senior Data Quality Strategist to lead our COE establishment and centralize all testing artifacts using Jira TTM. The ideal candidate will have 15+ years of QA experience, proficiency in Tosca DI, test case management, and test automation strategies.Estimated Salary: $150,000 - $180,000 per yearJob...

  • myGwork | Principal Engineer

    3 weeks ago

    bangalore, India myGwork Full time

    This job is with Amazon, an inclusive employer and a member of myGwork – the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly.DESCRIPTION:Principal Engineers provide technical leadership at Amazon.com. They help establish technical standards and drive Amazon's overall technical architecture,...

  • Principal Software Engineer

    2 months ago

    Bangalore Urban, India CAST Full time

    Job Overview: We are looking for a skilled Principal Software Engineer who is proficient in both front-end and back-end technologies, specifically in Java and React. The ideal candidate will have a deep understanding of the entire web development process, including design, development, and deployment. You will be responsible for building scalable web...

  • Principal Software Engineer

    2 months ago

    Bangalore Urban, India CAST Full time

    Job Overview:We are looking for a skilled Principal Software Engineer who is proficient in both front-end and back-end technologies, specifically in Java and React. The ideal candidate will have a deep understanding of the entire web development process, including design, development, and deployment. You will be responsible for building scalable web...

  • Principal Data Engineer

    3 months ago

    bangalore, India Taglynk Full time

    Job Summary:The Principal Data Engineer is responsible for leading the design, development, and optimization of scalable data pipelines, data lakes, and large-scale datasets. With over 9 years of experience in data engineering and expertise in tools like ElasticSearch, Redshift, and data lakes, this role ensures the efficient processing and storage of...

  • Principal Engineer: SOC Verification

    2 months ago

    Bangalore Urban, India Cyient Full time

    Dear Applicant,We, Cyient is hiring for Principal Engineer: SOC Verification for End to End Offshore-Onshore model based Global Product R&D Solution.Base Location: Bangalore/ HyderabadExp Range: 17-28 YrsBudget: Open for DiscussionKey Skills:1. Candidate must have strong experience in SOC/ IP Verification Flow from scratch.2. Candidate must have strong...

  • Principal Engineer: SOC Verification

    2 months ago

    Bangalore Urban, India Cyient Full time

    Dear Applicant, We, Cyient is hiring for Principal Engineer: SOC Verification for End to End Offshore-Onshore model based Global Product R&D Solution. Base Location: Bangalore/ Hyderabad Exp Range: 17-28 Yrs Budget: Open for Discussion Key Skills: 1. Candidate must have strong experience in SOC/ IP Verification Flow from scratch. 2. Candidate must have...

  • Freshworks - Principal Data Engineer - Synapse Analytics

    1 month ago

    Bangalore, India Freshworks Full time

    Job Description : We are seeking a dynamic and highly skilled Principal Data Engineer who has extensive experience building enterprise scale data platforms and lead these foundational efforts. This role demands someone who not only possesses a profound understanding of the data engineering landscape but is also at the forefront of their game.The ideal...

  • Kinara Capital | Principal Data Engineer | bangalore

    4 weeks ago

    bangalore, India Kinara Capital Full time

    About Company Kinara Capital is a FinTech NBFC dedicated to driving Financial Inclusion in the MSME sector. Our mission is to transform lives, livelihoods, and local economies by providing fast and flexible loans without property collateral to small business entrepreneurs. Led by a women-majority management team, Kinara Capital values diversity and...

  • Principal software engineer

    2 weeks ago

    Bangalore, India Borneo Full time

    Overview: Borneo is a fast-growing startup developing a cutting-edge data security and compliance automation platform, and we’re seeking experienced and hands-on Principal Engineers to drive innovation and contribute to our mission of transforming data security and privacy. As the Principal Engineer, you will be a driving force in shaping the technical...

  • Principal devops engineer

    2 weeks ago

    Bangalore, India Informatica Full time

    Build Your Career at Informatica We're looking for a diverse group of collaborators who believe data has the power to improve society. Adventurous minds who value solving some of the world's most challenging problems. Here, employees are encouraged to push their boldest ideas forward, united by a passion to create a world where data improves the quality...

  • Principal Software Engineer

    3 days ago

    Bangalore, India Nanonets Full time

    Nanonets has a vision to help computers see the world starting with reading and understanding documents.Machine Learning (ML) is no longer a futuristic concept—it's a present-day powerhouse transforming the business landscape. Nanonets is at the forefront of this transformation, offering innovative ML solutions designed to make document related...

  • Principal software engineer

    2 weeks ago

    Bangalore, India Oracle Full time

    Principal Software Development Engineer Cloud Engineering Infrastructure Development 8-10+ years of total experience in software development Bangalore location only As a Principal Software Engineer / Principal Member of Technical Staff, you will work with teams of software engineers responsible for the software design, development, and operations...

  • Principal Software Engineer

    3 weeks ago

    Bangalore, India Borneo Full time

    Overview: Borneo is a fast-growing startup developing a cutting-edge data security and compliance automation platform, and we’re seeking experienced and hands-on Principal Engineers to drive innovation and contribute to our mission of transforming data security and privacy. As the Principal Engineer, you will be a driving force in shaping the technical...

  • Principal Engineer

    2 months ago

    bangalore, India Angel One Full time

    About Angel one :Angel One Limited is a Fintech company providing broking services, margin trading facility, research services, depository services, investment education and financial products distribution to its clients, on a mission to become the No. 1 fintech organization in India. With about 23.0 million registered clients, we are onboarding an average...