Principal data engineer

4 weeks ago

Bengaluru, India Taglynk Full time
Job Summary:
The Principal Data Engineer is responsible for leading the design, development, and optimization of scalable data pipelines, data lakes, and large-scale datasets. With over 9 years of experience in data engineering and expertise in tools like Elastic Search, Redshift, and data lakes, this role ensures the efficient processing and storage of massive data streams, driving business intelligence and data-driven decision-making across the organization.
Key Responsibilities:
Data Architecture & Pipeline Development:
Design and implement highly scalable, reliable, and efficient data pipelines to process large datasets using modern data engineering tools and practices.
Architect and manage data lakes, enabling storage and querying of unstructured and structured data.
Lead initiatives to optimize data retrieval and integration using Elastic Search, Redshift, and other distributed systems.
Large Dataset Management:
Oversee the storage, processing, and analysis of large datasets, ensuring data quality, security, and performance.
Develop and maintain ETL processes to manage data flow across multiple sources, transforming and enriching data for analytics and reporting.
Collaborate with data scientists and analysts to provide seamless access to data and drive analytical outcomes.
Data Governance & Compliance:
Ensure data governance policies are adhered to, maintaining high standards of data quality, privacy, and security.
Implement and enforce best practices for data access control, data lineage, and monitoring.
Ensure compliance with industry standards and regulations regarding data protection (e.g., GDPR).
Performance Optimization:
Continuously monitor and optimize the performance of Elastic Search, Redshift, and other data infrastructure.
Identify and resolve bottlenecks, and work on improving query performance and cost efficiency.
Experiment with new technologies and tools to keep data engineering processes at the cutting edge.
Team Leadership & Mentorship:
Lead and mentor a team of data engineers, providing guidance on best practices, coding standards, and architecture design.
Collaborate with cross-functional teams (e.g., product, engineering, data science) to meet business requirements and drive data innovation.
Participate in code reviews and ensure adherence to coding standards and architectural guidelines.
Innovation & Strategic Planning:
Stay updated on emerging trends in big data technologies and proactively recommend improvements or changes to the existing data architecture.
Contribute to the long-term strategy for data architecture, ensuring scalability and flexibility for future growth.
Key Qualifications:
Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
9+ years of experience in data engineering, with at least 5 years of hands-on experience working with large datasets and managing data architectures involving Elastic Search, Redshift, and data lakes.
Proven experience in designing and implementing data pipelines, ETL processes, and distributed data systems.
Expertise in cloud-based data platforms (AWS, Azure, etc.), particularly in managing Redshift and data lake infrastructure.
Strong programming skills in Python, Java, or Scala, and experience with SQL.
Solid understanding of data modeling, database design, and performance tuning for large-scale systems.

  • Principal Data Architect

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Services Full time

    Job Title: Principal Data ArchitectWe are seeking a highly experienced Innovation and Technology Lead to join our team at NTT DATA Services. This role requires strategic thinking, excellent leadership skills, and the ability to drive technology adoption and efficiency improvements across the organization.

  • Principal Data Engineer

    6 months ago

    Bengaluru, India 14260 GSK India Global Services Private Limited Full time

    At GSK, we are dedicated to improving the quality of human life by enabling people to do more, feel better, and live longer. As a global leader in pharmaceuticals and healthcare, we have a rich history dating back to the 19th century and a relentless commitment to advancing healthcare for the betterment of humanity. Our mission is to help people around the...

  • Principal Data Engineer

    1 week ago

    Bengaluru, India Kinara Capital Full time

    About Company Kinara Capital is a FinTech NBFC dedicated to driving Financial Inclusion in the MSME sector. Our mission is to transform lives, livelihoods, and local economies by providing fast and flexible loans without property collateral to small business entrepreneurs. Led by a women-majority management team, Kinara Capital values diversity and...

  • Lead SAP BW/4HANA Developer and Data Architect

    2 weeks ago

    Bengaluru, Karnataka, India The Principal TAG Full time

    Company OverviewThe Principal TAG is a technology-driven company that requires a skilled Lead SAP BW/4HANA Developer and Data Architect to lead our data strategy.SalaryWe offer a competitive salary of $120,000 per annum, commensurate with experience.Job DescriptionAs a Lead SAP BW/4HANA Developer and Data Architect at The Principal TAG, you will be...

  • Data & Analytics Principal Consultant

    1 month ago

    Bengaluru, India NTT DATA Full time

    Job Description Req ID: 295091 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data & Analytics Principal Consultant to join our team in Bangalore, Karnātaka (IN-KA), India...

  • Data & Analytics Principal Consultant

    1 month ago

    Bengaluru, India NTT DATA Full time

    Job Description Req ID: 295075 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data & Analytics Principal Consultant to join our team in Bangalore, Karnātaka (IN-KA), India...

  • Principal Data Engineer

    2 months ago

    Bengaluru, India Rock interviews Full time

    What We Are Looking For : As a Principal Data Engineer, you will take ownership of the ideation, architecture, design, and development of our enterprise data platform. You will play a key role in collaborating with cloud and security architects to ensure a seamless integration within our broader technology strategy. We are looking for someone with a passion...

  • Principal Data Engineering Expert

    2 weeks ago

    Bengaluru, Karnataka, India Accolite Full time

    About the RoleAccolite is seeking a seasoned Principal Data Engineering Expert to lead our data engineering projects. This is an exciting opportunity for a talented individual with strong expertise in Python, Pyspark, and Pandas.ResponsibilitiesThe successful candidate will be responsible for:Designing and implementing scalable data pipelines using Python...

  • Data & Analytics Principal Consultant

    1 month ago

    Bengaluru, India NTT DATA Full time

    Job Description Req ID: 295146 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data & Analytics Principal Consultant to join our team in Bangalore, Karnātaka (IN-KA), India...

  • Principal Data Engineer

    6 months ago

    Bengaluru, India VISA Full time

    Job Description And Responsibilities Are you interested in building world class B2B data solutions & products serving 6 markets, 50+ countries fueling growth? With tremendous amount of data there is a huge untapped potential to uncover insights, architect state of the art data platforms & cutting-edge application development. Commercial and Money...

  • Data & Analytics Principal Consultant

    3 months ago

    Bengaluru, India NTT DATA Services Full time

    Req ID: 295075 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data & Analytics Principal Consultant to join our team in Bangalore, Karnātaka (IN-KA), India (IN).Job Duties: Must...

  • Data & Analytics Principal Consultant

    4 months ago

    Bengaluru, India NTT DATA Services Full time

    Req ID: 295142 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data & Analytics Principal Consultant to join our team in Bangalore, Karnātaka (IN-KA), India (IN).Job Duties: Must...

  • Data & Analytics Principal Consultant

    3 weeks ago

    Bengaluru, India NTT DATA Full time

    Job Description Req ID: 295145 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data & Analytics Principal Consultant to join our team in Bangalore, Karnātaka (IN-KA), India...

  • Data Engineering Principal

    3 weeks ago

    Bengaluru, India Metyis Full time

    What we offer   Interact with C-level at our clients on regular basis to drive their business towards impactful change. Lead your team in creating new business solutions. Seize opportunities at the client and at Metyis in our entrepreneurial environment. Become part of a fast-growing international and diverse team. What you will do We are...

  • Principal Consultant, Data Engineer

    3 weeks ago

    Bengaluru, India Genpact Full time

    Genpact (NYSE: G) is a global professional services and solutions firm delivering outcomes that shape the future. Our 125,000+ people across 30+ countries are driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients. Powered by our purpose – the relentless pursuit of a world that works better for people –...

  • Principal Engineer

    6 months ago

    Bengaluru, India Amazon Development Center India Private Limited - Kerala - P86 Full time

    Principal Engineers provide technical leadership at Amazon.com. They help establish technical standards and drive Amazon’s overall technical architecture, engineering practices, and engineering methodologies. They work on our hardest problems, building high quality, architecturally sound systems that are aligned with our business needs. They think globally...

  • Principal Data Engineering Specialist

    3 weeks ago

    Bengaluru, Karnataka, India Covenant Consultants Full time

    We are seeking an experienced Senior Software Engineer to join our team at Covenant Consultants, a top investment bank.Job Title: Principal Data Engineering Specialist - Data Engineering Team.About the Role:This is a critical position in our organization that requires expertise in building and running complex data pipelines and services. As a key member of...

  • Data & Analytics Principal Consultant

    1 month ago

    Bengaluru, India NTT DATA Services Full time

    Req ID: 295146 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data & Analytics Principal Consultant to join our team in Bangalore, Karnātaka (IN-KA), India...

  • Principal Consultant- Data Engineer

    4 months ago

    Bengaluru, India Genpact Full time

    Genpact (NYSE: G) is a global professional services and solutions firm delivering outcomes that shape the future. Our 125,000+ people across 30+ countries are driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients. Powered by our purpose – the relentless pursuit of a world that works better for people –...

  • Principal Data Infrastructure Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Almighty HRD Consultants Full time

    **Job Overview**We are seeking a seasoned data professional to join our client in Bangalore as a Principal Data Infrastructure Engineer. This is a full-time position ideal for someone with 7 to 10 years of experience in designing, building, and maintaining data infrastructure and systems.**Key Responsibilities**Design, develop, and deploy scalable and...