Principal Data Engineer

1 day ago


Bengaluru, Karnataka, India Draup Full time

About Draup:

We are a company at the forefront of data-driven innovation, harnessing cutting-edge technologies to extract valuable insights from the vast expanse of web data.

Job Summary:

This is an exceptional opportunity for a seasoned Principal Data Engineer with expertise in web scraping and AI-driven solutions to join our team. As a key member of our data engineering team, you will be responsible for designing and implementing distributed systems that can efficiently scrape large-scale datasets from diverse web sources.

Key Responsibilities:

  • Leverage your mastery of Python to architect and deploy scalable web scraping frameworks integrated with AI tools, enhancing automation capabilities.
  • Develop monitoring, logging, and alerting mechanisms to ensure high availability and performance of distributed systems.
  • Collaborate with cross-functional teams to design innovative AI-driven solutions for data extraction and automation.
  • Work with NoSQL databases like MongoDB to store and query scraped data efficiently.
  • Ensure data integrity and security while interacting with various web sources.

Requirements:

  • Extensive experience with Python and web frameworks such as Flask or Django.
  • Experience with AI tools and machine learning libraries to enhance and automate scraping processes.
  • Solid understanding of building and maintaining distributed systems, with hands-on experience in parallel programming (multithreading, asynchronous).
  • Working knowledge of asynchronous queue systems like Redis or RabbitMQ to handle distributed scraping tasks.
  • Proven experience with web mining, scraping tools (e.g., Scrapy, BeautifulSoup), and handling dynamic content.
  • Proficiency in working with NoSQL data storage systems like MongoDB, including querying and handling large datasets.
  • Estimated Salary: $145,000 per year


  • Bengaluru, Karnataka, India NTT DATA Services Full time

    Job Title: Principal Data ArchitectWe are seeking a highly experienced Innovation and Technology Lead to join our team at NTT DATA Services. This role requires strategic thinking, excellent leadership skills, and the ability to drive technology adoption and efficiency improvements across the organization.


  • Bengaluru, Karnataka, India Cymetrix Software Full time

    Job SummaryCymetrix Software is seeking a highly skilled Principal Data Engineer to lead the development of data pipelines by ingesting structured and unstructured data sources into Palantir Foundry.


  • Bengaluru, Karnataka, India The Principal TAG Full time

    Company OverviewThe Principal TAG is a technology-driven company that requires a skilled Lead SAP BW/4HANA Developer and Data Architect to lead our data strategy.SalaryWe offer a competitive salary of $120,000 per annum, commensurate with experience.Job DescriptionAs a Lead SAP BW/4HANA Developer and Data Architect at The Principal TAG, you will be...


  • Bengaluru, Karnataka, India Accolite Full time

    About the RoleAccolite is seeking a seasoned Principal Data Engineering Expert to lead our data engineering projects. This is an exciting opportunity for a talented individual with strong expertise in Python, Pyspark, and Pandas.ResponsibilitiesThe successful candidate will be responsible for:Designing and implementing scalable data pipelines using Python...


  • Bengaluru, Karnataka, India Squareroot Consulting Pvt Ltd Full time

    Principal Software Engineer Job Summary:We are looking for an exceptional Principal Software Engineer to lead the design and development of our Java-based applications. As a Principal Software Engineer, you will be responsible for architecting and implementing Microservices-based solutions using SpringBoot, utilizing NoSQL databases such as Scylla and Redis...


  • Bengaluru, Karnataka, India Covenant Consultants Full time

    We are seeking an experienced Senior Software Engineer to join our team at Covenant Consultants, a top investment bank.Job Title: Principal Data Engineering Specialist - Data Engineering Team.About the Role:This is a critical position in our organization that requires expertise in building and running complex data pipelines and services. As a key member of...


  • Bengaluru, Karnataka, India The Principal TAG Full time

    Job OverviewWe are seeking a highly skilled SAP BW/4HANA Developer to join our team at The Principal TAG. This is a 12-month remote opportunity that offers flexibility and the chance to work on challenging projects.Job DescriptionThe successful candidate will be responsible for designing, implementing, and optimizing large-scale data warehouses using SAP...


  • Bengaluru, Karnataka, India Talent21 Management and Shared Service Pvt Ltd Full time

    About Talent21 Management and Shared Service Pvt LtdWe are seeking a highly experienced Principal Software Engineering Lead to join our Epsilon Attribution/Forecasting Product Development team. As a Principal Software Engineering Lead, you will play a key role in designing, implementing, and optimizing data processing solutions for big data analysis,...


  • Bengaluru, Karnataka, India Almighty HRD Consultants Full time

    **Job Overview**We are seeking a seasoned data professional to join our client in Bangalore as a Principal Data Infrastructure Engineer. This is a full-time position ideal for someone with 7 to 10 years of experience in designing, building, and maintaining data infrastructure and systems.**Key Responsibilities**Design, develop, and deploy scalable and...


  • Bengaluru, Karnataka, India Iron Mountain Full time

    We are seeking a highly skilled Principal Data Architect to join our team at Iron Mountain. As a key member of our organization, you will play a pivotal role in designing and developing machine learning models and data processing algorithms for both new and existing customer implementations within the InSight platform.In this role, you will collaborate...


  • Bengaluru, Karnataka, India Continental Full time

    We are seeking a skilled Principal Data Architect to join our team at Continental. This role will involve owning modules of ContiTech's ONE Data & Analytics Platform and working closely with the platform architect to develop key features.The successful candidate will have a strong background in data engineering, with experience in designing, coding, and...


  • Bengaluru, Karnataka, India NTT DATA Services Full time

    About NTT DATA ServicesWe are a recognized leader in IT and business services, delivering innovative solutions to clients worldwide. As part of our team, you will have the opportunity to work on cutting-edge projects and contribute to our success.Job OverviewWe are currently seeking an experienced Sr. Cloud Native Developer to join our team in Bangalore,...


  • Bengaluru, Karnataka, India The Principal TAG Full time

    Job Title: Advanced SAP BW/4HANA Developer and Data ArchitectLocation: 100% RemoteDuration: 12 monthsBenefits:Flexible Work ScheduleJob Description:We are seeking an experienced SAP BW/4HANA Developer and Data Architect to join our team at The Principal TAG. As a key member of our data architecture team, you will be responsible for designing, implementing,...


  • Bengaluru, Karnataka, India MonoSpear Technologies Pvt Ltd Full time

    At MonoSpear Technologies Pvt Ltd, we are seeking a highly skilled Principal Data Engineer to lead our cloud and big data initiatives.Job SummaryWe are looking for an experienced data engineer who can design, develop, and maintain large-scale data processing systems using various technologies such as Apache Spark, Azure SQL Server, Cosmos DB/Mongo DB, and...


  • Bengaluru, Karnataka, India ThoughtFocus Full time

    We are seeking a highly skilled Principal Data Architect to join ThoughtFocus in Bangalore, India. As a Senior Data Engineer with 8-10 years of experience, you will design and optimize data pipelines, develop data infrastructure, and drive data integration projects to support business objectives.Key Responsibilities:Design, build, and maintain robust data...


  • Bengaluru, Karnataka, India Khatib & Alami Full time

    About the RoleWe are seeking a highly skilled Principal Engineering Manager to lead our engineering team at Khatib & Alami. This is a fantastic opportunity for an experienced Senior Project Engineer to take on a leadership role and drive our engineering efforts forward.


  • Bengaluru, Karnataka, India Luytens Technology Solutions Pvt. Ltd. Full time

    Job Summary: A seasoned Azure data expert is sought by Luytens Technology Solutions Pvt. Ltd. to lead the design, development, and optimization of large-scale data solutions on Microsoft Azure. Key responsibilities include leveraging Azure services for ETL processes, data warehousing, and analytics to ensure optimal performance and scalability.About Us: At...


  • Bengaluru, Karnataka, India Tata Communications Full time

    About the Role:As a Principal Data Science Leader at Tata Communications, you will play a critical role in shaping our AI engineering strategy and leading cross-functional teams to develop and deploy cutting-edge machine learning models. With over 10 years of experience in data science and a strong track record of delivering demonstrable outcomes, you will...


  • Bengaluru, Karnataka, India myGwork Full time

    About the RoleWe are seeking a highly skilled Principal Data Solutions Architect to join our team at Amazon. This is an exciting opportunity for a motivated and experienced professional to design and develop cutting-edge data solutions that drive business growth and innovation.


  • Bengaluru, Karnataka, India New Recruitments Hub Full time

    Job Title: Principal Software Engineering LeadAbout New Recruitments HubNew Recruitments Hub is a leading technology company that provides innovative solutions to businesses across the globe. We are currently seeking a highly skilled Principal Software Engineering Lead to join our team.Salary Range: $200,000 - $300,000 per annum (dependent on experience)Job...