AI - Data Engineer - Web Scraping

2 weeks ago

Anywhere in IndiaMultiple Locations Taiyo Full time ₹ 15,00,000 - ₹ 25,00,000 per year

Description :

About Taiyo.AI :

We empower the people who build the world.

Taiyo.AI is the world's first infrastructure intelligence platform. We are building the largest universal and industry standard database of opportunities (tenders, projects, news) and threats (economy, climate, geopolitics, finance, logistics, etc.) for real assets. Taiyo.AI has been instrumental in shaping how infrastructure companies (infra investors, engineering, procurement, construction, and infra insurers) benchmark new project development opportunities, get a panoramic and dynamic view of external risks, predict prices, identify drivers, and mitigate supply-side disruptions. We are seeking a candidate that is willing to learn and contribute to emerging technology and policy.

About The Team :

We are looking for the head of cloud backend engineering to oversee backend ops for managing and monitoring the data, related predictive analysis, provide insight into infrastructure projects related project screening and dynamically evaluate external risks, with a strong focus on supporting automation, process design, and resource planning.

Key responsibilities :

- Work on data sourcing

- Use web scrapers (Beautifulsoup, selenium, etc.)

- Manage the data normalization and standards validation

- Parametrize and automate the scrapers

- Develop and execute the processes for monitoring data sanity and checking for data availability and reliability

- Understand the business drivers and build insights through data

- Work with the stakeholders at all levels to establish current and ongoing data support and reporting needs

- Ensure continuous data accuracy and recognize data discrepancies in systems that require immediate attention/escalation

- Work and become an expert in the company's data warehouse and other data storage tools, understanding the definition, context, and proper use of all attributes and metrics

- Create dashboards based on business requirements

- Work on the distributed systems, scale, cloud, caching, CI/CD (continuous integration and deployment), distributed logging, data pipeline, REST API)

Who can apply :

- Creativity & complex problem-solving skills

- Exceptional and scalable web scraping skills

- Passion and interest in doing ETL jobs

- Good English speaking and communication skills

- Ability to work with a global remote culture

- Initiative and entrepreneurship skills

- Experience with microservices architecture and writing REST APIs

- Knowledge of Kubernetes, Docker and Airflow

- Prior experience with Python, Django and Gunicorn

- Independent work ethic with an ability to work in a fast-paced environment

We are looking for data engineers with Python scripting practices and scalable web scraping skills, including monitoring ingestion of data, adhering to data standards, and solid knowledge of data and cloud workflow orchestration

  • AI Web Scraping Engineer

    2 weeks ago

    india, IN S2T AI - AI-Powered Investigations Full time

    We're seeking a forward-thinking Web Scraping Engineer who leverages AI tools to accelerate development and streamline data extraction processes. Join our India team and work at the intersection of traditional scraping expertise and cutting-edge AI-powered development.The Role:Design and implement scalable data extraction solutions using AI to rapidly...

  • Ai web scraping engineer

    2 weeks ago

    India S2T AI - AI-Powered Investigations Full time

    We're seeking a forward-thinking Web Scraping Engineer who leverages AI tools to accelerate development and streamline data extraction processes. Join our India team and work at the intersection of traditional scraping expertise and cutting-edge AI-powered development. The Role: Design and implement scalable data extraction solutions using AI to rapidly...

  • Ai web scraping engineer

    3 weeks ago

    India S2T AI - AI-Powered Investigations Full time

    We're seeking a forward-thinking Web Scraping Engineer who leverages AI tools to accelerate development and streamline data extraction processes. Join our India team and work at the intersection of traditional scraping expertise and cutting-edge AI-powered development. The Role: Design and implement scalable data extraction solutions using AI to...

  • AI Web Scraping Engineer

    3 weeks ago

    India S2T AI - AI-Powered Investigations Full time

    We're seeking a forward-thinking Web Scraping Engineer who leverages AI tools to accelerate development and streamline data extraction processes. Join our India team and work at the intersection of traditional scraping expertise and cutting-edge AI-powered development. The Role: Design and implement scalable data extraction solutions using AI to...

  • AI Web Scraping Engineer

    3 weeks ago

    India S2T AI - AI-Powered Investigations Full time

    We're seeking a forward-thinking Web Scraping Engineer who leverages AI tools to accelerate development and streamline data extraction processes. Join our India team and work at the intersection of traditional scraping expertise and cutting-edge AI-powered development. The Role: Design and implement scalable data extraction solutions using AI to rapidly...

  • AI Web Scraping Engineer

    3 weeks ago

    India S2T AI - AI-Powered Investigations Full time

    We're seeking a forward-thinking Web Scraping Engineer who leverages AI tools to accelerate development and streamline data extraction processes. Join our India team and work at the intersection of traditional scraping expertise and cutting-edge AI-powered development.The Role:Design and implement scalable data extraction solutions using AI to rapidly...

  • 15h Left: AI Web Scraping Engineer

    3 weeks ago

    India S2T AI - AI-Powered Investigations Full time

    We're seeking a forward-thinking Web Scraping Engineer who leverages AI tools to accelerate development and streamline data extraction processes. Join our India team and work at the intersection of traditional scraping expertise and cutting-edge AI-powered development. The Role: - Design and implement scalable data extraction solutions using AI to rapidly...

  • Senior web scraping engineer

    2 weeks ago

    India S2T AI - AI-Powered Investigations Full time

    We are on the lookout for a highly competent, self-motivated Senior Web Scraping Engineer with real-world experience in API Scraping or Mobile Scraping to join our India team. The Role: Gather and process raw data at scale (including writing scripts, web scraping, calling APIs) Work in a larger team. Able to work independently to complete...

  • Senior Web Scraping Engineer

    1 week ago

    India S2T AI - AI-Powered Investigations Full time

    We are on the lookout for a highly competent, self-motivated Senior Web Scraping Engineer with real-world experience in API Scraping or Mobile Scraping to join our India team. The Role: Gather and process raw data at scale (including writing scripts, web scraping, calling APIs) Work in a larger team. Able to work independently to complete assigned tasks The...

  • Senior Web Scraping Engineer

    2 weeks ago

    india, IN S2T AI - AI-Powered Investigations Full time

    We are on the lookout for a highly competent, self-motivated Senior Web Scraping Engineer with real-world experience in API Scraping or Mobile Scraping to join our India team.The Role:Gather and process raw data at scale (including writing scripts, web scraping, calling APIs)Work in a larger team.Able to work independently to complete assigned tasksThe...