ibou’s four crawlers

Our crawlers explore the public web to help you find pages, articles and images in ibou’s services. Each has its own role.

The four crawlers, their roles, user agents and IP ranges.
Service and role	User-agent	IP ranges
RecherchePages for ibou’s search engine.	`Mozilla/5.0 (compatible; ibou-search/2.0; +https://www.ibou.com/crawlers.html)`	IP list (JSON)
DécouverteArticles for ibou Découverte.	`Mozilla/5.0 (compatible; ibou-discovery/2.0; +https://www.ibou.com/crawlers.html)`	IP list (JSON)
ImagesFinding and cataloguing images.	`Mozilla/5.0 (compatible; ibou-images/2.0; +https://www.ibou.com/crawlers.html)`	Coming soon
ActualitésArticles for ibou Actualités.	`Mozilla/5.0 (compatible; ibou-news/2.0; +https://www.ibou.com/crawlers.html)`	IP list (JSON)

The user-agent is the identification string a crawler sends to your server. It can be copied, so it does not prove that a visit comes from ibou. IP lists may change.

Your site, your rules

All four crawlers follow robots.txt, the file where you specify which parts of your site they may explore. We link back to the sites we cite, and we do not train AI models with the data collected. Read our commitments.

Les quatre robots respectent robots.txt, le fichier où vous indiquez les parties de votre site qu’ils peuvent explorer. Nous renvoyons vers les sites que nous citons et n’entraînons pas de modèles d’IA avec les données collectées. Lire nos engagements.

Technical details

Limit or block crawling

To block all four crawlers, add this group to your site’s /robots.txt file:

Pour bloquer les quatre robots, ajoutez ce groupe au fichier /robots.txt de votre site :

User-agent: ibou-search
User-agent: ibou-discovery
User-agent: ibou-images
User-agent: ibou-news
Disallow: /

To target just one crawler, keep only its user-agent line. To block just one directory, replace / with its path, for example /private/.

Pour ne viser qu’un robot, gardez uniquement sa ligne User-agent. Pour exclure un seul dossier, remplacez / par son chemin, par exemple /prive/.

ibou-search: timing and verification

The following information applies only to ibou-search. The other three crawlers do not support Crawl-delay.

Ces informations concernent uniquement ibou-search. Les trois autres robots ne prennent pas en charge Crawl-delay.

Slow down visits

ibou-search waits 5 seconds between requests to the same host, and 2.5 seconds between requests to the same IP within a domain. To increase the delay to 10 seconds per host, use this rule in robots.txt:

ibou-search attend 5 secondes entre deux requêtes sur un même hôte, et 2,5 secondes sur une même IP d’un même domaine. Pour porter le délai à 10 secondes par hôte, utilisez cette règle dans robots.txt :

User-agent: ibou-search
Crawl-delay: 10

If you already have a group for ibou-search, add the delay there alongside its existing rules.

Si un groupe vise déjà ibou-search, ajoutez-y le délai en conservant ses autres règles.

Verify a visit

Run a reverse DNS lookup on the source IP. The hostname must end in .ibou.io.
Resolve that hostname to its IP addresses (A for IPv4, AAAA for IPv6). The original IP must be among them.

Both checks are needed. The ibou-search IP ranges (JSON) are also available; this list may change.

Need to adjust visits or report a problem? Write to network-abuse@ibou.com.

Include the domain, source IP address and a few lines from your server logs.

ibou’s four crawlers

Les quatre robots d’ibou

Your site, your rules

Votre site, vos règles

Technical details

Les détails techniques

Slow down visits

Ralentir les visites

Verify a visit

Vérifier une visite