Is there any specification regarding robots.txt for Gemini crawlers? I've recently changed some URLs on Yesterweb and am now considering how to implement redirection without keeping the old files.
Mar 28 · 3 months ago
4 Comments ↓
🏕️ skyjake [mod...] · Mar 29 at 04:29:
— geminiprotocol.net/docs/companion/robots.gmi
wow, thanks! however, the protocol does not mention anything about the redirects, correct?
🏕️ skyjake [mod...] · Mar 30 at 17:20:
If you're referring to redirecting away from obsolete URLs, just ensure you use the permanent redirection status and it should be fine.
@skyjake unfortunately I can't operate with the status code there,
thoughts to keep old document for some time, with new location reference in body.
but my fs directory looks like big trash with that. Sometimes I miss .htaccess, lol
Source