以下代码是学做网站网搜集的屏蔽无用和恶意蜘蛛的代码，想添加禁止爬取的蜘蛛只需要修改中间部分的蜘蛛名称即可。
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} “^$|^-$|Chrome/92.0.4515.159|compatible|BLEXBot/1.0|MSNbot|Webdup|AcoonBot|SemrushBot|CrawlDaddy|DotBot|Applebot|AhrefsBot|Ezooms|EdisterBot|EC2LinkFinder|jikespider|Purebot|MJ12bot|DingTalkBot|DuckDuckBot|WangIDSpider|WBSearchBot|Wotbox|xbfMozilla|Yottaa|YandexBot|Barkrowler|SeznamBot|Jorgee|CCBot|SWEBot|PetalBot|spbot|TurnitinBot-Agent|mail.RU|curl|perl|Python|Wget|Xenu|ZmEu|EasouSpider|YYSpider|python-requests|oBot|MauiBot” [NC]
RewriteRule !(^robots\.txt$) http://en.wikipedia.org/wiki/Robots_exclusion_standard [R=403,L]
将上述内容保存在名为 .htaccess 的文件，上传到网站的根目录中即可。
注意：Windows无法起一个点(.)开头的文件名。现在本地创建 htaccess.txt 的记事本文件，再将记事本上传到网站根目录后用FTP改下名字即可！
