下面这个robots是什么 意思?User-agent:* Disallow:/plus/ad_js.phpDisallow:/plus/advancedsearch.phpDisallow:/plus/car.phpDisallow:/plus/carbuyaction.phpDisallow:/plus/shops_buyaction.phpDisallow:/plus/erraddsave.phpDisallow:/plus/posttocar.phpD
来源:学生作业帮助网 编辑:作业帮 时间:2024/07/04 03:10:48
![下面这个robots是什么 意思?User-agent:* Disallow:/plus/ad_js.phpDisallow:/plus/advancedsearch.phpDisallow:/plus/car.phpDisallow:/plus/carbuyaction.phpDisallow:/plus/shops_buyaction.phpDisallow:/plus/erraddsave.phpDisallow:/plus/posttocar.phpD](/uploads/image/z/14745208-40-8.jpg?t=%E4%B8%8B%E9%9D%A2%E8%BF%99%E4%B8%AArobots%E6%98%AF%E4%BB%80%E4%B9%88+%E6%84%8F%E6%80%9D%3FUser-agent%3A%2A+Disallow%3A%2Fplus%2Fad_js.phpDisallow%3A%2Fplus%2Fadvancedsearch.phpDisallow%3A%2Fplus%2Fcar.phpDisallow%3A%2Fplus%2Fcarbuyaction.phpDisallow%3A%2Fplus%2Fshops_buyaction.phpDisallow%3A%2Fplus%2Ferraddsave.phpDisallow%3A%2Fplus%2Fposttocar.phpD)
下面这个robots是什么 意思?User-agent:* Disallow:/plus/ad_js.phpDisallow:/plus/advancedsearch.phpDisallow:/plus/car.phpDisallow:/plus/carbuyaction.phpDisallow:/plus/shops_buyaction.phpDisallow:/plus/erraddsave.phpDisallow:/plus/posttocar.phpD
下面这个robots是什么 意思?
User-agent:*
Disallow:/plus/ad_js.php
Disallow:/plus/advancedsearch.php
Disallow:/plus/car.php
Disallow:/plus/carbuyaction.php
Disallow:/plus/shops_buyaction.php
Disallow:/plus/erraddsave.php
Disallow:/plus/posttocar.php
Disallow:/plus/disdls.php
Disallow:/plus/feedback_js.php
Disallow:/plus/mytag_js.php
Disallow:/plus/rss.php
Disallow:/plus/search.php
Disallow:/plus/recommend.php
Disallow:/plus/stow.php
Disallow:/plus/count.php
Disallow:/include
Disallow:/templets
如果下面这样写的话,会有什么危害吗?
User-Agent:*
Allow:/
下面这个robots是什么 意思?User-agent:* Disallow:/plus/ad_js.phpDisallow:/plus/advancedsearch.phpDisallow:/plus/car.phpDisallow:/plus/carbuyaction.phpDisallow:/plus/shops_buyaction.phpDisallow:/plus/erraddsave.phpDisallow:/plus/posttocar.phpD
User-agent:* 这里的*代表的所有的搜索引擎,*是一个通配符.
Disallow:/plus/ad_js.php Disallow是不允许、禁止的意思,这里定义是禁止爬寻plus目录下的ad_js.php文件.
下边的Disallow的意思也是禁止所有搜索引擎都不允许爬取的文件或目录.
User-Agent:*
Allow:/ 的意思是所有的搜索引擎都可以爬取你网站的所有内容.
Robots协议用来告知搜索引擎哪些页面能被抓取,哪些页面不能被抓取;可以屏蔽一些网站中比较大的文件,如:图片,音乐,视频等,节省服务器带宽;可以屏蔽站点的一些死链接.方便搜索引擎抓取网站内容;设置网站地图连接,方便引导蜘蛛爬取页面.
如果你设置了所有都可以爬取,那就不必要设置robots,上述的所有robots的作用也都没用.
Disallow:表示禁止抓取的!
Allow: /表示整站任意页面可抓取
危害么就是把你的模板页面都给抓出来了~有的不想给别人看的东西也被搜出来了