设置教程七牛云存储robots.txt搜索引擎规则

设置教程七牛云存储robots.txt搜索引擎规则

默认robots.txt

# robots.txt generated at http://portal.qiniu.com
User-agent: Baiduspider
Disallow: /
User-agent: Sosospider
Disallow: /
User-agent: sogou spider
Disallow: /
User-agent: YodaoBot
Disallow: /
User-agent: Googlebot
Disallow: /
User-agent: Bingbot
Disallow: /
User-agent: Slurp
Disallow: /
User-agent: MSNBot
Disallow: /
User-agent: googlebot-image
Disallow: /
User-agent: googlebot-mobile
Disallow: /
User-agent: yahoo-blogs/v3.9
Disallow: /
User-agent: psbot
Disallow: /
User-agent: *
Disallow: /

修改以后的

# robots.txt generated at http://portal.qiniu.com
# edit for statice domain
User-agent: Baiduspider
Allow: /wp-content/uploads/sites/2/*.jpg$
Allow: /wp-content/uploads/sites/2/*.png$
Disallow: /
User-agent: Baiduspider-image
Allow: /wp-content/uploads/sites/2/*.jpg$
Allow: /wp-content/uploads/sites/2/*.png$
Disallow: /
User-agent: 360Spider
Allow: /wp-content/uploads/sites/2/*.jpg$
Allow: /wp-content/uploads/sites/2/*.png$
Disallow: /
User-agent: 360Spider-Image
Allow: /wp-content/uploads/sites/2/*.jpg$
Allow: /wp-content/uploads/sites/2/*.png$
Disallow: /
User-agent: Sosospider
Disallow: /
User-agent: sogou spider
Disallow: /
User-agent: YodaoBot
Disallow: /
User-agent: Googlebot
Disallow: /
User-agent: Bingbot
Disallow: /
User-agent: Slurp
Disallow: /
User-agent: MSNBot
Disallow: /
User-agent: googlebot-image
Disallow: /
User-agent: googlebot-mobile
Disallow: /
User-agent: yahoo-blogs/v3.9
Disallow: /
User-agent: psbot
Disallow: /
User-agent: *
Disallow: /

# robots.txt generated at http://portal.qiniu.com
User-agent: Baiduspider
Allow: /wp-content/uploads/*.jpg$
Allow: /wp-content/uploads/*.png$
Disallow: /
User-agent: 360Spider
Allow: /wp-content/uploads/*.jpg$
Allow: /wp-content/uploads/*.png$
Disallow: /
User-agent: Baiduspider-image
Allow: /wp-content/uploads/*.jpg$
Allow: /wp-content/uploads/*.png$
Disallow: /
User-agent: 360Spider-Image
Allow: /wp-content/uploads/*.jpg$
Allow: /wp-content/uploads/*.png$
Disallow: /
User-agent: *
Disallow: /

除了百度360以外还带神马搜索蜘蛛/搜狗/

# robots.txt generated at http://portal.qiniu.com
User-agent: Baiduspider
Allow: /wp-content/uploads/*.jpg$
Allow: /wp-content/uploads/*.png$
Disallow: /
User-agent: Baiduspider-image
Allow: /wp-content/uploads/*.jpg$
Allow: /wp-content/uploads/*.png$
Disallow: /
User-agent: 360Spider
Allow: /wp-content/uploads/*.jpg$
Allow: /wp-content/uploads/*.png$
Disallow: /
User-agent: YisouSpider
Allow: /wp-content/uploads/*.jpg$
Allow: /wp-content/uploads/*.png$
Disallow: /
User-agent: Sosospider
Disallow: /
User-agent: sogou spider
Allow: /wp-content/uploads/*.jpg$
Allow: /wp-content/uploads/*.png$
Disallow: /
User-agent: YodaoBot
Disallow: /
User-agent: Googlebot
Disallow: /
User-agent: Bingbot
Disallow: /
User-agent: Slurp
Disallow: /
User-agent: MSNBot
Disallow: /
User-agent: googlebot-image
Disallow: /
User-agent: googlebot-mobile
Disallow: /
User-agent: yahoo-blogs/v3.9
Disallow: /
User-agent: psbot
Disallow: /
User-agent: *
Disallow: /