Disallow robots from indexing /interact/ (#10666) · c4f2433300 - mastodon - 碧茶

Disallow robots from indexing /interact/ (#10666)

This does not provide any new information and may just triple the number
of crawled pages

pull/4/head

ThibG 5 years ago

committed by

Eugen Rochko

parent

0db269f3dc

commit

c4f2433300

1 changed files with 1 additions and 0 deletions

Split View

							
								+ 1
								
								- 0
							
public/robots.txtView File
								
					@ -2,3 +2,4 @@
				
					User-agent: *
				
					Disallow: /media_proxy/
				
					Disallow: /interact/