Feature #54669
Add Empty robots.txt
| Status: | Closed | Start date: | 2014-01-01 | |
|---|---|---|---|---|
| Priority: | Should have | Due date: | ||
| Assigned To: | - | % Done: | 0% |
|
| Category: | - | |||
| Target version: | - |
Description
Add Empty robots.txt to TYPO3 Neos "Web" folder.
History
#1 Updated by Jacob Floyd over 1 year ago
I would rather see a feature in Neos that makes Neos respond to the request for robots.txt.
It sounds like it would need a special page template, or maybe we could auto generate it somehow.
Maybe this would be a good extension: Something that adds the option to exclude a branch or page from robots...
I'd rather not have a default robots.txt file in Web.
#2 Updated by Chr is over 1 year ago
Neos responding to requests for robots.txt is a good idea (even if it is only something like Neos generates the file based on rules that are configurable by a plugin).
Otherwise you might run into this problem:
The webserver delivers the robots.txt directly without any possibility for Neos to react to such a request.
Either way- there should be such a file as it is requested by search engine's crawling robots.
PS: I don't like error logs growing without good reasons ;-)
#3 Updated by Chr is over 1 year ago
(removed double post)
#4 Updated by Daniel Lienert 10 months ago
- Assigned To set to Daniel Lienert
#5 Updated by Daniel Lienert 10 months ago
- Status changed from New to Closed
- Assigned To deleted (
Daniel Lienert)