The robots.txt is a text file placed at the root of a website allowing you to give instructions to search engines crawlers indicating the pages that can be indexed. It is also used for SEO purposes.
To add a robots.txt file, access the Toolbar and click on Settings.
And then on ‘SEO’ on the menu bar on the left. The ‘Yes’ field is checked by default.
Changing the content of the robots.txt file can harm your SEO. Do not modify the content of robots.txt unless you are an experienced Webmaster.