Use Robots.txt Generator tool to generate robots.txt file that helps ensure Google and other search engines are crawling and indexing your site properly.
Search Engines are using robots (or so-called User-Agents) to crawl your pages. The robots.txt file is a text file that defines which parts of a domain can be crawled by a robot. In addition, the robots.txt file can include a link to the XML-sitemap.
If you are a Getshop Today user, you don't have to worry about this, as Getshop Today generated robots.txt file automatically.