SEO - Robots and Sitemap.xml

robots.txt

Easiest improvement for robots is add file robots.txt and following content. It means allow any robots engine to crawl your site, and don't index your /keystone and /admin root folder. One of the reason you do that is then these engine won't check your page so first your page speed is better.

User-agent: *
Disallow: /keystone
Disallow: /admin

More information about robots, see Create robots.txt file or robots database

sitemap.xml

sitemap.xml is another important thing to improve google crawl your website, I'm using following to auto generate sitemap. sitemap generator