Gatsby Robots file
by John Vincent
Posted on September 20, 2019
Building a robot.txt
file with Gatsby is straightforward.
Gatsby Robots Plugin
Using the plugin gatsby-plugin-robots-txt
, add to gatsby-config.js
{
resolve: `gatsby-plugin-robots-txt`,
options: {
host: siteUrl,
sitemap: `${baseUrl}sitemap.xml`,
policy: [{ userAgent: `*`, allow: `/` }],
},
},
The robots.txt
file will be build only for production builds.