Gatsby Robots file

Building a robots.txt

Gatsby Robots file

by John Vincent


Posted on September 20, 2019


Building a robot.txt file with Gatsby is straightforward.

Gatsby Robots Plugin

Using the plugin gatsby-plugin-robots-txt, add to gatsby-config.js

{
    resolve: `gatsby-plugin-robots-txt`,
    options: {
        host: siteUrl,
        sitemap: `${baseUrl}sitemap.xml`,
        policy: [{ userAgent: `*`, allow: `/` }],
    },
},

The robots.txt file will be build only for production builds.