Gatsby Robots file

Building a robots.txt

Gatsby Robots file

by John Vincent

Posted on September 20, 2019

Building a robot.txt file with Gatsby is straightforward.

Gatsby Robots Plugin

Using the plugin gatsby-plugin-robots-txt, add to gatsby-config.js

    resolve: `gatsby-plugin-robots-txt`,
    options: {
        host: siteUrl,
        sitemap: `${baseUrl}sitemap.xml`,
        policy: [{ userAgent: `*`, allow: `/` }],

The robots.txt file will be build only for production builds.