Gatsby Robots file

Building a robots.txt

Gatsby Robots file

by John Vincent

Posted on September 20, 2019

Building a robot.txt file with Gatsby is straightforward.

Gatsby Robots Plugin

Using the plugin gatsby-plugin-robots-txt, add to gatsby-config.js

	resolve: `gatsby-plugin-robots-txt`,
	options: {
		host: siteUrl,
		sitemap: `${baseUrl}sitemap.xml`,
		policy: [{ userAgent: `*`, allow: `/` }],

The robots.txt file will be build only for production builds.