Gatsby Robots file

Building a robots.txt

Gatsby Robots file

by John Vincent


Posted on September 20, 2019


Building a robot.txt file with Gatsby is straightforward.

Gatsby Robots Plugin

Using the plugin gatsby-plugin-robots-txt, add to gatsby-config.js

{
	resolve: `gatsby-plugin-robots-txt`,
	options: {
		host: siteUrl,
		sitemap: `${baseUrl}sitemap.xml`,
		policy: [{ userAgent: `*`, allow: `/` }],
	},
},

The robots.txt file will be build only for production builds.