Our crawler
If you publish a blog or a newsletter, you may have seen Everett in your server logs. This page explains who we are, why we fetched your pages, what we do with them, and how to tell us to stop. The short version: we fetch your writing because one of our readers follows you, we never republish it, and a line in your robots.txt keeps us out.
Everett is a reading and thinking tool run by Seattle Thinking Tools LLC. Readers save what they read and follow the writers they subscribe to, and Everett helps them see how it all connects. When a reader follows your publication, Everett reads your feed on their behalf, the way a feed reader always has.
Two agents, two jobs
The feed reader. Identifies itself as Everett/1.0 (+https://everett.tools/bot; N followers), where N is how many Everett readers follow you. It fetches your RSS or Atom feed a few times a day, honours ETag and Last-Modified so unchanged feeds cost you almost nothing, and fetches nothing else. Like any subscriber's feed reader, it is not governed by robots.txt: it asks only for the feed you publish for subscribers to read.
The crawler. Identifies itself as EverettBot/1.0 (+https://everett.tools/bot). When a feed carries only an excerpt, the crawler fetches the linked post so a follower can search it and ask questions of it. It obeys robots.txt, including Crawl-delay, makes one request at a time to any site, and is capped at a fixed number of pages a day across every site we follow. It never signs in, never presents a reader's credentials, and never goes past a paywall: if your post is for paying subscribers, we see exactly what a stranger sees and index only that.
Both agents run from our servers in the United States. Neither executes advertising scripts, submits forms, or follows links beyond the posts your feed names.
What we do with your writing
- We index it for your followers. A reader who follows you can find your posts in search and ask questions that draw on them, with the answer pointing back to you. Nobody who does not follow you sees a word of it.
- We send readers to you. A follower who opens one of your posts in Everett is shown a summary and a link to read it on your site. The full text stays yours; we do not republish it, and the pages that list your posts are kept out of search engines.
- A reader may save a copy. When a follower saves a post to their own library, Everett keeps a copy for them to highlight and annotate, the same as if they had saved the page from their browser. That copy is private to them.
- We do not train on it. We do not train AI models on your writing, and we do not use it to improve models for anyone else. Our privacy policy describes the AI providers we use and their zero-retention terms.
How to opt out
Keep the crawler out. Add two lines to your robots.txt and EverettBot will stop fetching your posts within a day. Your feed will still be read, and followers will see only what the feed itself carries.
User-agent: EverettBot Disallow: /
Stop the feed reader too. Email hello@everett.tools with your site's address and we will remove your publication from Everett. Readers who follow you will stop receiving new posts.
What opting out does not do. A reader who visits your page in their own browser can still save it to their library, exactly as they can print it or bookmark it. Saving is the reader's act, not ours, and it is not something a crawler policy can reach.
Is this your publication?
Writers can claim their publication on Everett to see who follows them and to edit how it appears. If you would like to, or if you have any question about traffic you have seen from us, email hello@everett.tools. A person reads every message.