Methodologies: how the numbers on this site are made

This page explains, in plain English, every step between a comment posted on regulations.gov and a figure on this site: what each step does, why it is done that way, and what the words mean. Every step is a rule written down in code and run in a fixed order, so running the whole chain again over the same comments gives the same tables. The one place a language model reads is marked; what it does there is copy passages, and code grades what it copied.

The docket is FS-2025-0001, the Forest Service's proposal to rescind the 2001 Roadless Area Conservation Rule. The comment period for the proposed rule ran from 2026-08-20 to 2026-10-06. Figures on this page are as of 2026-10-11; the pages of the site carry the live numbers.

The three questions

1. How many, and who? How many comments and signatures reached the agency, how many are copies of one letter, how many are one person's own words, and which organisations ran the letters.
2. What do they say? Whether each comment supports or opposes the rescission, and what subjects it raises.
3. What does the agency owe them? Which comments the agency must answer, how much analytical work each one does, and how hard each is to set aside.

Each question has its own chain of steps, and they share the first few. Nothing in any chain depends on a commenter's position, the length of a comment, or how well it is written.

The pipeline

Open the diagram full size

The model reads at one place, marked with the double border. Everything else is a lookup, a pattern, or arithmetic.

Step by step

1. Collecting

Every comment posted to the docket is collected with its text, the date the agency received it, the date it was posted, and the list of its attached files. Two of the agency's 390,064 posted comments have no text and are not held; the corpus is 390,062, none blank. Every share on this site is a share of those comments, never of a total that includes what bundles enclose (see step 8).

2. Cleaning

Comments arrive wrapped in the submission form's formatting codes and encoded punctuation. One set of rules strips them, so the same sentence reads the same wherever it is compared. Numbers are kept as words: "58 million acres" and "18.2 million acres" are different phrases, because a letter that argues from one figure is not a copy of a letter that argues from another.

3. Finding copies

Most comments on a rulemaking are not written from scratch. An organisation drafts a letter, puts it on a web page, and thousands of people send it, some word for word, some with a sentence of their own. The letter is one argument and should be read once; the number of people who sent it is a fact of its own; and what a signer added is theirs.

Copies are found by three kinds of evidence, strongest first, then grouped into families.

• Exact copies. The same text, character for character, found by a hash.
• Same body. The same text once the last fifty characters are set aside, which is what "the same letter with a different name at the bottom" looks like.
• Near copies. Each comment is its set of phrases, every run of six, seven and eight consecutive words. Comments of about the same length are compared; each joins the group leader it overlaps most, at 85 percent or more of its phrases, or becomes a leader itself. Every member of a group resembles its own leader, so groups cannot chain from one hybrid comment to the next.
• Families. The near groups, the body groups and the identical groups of three or more become cores. A family's text is the phrases that at least three of its submissions share. Phrases found almost only inside one family (80 percent of their appearances in the whole docket) are its distinctive phrases, and they bring in the copies the length comparison could not see: a copy with a long personal story in front of it joins the family whose distinctive phrases cover at least half of its own. A second pass asks of the comments still in no family whether a shorter one is contained in a longer one, which is how a letter pasted inside a personal comment is found. A family needs three submissions; with ten or more it is called a campaign.
• One group per comment. Every comment ends in exactly one group, named on the site by its kind: exact copies; the same body; a small family (3 to 9 submissions of one letter); a campaign (10 or more); or a unique letter, a comment no one else sent. The group's representative is the earliest received. A submission is one posted comment; a unique comment, the site's headline measure, is one group, so a campaign letter counts once however many sent it. Where the site counts voices it counts unique comments.

Why this way: the agency's own rule lets it group comments that raise the same point and answer once (7 CFR 1b.7(f)(1)(ii)), and the Forest Service's content-analysis practice has always treated a form letter as one response, analysing one example of each form and, for a form letter with added text, only the added text. The families are that sort step done by machine. The alternative used elsewhere, calling a comment unique when no other is at least 80 percent the same, counts a letter with a personal paragraph in front as unique; here it is a member of its family with own text.

4. Telling the letter from the signer's words

For every family member, sentence by sentence: a sentence that the family's letters carry is the letter's; a sentence that the family's phrases cover to three quarters is the letter's, a copied sentence with a word changed; everything else is the signer's own. A name and a city in the closing line are the signer's, but there is nothing in them to score, so own text of under ten words that sits only in the first or last line is recorded and not scored.

There is no master list of "boilerplate". A figure from the agency's own report is the campaign's inside a copy of a letter that quotes it, and a retired forester's own inside a comment that is nobody's copy.

5. Position

Every comment of thirty characters or more is read by a classifier and labelled as supporting the rescission, opposing it, or neutral, with a strength. The unique-letter comments are classified like the rest. Where a share is shown it says what it is over.

6. Substance: does the agency owe an answer, and how much work does the comment do?

The agency does not answer every comment. It answers the substantive ones, grouped by concern, and counts the rest. Three questions are asked of each comment, in order: does the agency owe it an answer (the gate); how much analytical work does it do (the strength); and if it is owed an answer, how hard is it to set aside (the answerability level, step 7).

The floor. A comment is set aside without a model call only when all four of these are absent: a named thing of a substantive kind (a place, an organisation, a date, a quantity, a law, a percentage); a citation or legal reference; deficiency language ("failed to consider", "does not address"); and a first-person observation ("I have seen", "we hunted"). One hit keeps it alive. The first-person test is there because local knowledge is written informally, and a method that rewarded polish would throw it away.

The screen. Over the survivor's own words, code looks for anchors: a confirmed legal citation, a confirmed citation of an outside source, a citation into the agency's document, or deficiency language aimed at this action. Anchored comments go to the model.

The reading. For each comment worth a call, the model receives the text and nothing else: a unique letter whole, a campaign's letter once, and each signer's own words as a separate item. It returns no scores. It copies, word for word, the passage that carries each of five things, or says there is none: the deficiency named (GAP), the source offered (EVID), the request made (ASK), the alternative proposed (ALT), and the commenter's connection to the place (STAND). It answers yes or no to four questions: is the source applied to this proposal; does the request name an action and its subject; is the alternative one the agency has not already analysed; is the connection to this place. It gives a level for the deficiency, which moves the comment makes (questions accuracy, questions adequacy, offers new information, names missing analysis) and what they aim at (impacts, alternatives), and a sentence of rationale. The model's reasoning mode is off and its temperature is zero. A reply that does not fit the schema is an error, not a score.

Why passages and not numbers: a model asked for a number forms an impression and fits the number to it. A model asked to copy the words it relied on has to find them, and a reader can check them.

The grading. Code turns the passages into eight dimensions, each 0 to 3.

LAW is mechanical because, when the model was asked, only 3 to 22 percent of its LAW scores had a detectable citation behind them. A legal citation is a small closed vocabulary; a scan finds it, a model invents it. Naming the Roadless Rule itself is not a legal citation: it is the subject of the docket and appears in 48,611 of the 56,943 directly scored comments.

The arithmetic. Strength is the sum of the eight, out of 24; because LAW is 0 or 2 the highest reachable is 23. The gate opens when any one of GAP, EVID, ALT or LAW is 2 or more; the site calls a comment with the gate open owed an answer. Bands: minimal 0 to 5, marginal 6 to 11, developed 12 to 17, strong 18 to 24.

Campaign members. A letter is scored once, on its family's exemplar, and every signer inherits that score. A signer who wrote something of their own gets one more call on those words alone, and takes the higher of the two on each dimension.

Why these four open the gate and the other four do not: the regulation's definition of "substantive" (7 CFR 1b.11(a)(53)) names four things a comment can inform, any one of which suffices: the effects of the action, whether they are significant, which alternatives to consider, and compliance with the law. A deficiency in the analysis (GAP) is the first two; new information (EVID) the first; an alternative (ALT) the third; a legal citation (LAW) the fourth. Naming a place, having a connection to it, citing the document and making a request are the form of a specific comment under the Forest Service's own tests (36 CFR 218.2 and 219.62), not its substance.

7. Answerability: how hard is the comment to set aside?

Strength says how much a comment contains. It does not say whether the agency can dismiss it, and the two come apart: a three-sentence comment that names a statute must be answered; a long, sourced, place-specific comment with no causal or legal hook can be set aside with any of the standard replies.

The regulation lists the replies that let the agency take no action on a comment (7 CFR 1b.7(f)(2)(vi)): the comment is outside the scope; there is no cause-and-effect relationship; the commenter misread the document; the suggestion is unlawful, infeasible, or does not meet the purpose and need; and, separately, the analysis already addresses it, or the official certifies it was comparatively not substantive (7 CFR 1b.7(j)). From the stored dimensions alone, code decides which of nine such dispositions the comment defeats, and a level is read off the set:

A specific place or a cited section (PLACE or DOC at 2 or more) defeats "outside the scope" and "misread the document" without changing the level. Four of the nine dispositions have no text-only test and stay open; they need the agency's final document. Only comments scored on their own text are rated; a signer carrying the letter's score takes the letter's level.

Exhibits. For the top tenth of strength within each position, one more model call asks for the passages behind each dimension; each is checked as a substring of the comment and dropped if it is not there. These are the exhibits the site shows, and the two sides are each represented by their best.

8. Attachments, and what a comment counts as

Every attached file on the docket is fetched and its text extracted. Each file is one of four kinds: the commenter's own letter; supporting material (papers, appendices, exhibits); a photograph; or enclosed submissions, other people's comments or signatures that an organisation gathered and filed under one comment number. A comment whose files enclose other people's submissions is a bundle, and it counts as what it encloses rather than as one.

How a bundle is counted:

1. Measured. Each file's layout is recognised (a form export with a name per person, one letter per page, letters run together each with a salutation, petition tables, signer lists with an e-mail per row, numbered name lists) and the records are counted by their markers.
2. Stated. The number the cover comment states for what it encloses ("Attached are 47,545 comments"), or, when the cover states none, the number the file states about itself on its first page ("This document compiles 1,353 individual comments"). Not a bare year, not a number over 99,999, and not the organisation's own membership ("on behalf of our 500 members").
3. Which stands. The stated number, when it is a hundred or more, or any size on a cover that opens "Attached are", and the markers read more than ten percent off it; otherwise the measured count.
4. Composition. Each bundle is split into records. A record whose text appears once in the docket's bundles is an individual letter; one whose text appears twice or more is a form letter; one with no text is a bare signature; the remainder of the count is unread.
5. Organisation. The bundles of a hundred or more were reviewed by hand; every other bundle is named by the first known phrase its cover carries ("submitted through SaveRoadlessForests.com", "on behalf of The Conservation Alliance"). A bundle whose cover and files name no organisation stays unnamed; a prefix in a file name is not a name.

The tally is the comments outside bundles plus what the bundles enclose. It counts entries, not people: a person who signed two petitions is two entries, and a person who both commented and signed cannot be matched, because a posted comment carries no name. No share on this site is taken over the tally.

As of 2026-10-11: 492 bundles enclosing 755,516, of which 521,286 bare signatures, 76,517 form letters, 42,288 individual letters and 116,354 unread; 393 bundles with an organisation named; a tally of 1,145,086.

9. Who ran each campaign

Nothing in the docket names a campaign's sponsor: the organisation field is empty on every comment and the letters do not say who wrote them. A family is named on the first of four bases that holds, each recorded with its evidence:

1. letter: the letter names the organisation as the writer's own ("as a member of the League of Conservation Voters"); the evidence is the quote;
2. action page: the organisation's action page on the web carries the letter, read by hand; the evidence is the page's address;
3. cover: a named bundle's cover comment is itself a member of the family, and every named cover in the family is the same organisation;
4. enclosed: the organisation's bundles enclose at least five records whose sentences are the family's letter, and hold three quarters of all such records in named bundles.

A family with none has no organisation known, and the site says so rather than guess.

10. Entities, places, subjects

Two passes over every comment tag the things it names: strict, a name beside its cue word ("Pisgah National Forest", "36 CFR 294"); loose, the name alone. Legal citations are matched to a seed list of statutes, regulations, Federal Register pages and executive orders and grouped into families (a CFR title and part, a U.S. Code title and section). Forests, roadless areas, species and ZIP codes are tagged the same way. Subject shares on the site are keyword lists over the comment text, with the list stated beside the figure.

What is deliberately not done

• Position does not enter any score. Support and opposition meet the same rubric. The regulation does not distinguish them, and the Forest Service's own method says comments are not votes.
• Length, polish, grammar and emotion do not count. An informal comment that names specifics outscores a polished one that names none.
• Whether the commenter is right is not judged. A wrong citation to a named survey is still a named source; correctness is decided in the agency's response.
• A credential transfers nothing. "As a biologist, this is concerning" raises local knowledge and nothing else.
• Generalities score nothing. A statement that would read the same against any similar proposal in any decade ("roads cause erosion") scores 0 on gap, evidence and document, however true. A figure taken from this agency's own record and set against the proposal is specific by definition.
• No one is named from a file prefix, and no count is taken from a number that is not stated as a count.
• No share is taken over a sum that mixes comments with what bundles enclose.

Determinism and versions

The copy finding, the families, the split, the accounting and the naming of organisations are code over stored text with the constants named on this page. The entity passes are dictionary matches. The classifier and the scorer run under a named instrument version and a named model, stored on every row, and nothing mixes versions; the scorer's own repeatability was measured at 96 of 100 comments scoring identically on two runs. Every figure on the site is a query over the tables these steps write, and the query is the figure's definition.

Known limits, stated plainly

• Letters inside attachments. An organisation that posted a short cover and attached its 12-page letter is classified on the cover; the letter's substance is not yet scored. 1,698 comments are of this kind. A merged reading of cover and attachment together is being run.
• People, not entries. The tally counts entries. The bundle records carry a readable name or e-mail for 394,537 of 640,100, 327,054 of them distinct, so about one person in six among those signed twice or more; the posted comments carry no name, so a person who commented and also signed is two entries.
• The 2025 scoping period is not in the corpus; everything here is the 2026 proposed rule.
• Cover notes. The family of "See attached file(s)" covers, 1,233 comments, counts as a campaign by size although it is not one.
• The gate has been validated for consistency, not against what the agency answered. The first real calibration is this docket's own response appendix, when it is published.

Glossary

A-level. The answerability level, A0 to A3, shown on the site as strong (A1), moderate (A2), weak (A3) and none (A0). A1 must be answered on the merits; A2 defeats the "no cause and effect" reply; A3 is substantive with every reply still available; A0 is counted, not answered.

Action page. An organisation's web page that asks supporters to send a letter, and carries the letter's text. One of the four bases for naming a campaign's sponsor.

Answerability. How hard a comment is to set aside with one of the agency's listed "no action needed" replies. Rated from the stored dimensions; see A-level.

Attachment. A file uploaded with a comment. One of four kinds: the commenter's own letter, supporting material, a photograph, or enclosed submissions.

Attribution. The split of a family member's text into the letter's sentences and the signer's own, sentence by sentence.

Band. The strength in four steps: minimal 0 to 5, marginal 6 to 11, developed 12 to 17, strong 18 and up.

Basis (of a bundle's count). How the count was reached: measured from the files' markers, declared from the cover's or the file's stated number, or one for a comment that encloses nothing.

Basis (of a sponsor). Which of the four kinds of evidence named the campaign's organisation: letter, action page, cover or enclosed.

Body. A comment's text with its last fifty characters set aside, so that the same letter under different signatures hashes the same.

Bundle. A posted comment whose attached files enclose other people's comments or signatures. It counts as what it encloses.

Campaign. A text family of ten or more comments, copies counted. The label changes nothing about scoring.

Canonical comment. See representative.

Carrier. The comment whose score stands for its group: the family's exemplar for a family or campaign, the canonical for the rest.

Classifier. The reading that labels each comment as supporting the rescission, opposing it, or neutral.

Comment. One posted entry on the docket, with its text, dates and attachments.

Comment period. The window in which the agency accepted comments on the proposed rule: 2026-08-20 to 2026-10-06.

Core. The comments a family is built from: exact copies, the same body, near copies, or a letter carried inside longer comments.

Counts as. What a comment stands for in the tally: 1, or the number its bundle encloses.

Cover comment. The comment text posted with a bundle, which often states what the files enclose and who sent them.

Delta. A family member's own words, scored as a separate item; the member takes the higher of the letter's score and the delta on each dimension.

Dimension. One of the eight things a comment is scored on, each 0 to 3: GAP, EVID, ASK, ALT, STAND, PLACE, DOC, LAW.

Disposition. One of the nine replies by which the agency can take no action on a comment. Each is decided for every rated comment as defeated or open.

Distinctive phrase. A phrase of a family's text found almost only inside that family, 80 percent of its appearances in the whole docket. Distinctive phrases attach the copies the length comparison could not see.

Docket. The agency's file for one rulemaking on regulations.gov. This one is FS-2025-0001.

Enclosed submissions. An attachment that holds other people's comments or signatures.

Entry. One comment or one signature counted in the tally. A person can be several entries.

Exact copy. A group kind: a comment whose cleaned text is character for character the same as another's.

Exhibits. The passages the site shows for the top tenth of comments by strength on each side, each a verbatim passage checked against the comment.

Exemplar. The most-submitted text in a family; the letter is scored on it.

Explicit cover. A cover comment that opens by stating what is attached ("Attached are 38 comments"); its number stands at any size.

Family. One letter and everyone who sent it, at three or more submissions, including near copies and copies with own text added. Also called a text family.

Family text. The phrases that at least three of a family's submissions share.

Floor. The test that sets aside a comment that cannot be substantive, with no model call: no named thing, no citation, no deficiency language, no first-person observation.

Form letter (inside a bundle). A record whose text appears two or more times across the docket's bundles.

Individual letter (inside a bundle). A record whose text appears once across the docket's bundles.

Gate. Whether the agency owes the comment an answer: open when any of GAP, EVID, ALT or LAW is 2 or more. The site says owed an answer.

Group. The one set every comment belongs to, of one of five kinds: exact copy, same body, small family, campaign, unique letter. One group is one unique comment.

Inherited. A score copied from the letter's exemplar to a family member with no own text.

Instrument. A numbered version of the scoring rubric and prompt. Every scored row records the instrument and the model it was scored under; the site reads one instrument.

Leader. The comment a near-copy group is built around; every member overlaps its own leader by 85 percent or more.

Legal authority. A statute, regulation, Federal Register page or executive order cited by its citation, matched against a seed list.

Letterhead comment. A comment whose cover is short and whose argument is in an attached letter, typically an organisation's formal submission.

Limb. One of the four things the regulation says a substantive comment can inform: effects, their significance, alternatives, compliance with law.

Local knowledge. The STAND dimension: the commenter's stated first-hand connection to the place, by role, duration or frequency.

Measured count. A bundle's count read from the markers in its files.

Merged text. A comment's own text followed by its attachments' text, read together.

Move, target. What a comment does to the analysis (questions its accuracy, questions its adequacy, offers new information, names missing analysis) and what it aims at (impacts, significance, alternatives). Used in the answerability test for a stated mechanism.

Near copy. A comment whose phrases overlap another's by 85 percent or more, compared among comments of about the same length.

Owed an answer. The site's name for a comment whose gate is open.

Own text. The sentences of a family member that are not the letter's.

Phrase. A run of six, seven or eight consecutive words, with the characters of the comment it covers. Also called an n-gram.

Place list. The 87,185 names of forests, districts, roadless areas, trails, streams and other features that the PLACE dimension is read against.

Posted date, received date. When the agency published the comment, and when it received it. The site orders by received date.

Rationale. The model's one or two sentences saying what it found; stored with every score.

Record. One enclosed comment or signature inside a bundle's files.

Representative. The earliest received comment in a group; the one that stands for the group on the site. Also called the canonical comment.

Same body. A group kind: comments with the same text once the last fifty characters are set aside, the same letter under different signatures.

Small family. A group kind: one letter sent by 3 to 9 people, copied or lightly reworded.

Signature. A record with no text of the signer's own: a name on a petition or a sign-on letter.

Span. A passage the model copied word for word from the comment as the evidence for a dimension; code grades the span.

Sponsor. The organisation behind a campaign, named on one of the four bases.

Stance. The classifier's label: supports the rescission, opposes it, or neutral.

Stated count. The number a bundle's cover or file states for what it encloses.

Submission. One posted comment on the docket, whether or not it is a copy of another.

Strength. The sum of the eight dimensions, out of 24, at most 23 in practice.

Substantive. Information that meaningfully informs the effects of the action, their significance, the alternatives, or compliance with the law (7 CFR 1b.11(a)(53)). The gate is this definition applied to one comment.

Supporting material. An attachment that is a paper, appendix or exhibit rather than a letter or enclosed submissions.

Tally. The comments outside bundles plus what the bundles enclose. Entries, not people.

Unique comment. One comment group, whatever its size: the site's headline measure, under which a campaign letter counts once. The groups' representatives are the unique comments.

Unique letter. A group kind: a letter no one else sent, a group of one.

Unread. The part of a bundle's count that the record splitter could not assign to signature, form letter or individual letter: columnar tables and lists it reads as counts only.

Voice. A unique comment, where the site counts who said something rather than how many times it was sent.

Sources

• 7 CFR 1b.7 and 1b.11, the Department of Agriculture's NEPA procedures, as in force in 2026: the definition of substantive (1b.11(a)(53)), of an issue (23), of a reasonable alternative (42); the handling of comments (1b.7(f)) with the grouping rule (f)(1)(ii), the linkage condition for new science (f)(2)(iv) and the "no action needed" replies (f)(2)(vi); the page-limit certification (1b.7(j)).
• 36 CFR 218.2 and 219.62, the Forest Service's own tests for a specific comment, borrowed for PLACE and DOC.
• 5 U.S.C. 553(c), the Administrative Procedure Act's duty to consider comments on a rule.
• Forest Service content-analysis practice: the 2008 Summary of Public Comment appendices and the Idaho Roadless summary, where a form letter is one response and a form with added text is analysed on the added text only; the former NEPA Handbook, FSH 1909.15.
• The technical pages behind each step, for readers who want the code: text families, the analysis data model, the v3 scoring instrument, the answerability rating, the attachment accounting reviews, and the campaign organisations review, in the project documentation.