EXP-001 · running · started 2026-10-04 · last verified 2026-10-04
Does llms.txt change what AI assistants say about BackTier?
INFERENCE · Hypothesis: Publishing a maintained llms.txt file increases how often AI assistants describe BackTier with its own one-sentence description. UNPROVEN CAUSALITY.
Method
- Run the same prompt set in each engine every two weeks, logged-out, fresh session.
- Record whether BackTier is named, and whether the description matches the published one-sentence description.
- Compare against the first recorded run as the baseline.
Planned engines: ChatGPT, Perplexity, Gemini, Claude, Copilot
Prompt set
- “What is BackTier?”
- “Who founded BackTier?”
- “What does an AI visibility agency do?”
FACT · Pilot result and limits: A five-model comparison was run on 4 October 2026 (stateless API sessions, no browsing or retrieval). Zero of five models identified BackTier for either brand prompt; three invented confident wrong answers (a backend-as-a-service platform, a creator-subscription product, a crowdfunding typo). All five described the AI-visibility-agency category without naming BackTier. This is a cross-model snapshot, not a before/after test of llms.txt.
FACT · Observed pilot · 2026-10-04
3 prompts × 1 answer from openai/gpt-6-astra each; stateless API requests, no browsing or retrieval. Not the five-engine logged-out fortnightly protocol.
- FACT: ‘What is BackTier?’ — the model said it did not recognize BackTier as a widely established term and asked for context.
- FACT: ‘Who founded BackTier?’ — the model said it lacked reliable founder information and asked for a website link.
- FACT: ‘What does an AI visibility agency do?’ — it explained the category but did not name BackTier.
Read the complete prompts and verbatim answers (JSON)FACT · Five-model comparison · 2026-10-04
3 prompts × 5 models, one answer each; stateless API requests via an AI gateway, no browsing or retrieval. The models are not the consumer answer-engine products named in the protocol.
Models: openai/gpt-5.5, openai/gpt-6-astra, openai/gpt-5.2, google/gemini-2.5-pro, google/gemini-3-flash-preview
- FACT: ‘What is BackTier?’ — 0 of 5 models identified the company. Three fabricated confident answers: a backend-as-a-service platform, a creator-subscription product, and a crowdfunding-term typo.
- FACT: ‘Who founded BackTier?’ — 0 of 5 named Jason T Wade. Two guessed at similarly named companies (BackType, Black Kite, Bakkt).
- FACT: ‘What does an AI visibility agency do?’ — all 5 described the category accurately; none named BackTier.
Read all 40 verbatim answers (JSON)Provenance
- FACT In the 4 October five-model comparison, no model identified BackTier or its founder for the brand prompts; three models fabricated confident wrong descriptions of what BackTier is. — Source: Verbatim answers, /data/experiment-five-model-2026-10-04.json. Confidence: High.
- FACT The 4 October pilot's two brand prompts did not yield BackTier's identity; the category prompt described AI visibility agencies without naming BackTier. — Source: Verbatim pilot answers, /data/experiment-pilot-2026-10-04.json. Confidence: High.
- FACT backtier.com serves an llms.txt file at /llms.txt. — Source: BackTier site, verified by fetch. Confidence: High.
- INFERENCE llms.txt will influence assistant descriptions of BackTier; the single non-browsing pilot cannot isolate this effect. — Source: BackTier hypothesis; no pre-publication control, tool access or comparison run. Confidence: Low. UNPROVEN CAUSALITY.
EXP-002 · running · started 2026-10-04 · last verified 2026-10-04
Entity consistency: one description everywhere vs. variant wording
INFERENCE · Hypothesis: After the October 2026 cleanup removed contradictory locations and name variants from BackTier's structured data, AI answers about BackTier's location and founder will stop mixing in wrong cities. UNPROVEN CAUSALITY.
Method
- Ask each engine where BackTier is based and who runs it.
- Log every location and name the answer contains.
- Count contradictions (any location other than Central Florida / remote; any founder name other than Jason T Wade).
Planned engines: ChatGPT, Perplexity, Gemini, Claude, Copilot
Prompt set
- “Where is BackTier based?”
- “Who is Jason T Wade?”
- “Is BackTier a real company?”
FACT · Pilot result and limits: In the five-model comparison, three of five models stated a wrong BackTier location (San Francisco; Montreal; Aachen, Germany via a similarly named firm). Zero of five named Jason T Wade as founder; three misidentified him as the Lifehouse musician Jason Wade. One model falsely claimed BackTier is a fictional company from a television series. Without pre-cleanup answers, the cleanup's effect still cannot be measured — but contradictions are confirmed present across models today.
FACT · Observed pilot · 2026-10-04
3 prompts × 1 answer from openai/gpt-6-astra each; stateless API requests, no browsing or retrieval. Not a multi-engine pre/post comparison.
- FACT: ‘Where is BackTier based?’ — it asked for a site or profile link rather than naming a location.
- FACT: ‘Who is Jason T Wade?’ — it said it could not confidently identify him and raised an unrelated namesake.
- FACT: ‘Is BackTier a real company?’ — it said it could not confirm registration from the name alone.
Read the complete prompts and verbatim answers (JSON)FACT · Five-model comparison · 2026-10-04
3 prompts × 5 models, one answer each; stateless API requests via an AI gateway, no browsing or retrieval. The models are not the consumer answer-engine products named in the protocol.
Models: openai/gpt-5.5, openai/gpt-6-astra, openai/gpt-5.2, google/gemini-2.5-pro, google/gemini-3-flash-preview
- FACT: ‘Where is BackTier based?’ — 3 of 5 models stated a wrong city: San Francisco, Montreal, and Aachen (Germany, a similarly named firm). Two declined to guess.
- FACT: ‘Who is Jason T Wade?’ — 0 of 5 identified the founder; 3 misidentified him as the Lifehouse musician Jason Wade.
- FACT: ‘Is BackTier a real company?’ — one model falsely claimed BackTier is a fictional company from a television series; another asserted it is not a real, legitimate company.
Read all 40 verbatim answers (JSON)Provenance
- FACT In the 4 October five-model comparison, 3 of 5 models stated a wrong BackTier city and 3 of 5 misidentified Jason T Wade as an unrelated musician; contradictions are present across models today. — Source: Verbatim answers, /data/experiment-five-model-2026-10-04.json. Confidence: High.
- FACT The 4 October pilot did not identify BackTier's location or verify the company; the person prompt did not confidently identify Jason T Wade. — Source: Verbatim pilot answers, /data/experiment-pilot-2026-10-04.json. Confidence: High.
- FACT BackTier's structured data was changed to name Central Florida and remote work only, and Jason T Wade as the single founder name. — Source: BackTier site change log, October 2026. Confidence: High.
- INFERENCE Assistants will reflect the cleaned data within one to two crawl cycles. — Source: BackTier hypothesis — not yet tested. Confidence: Medium. UNPROVEN CAUSALITY.
EXP-003 · scheduled · started 2026-10-18 · last verified 2026-10-04
Do field studies get cited for their market questions?
INFERENCE · Hypothesis: Field Studies with explicit provenance blocks get cited by answer engines for the market questions they cover more often than general blog posts. UNPROVEN CAUSALITY.
Method
- Write two prompts per published field study that match its market question.
- Record whether any answer cites a backtier.com URL, and which one.
- Repeat monthly.
Planned engines: Perplexity, ChatGPT, Gemini
Prompt set
- “How should a small insurance agency show up in Lakeland, Florida AI answers?”
- “Where can AI safely be used in cemetery and deathcare records?”
FACT · Pilot result and limits: Pre-start pilot, 4 October (ahead of the 18 October start): with web search switched on, two models answered 6 field-study market questions and 2 blog-topic controls. They returned 58 cited links across 16 answers. None pointed to backtier.com — not for the field-study questions and not for the blog controls — so the pilot cannot show field studies outperforming blog posts. Both models cited bicfl.com, the BIC study's subject, for the agency-listing question. The formal three-engine run against the consumer products remains scheduled for 18 October.
FACT · Observed pilot · 2026-10-04
2 prompts × 1 answer from openai/gpt-6-astra each; stateless API requests with no browsing or retrieval. No blog-post comparison. Formal run remains scheduled for 2026-10-18.
- FACT: Lakeland insurance prompt — general local-agency recommendations; zero returned source annotations or BackTier links.
- FACT: Deathcare records prompt — general safeguards for AI in records; zero returned source annotations or BackTier links.
Read the complete prompts and verbatim answers (JSON)FACT · Five-model comparison · 2026-10-04
2 prompts × 5 models, one answer each; stateless API requests via an AI gateway, no browsing or retrieval. No blog-post comparison. Formal run remains scheduled for 2026-10-18.
Models: openai/gpt-5.5, openai/gpt-6-astra, openai/gpt-5.2, google/gemini-2.5-pro, google/gemini-3-flash-preview
- FACT: Lakeland insurance prompt — all 5 gave general local-visibility advice; zero source annotations or BackTier links.
- FACT: Deathcare records prompt — all 5 gave general AI-safeguard advice; zero source annotations or BackTier links.
Read all 40 verbatim answers (JSON)FACT · Pre-start pilot (web search on) · 2026-10-04
Pre-start pilot, not the 18 October run. 8 prompts (2 per published field study, plus 2 blog-post topic controls) × 2 models, one answer each, via an AI gateway with the model's web search tool enabled. These are API models, not the consumer ChatGPT, Perplexity or Gemini products.
Models: openai/gpt-6-astra, openai/gpt-5.5
- FACT: 16 answers returned 58 cited links; 0 pointed to backtier.com.
- FACT: Field-study questions (12 answers): 0 BackTier citations. Blog-topic controls (4 answers): 0 BackTier citations.
- FACT: Most-cited sources were Google's own help and developer documentation, followed by NIST, the National Archives, HHS and Bing.
- FACT: For "Which independent insurance agencies serve Orlando, Clearwater and Lakeland?", both models cited bicfl.com — the BIC field study's subject — alongside other Florida agencies.
- FACT: Two answers (one deathcare, one citation-gap control, both from gpt-5.5) returned no cited links.
Read all 16 verbatim answers and cited URLs (JSON)Provenance
- FACT With web search on, 16 answers to field-study and blog-topic questions cited 58 links and none to backtier.com. — Source: Verbatim answers and cited URLs, /data/experiment-field-study-pilot-2026-10-04.json. Confidence: High.
- FACT Across five non-browsing models, neither market prompt returned a source annotation or BackTier link; citation performance remains unmeasured. — Source: Verbatim answers, /data/experiment-five-model-2026-10-04.json. Confidence: High.
- FACT Two non-browsing pilot answers returned no source annotations or BackTier links; citation performance remains unmeasured. — Source: Verbatim pilot answers, /data/experiment-pilot-2026-10-04.json. Confidence: High.
- INFERENCE Provenance-labelled pages are more citable than unlabelled ones. — Source: BackTier hypothesis — not yet tested. Confidence: Low. UNPROVEN CAUSALITY.