Chapter 7. Methodology, Definitions, and Data: How the AI Shopping Visibility Study 2026 Was Built

AEOsome Research. Vijaya Kumar Channalli. Chapter 7 of the AI Shopping Visibility Study 2026, version 2026-10. The hub carries the abstract, the ten headline findings, and the other chapters. This chapter is the reference for every definition, rule, and limit the other six rely on.

Abstract

The study re-analyzes two collections of AI shopping answers under one code base and one set of definitions. Study 1 holds 8,520 ChatGPT answers to women's-dress questions collected on February 28 and March 1, 2026. Study 2 holds 27,255 answers from ChatGPT, Google AI Mode, Microsoft Copilot, and Perplexity to 5,584 questions across seven categories, collected May 5 to 14, 2026, reduced to 22,335 by keeping one answer per question and engine. This chapter documents the collections, the de-duplication rule and its reconciliation against the original summaries, the product-card definition applied to each engine's response format, the brand resolver and its resolution rates, the domain-type rule set that replaced the engine's own classifier, the 23 tagged query signals and 14 computed query features, the 19 answer-shape features, the statistical methods and the three confidence labels, the comparability of the two studies, ten known limits, and twelve gaps the data cannot close. It ends with the data availability, license, and citation terms.

1. The Two Collections

Table 1. Study design.

Questions in both studies were generated to cover the nine stages of the eCommerce shopping-intent framework in several phrasing styles. Study 1 holds 900 to 1,260 questions per stage except Post Purchase (427). Study 2 holds 2,000 to 2,800 answers per stage on the de-duplicated basis; Home and Toys have no Budget Framing or Post Purchase questions and Toys has no Validation questions, so five of the 63 category-and-stage cells are empty.

Table 2. Study 2 subcategories. Answers on the de-duplicated basis, four engines pooled.

Served models. Both collections requested one ChatGPT model and were served more than one. In Study 1 the two served models have the same carousel rate (59.7% and 59.1%), so the mixture does not affect its results. In Study 2 the three served models have card rates of 3.7%, 65.3%, and 72.2%, and routing was by collection day rather than by question (every stage and style received the same mix), so the served model is treated as a variable throughout and every ChatGPT figure in the study is reported by model. gpt-5-3 answered only on May 5 (Electronics), which confounds it with that category; the contrast between gpt-5-3-mini and gpt-5-5, which share collection days and categories, is clean.

2. De-duplication

Study 2's collection re-submitted some questions, almost all in Electronics, and stored every attempt. The analysis keeps one answer per (engine, question): the last attempt. This is the rule the original collection summary used, and the re-analysis reproduces that summary exactly.

Table 3. De-duplication reconciliation.

The removed retries are not discarded. They are the basis of the stability analysis in chapter 5, which compares the first and last attempt of each retry group. Every other Study 2 rate in the study is on the 22,335-answer basis. Rates on the 27,255 collected answers are never published; where one appeared in an earlier article, chapter 6 corrects it.

Study 1 has no retries at the question level (8,520 distinct question identifiers), but 679 of its questions repeat another question's text, which gives 1,286 pairs of identical questions and a second stability test.

3. The Product-Card Definition

An answer "shows a product card" when the engine's structured response contains at least one product object. A product mentioned in the text does not count. The rule is the one the original Study 2 collection used, applied identically to Study 1.

Table 4. Product surfaces by engine.

Each product object yields one slot with, where the engine provides them, title, price, rating, review count, merchant, and link. Google AI Mode's links are search redirects, so its merchant is known only from the store name and its host domain is unknown. "Shelf", "carousel", "card strip", and "product card" are used interchangeably in the chapters; "carousel" is Study 1's name for ChatGPT's eight-slot strip.

4. Brand Resolution

The engines' own brand fields are unreliable: Study 1's is the first word of the product title, so "Old Navy" becomes "Old"; Copilot carries a brand field on about one slot in eight, ChatGPT's offer data on one in ten, and Google AI Mode and Perplexity carry none. One resolver was written for both studies and applied to all 96,503 slots (56,000 in Study 2, 40,544 in Study 1, including retries). It tries five methods in order and records which one fired and a confidence level.

Table 5. Brand resolution methods, share of slots.

Sub-brands roll up to the parent (Samsung Galaxy to Samsung, Google Nest to Google). The lexicon of 1,200 brands and its matching counts are a published table. Perplexity's low resolution reflects its merchant network of small stores whose titles do not carry a recognizable brand. Per-brand counts in the chapters use all resolved slots; concentration measures (HHI, Gini, top-k shares) exclude the first-word fallback.

5. Source Typing

Each cited source has a URL, a title, a position in the engine's list, and, for ChatGPT only, a publication date. The registrable domain of every cited source was typed with one rule set, replacing the engine's own classifier, which had typed 14,898 of Study 1's 15,034 domains as Retail. The 300 most-cited domains were reviewed by hand and the rules adjusted until they agreed.

Table 6. Domain types.

Table 7. Citations by source type and engine. Share of cited sources; Study 2 on the de-duplicated basis.

The same rule set types the host domain of every product slot, which is how the chapters separate brand-owned, retailer, and marketplace hosting of cards. Study 1's engine taxonomy is kept in the research folder as the record of the discrepancy and is not used.

6. Query Features

Two feature sets describe the questions.

Tagged signals. The question generator tagged 23 signals on each question: brand mentioned, price mentioned and a price cap, deal-seeking, urgency, explicit purchase, use case, comparison, problem, conversational tone, attribute count, ambiguity score, product specificity, and others. All 8,520 Study 1 questions carry them; in Study 2 they are present for six categories and absent for Electronics (894 questions, 3,599 answers), so every signal analysis in Study 2 uses the 18,736 tagged answers. In Study 1 four signals are constant (typo, local intent, model name, SKU all zero; specificity always "Generic") and are not analyzed.

Computed features. To cover every question in both studies with one definition, 14 features were computed from the question text by the study's own code: token count, character count, a question mark, a wh-word opening, and the presence of price, compare, deal, urgency, buy, "best", problem, and use-case words, all-lowercase phrasing, missing end punctuation, and dollar amounts. These are the "wording features" in chapter 4.

Table 8. Feature coverage.

7. Answer-Shape Features

Nineteen features were computed from each answer's text: characters, words, sentences, bullets, headers, table rows, bold spans, links, hedging phrases ("it depends", "may vary", "generally", and similar) per 100 words, whether the answer asks a clarifying question, whether it ends with a question, whether it names a single recommendation, whether it names a retailer, dollar mentions and the minimum and maximum price in the text, and question-mark count. Markdown structure (bullets, headers, tables) is available for Study 2 only; Study 1 stored plain text. ChatGPT also returns a list of named product entities, used in the text-against-cards comparison in chapter 5.

8. Statistics and Confidence Labels

Rates and intervals. Every rate is a share of answers (or slots) in a defined cell. Headline rates carry a 95% confidence interval: Wilson intervals for simple proportions and bootstrap intervals (1,000 resamples) for the published platform, stage, and category rates. The platform rates have half-widths near 1.2 points, Study 2 stage rates 1.6 to 2.0 points, and Study 1 stage rates 1.6 to 3.1 points; the intervals are listed in the hub's Appendix A.

Multiple comparisons. The 36 pairwise stage differences in Study 2 were tested with a Bonferroni correction at a family-wise 0.05; 31 survive. The five that do not are adjacent stages within four points and the chapters never call them differences. Every platform and category difference survives.

Models. Chapter 4 fits a logistic regression (odds ratios with intervals) and a gradient-boosted classifier (predictive power, permutation importance) for three targets, evaluated with five-fold cross-validation. Study 2 folds are grouped by question so the same question never appears on both sides of a split; rank-one models are grouped by carousel. Permutation importance is the out-of-sample AUC lost when a feature's values are shuffled, averaged over five repeats.

Overlap and concentration. Jaccard overlap between two sets is the size of their intersection over the size of their union. HHI is the sum of squared brand shares of slots (0 for many equal brands, 1 for one brand); Gini measures inequality of brand slot counts; top-k share is the share of slots held by the k largest brands.

Table 9. Confidence labels used in every chapter.

Cells below 60 answers are never called Confirmed. Where a difference lies inside its interval the text says so.

9. What Study 1 and Study 2 Can Be Compared On

The two studies share definitions and code, so these are comparable: whether an answer carries a card; products per answer; the brand, merchant, and host mix of the cards; the source-type mix of citations; answer shape; the price, rating, and review profile of surfaced products; the 14 computed wording features; and stage and style effects within each study.

The level of any rate is not comparable. Study 1 is one category in February in one collection run with two served models; Study 2's ChatGPT is seven other categories in May with three served models, 60% of its answers from a small model that rarely shows cards. Study 1's 59.5% carousel rate against Study 2 ChatGPT's 28.5% is a difference between two snapshots, not a trend. Restricted to its two full models, Study 2 ChatGPT's card rate is 66.9%, within eight points of Study 1. Chapter 2 compares the two ChatGPT snapshots metric by metric on the full-model subset as well as the whole.

10. Known Limits

1. Every Study 2 engine comparison uses the de-duplicated set, and the 30 incomplete answers count as "no card".
2. Tagged signals are absent for Electronics; signal analyses in Study 2 cover 18,736 answers and exclude the category with the highest card rate.
3. Study 1's engine brand field and domain classifier were discarded; published figures that relied on them are corrected in chapter 6.
4. ChatGPT's served model is a confound in Study 2 and is reported as a variable; gpt-5-3 is confounded with the Electronics collection day.
5. Study 2 records batch queue time (median 2.3 hours), not response latency; Study 1 records request time (median 35 to 45 seconds). Speed is not compared.
6. Google AI Mode's product links are redirects, so its card hosts are unknown and it is excluded from host-domain measures.
7. Perplexity's cards include stores outside the United States, about one slot in twelve, with prices printed in dollars that are not dollars. Its price figures use medians and are flagged wherever they appear.
8. Source publication dates exist only for ChatGPT, on 57% of its sources. Freshness is a ChatGPT-only analysis.
9. No page content was stored for cited sources; comparisons of what a source says against what the answer says use source titles only.
10. All answers are United States, English, logged out, and captured from the engines' consumer web interfaces by an automated collection service; no engine's developer API was used. Personalization, location, and mobile surfaces are unobserved.

11. What the Data Cannot Answer

Table 10. Twelve gaps and what would close them.

12. Data Availability, License, and Citation

Availability. The study will publish its aggregated tables as a downloadable package under a Creative Commons Attribution 4.0 license (CC BY 4.0): rates by stage, engine, and category with intervals; engine agreement and overlap; category and subcategory profiles; the leading brands and cited domains by stage; the model summaries; and a column glossary. The package will never include question texts, answer texts, individual product URLs, or any per-answer row. Until the package is released, the tables behind any figure in the study are available on request through the contact on the AEOsome site. The research folder that produced the study holds 130 tables, 15 scripts, and 19 figures; the scripts run in order from the raw collections and were used for every number on these pages.

Versioning. This is version 2026-10. A re-collection or a recut produces a new version with a changelog in chapter 6. Numbers on earlier AEOsome articles were brought into line with this version in October 2026, with a dated note on each.

Citation. AEOsome Research (2026). AI Shopping Visibility Study 2026: How four AI engines answer 30,855 shopping questions. Version 2026-10. https://www.aeosome.com/blog/research/ai-shopping-visibility-study-2026/

Quote any figure with its study label, its basis, and, for ChatGPT, its served model, as shown on the page. Figures and tables may be reproduced with a link to the hub.

13. Glossary

Table 11. Terms.

Table 12. The nine intent stages.

Navigation. Previous: Chapter 6. Corrections and changes · Next: Study hub · AI Shopping Visibility Study 2026 (hub, abstract, and all chapters).

About the author. Vijaya Kumar Channalli is a serial SaaS founder and the founder of AEOsome (Tartu, Estonia), which helps eCommerce brands get their products recommended when shoppers ask AI what to buy. Read how we write, source and correct posts in our editorial policy.

Sources: AEOsome Research, AI Shopping Visibility Study 2026, chapter 7. Study 1: 8,520 ChatGPT answers, women's dresses, United States, February 28 to March 1, 2026. Study 2: 22,335 de-duplicated answers across ChatGPT, Google AI Mode, Microsoft Copilot, and Perplexity, seven categories, United States, May 5 to May 14, 2026. Hub: https://www.aeosome.com/blog/research/ai-shopping-visibility-study-2026/