From Keywords to Decisions: Building SEO Automation That Improves Research

The easiest SEO workflow to automate is a report nobody needs. Data arrives, a model writes a summary, and a dashboard grows. The difficult work is deciding what should change on a website and why that change deserves attention before other work.

Useful automation reduces the cost of reaching a defensible decision. It connects discovery, intent analysis, competitor research, technical evidence, and business value. Language models can assist that process, but they should not become the source of facts that the system never collected.

Start with a business question and a decision queue

Suppose a small team operates an English and Polish information service for people living in the UK. Its questions might include: which existing guides need improvement, which recurring customer problems are insufficiently covered, and which technical defects are preventing useful pages from being found?

Those questions suggest three distinct queues: update, create, and repair. Each queue needs an owner, supporting evidence, an expected benefit, and a review date. A list of 20,000 keywords is an input to these queues, not a strategy by itself.

Begin with the site's existing Search Console data, customer language, page inventory, and known competitors. Add external keyword discovery where it fills a specific gap. This sequence keeps the work connected to the organisation's actual audience.

Preserve the meaning and origin of keyword metrics

Discovery data can come from search suggestions, related questions, competitor databases, internal search, support enquiries, and keyword-planning systems. These sources have different biases. Existing-site queries reflect what the site already reaches; competitor databases reflect the provider's collection universe; model-generated phrases reflect linguistic plausibility.

Maintain source tags rather than collapsing everything into an apparently uniform list. An LLM-suggested phrase should remain a candidate until demand or usefulness is established elsewhere.

Google Ads historical metrics include approximate search volumes, advertising competition, and bid information. The documented response also exposes close variants. These are useful planning inputs, but advertising competition should not be relabelled as organic ranking difficulty. Google Ads historical metrics documentation.

For multilingual research, preserve geography, language settings, reporting period, and any variant grouping returned by the source. Identical volumes across requests are not proof of independent audiences. Do not sum them until their definitions establish that the underlying populations are additive.

In practical terms, a query table should retain both the original phrase and a normalized key. Preserve Polish diacritics in the original. Keep translated or semantically related phrases connected through a relationship table rather than replacing them with one English label.

Use semantic similarity to find candidates, then inspect search intent

Embeddings represent text numerically so that related phrases can be compared. They are useful for reducing a large vocabulary into manageable candidate groups. Sentence Transformers provides working examples of k-means, agglomerative clustering, and community-based approaches, making it a practical starting point for this stage. Sentence Transformers clustering examples.

Semantic closeness, however, does not settle whether two queries deserve the same landing page. “Payroll software UK”, “how payroll works”, and “payroll jobs London” share a topic while implying different tasks.

A proposed workflow therefore has two stages. First, generate semantic neighbourhoods within each language and market. Second, validate selected neighbours against the search results returned for them under comparable conditions.

One transparent SERP similarity measure is Jaccard overlap:

SERP overlap(A, B) = shared normalized URLs / distinct URLs across both sets

If two sets of ten distinct URLs share five URLs, their Jaccard score is 5/15, approximately 0.33. The simpler “five shared out of ten” measure would be 0.50. Neither is inherently wrong, but they must not share an ambiguous label.

URL normalization should remove known tracking noise without collapsing genuinely different pages. Preserve the original URLs so the transformation can be audited. Compare domains separately if useful: overlap at domain level answers a different question from overlap at page level.

There is no universal threshold proving that two keywords belong on the same page. Calibrate a candidate threshold using examples reviewed by someone who understands the market. Give reviewers query pairs, the observed results, and a concrete question: would one well-designed page satisfy both needs?

Avoid a common graph-clustering mistake. If A resembles B and B resembles C, it does not necessarily follow that A and C belong together. Review cluster coherence and split broad groups where the shared intent disappears.

Turn clusters into an editorial map

A useful topical map represents audience tasks and page responsibilities. It is not simply a taxonomy of every phrase a model can generate.

For each cluster, record the intended audience, dominant task, market, candidate page type, existing URL, evidence of demand, and unanswered questions. A cluster may map to a service page, a calculator, a comparison, a directory, or a guide. Creating another article is only one possible response.

Content-gap analysis should also distinguish absence from weakness. A competitor may have a useful comparison that the site lacks. Alternatively, the site may already cover the subject but hide it behind poor navigation, an outdated example, or a page whose main purpose is unclear.

The following decision table is a recommended interpretation layer:

This map also provides a better input to writers. A brief can explain the audience's task, the evidence available, and what would make the result useful. It need not prescribe an arbitrary word count or imitate the headings of every ranking page.

Join datasets at a grain they can support

Combining Search Console, analytics, crawl data, and business records is powerful because each source answers a different question. Search Console describes search exposure and clicks for a property. Analytics describes measured site activity. A crawl records technical and content observations. Business systems record outcomes such as qualified leads or paid transactions.

The Search Analytics API supports grouped queries and pagination, with a maximum request row limit of 25,000. Crucially, its documentation says that it does not guarantee all rows, but rather top rows. Paginating until a response is empty does not establish that every underlying query has been recovered. Search Analytics API reference.

Google's bulk export to BigQuery provides a separate route that avoids the ordinary daily row limitation, while preserving privacy-related limitations on query information. The feature dates to February 2023 but remains a foundational architectural reference because it changes how larger properties can retain and analyse performance data. Search Console bulk export announcement.

The recommended join is initially page–date–market, adding device where data coverage permits. Keep canonical mappings, redirects, time zones, and attribution definitions explicit. A daily Search Console aggregate and a session-level analytics export do not become compatible merely because they contain the same URL.

Most importantly, do not attach every conversion on a landing page to every query that led to that page. That duplicates outcomes. If query-level conversion information is unavailable, label the result as page-level business context rather than keyword-level revenue.

Prioritise with ranges and explicit assumptions

A useful prioritisation model combines relevance, opportunity, feasibility, and confidence. The point is not to generate a mathematically impressive score. It is to expose why one action is preferred.

For work with enough data, use a scenario calculation:

Illustrative incremental contribution =
additional qualified visits
× assumed conversion rate
× contribution per conversion
− implementation and maintenance cost

Use low, central, and high assumptions. If conversion data exists only at page level, say so. If additional visits are inferred from a ranking scenario, state the click-through assumptions and account for the observed result layout. The result is a planning model, not a forecast validated by the search engine.

For early-stage topics without adequate data, use a small rubric instead: strong or weak business fit, clear or uncertain demand, attainable or difficult execution, high or low evidence confidence. Keep these labels visible. An uncertain high-volume keyword should not automatically outrank a smaller opportunity backed by repeated customer requests.

Detect changes before asking a model to explain them

An anomaly pipeline should first validate the input. A sudden decline may be a parser failure, an expired credential, a changed filter, or incomplete recent data. A plausible narrative generated before those checks can send the team in the wrong direction.

After data validation, compare like with like: the same market, device, query panel, collection policy, and reporting window. Use weekday-aware baselines for traffic metrics and keep known launches, migrations, holidays, and content changes in an event log.

For rankings, look at distribution changes and sustained movement across related queries. For page performance, separate declining exposure from declining click-through rate. For AI citations, check whether the prompt set or collection route changed before attributing a movement to content.

A recommended alert contains the observation, scope, evidence links, alternative explanations, suggested next step, and owner. “Investigate a sustained loss across this cluster” is more useful than “SEO visibility is down 12%” with no denominator or collection context.

Give language models bounded analytical jobs

LLMs are well suited to proposing labels, drafting summaries, extracting candidate entities, and explaining a set of already verified observations. They are less trustworthy when asked to invent missing metrics, declare causation, or decide that a vendor's data is correct without inspection.

Retrieval-augmented generation can help by giving the model relevant stored observations and source excerpts. It does not guarantee a faithful answer. Require references to observation IDs, validate that those references exist, and check that the cited evidence supports the claim.

For a structured opportunity record, require fields such as proposed_action, supporting_observation_ids, assumptions, uncertainties, and review_status. Calculate costs, rates, and totals in ordinary code or database queries. Let the model explain the validated numbers.

Keep retrieved web content outside the instruction boundary. A page being analysed must not be allowed to tell an agent to reveal credentials, purchase more data, or publish a recommendation. Set explicit budgets and tool permissions for any agent that can schedule additional research.

These are engineering recommendations, not claims that a particular model has passed an evaluation for this use case.

Evaluate decisions, not writing fluency

The most useful evaluation set is a sample of realistic research tasks with independently reviewed outcomes. Include both Polish and English examples, ambiguous intent, mixed-intent results, missing data, and cases where the correct decision is to take no action.

Measure cluster pair accuracy, unsupported factual claims, missed high-priority issues, reviewer acceptance, and time to a verified decision. Audit language segments separately. A system that performs well on English examples may still need different preprocessing, prompts, or review thresholds for Polish.

These assessments describe appropriate use, not measured performance of an installed system. The objective is a research process that produces fewer weak recommendations and more useful decisions—not simply more content or more automated messages.