Monitor Your Website for Search Problems and Outages
What you get
Custd audits each domain you verify once a week. The audit checks what Google, Bing and AI search tools (ChatGPT, Claude, Perplexity, Copilot) need: that pages answer, can be crawled and indexed, carry a title, description, canonical address and structured data, show their text without JavaScript, and load quickly. Each problem becomes a finding with the page, what was seen, how to fix it and a source to read.
A finding stays open until an audit no longer sees it, then it shows as fixed. If it comes back, it reopens.
Custd also checks each verified domain's home page every five minutes and its HTTPS
certificate every six hours. Two failed checks in a row open an outage; the first check
that passes closes it. Outages, slow responses and certificate problems are findings too,
under the uptime.* and tls.* rules, and they are fixed by the checks themselves rather
than by the weekly audit.
Find your way around
Site Insights in the top navigation lists every site you added, the one that most needs you first. Each row says whether the site is up, one sentence on what is left to do, and its first action as a link straight to where it is done.
A site's page opens on Overview: one sentence on whether the site works and how much is left, when it was last checked and what Custd cannot check yet, then the three most important actions, each with why it matters and one control (View incident, Go to the finding, Check again, Connect search or Reconnect). Below them, one line each says whether the site is working, how search reaches people and how much is left to fix, with a link to the detail.
The detail sits in four more views, chosen from the tabs under the site's name (on a phone, the View list):
What to fix (what the latest audit changed, the findings, every page read and a Tools fold with campaign links),
Search (search numbers, with countries and devices, keyword rankings, telling search engines a page changed, AI
visibility and competitors as folds that open on request), Reliability (uptime and certificate, page speed and site
traffic) and Reports. Settings, beside the site's name, holds only setup, in three groups: Connections
(Google, Bing and the IndexNow key), Checks and Business details. Each view's address can be copied and shared
with a colleague who has access, and the browser's back button returns to the view you came from. Links to the old
sections, such as #uptime or #competitors, still open the view that holds them.
1. Add your domain
Open Site Insights, choose Add site, paste the site's address (a full link such as
https://www.example.com/pricing is fine: Custd keeps only www.example.com) and choose
Add site. Custd says which Site the address goes to: the Site that already holds the same
address with or without www., a Site named after it, or your only Site. To pick another, open
Add it to another Site. An address you already added is not added twice; the dialog links to it.
The site's page then opens with the proof steps and a Getting started list: add the site, prove you own it, the first check, and connecting search (optional). Each step says where it stands from what Custd has recorded, so you can leave and come back to it. The list goes once ownership is proved and the first check has finished.
If your organisation has no Site yet, adding the first address creates one, named after the address
without www., and stores its page views in a destination of its own. This needs a tenant
administrator. The Site starts with no environment, allowed origins or usable write key, which
only the browser script needs: to install it later, set those in Admin → Sites and rotate the
write key there.
Or from the command line:
custdctl sites domains add --profile acme --host www.example.com
Without --site, Custd places the address by the same rules; if it cannot tell which of several
Sites you mean, it answers site_choice_required and you add --site <site-uuid>.
2. Prove you control it
Custd fetches nothing on a schedule until you prove the domain is yours. Do either one:
- add a DNS TXT record at
_custd-verification.<your host>with the value shown, or - publish a file at
https://<your host>/.well-known/custd-verification.txtcontaining the token shown.
Then choose Verify now, or run custdctl sites domains verify <domain-uuid>. If the
check fails, the page shows what each proof returned. The file must be served directly:
Custd does not follow a redirect for it.
Verifying a domain also verifies its subdomains in the same account: once example.com is
verified, www.example.com and shop.eu.example.com are too (custdctl sites domains show
prints verified by parent and the parent's UUID). It never works the other way, and
myexample.com is not a subdomain of example.com. If the parent is removed, subdomains
verified only through it stop being verified.
3. Read the findings
The first audit runs within a few minutes of verification, then weekly. Check whole site
(or custdctl sites domains check <domain-uuid>) queues another audit for the next run.
After fixing one page, Check again on its finding re-reads just that page; the finding
reads fixed only once that check saw it gone. When the same problem is on several pages and the
check can confirm it, Check all N pages again re-reads each of them and says what it saw, such
as "3 of 5 fixed, 2 still open". A site gets 30 page checks an hour; if they run out, the screen
says how many pages were queued, and you check the rest later.
With Search Console connected, a page's own screen also lists Searches that show this page: Google's busiest searches for it over the newest 28 days (up to 20), where the page already uses each, title ideas built from them and your brand name, and other pages of the site that mention one and could link to it. The same searches can be counted in a draft before you publish. Until Search Console is connected or the first import finishes, the screen says so instead.
Copy for my developer copies the problem as plain text for whoever edits the site: the pages, what was found on each, the code to add and how to check the fix from the public page. It holds no Custd login, link or id, so you can paste it into an email or a ticket. What it holds shows the text before you send it.
What to fix lists the open findings, most serious first. Use Show to see fixed or dismissed findings. If an audit finding is intentional, open its rule and select Dismiss rule for all URLs on this domain, or Dismiss this URL for one affected page. Enter a reason. Use Show → Dismissed to review the reason and restore the selected scope. Audits keep recording what they observe. A restored finding returns to Open if it is still observed.
From the command line:
custdctl sites findings --profile acme <domain-uuid>
custdctl sites findings --profile acme --state all --json <domain-uuid>
A fixed finding has Search before and after the fix: its page's clicks and impressions from
each connected search provider over the same number of days (28 by default) before and after the
day an audit first saw it fixed. A window reads "not enough data" until it has ended and been
imported, or while days or the page's rows are missing; it never shows that as zero. Other changes
and ranking updates move these numbers too, so the difference is not proof the fix caused it.
custdctl sites impact --profile acme --days 28 <domain-uuid> <finding-uuid> reads the same.
To see what to do first, open the domain's Overview, or read its next actions: whether the site is up, what is not checked yet, and its three most important actions in a fixed order (prove ownership, an outage, a certificate that is invalid or expires within 14 days, refused search access, serious findings by pages affected, the first audit, connecting search, then findings worth fixing). The portfolio summary gives every domain's status and first action, worst first; the Site Insights list shows it. The client report lists the same three actions.
custdctl sites domains next --profile acme <domain-uuid>
custdctl sites domains portfolio --profile acme
A product that shows every Site of a tenant, such as a hub, provisions a read-only credential for
these two reads with the site-insights-read scope template; a site-bound credential reads them
for its own Site only.
high findings stop a page being found or read at all; fix those first. info findings
are choices, such as blocking an AI crawler, not mistakes.
To be told when an audit opens a new high finding, set an email address or your webhooks
in the Alerts form on the Site Insights page, or run
custdctl notification destinations set --event site_finding_opened --channel email --email seo@acme.example.
Custd sends at most three alerts per audit, and none for a domain's first successful audit,
which records what is already there.
Optional AI suggestions in Awthy
AI starts Off. If your company has own-key access, open Manage AI settings in Custd from Awthy Hub's Reporting → Your sites in search as the account owner. Read the DeepSeek consent notice, then choose Save consent if you agree. Enter or replace your DeepSeek API key in Custd and choose Save AI settings. Reopen the settings from Hub if the session has expired.
A supported finding such as a missing page title can show a suggested explanation and next step in Awthy → Search & speed. Hub shows confirmed and estimated provider costs and the monthly allowance. In Custd, Turn off AI and remove key stops suggestions. Revoke consent and remove key withdraws provider permission as well. If the key was already removed, choose Revoke consent.
4. Uptime, certificate and warming
The domain's Reliability view starts with Uptime and certificate: one
status (Up, Failing, Down, Checks paused or No recent check), the last check with its HTTP
status and response time, the certificate's expiry and issuer, and the recent outages.
From the command line, custdctl sites domains show <domain-uuid> prints the same state and
custdctl sites incidents --profile acme <domain-uuid> lists the outages.
A check passes when the home page answers with a status from 200 to 399 within ten seconds, following up to three redirects. The findings are:
| Rule | Severity | When |
|---|---|---|
uptime.down | high | an outage is open |
uptime.slow | medium | the home page takes more than three seconds on average |
tls.invalid | high | the certificate is expired, untrusted or for another host |
tls.expiring-soon | high | the certificate expires within seven days |
tls.expiring | medium | the certificate expires within thirty days |
Choose how the domain is checked in the domain's Settings → Checks (network probes are under
Advanced: network probes), or
custdctl sites domains checks --profile acme --mode <mode> <domain-uuid>:
uptime(the default): the home page every five minutes and the certificate every six hours.warm: the same, plus one anonymous home page request every minute, so a site that goes cold when idle is kept ready for its next visitor. The page shows the average response time without warming and with it, so you can see what warming buys before keeping it on.paused: no requests at all. Custd cannot tell that a site is stopped, so whoever stops it sets the domain topaused; nothing then wakes the site until the mode is set back. HaakCo Hosting does this for a paused environment.
To verify a public recovery without waiting for the regular interval, select
Reliability → Public network uptime → Check public uptime now or run
custdctl sites domains uptime-check DOMAIN_UUID. The request is queued for the next
scheduler tick. The displayed status changes when the check finishes. The domain must
be verified with uptime checks enabled.
For a site reachable from an internal network, open Reliability → Private network uptime
and select Set up private probe. Enter a name that identifies the network. Save the
one-time credential and report URL shown after setup in the probe's configuration. Run the
probe where it can reach both the internal site and the Custd report URL. Configure your
service manager to supply the credential in CUSTD_PRIVATE_UPTIME_TOKEN, then run:
custdctl sites run-private-uptime \
--report-url 'https://custd.com/api/v1/site-private-uptime-reports/company/monitor' \
--token-env CUSTD_PRIVATE_UPTIME_TOKEN \
'https://internal.example.net/'
Replace the example URLs with the report URL from setup and the internal site's HTTPS URL.
For CLI setup, run custdctl sites domains private-uptime-create --network Office --credential-file /secure/probe-token DOMAIN_UUID. The command saves the one-time token
in the new private file and prints the report path. Use the Custd API origin plus that path
as the runner's report URL.
Keep one runner active per monitor. It checks once a minute by default and polls for
Check private uptime now requests every 15 seconds. From the CLI, run
custdctl sites domains private-uptime-check DOMAIN_UUID to queue a check or
custdctl sites domains private-uptime DOMAIN_UUID to see the latest result. The
panel shows the network name, observation time and result. A site can be Up from inside
the network while its public check shows Down. In Private probe settings, rotate the
credential to replace it or remove the monitor to stop receiving its results. The CLI command
custdctl sites domains private-uptime-rotate --monitor MONITOR_UUID --credential-file /secure/new-token DOMAIN_UUID
revokes the previous token and saves the replacement. Update the runner's service configuration
with the replacement token and restart it.
To retire a monitor, run custdctl sites domains private-uptime-remove --monitor MONITOR_UUID DOMAIN_UUID
and stop the runner. Removal revokes its token.
From Reliability → Configure network probes, enable DNS and optionally enter the public
IPv4 and IPv6 addresses you expect. Reliability shows missing and unexpected addresses after
a check. Leave the list blank to check only that the name resolves to public addresses.
For automation, use custdctl sites probes --set 'dns:1.1.1.1;2606:4700::1111'
with the domain UUID. The --set option replaces the probe list.
To catch an error page that still returns HTTP 200, open Reliability → HTTP expectations. You can check the final HTTP status after redirects, limit the time to response headers, require public text in the response, or match the exact string value of a top-level JSON field. Saving blank fields restores the default HTTP 200–399 check. Reliability shows the reason for a failed expectation without storing the response body.
In Reliability, Domain registration shows the public registry's expiry date, its source, and the last check time. If no recent date is available, the section shows Expiry unknown. Within 30 days of the published date, Custd opens a renewal finding and uses the configured finding alert destination. Confirm the actual renewal date with your registrar.
For a scheduled job, open Reliability → Scheduled jobs → Add heartbeat monitor. Enter the
job's name, five-field cron schedule, timezone, and grace period. The monitor shows the next
expected run and whether the latest scheduled occurrence is waiting, on time, late, missed,
or failed. The setup panel displays the report URL and a one-time write token. Store the token
in the job's secret manager. You can also manage monitors through the Admin API.
Send a JSON report to POST /api/v1/site-heartbeat-reports/{companySlug}/{monitorUuid}
with the write token in the Authorization: Bearer header.
{"scheduledFor":"2026-11-20T00:00:00Z","outcome":"succeeded"}
Use the intended scheduled time, even if the job started late. Report failed for a failed job.
An exact retry returns the original receipt. Once a newer run has reported, an older report
returns a conflict. Rotation replaces the credential immediately. Configure the existing Site
outage alert destination to receive one message when a job misses its deadline or reports failure,
and one when a successful report clears the incident. The alert links directly to Scheduled jobs.
For automation, use custdctl sites heartbeat list, create, rotate, or remove. The
create and rotate commands save the one-time token to a new --credential-file. Load the token
from your secret manager into an environment variable for the scheduled job. Use
custdctl sites heartbeat report --report-url URL --token-env ENV_NAME --scheduled-for TIME --outcome succeeded
to submit the result.
For planned public downtime, open Reliability → Maintenance windows and enter the start and end in UTC. During the window, Custd pauses public uptime, certificate, warming and network checks, and shows the window's end time. Checks resume on the next scheduler tick after the window ends or is cancelled. Cancelled windows stay in the list.
For automation, use custdctl sites maintenance schedule with --request-file containing
windowUuid, startsAt and endsAt. Use RFC 3339 timestamps and reuse the same UUID when you
retry the same window. Use list to read the domain's windows and cancel --window WINDOW_UUID
to stop one. Pass the domain UUID after the flags.
During an open outage, select Acknowledge outage in Reliability to record that it has been seen. Checks continue and the next passing check closes the outage.
To be told when a site goes down and when it is back up, set the Outage alerts form under
Site Insights settings on the Site Insights page, or run
custdctl notification destinations set --event site_incident --channel email --email ops@acme.example.
Custd sends one alert when an outage opens and one when it closes, never one per failed
check. A certificate that becomes high (expiring within seven days, or invalid) sends a
finding alert to the site_finding_opened destination.
Read real-user speed
Open Reliability → Page speed → Real-user Web Vitals for Chrome UX Report field data. The section shows the collection period and the 75th-percentile values across devices for LCP, INP and CLS. A metric can be missing even if other metrics are available. Lighthouse results below it show Custd's scheduled tests of the home page.
5. Let a plugin or server manage one site
A WordPress plugin or a hosting environment should not hold your administrator login, and it
should not see your other sites. Give it a site-bound credential instead: it holds
site_domains.manage, and it can add, verify, check and remove the domains of one Site and
read their findings, checks and outages, and nothing else. Naming another Site, or another
Site's domain, answers as if it did not exist.
- For an existing Site: open Credentials, choose the purpose Site Insights (one
Site), enter the Site UUID and create it, or run
custdctl sites domains credential --profile acme --company-slug acme --site <site-uuid> --client-id acme-wordpress. The secret is shown once. Deleting the credential keeps the Site. - From a server that provisions per site (a broker holding
producers.provision, such as a WordPress hub or a hosting control panel): call producer provisioning with"scopeTemplate": "site-insights". Custd creates a Site named afterdisplayNamefor the credential. Repeating the call keeps the same Site; revoking the producer removes the Site and its domains, so a disconnected install leaves nothing behind.
A hosting provider that already proves ownership itself (a TXT challenge, registrar control or
managed DNS) does not need the Custd token. HaakCo can enable its broker for the
site-insights-attesting template, whose credential also holds site_domains.attest. That
credential attests a domain of its Site with POST /api/v1/site-domains/<domain-uuid>/attestation
(or custdctl sites domains attest <domain-uuid>): the domain, and its subdomains, are then
verified "by attestation", with the credential and time recorded and written to the audit log.
DELETE on the same path (custdctl sites domains unattest) withdraws it; so does revoking the
credential or removing the scope from it.
When HaakCo hosts your site but its DNS is with another provider and you cannot publish the record,
HaakCo can confirm the domain for you. It then reads verifiedBy: operator. A passing DNS or file
check later replaces that, and HaakCo can withdraw the confirmation if it stops being true.
If you look after sites for your own customers, send each Site's finding and outage alerts to
the customer instead of to you: set Alert email for this Site under Site Insights
settings on the Site Insights page, run custdctl sites alerts --site <site-uuid> --email owner@customer.example
(--clear goes back to your destinations), or PUT /api/v1/site-domains/alert-destination
with {"emailRecipient": "..."}; a site-bound credential can set its own Site's. Webhook
alerts need no override: each one names the domain, so your receiver can route it.
With that credential the plugin calls the same Site Insights operations without naming its
Site: POST /api/v1/site-domains with just {"host": "www.example.com"}, then verify,
findings, checks and incidents on the domain it got back. In the PHP SDK that is
adminSiteDomains()->create(CreateSiteDomainRequest::forOwnSite('www.example.com')) and
list(); in Go, SiteDomains.Create with an empty SiteUUID and SiteDomains.List(ctx, "").
6. Search performance from Google and Bing
Custd can import how a verified domain does in search: clicks, impressions, click-through rate (CTR) and average position, by page and by search query. The domain's Search view shows the numbers, each with its source, the days it covers and when it was last imported. Custd imports once a day and keeps sixteen months, the same as Search Console.
Google Search Console
Custd reads Search Console as its own service account, so you never share a Google password or approve an app.
- Find the service account's address: open the domain's Settings → Connections, where
Copy copies it, or run
custdctl sites search account --profile acme. - In Search Console, open your property, go to Settings → Users and permissions → Add user, enter that address and choose Restricted permission. Custd only reads.
- Connect the domain with the property's name exactly as Search Console shows it: a
domain property is
sc-domain:example.com, a URL-prefix property is the full address ending in/, such ashttps://www.example.com/.
custdctl sites search connect --profile acme --provider google \
--property sc-domain:example.com DOMAIN_UUID
Search Console's numbers are final after about three days, so the newest day shown is usually three days ago.
Bing Webmaster Tools
Bing uses your own API key. In Bing Webmaster Tools, open Settings → API access → API Key, generate a key, and connect with the site's address as Bing lists it. Custd stores the key encrypted and never shows it again; put it in an environment variable rather than on the command line:
BING_KEY='…' custdctl sites search connect --profile acme --provider bing \
--property https://www.example.com/ --api-key-env BING_KEY DOMAIN_UUID
Bing reports weekly totals, so its numbers are marked weekly.
When an import keeps failing
A failed import is retried by itself, waiting longer each time, up to a day. Once you have fixed the cause (for
example given Custd's account access to the property again), custdctl sites search retry --provider google <domain-uuid> runs it on the next scheduler tick instead
(POST /api/v1/site-domains/<domain-uuid>/search/<provider>/retry). A connection the provider refused shows as
revoked and is not retried: connect it again.
Reading it and the advice
custdctl sites search show DOMAIN_UUID (or the domain's Search view) shows each
source's totals, the busiest pages and queries, and each connection's state:
| State | Meaning |
|---|---|
pending | Connected; the first import runs within fifteen minutes. |
active | The last import worked. |
failing | The provider failed for a reason that may pass (an outage, a rate limit). Custd retries after an hour, then less often, at most a day apart. |
revoked | The provider refused access: the service account was removed from the property, the property name is wrong, or the Bing key was revoked. Fix access and connect again. |
Custd keeps up to 500 pages and 500 queries per provider per day, and Search Console's
countries and devices. To page through all of them, run
custdctl sites search rows --dimension country DOMAIN_UUID or call
GET /api/v1/site-domains/DOMAIN_UUID/search/rows with dimension (page, query,
country or device), and optionally provider, days, offset and limit (at most
1,000). The answer's total says how many rows match. Bing does not report countries or
devices, so the answer lists it under unavailable rather than showing zero traffic.
Which search pages lead to goals
When the Site runs the Custd browser tracker, name the events that count as goals:
custdctl sites value --profile acme --goal signup --goal purchase --days 28 DOMAIN_UUID
(GET /api/v1/site-domains/DOMAIN_UUID/search-value?goal=signup&goal=purchase). Each of the
50 busiest search pages (across providers) shows its clicks and impressions next to the visitors
whose first page view in the window was that page and the goals they reached. A visitor may have
come from anywhere, not only search. A page nobody landed on has no goal numbers rather than zero,
and partial means the goal numbers are incomplete. Goals are read with your own reporting access
(a tenant administrator or analyst); without it goalStatus is unavailable and says why.
Search queries are not tied to goals, because the providers report pages and queries separately.
Inspecting one URL
To see how Google has one page in its index, ask for a URL Inspection of a URL on a verified domain connected to Search Console:
custdctl sites search inspect --profile acme --url https://www.example.com/pricing DOMAIN_UUID
custdctl sites search inspections --profile acme DOMAIN_UUID
The Admin API is POST /api/v1/site-domains/DOMAIN_UUID/search/inspections with
{"url": "https://www.example.com/pricing"}, and GET on the same path for the latest fifty.
Custd asks Google within about fifteen minutes, as the same service account that imports your
numbers. The answer shows Google's own values: the verdict (PASS, PARTIAL, FAIL or
NEUTRAL), the coverage text, whether a noindex rule or robots.txt blocks the page, whether
Google could fetch it, when it last crawled it and as which crawler, the canonical Google chose
and the one the page declares, and the mobile usability verdict while Google still reports one.
A field Google does not report is left out, never guessed.
Each tenant may ask for 200 inspections in any 24 hours, well inside Google's own limit. A request Google refuses (the URL is outside the property, or the service account may not inspect it) fails with the reason and does not disconnect the domain; a rate limit or outage is tried again up to three times. Inspections are kept for ninety days.
Telling search engines a page changed (IndexNow)
IndexNow tells Bing, Yandex, Seznam, Naver and the other engines that take part that pages were added, changed or deleted, so they can crawl them sooner. Google does not use it.
In Site Insights, set it up once in the domain's Settings → Connections under Tell Bing and others a page changed (IndexNow): Turn on IndexNow, publish the key file it shows, then Verify key file. After that, submit changed pages from the Search view's Tell search engines a page changed, which also lists what was sent. From the command line:
- Give the verified domain a key:
custdctl sites indexnow enable DOMAIN_UUID(orPUT /api/v1/site-domains/DOMAIN_UUID/indexnow). The answer names the key and itskeyLocation, such ashttps://www.example.com/0123456789abcdef0123456789abcdef.txt. - Publish a text file at that address holding only the key, then run
custdctl sites indexnow verify DOMAIN_UUID(POST .../indexnow/verify). Custd fetches the file from the public internet without following redirects and says what it found. - Submit changed URLs on the domain's own host, up to 100 at a time:
custdctl sites indexnow submit --url https://www.example.com/a --url https://www.example.com/b DOMAIN_UUID(POST .../indexnow/submissionswith{"urls": [...]}).
Custd sends each batch to IndexNow within about fifteen minutes. custdctl sites indexnow show
(GET .../indexnow) lists the latest batches:
| Status | Meaning |
|---|---|
queued | Waiting to be sent, or to be tried again after a rate limit or outage. |
accepted | IndexNow received the URLs (HTTP 200, or 202 while it checks the key). It does not mean any engine has indexed them. |
rejected | IndexNow refused the batch and says why. A refused key is marked unverified until you verify it again. |
failed | It could not be sent after three tries, or the key is no longer verified. |
Whether a page is indexed is up to each engine: use a URL inspection for Google, and the
search numbers for both providers. Each tenant may submit 1,000 URLs in any 24 hours, and
batches are kept for ninety days. custdctl sites indexnow disable deletes the key and its
batches; remove the key file too.
From the imported numbers Custd adds two findings to the list:
search.low-ctr: a page shown at least 200 times in 28 days, on the first results page, that fewer than 1% of searchers click. Rewrite its title and description.search.position-drop: a page whose average position fell by five places or more compared with the 28 days before.
They are fixed when the numbers no longer show the problem. Disconnecting a provider
(custdctl sites search disconnect --provider bing DOMAIN_UUID) deletes its imported
numbers.
Neither Google's nor Bing's API reports AI Overviews, AI Mode or Copilot citations yet, so Custd shows only what the APIs provide.
Tracking keyword rankings
Rank tracking follows your own keywords through your own Search Console data. Custd does not scrape search results, so it never shows a competitor's position.
- Connect the domain to Google Search Console (above).
- Track a keyword:
custdctl sites rankings track --keyword "custd pricing" --country zaf --device mobile DOMAIN_UUID(orPOST /api/v1/site-domains/DOMAIN_UUID/rankings/keywords). The country is a three-letter ISO code orall; the device isdesktop,mobile,tabletorall. A Site tracks up to 50 keywords over all its domains. - Read them:
custdctl sites rankings show DOMAIN_UUID(GET .../rankings?days=28).
Each daily import gives every tracked keyword one entry per day:
| State | Meaning |
|---|---|
ranking | Your site appeared. The entry names the best-placed page, its average position, and the keyword's clicks and impressions. |
not_ranking | Search Console was read for that day and your site did not appear for the keyword. |
no_data | Nothing was read: Search Console is not connected or failing, the day is not imported yet, or the keyword was paused or not tracked yet. |
Search Console's final numbers run three days behind, so the newest day is three days ago. A keyword is read even when it is outside the 500 busiest queries Custd keeps for a day.
When a keyword moves across its alert position (10 by default, --notify-position to change
it), Custd sends an alert to the Site's alert address: when it leaves the top 10, stops
ranking, or comes back in. The first day it is observed sets the baseline, and one import
sends at most three alerts. Pause a keyword with custdctl sites rankings update --paused DOMAIN_UUID KEYWORD_UUID, which stops reading it and stops its alerts; run the same command
without --paused to resume. custdctl sites rankings untrack DOMAIN_UUID KEYWORD_UUID stops
tracking it and deletes its history. History is kept for sixteen months.
The shared search pool
The shared search pool is opt-in and off unless you join. Organisations that join pool their Google Search Console searches, and only they can read the pool: give to get. Joining accepts the shared search pool terms for your organisation.
- What you share: the searches your sites appear for, with how often they were shown and at what position, over the newest 28 days with final data.
- What anyone in the pool sees: only a search that at least five organisations share, with at least 100 times shown between them. It shows as a range of how often it was shown (100 to 999, 1,000 to 10,000, 10,000 to 100,000, or over 100,000 times) and the share of those at positions 1 to 3, 4 to 10, 11 to 20 and after 20, rounded to 5%, over every country and device. The pool never holds your organisation, sites, pages, clicks or exact numbers.
- What you get: the pool for any search, and its numbers beside your tracked keywords. The pool covers every country and device, even for a keyword you track in one. A tracked keyword too few organisations share says so instead.
- Leaving: you stop reading the pool at once, and your searches leave it at its next rebuild. The pool is rebuilt every six hours.
Join or leave under Site Insights settings on the Site Insights page, or
custdctl sites search-pool join --accept-terms 2026-11-18 and custdctl sites search-pool leave
(PUT /api/v1/site-domains/search-pool with {"joined": true, "termsVersion": "2026-11-18"} or
{"joined": false}); a tenant admin or ingest manager makes the change and it is audited with the
terms version accepted. A join that does not name the current terms version is refused.
custdctl sites search-pool show says where you stand.
custdctl sites search-pool queries --search pricing reads the pool
(GET /api/v1/site-domains/search-pool/queries?search=&limit=), busiest first; an organisation that has not
joined gets 403 site_search_pool_not_joined.
Google Analytics 4
Custd reads GA4 as the same service account it reads Search Console with, so there is nothing new to share.
- In GA4, open Admin → Property access management, add the service account's address (step 1 above) and choose the Viewer role.
- Copy the property's numeric ID from Admin → Property details.
- Connect the domain:
custdctl sites analytics connect --profile acme --property 123456789 DOMAIN_UUID(PUT /api/v1/site-domains/<domain-uuid>/analytics/ga4with{"propertyId": "123456789"}).
Once a day Custd imports the newest 28 complete days: the busiest 100 landing pages from Organic Search with their
sessions, engaged sessions and key events, organic key events by name, and sessions by channel for all traffic.
Each import replaces the last. When GA4 recorded no organic key event in those days, key events show as not
recorded, never as zero. Ecommerce and audience reports are not imported and say so. If GA4 refuses the
service account (it was removed, or the property ID is wrong), the connection reads revoked and Custd stops
asking until you connect again. custdctl sites analytics show DOMAIN_UUID (GET …/analytics) shows the
connection and the reports; custdctl sites analytics disconnect removes the property and its imported rows.
7. Save and share reports
Reports on a domain's page opens its Reports view. Your saved reports come first, each with its template, search period, when it was built and who can see it (Private until you share it, or how many live links open it). Below them is the live report, read now: uptime now and over 24 hours, 7 and 30 days with ninety daily bars, the certificate's expiry, the latest desktop and mobile Lighthouse scores, 28 days of search numbers and up to fifty open findings with their fixes.
Create report builds the report from today's numbers and keeps that copy under a name:
- Executive summary keeps the glance, uptime percentages, speed scores, search totals and the five most serious things to fix.
- Summary and technical appendix adds the daily uptime bars, Lighthouse timings, the search tables and all open findings, up to fifty.
- Search period is the last 7, 28 or 90 days.
A saved report does not change until you choose Replace with current numbers, which rebuilds it in place: its links stay and show the new version, and the screen says how many do. Delete removes the copy and stops every link that opened it. A domain keeps up to 20 saved reports, each with its own name.
- Print or save as PDF prints the report on its own, without the Custd navigation.
- Download findings (CSV) and Download search queries (CSV) give the rows for a spreadsheet.
- Share link creates a link anyone can open without a Custd login, for 7, 30 or 90 days. On a saved report the link opens that saved copy; on the live report it opens the report as it is when the link is opened. Copy it when it is created: Custd keeps only a fingerprint of it and cannot show it again. The dialog lists the live links; Revoke stops one at once.
A report never shows the domain's verification token, its Site, check failure details or
search connection details. The same operations are on the Admin API
(GET /api/v1/site-domains/{domainUuid}/report, /saved-reports and /report-shares) and in
custdctl sites reports.
Publish a status page
Under Reports → Status page, choose which verified domains to show and give each one a customer-facing label. Select Preview status page to see the labels, public uptime and last public check times. Selected components show Maintenance during an active window. The preview is private. Select Publish status page and confirm to make the page available to anyone with its address. Use Copy public link to share it. Unpublish closes that address and keeps the saved title and component labels so you can publish it again.
Readers can use Get status updates by email on the public status page. They confirm their address through the email link. A public update is also queued for confirmed subscribers. Each update email includes an unsubscribe link.
During an outage, Reliability → Post customer update opens the status page editor. Write the customer-facing message, select Review customer update, and confirm to post it. The public page shows the latest 20 messages. Include the recovery in a new message when the service is back.
For automation, custdctl sites status-page show reads the private settings. Use preview
and publish with --request-file containing a JSON object with title and components.
Each component needs a verified domainUuid and a customer-facing label. Pass the domain UUID
after the flags. Use post-update with a message and a stable requestUuid in the request
file so retries keep the same update. Use unpublish to close the public link.
Campaign links
What to fix → Tools → Campaign links saves a link to one of the domain's pages tagged with utm_source,
utm_medium and utm_campaign (and optionally utm_content). Custd stores the values in lower case with spaces as
_, and suggests common source and medium names, so a report does not count "Facebook" and "facebook" as two sources.
Copy the link from the list; Archive retires it and keeps it on record. When the site sends page views to Custd (the
browser script or an SDK), visits through a tagged link show under that campaign in your web analytics reports. The same
operations are on the Admin API (/api/v1/site-domains/{domainUuid}/campaign-links) and in custdctl sites campaigns.
8. Tell Custd about the business
Search research needs to know the business behind a site, not just its pages. Each Site keeps a project context, shared by all its domains:
- up to 5 brand names people search for it by;
- up to 10 competitor domains you name, such as
rival.example; - up to 10 markets, each a country and a language, such as South Africa in English
(
en-ZA).
Open a domain's Settings → Business details and fill in Project context, or run
custdctl sites context --site <site-uuid> --brand Acme --competitor rival.example --market en-ZA.
Each list flag replaces only that list, so --market af-ZA --market en-ZA keeps the brands and
competitors; --clear removes everything. The Admin API is
GET and PUT /api/v1/site-domains/project-context?siteUuid=<site-uuid> with
{"brandNames": [...], "competitorDomains": [...], "markets": [{"country": "ZA", "language": "en"}]};
saving empty lists clears it. A site-bound credential reads and sets only its own Site's.
A competitor must be a plain DNS name, a country an ISO 3166-1 alpha-2 code and a language an
ISO 639 code; anything else is refused with the reason.
9. Research keywords and competitors
Once the project context names competitors, Custd reads public data about them and about your own verified domains. It runs when your Custd operator has turned it on; until then everything below reads as pending.
- Page inventory. Each domain's page URLs from its sitemaps and the Wayback Machine, with
where each was found and the first and last day it was seen. A site whose
robots.txtrefuses Custd, or that publishes no sitemap, shows that source asunavailable.custdctl sites competitors --site <site-uuid>shows every domain, andcustdctl sites web-pages --site <site-uuid> --domain rival.examplepages through its URLs. - Popularity band. Each domain's band on the daily Tranco list (
top-1ktotop-1m).unrankedmeans outside the top million;unknownmeans not read yet. It is a band, not visits. - Topic trends. Monthly Wikipedia pageviews for your brand names and your keyword lists'
seed terms, from the Wikipedia of your first market.
no_articlemeans Wikipedia has no such article; it does not mean nobody is interested.custdctl sites trends --site <site-uuid>. - Keyword lists. Up to 50 named lists of up to 500 keywords each. Repeats that differ only in
case or spacing are dropped. Lists belong to the Site, so disconnecting and reconnecting Search
Console keeps them. In the Search view, under Competitors and links → Keyword lists, New keyword list
saves one (a name and one keyword a line), and Edit and Delete change the list shown.
custdctl sites keywords --site <site-uuid> --name Trail --seed "trail running" --keyword "trail shoes"saves one,--deleteremoves it, andcustdctl sites clusters --site <site-uuid> --name Trailgroups its keywords by shared word with your own search numbers for the last 90 days. A keyword your search connections never reported shows no numbers, not zero. With Bing connected and a market set in the project context, Custd also asks Bing's keyword research about each saved keyword in your first market once a week: the clusters read adds Bing's searches in the latest week as a separate number, and a list's seed term gets the keywords Bing relates to it. A keyword Bing reported nothing for has no number, and with no market or no Bing connection the read says which is missing. - Content gap. Topics your competitors publish pages about that you cover with neither a page
nor a search query. Each topic lists the competitors and pages, with their sources and days.
custdctl sites gap --site <site-uuid> --seed "trail running"keeps only pages about the seed. Until Custd knows words from your own pages or queries (a home page alone has none) the gap readsunavailableand lists nothing.partialmeans an inventory behind it is still running or stopped at its page limit. - Links. Who links to one of your own or competitor domains, from three sources, each total
labelled with what it covers and when it was read.
bing: the inbound links Bing Webmaster Tools lists for an own domain you connected to Bing, read weekly.commoncrawl: the domains linking to it in Common Crawl's quarterly domain graph, up to 1,000 best ranked, with its own harmonic centrality and PageRank positions.crawl: links Custd's audits of the Site's other domains saw.lostlists the domains that linked last quarter and no longer do; it needs two quarters read. A source that cannot read a domain showsunavailable, and one not read yetpending, never zero.custdctl sites links --site <site-uuid> --domain www.example.com.
The Admin API reads are GET /api/v1/site-domains/web-domains, web-pages, topic-trends,
content-gap, keyword-clusters and referring-domains (?domain=<host>); lists use GET, PUT
and DELETE /api/v1/site-domains/keyword-lists.
All take ?siteUuid=<site-uuid>, and a site-bound credential reaches only its own Site.
Prepare a sourced brief
- In Site Insights, open Settings → Project context and set your market. Google Trends currently covers South Africa, the UK, and the US. For a technical audience, select Include Hacker News topics for this Site’s technical audience.
- Save the phrases your readers use in a keyword list. Open Search → Emerging topics to see which public topics match those saved terms.
- Open the original source and check its date and relevance. Expand Prepare a brief to copy the topic, matched term, date, and source links into your outline.
- Choose one question your readers need answered. Add a dated observation from your own work and link to an existing page that helps the reader take the next step.
For example, if a saved term about web performance matches a browser release, read the release notes before proposing an article. Run a check on a page you can discuss publicly and record the date, device, and result. Use that observation to explain what readers can check on their own pages. A trend match shows recent attention. Search volume and rankings come from the separate keyword and search reports.
Export your data
custdctl sites export --output www-example.json <domain-uuid> writes one domain's and its Site's Site Insights
data into one JSON file: findings open and fixed with when each was first and last seen (up to 10,000, most severe
first; the file says when there are more), outages, the 90-day uptime history, every measured page's Lighthouse
runs for 90 days, keyword rankings for the sixteen months kept, URL inspections, IndexNow, saved reports with their
content, campaign links (archived ones too), project context, keyword lists, competitor web domains, topic trends
and every stored day of search clicks, impressions and positions. The file must not exist yet and is written
readable only by you. Sealed Bing API keys and report share secrets are never exported. Access-log traffic, the
latest audit's crawled pages and links, page facts, competitor page lists and network probes are not in the file;
read them with their own commands. Run it once per domain to export a whole tenant (custdctl sites domains list).
The Admin API reads are GET /api/v1/site-domains/<domain-uuid>/export and
GET /api/v1/site-domains/<domain-uuid>/export/search-metrics?offset=<n>&limit=<1-10000>, which pages the daily
search metrics in a fixed order; the export's search.metricRows says how many there are.
Removing a domain removes everything kept for it, and deleting a Site removes its domains and its project context, keyword lists, web data, topic trends and alert destination. Offboarding a tenant removes all of it.
Limits
- An audit reads at most 200 pages, one request at a time, and skips paths your
robots.txtdisallows. When it stops at that limit it marks nothing fixed, because the pages it did not read may still have the problem. - Only public addresses are audited; a domain that resolves to a private address is refused.
- Removing a domain removes its findings, check state and outages.
- Custd keeps the latest check and the twenty most recent outages for each domain, not a history of every check.