Skip to main content

Monitor Your Website for Search Problems and Outages

What you get​

Custd audits each domain you verify once a week. The audit checks what Google, Bing and AI search tools (ChatGPT, Claude, Perplexity, Copilot) need: that pages answer, can be crawled and indexed, carry a title, description, canonical address and structured data, show their text without JavaScript, and load quickly. Each problem becomes a finding with the page, what was seen, how to fix it and a source to read.

A finding stays open until an audit no longer sees it, then it shows as fixed. If it comes back, it reopens.

Custd also checks each verified domain's home page every five minutes and its HTTPS certificate every six hours. Two failed checks in a row open an outage; the first check that passes closes it. Outages, slow responses and certificate problems are findings too, under the uptime.* and tls.* rules, and they are fixed by the checks themselves rather than by the weekly audit.

Find your way around​

Site Insights in the top navigation lists every site you added, the one that most needs you first. Each row says whether the site is up, one sentence on what is left to do, and its first action as a link straight to where it is done.

A site's page opens on Overview: one sentence on whether the site works and how much is left, when it was last checked and what Custd cannot check yet, then the three most important actions, each with why it matters and one control (View incident, Go to the finding, Check again, Connect search or Reconnect). Below them, one line each says whether the site is working, how search reaches people and how much is left to fix, with a link to the detail.

The detail sits in four more views, chosen from the tabs under the site's name (on a phone, the View list): What to fix (what the latest audit changed, the findings, every page read and a Tools fold with campaign links), Search (search numbers, with countries and devices, keyword rankings, telling search engines a page changed, AI visibility and competitors as folds that open on request), Reliability (uptime and certificate, page speed and site traffic) and Reports. Settings, beside the site's name, holds only setup, in three groups: Connections (Google, Bing and the IndexNow key), Checks and Business details. Each view's address can be copied and shared with a colleague who has access, and the browser's back button returns to the view you came from. Links to the old sections, such as #uptime or #competitors, still open the view that holds them.

1. Add your domain​

Open Site Insights, choose Add site, paste the site's address (a full link such as https://www.example.com/pricing is fine: Custd keeps only www.example.com) and choose Add site. Custd says which Site the address goes to: the Site that already holds the same address with or without www., a Site named after it, or your only Site. To pick another, open Add it to another Site. An address you already added is not added twice; the dialog links to it.

The site's page then opens with the proof steps and a Getting started list: add the site, prove you own it, the first check, and connecting search (optional). Each step says where it stands from what Custd has recorded, so you can leave and come back to it. The list goes once ownership is proved and the first check has finished.

If your organisation has no Site yet, adding the first address creates one, named after the address without www., and stores its page views in a destination of its own. This needs a tenant administrator. The Site starts with no environment, allowed origins or usable write key, which only the browser script needs: to install it later, set those in Admin → Sites and rotate the write key there.

Or from the command line:

custdctl sites domains add --profile acme --host www.example.com

Without --site, Custd places the address by the same rules; if it cannot tell which of several Sites you mean, it answers site_choice_required and you add --site <site-uuid>.

2. Prove you control it​

Custd fetches nothing on a schedule until you prove the domain is yours. Do either one:

  • add a DNS TXT record at _custd-verification.<your host> with the value shown, or
  • publish a file at https://<your host>/.well-known/custd-verification.txt containing the token shown.

Then choose Verify now, or run custdctl sites domains verify <domain-uuid>. If the check fails, the page shows what each proof returned. The file must be served directly: Custd does not follow a redirect for it.

Verifying a domain also verifies its subdomains in the same account: once example.com is verified, www.example.com and shop.eu.example.com are too (custdctl sites domains show prints verified by parent and the parent's UUID). It never works the other way, and myexample.com is not a subdomain of example.com. If the parent is removed, subdomains verified only through it stop being verified.

3. Read the findings​

The first audit runs within a few minutes of verification, then weekly. Check whole site (or custdctl sites domains check <domain-uuid>) queues another audit for the next run. After fixing one page, Check again on its finding re-reads just that page; the finding reads fixed only once that check saw it gone. When the same problem is on several pages and the check can confirm it, Check all N pages again re-reads each of them and says what it saw, such as "3 of 5 fixed, 2 still open". A site gets 30 page checks an hour; if they run out, the screen says how many pages were queued, and you check the rest later.

With Search Console connected, a page's own screen also lists Searches that show this page: Google's busiest searches for it over the newest 28 days (up to 20), where the page already uses each, title ideas built from them and your brand name, and other pages of the site that mention one and could link to it. The same searches can be counted in a draft before you publish. Until Search Console is connected or the first import finishes, the screen says so instead.

Copy for my developer copies the problem as plain text for whoever edits the site: the pages, what was found on each, the code to add and how to check the fix from the public page. It holds no Custd login, link or id, so you can paste it into an email or a ticket. What it holds shows the text before you send it.

What to fix lists the open findings, most serious first. Use Show to see fixed or dismissed findings. If an audit finding is intentional, open its rule and select Dismiss rule for all URLs on this domain, or Dismiss this URL for one affected page. Enter a reason. Use Show → Dismissed to review the reason and restore the selected scope. Audits keep recording what they observe. A restored finding returns to Open if it is still observed.

From the command line:

custdctl sites findings --profile acme <domain-uuid>
custdctl sites findings --profile acme --state all --json <domain-uuid>

A fixed finding has Search before and after the fix: its page's clicks and impressions from each connected search provider over the same number of days (28 by default) before and after the day an audit first saw it fixed. A window reads "not enough data" until it has ended and been imported, or while days or the page's rows are missing; it never shows that as zero. Other changes and ranking updates move these numbers too, so the difference is not proof the fix caused it. custdctl sites impact --profile acme --days 28 <domain-uuid> <finding-uuid> reads the same.

To see what to do first, open the domain's Overview, or read its next actions: whether the site is up, what is not checked yet, and its three most important actions in a fixed order (prove ownership, an outage, a certificate that is invalid or expires within 14 days, refused search access, serious findings by pages affected, the first audit, connecting search, then findings worth fixing). The portfolio summary gives every domain's status and first action, worst first; the Site Insights list shows it. The client report lists the same three actions.

custdctl sites domains next --profile acme <domain-uuid>
custdctl sites domains portfolio --profile acme

A product that shows every Site of a tenant, such as a hub, provisions a read-only credential for these two reads with the site-insights-read scope template; a site-bound credential reads them for its own Site only.

high findings stop a page being found or read at all; fix those first. info findings are choices, such as blocking an AI crawler, not mistakes.

To be told when an audit opens a new high finding, set an email address or your webhooks in the Alerts form on the Site Insights page, or run custdctl notification destinations set --event site_finding_opened --channel email --email seo@acme.example. Custd sends at most three alerts per audit, and none for a domain's first successful audit, which records what is already there.

Optional AI suggestions in Awthy​

AI starts Off. If your company has own-key access, open Manage AI settings in Custd from Awthy Hub's Reporting → Your sites in search as the account owner. Read the DeepSeek consent notice, then choose Save consent if you agree. Enter or replace your DeepSeek API key in Custd and choose Save AI settings. Reopen the settings from Hub if the session has expired.

A supported finding such as a missing page title can show a suggested explanation and next step in Awthy → Search & speed. Hub shows confirmed and estimated provider costs and the monthly allowance. In Custd, Turn off AI and remove key stops suggestions. Revoke consent and remove key withdraws provider permission as well. If the key was already removed, choose Revoke consent.

4. Uptime, certificate and warming​

The domain's Reliability view starts with Uptime and certificate: one status (Up, Failing, Down, Checks paused or No recent check), the last check with its HTTP status and response time, the certificate's expiry and issuer, and the recent outages. From the command line, custdctl sites domains show <domain-uuid> prints the same state and custdctl sites incidents --profile acme <domain-uuid> lists the outages.

A check passes when the home page answers with a status from 200 to 399 within ten seconds, following up to three redirects. The findings are:

RuleSeverityWhen
uptime.downhighan outage is open
uptime.slowmediumthe home page takes more than three seconds on average
tls.invalidhighthe certificate is expired, untrusted or for another host
tls.expiring-soonhighthe certificate expires within seven days
tls.expiringmediumthe certificate expires within thirty days

Choose how the domain is checked in the domain's Settings → Checks (network probes are under Advanced: network probes), or custdctl sites domains checks --profile acme --mode <mode> <domain-uuid>:

  • uptime (the default): the home page every five minutes and the certificate every six hours.
  • warm: the same, plus one anonymous home page request every minute, so a site that goes cold when idle is kept ready for its next visitor. The page shows the average response time without warming and with it, so you can see what warming buys before keeping it on.
  • paused: no requests at all. Custd cannot tell that a site is stopped, so whoever stops it sets the domain to paused; nothing then wakes the site until the mode is set back. HaakCo Hosting does this for a paused environment.

To verify a public recovery without waiting for the regular interval, select Reliability → Public network uptime → Check public uptime now or run custdctl sites domains uptime-check DOMAIN_UUID. The request is queued for the next scheduler tick. The displayed status changes when the check finishes. The domain must be verified with uptime checks enabled.

For a site reachable from an internal network, open Reliability → Private network uptime and select Set up private probe. Enter a name that identifies the network. Save the one-time credential and report URL shown after setup in the probe's configuration. Run the probe where it can reach both the internal site and the Custd report URL. Configure your service manager to supply the credential in CUSTD_PRIVATE_UPTIME_TOKEN, then run:

custdctl sites run-private-uptime \
--report-url 'https://custd.com/api/v1/site-private-uptime-reports/company/monitor' \
--token-env CUSTD_PRIVATE_UPTIME_TOKEN \
'https://internal.example.net/'

Replace the example URLs with the report URL from setup and the internal site's HTTPS URL. For CLI setup, run custdctl sites domains private-uptime-create --network Office --credential-file /secure/probe-token DOMAIN_UUID. The command saves the one-time token in the new private file and prints the report path. Use the Custd API origin plus that path as the runner's report URL.

Keep one runner active per monitor. It checks once a minute by default and polls for Check private uptime now requests every 15 seconds. From the CLI, run custdctl sites domains private-uptime-check DOMAIN_UUID to queue a check or custdctl sites domains private-uptime DOMAIN_UUID to see the latest result. The panel shows the network name, observation time and result. A site can be Up from inside the network while its public check shows Down. In Private probe settings, rotate the credential to replace it or remove the monitor to stop receiving its results. The CLI command custdctl sites domains private-uptime-rotate --monitor MONITOR_UUID --credential-file /secure/new-token DOMAIN_UUID revokes the previous token and saves the replacement. Update the runner's service configuration with the replacement token and restart it. To retire a monitor, run custdctl sites domains private-uptime-remove --monitor MONITOR_UUID DOMAIN_UUID and stop the runner. Removal revokes its token.

From Reliability → Configure network probes, enable DNS and optionally enter the public IPv4 and IPv6 addresses you expect. Reliability shows missing and unexpected addresses after a check. Leave the list blank to check only that the name resolves to public addresses. For automation, use custdctl sites probes --set 'dns:1.1.1.1;2606:4700::1111' with the domain UUID. The --set option replaces the probe list.

To catch an error page that still returns HTTP 200, open Reliability → HTTP expectations. You can check the final HTTP status after redirects, limit the time to response headers, require public text in the response, or match the exact string value of a top-level JSON field. Saving blank fields restores the default HTTP 200–399 check. Reliability shows the reason for a failed expectation without storing the response body.

In Reliability, Domain registration shows the public registry's expiry date, its source, and the last check time. If no recent date is available, the section shows Expiry unknown. Within 30 days of the published date, Custd opens a renewal finding and uses the configured finding alert destination. Confirm the actual renewal date with your registrar.

For a scheduled job, open Reliability → Scheduled jobs → Add heartbeat monitor. Enter the job's name, five-field cron schedule, timezone, and grace period. The monitor shows the next expected run and whether the latest scheduled occurrence is waiting, on time, late, missed, or failed. The setup panel displays the report URL and a one-time write token. Store the token in the job's secret manager. You can also manage monitors through the Admin API. Send a JSON report to POST /api/v1/site-heartbeat-reports/{companySlug}/{monitorUuid} with the write token in the Authorization: Bearer header.

{"scheduledFor":"2026-11-20T00:00:00Z","outcome":"succeeded"}

Use the intended scheduled time, even if the job started late. Report failed for a failed job. An exact retry returns the original receipt. Once a newer run has reported, an older report returns a conflict. Rotation replaces the credential immediately. Configure the existing Site outage alert destination to receive one message when a job misses its deadline or reports failure, and one when a successful report clears the incident. The alert links directly to Scheduled jobs.

For automation, use custdctl sites heartbeat list, create, rotate, or remove. The create and rotate commands save the one-time token to a new --credential-file. Load the token from your secret manager into an environment variable for the scheduled job. Use custdctl sites heartbeat report --report-url URL --token-env ENV_NAME --scheduled-for TIME --outcome succeeded to submit the result.

For planned public downtime, open Reliability → Maintenance windows and enter the start and end in UTC. During the window, Custd pauses public uptime, certificate, warming and network checks, and shows the window's end time. Checks resume on the next scheduler tick after the window ends or is cancelled. Cancelled windows stay in the list.

For automation, use custdctl sites maintenance schedule with --request-file containing windowUuid, startsAt and endsAt. Use RFC 3339 timestamps and reuse the same UUID when you retry the same window. Use list to read the domain's windows and cancel --window WINDOW_UUID to stop one. Pass the domain UUID after the flags.

During an open outage, select Acknowledge outage in Reliability to record that it has been seen. Checks continue and the next passing check closes the outage.

To be told when a site goes down and when it is back up, set the Outage alerts form under Site Insights settings on the Site Insights page, or run custdctl notification destinations set --event site_incident --channel email --email ops@acme.example. Custd sends one alert when an outage opens and one when it closes, never one per failed check. A certificate that becomes high (expiring within seven days, or invalid) sends a finding alert to the site_finding_opened destination.

Read real-user speed​

Open Reliability → Page speed → Real-user Web Vitals for Chrome UX Report field data. The section shows the collection period and the 75th-percentile values across devices for LCP, INP and CLS. A metric can be missing even if other metrics are available. Lighthouse results below it show Custd's scheduled tests of the home page.

5. Let a plugin or server manage one site​

A WordPress plugin or a hosting environment should not hold your administrator login, and it should not see your other sites. Give it a site-bound credential instead: it holds site_domains.manage, and it can add, verify, check and remove the domains of one Site and read their findings, checks and outages, and nothing else. Naming another Site, or another Site's domain, answers as if it did not exist.

  • For an existing Site: open Credentials, choose the purpose Site Insights (one Site), enter the Site UUID and create it, or run custdctl sites domains credential --profile acme --company-slug acme --site <site-uuid> --client-id acme-wordpress. The secret is shown once. Deleting the credential keeps the Site.
  • From a server that provisions per site (a broker holding producers.provision, such as a WordPress hub or a hosting control panel): call producer provisioning with "scopeTemplate": "site-insights". Custd creates a Site named after displayName for the credential. Repeating the call keeps the same Site; revoking the producer removes the Site and its domains, so a disconnected install leaves nothing behind.

A hosting provider that already proves ownership itself (a TXT challenge, registrar control or managed DNS) does not need the Custd token. HaakCo can enable its broker for the site-insights-attesting template, whose credential also holds site_domains.attest. That credential attests a domain of its Site with POST /api/v1/site-domains/<domain-uuid>/attestation (or custdctl sites domains attest <domain-uuid>): the domain, and its subdomains, are then verified "by attestation", with the credential and time recorded and written to the audit log. DELETE on the same path (custdctl sites domains unattest) withdraws it; so does revoking the credential or removing the scope from it.

When HaakCo hosts your site but its DNS is with another provider and you cannot publish the record, HaakCo can confirm the domain for you. It then reads verifiedBy: operator. A passing DNS or file check later replaces that, and HaakCo can withdraw the confirmation if it stops being true.

If you look after sites for your own customers, send each Site's finding and outage alerts to the customer instead of to you: set Alert email for this Site under Site Insights settings on the Site Insights page, run custdctl sites alerts --site <site-uuid> --email owner@customer.example (--clear goes back to your destinations), or PUT /api/v1/site-domains/alert-destination with {"emailRecipient": "..."}; a site-bound credential can set its own Site's. Webhook alerts need no override: each one names the domain, so your receiver can route it.

With that credential the plugin calls the same Site Insights operations without naming its Site: POST /api/v1/site-domains with just {"host": "www.example.com"}, then verify, findings, checks and incidents on the domain it got back. In the PHP SDK that is adminSiteDomains()->create(CreateSiteDomainRequest::forOwnSite('www.example.com')) and list(); in Go, SiteDomains.Create with an empty SiteUUID and SiteDomains.List(ctx, "").

6. Search performance from Google and Bing​

Custd can import how a verified domain does in search: clicks, impressions, click-through rate (CTR) and average position, by page and by search query. The domain's Search view shows the numbers, each with its source, the days it covers and when it was last imported. Custd imports once a day and keeps sixteen months, the same as Search Console.

Google Search Console​

Custd reads Search Console as its own service account, so you never share a Google password or approve an app.

  1. Find the service account's address: open the domain's Settings → Connections, where Copy copies it, or run custdctl sites search account --profile acme.
  2. In Search Console, open your property, go to Settings → Users and permissions → Add user, enter that address and choose Restricted permission. Custd only reads.
  3. Connect the domain with the property's name exactly as Search Console shows it: a domain property is sc-domain:example.com, a URL-prefix property is the full address ending in /, such as https://www.example.com/.
custdctl sites search connect --profile acme --provider google \
--property sc-domain:example.com DOMAIN_UUID

Search Console's numbers are final after about three days, so the newest day shown is usually three days ago.

Bing Webmaster Tools​

Bing uses your own API key. In Bing Webmaster Tools, open Settings → API access → API Key, generate a key, and connect with the site's address as Bing lists it. Custd stores the key encrypted and never shows it again; put it in an environment variable rather than on the command line:

BING_KEY='…' custdctl sites search connect --profile acme --provider bing \
--property https://www.example.com/ --api-key-env BING_KEY DOMAIN_UUID

Bing reports weekly totals, so its numbers are marked weekly.

When an import keeps failing​

A failed import is retried by itself, waiting longer each time, up to a day. Once you have fixed the cause (for example given Custd's account access to the property again), custdctl sites search retry --provider google <domain-uuid> runs it on the next scheduler tick instead (POST /api/v1/site-domains/<domain-uuid>/search/<provider>/retry). A connection the provider refused shows as revoked and is not retried: connect it again.

Reading it and the advice​

custdctl sites search show DOMAIN_UUID (or the domain's Search view) shows each source's totals, the busiest pages and queries, and each connection's state:

StateMeaning
pendingConnected; the first import runs within fifteen minutes.
activeThe last import worked.
failingThe provider failed for a reason that may pass (an outage, a rate limit). Custd retries after an hour, then less often, at most a day apart.
revokedThe provider refused access: the service account was removed from the property, the property name is wrong, or the Bing key was revoked. Fix access and connect again.

Custd keeps up to 500 pages and 500 queries per provider per day, and Search Console's countries and devices. To page through all of them, run custdctl sites search rows --dimension country DOMAIN_UUID or call GET /api/v1/site-domains/DOMAIN_UUID/search/rows with dimension (page, query, country or device), and optionally provider, days, offset and limit (at most 1,000). The answer's total says how many rows match. Bing does not report countries or devices, so the answer lists it under unavailable rather than showing zero traffic.

Which search pages lead to goals​

When the Site runs the Custd browser tracker, name the events that count as goals:

custdctl sites value --profile acme --goal signup --goal purchase --days 28 DOMAIN_UUID

(GET /api/v1/site-domains/DOMAIN_UUID/search-value?goal=signup&goal=purchase). Each of the 50 busiest search pages (across providers) shows its clicks and impressions next to the visitors whose first page view in the window was that page and the goals they reached. A visitor may have come from anywhere, not only search. A page nobody landed on has no goal numbers rather than zero, and partial means the goal numbers are incomplete. Goals are read with your own reporting access (a tenant administrator or analyst); without it goalStatus is unavailable and says why. Search queries are not tied to goals, because the providers report pages and queries separately.

Inspecting one URL​

To see how Google has one page in its index, ask for a URL Inspection of a URL on a verified domain connected to Search Console:

custdctl sites search inspect --profile acme --url https://www.example.com/pricing DOMAIN_UUID
custdctl sites search inspections --profile acme DOMAIN_UUID

The Admin API is POST /api/v1/site-domains/DOMAIN_UUID/search/inspections with {"url": "https://www.example.com/pricing"}, and GET on the same path for the latest fifty. Custd asks Google within about fifteen minutes, as the same service account that imports your numbers. The answer shows Google's own values: the verdict (PASS, PARTIAL, FAIL or NEUTRAL), the coverage text, whether a noindex rule or robots.txt blocks the page, whether Google could fetch it, when it last crawled it and as which crawler, the canonical Google chose and the one the page declares, and the mobile usability verdict while Google still reports one. A field Google does not report is left out, never guessed.

Each tenant may ask for 200 inspections in any 24 hours, well inside Google's own limit. A request Google refuses (the URL is outside the property, or the service account may not inspect it) fails with the reason and does not disconnect the domain; a rate limit or outage is tried again up to three times. Inspections are kept for ninety days.

Telling search engines a page changed (IndexNow)​

IndexNow tells Bing, Yandex, Seznam, Naver and the other engines that take part that pages were added, changed or deleted, so they can crawl them sooner. Google does not use it.

In Site Insights, set it up once in the domain's Settings → Connections under Tell Bing and others a page changed (IndexNow): Turn on IndexNow, publish the key file it shows, then Verify key file. After that, submit changed pages from the Search view's Tell search engines a page changed, which also lists what was sent. From the command line:

  1. Give the verified domain a key: custdctl sites indexnow enable DOMAIN_UUID (or PUT /api/v1/site-domains/DOMAIN_UUID/indexnow). The answer names the key and its keyLocation, such as https://www.example.com/0123456789abcdef0123456789abcdef.txt.
  2. Publish a text file at that address holding only the key, then run custdctl sites indexnow verify DOMAIN_UUID (POST .../indexnow/verify). Custd fetches the file from the public internet without following redirects and says what it found.
  3. Submit changed URLs on the domain's own host, up to 100 at a time: custdctl sites indexnow submit --url https://www.example.com/a --url https://www.example.com/b DOMAIN_UUID (POST .../indexnow/submissions with {"urls": [...]}).

Custd sends each batch to IndexNow within about fifteen minutes. custdctl sites indexnow show (GET .../indexnow) lists the latest batches:

StatusMeaning
queuedWaiting to be sent, or to be tried again after a rate limit or outage.
acceptedIndexNow received the URLs (HTTP 200, or 202 while it checks the key). It does not mean any engine has indexed them.
rejectedIndexNow refused the batch and says why. A refused key is marked unverified until you verify it again.
failedIt could not be sent after three tries, or the key is no longer verified.

Whether a page is indexed is up to each engine: use a URL inspection for Google, and the search numbers for both providers. Each tenant may submit 1,000 URLs in any 24 hours, and batches are kept for ninety days. custdctl sites indexnow disable deletes the key and its batches; remove the key file too.

From the imported numbers Custd adds two findings to the list:

  • search.low-ctr: a page shown at least 200 times in 28 days, on the first results page, that fewer than 1% of searchers click. Rewrite its title and description.
  • search.position-drop: a page whose average position fell by five places or more compared with the 28 days before.

They are fixed when the numbers no longer show the problem. Disconnecting a provider (custdctl sites search disconnect --provider bing DOMAIN_UUID) deletes its imported numbers.

Neither Google's nor Bing's API reports AI Overviews, AI Mode or Copilot citations yet, so Custd shows only what the APIs provide.

Tracking keyword rankings​

Rank tracking follows your own keywords through your own Search Console data. Custd does not scrape search results, so it never shows a competitor's position.

  1. Connect the domain to Google Search Console (above).
  2. Track a keyword: custdctl sites rankings track --keyword "custd pricing" --country zaf --device mobile DOMAIN_UUID (or POST /api/v1/site-domains/DOMAIN_UUID/rankings/keywords). The country is a three-letter ISO code or all; the device is desktop, mobile, tablet or all. A Site tracks up to 50 keywords over all its domains.
  3. Read them: custdctl sites rankings show DOMAIN_UUID (GET .../rankings?days=28).

Each daily import gives every tracked keyword one entry per day:

StateMeaning
rankingYour site appeared. The entry names the best-placed page, its average position, and the keyword's clicks and impressions.
not_rankingSearch Console was read for that day and your site did not appear for the keyword.
no_dataNothing was read: Search Console is not connected or failing, the day is not imported yet, or the keyword was paused or not tracked yet.

Search Console's final numbers run three days behind, so the newest day is three days ago. A keyword is read even when it is outside the 500 busiest queries Custd keeps for a day.

When a keyword moves across its alert position (10 by default, --notify-position to change it), Custd sends an alert to the Site's alert address: when it leaves the top 10, stops ranking, or comes back in. The first day it is observed sets the baseline, and one import sends at most three alerts. Pause a keyword with custdctl sites rankings update --paused DOMAIN_UUID KEYWORD_UUID, which stops reading it and stops its alerts; run the same command without --paused to resume. custdctl sites rankings untrack DOMAIN_UUID KEYWORD_UUID stops tracking it and deletes its history. History is kept for sixteen months.

The shared search pool​

The shared search pool is opt-in and off unless you join. Organisations that join pool their Google Search Console searches, and only they can read the pool: give to get. Joining accepts the shared search pool terms for your organisation.

  • What you share: the searches your sites appear for, with how often they were shown and at what position, over the newest 28 days with final data.
  • What anyone in the pool sees: only a search that at least five organisations share, with at least 100 times shown between them. It shows as a range of how often it was shown (100 to 999, 1,000 to 10,000, 10,000 to 100,000, or over 100,000 times) and the share of those at positions 1 to 3, 4 to 10, 11 to 20 and after 20, rounded to 5%, over every country and device. The pool never holds your organisation, sites, pages, clicks or exact numbers.
  • What you get: the pool for any search, and its numbers beside your tracked keywords. The pool covers every country and device, even for a keyword you track in one. A tracked keyword too few organisations share says so instead.
  • Leaving: you stop reading the pool at once, and your searches leave it at its next rebuild. The pool is rebuilt every six hours.

Join or leave under Site Insights settings on the Site Insights page, or custdctl sites search-pool join --accept-terms 2026-11-18 and custdctl sites search-pool leave (PUT /api/v1/site-domains/search-pool with {"joined": true, "termsVersion": "2026-11-18"} or {"joined": false}); a tenant admin or ingest manager makes the change and it is audited with the terms version accepted. A join that does not name the current terms version is refused. custdctl sites search-pool show says where you stand. custdctl sites search-pool queries --search pricing reads the pool (GET /api/v1/site-domains/search-pool/queries?search=&limit=), busiest first; an organisation that has not joined gets 403 site_search_pool_not_joined.

Google Analytics 4​

Custd reads GA4 as the same service account it reads Search Console with, so there is nothing new to share.

  1. In GA4, open Admin → Property access management, add the service account's address (step 1 above) and choose the Viewer role.
  2. Copy the property's numeric ID from Admin → Property details.
  3. Connect the domain: custdctl sites analytics connect --profile acme --property 123456789 DOMAIN_UUID (PUT /api/v1/site-domains/<domain-uuid>/analytics/ga4 with {"propertyId": "123456789"}).

Once a day Custd imports the newest 28 complete days: the busiest 100 landing pages from Organic Search with their sessions, engaged sessions and key events, organic key events by name, and sessions by channel for all traffic. Each import replaces the last. When GA4 recorded no organic key event in those days, key events show as not recorded, never as zero. Ecommerce and audience reports are not imported and say so. If GA4 refuses the service account (it was removed, or the property ID is wrong), the connection reads revoked and Custd stops asking until you connect again. custdctl sites analytics show DOMAIN_UUID (GET …/analytics) shows the connection and the reports; custdctl sites analytics disconnect removes the property and its imported rows.

7. Save and share reports​

Reports on a domain's page opens its Reports view. Your saved reports come first, each with its template, search period, when it was built and who can see it (Private until you share it, or how many live links open it). Below them is the live report, read now: uptime now and over 24 hours, 7 and 30 days with ninety daily bars, the certificate's expiry, the latest desktop and mobile Lighthouse scores, 28 days of search numbers and up to fifty open findings with their fixes.

Create report builds the report from today's numbers and keeps that copy under a name:

  • Executive summary keeps the glance, uptime percentages, speed scores, search totals and the five most serious things to fix.
  • Summary and technical appendix adds the daily uptime bars, Lighthouse timings, the search tables and all open findings, up to fifty.
  • Search period is the last 7, 28 or 90 days.

A saved report does not change until you choose Replace with current numbers, which rebuilds it in place: its links stay and show the new version, and the screen says how many do. Delete removes the copy and stops every link that opened it. A domain keeps up to 20 saved reports, each with its own name.

  • Print or save as PDF prints the report on its own, without the Custd navigation.
  • Download findings (CSV) and Download search queries (CSV) give the rows for a spreadsheet.
  • Share link creates a link anyone can open without a Custd login, for 7, 30 or 90 days. On a saved report the link opens that saved copy; on the live report it opens the report as it is when the link is opened. Copy it when it is created: Custd keeps only a fingerprint of it and cannot show it again. The dialog lists the live links; Revoke stops one at once.

A report never shows the domain's verification token, its Site, check failure details or search connection details. The same operations are on the Admin API (GET /api/v1/site-domains/{domainUuid}/report, /saved-reports and /report-shares) and in custdctl sites reports.

Publish a status page​

Under Reports → Status page, choose which verified domains to show and give each one a customer-facing label. Select Preview status page to see the labels, public uptime and last public check times. Selected components show Maintenance during an active window. The preview is private. Select Publish status page and confirm to make the page available to anyone with its address. Use Copy public link to share it. Unpublish closes that address and keeps the saved title and component labels so you can publish it again.

Readers can use Get status updates by email on the public status page. They confirm their address through the email link. A public update is also queued for confirmed subscribers. Each update email includes an unsubscribe link.

During an outage, Reliability → Post customer update opens the status page editor. Write the customer-facing message, select Review customer update, and confirm to post it. The public page shows the latest 20 messages. Include the recovery in a new message when the service is back.

For automation, custdctl sites status-page show reads the private settings. Use preview and publish with --request-file containing a JSON object with title and components. Each component needs a verified domainUuid and a customer-facing label. Pass the domain UUID after the flags. Use post-update with a message and a stable requestUuid in the request file so retries keep the same update. Use unpublish to close the public link.

What to fix → Tools → Campaign links saves a link to one of the domain's pages tagged with utm_source, utm_medium and utm_campaign (and optionally utm_content). Custd stores the values in lower case with spaces as _, and suggests common source and medium names, so a report does not count "Facebook" and "facebook" as two sources. Copy the link from the list; Archive retires it and keeps it on record. When the site sends page views to Custd (the browser script or an SDK), visits through a tagged link show under that campaign in your web analytics reports. The same operations are on the Admin API (/api/v1/site-domains/{domainUuid}/campaign-links) and in custdctl sites campaigns.

8. Tell Custd about the business​

Search research needs to know the business behind a site, not just its pages. Each Site keeps a project context, shared by all its domains:

  • up to 5 brand names people search for it by;
  • up to 10 competitor domains you name, such as rival.example;
  • up to 10 markets, each a country and a language, such as South Africa in English (en-ZA).

Open a domain's Settings → Business details and fill in Project context, or run custdctl sites context --site <site-uuid> --brand Acme --competitor rival.example --market en-ZA. Each list flag replaces only that list, so --market af-ZA --market en-ZA keeps the brands and competitors; --clear removes everything. The Admin API is GET and PUT /api/v1/site-domains/project-context?siteUuid=<site-uuid> with {"brandNames": [...], "competitorDomains": [...], "markets": [{"country": "ZA", "language": "en"}]}; saving empty lists clears it. A site-bound credential reads and sets only its own Site's. A competitor must be a plain DNS name, a country an ISO 3166-1 alpha-2 code and a language an ISO 639 code; anything else is refused with the reason.

9. Research keywords and competitors​

Once the project context names competitors, Custd reads public data about them and about your own verified domains. It runs when your Custd operator has turned it on; until then everything below reads as pending.

  • Page inventory. Each domain's page URLs from its sitemaps and the Wayback Machine, with where each was found and the first and last day it was seen. A site whose robots.txt refuses Custd, or that publishes no sitemap, shows that source as unavailable. custdctl sites competitors --site <site-uuid> shows every domain, and custdctl sites web-pages --site <site-uuid> --domain rival.example pages through its URLs.
  • Popularity band. Each domain's band on the daily Tranco list (top-1k to top-1m). unranked means outside the top million; unknown means not read yet. It is a band, not visits.
  • Topic trends. Monthly Wikipedia pageviews for your brand names and your keyword lists' seed terms, from the Wikipedia of your first market. no_article means Wikipedia has no such article; it does not mean nobody is interested. custdctl sites trends --site <site-uuid>.
  • Keyword lists. Up to 50 named lists of up to 500 keywords each. Repeats that differ only in case or spacing are dropped. Lists belong to the Site, so disconnecting and reconnecting Search Console keeps them. In the Search view, under Competitors and links → Keyword lists, New keyword list saves one (a name and one keyword a line), and Edit and Delete change the list shown. custdctl sites keywords --site <site-uuid> --name Trail --seed "trail running" --keyword "trail shoes" saves one, --delete removes it, and custdctl sites clusters --site <site-uuid> --name Trail groups its keywords by shared word with your own search numbers for the last 90 days. A keyword your search connections never reported shows no numbers, not zero. With Bing connected and a market set in the project context, Custd also asks Bing's keyword research about each saved keyword in your first market once a week: the clusters read adds Bing's searches in the latest week as a separate number, and a list's seed term gets the keywords Bing relates to it. A keyword Bing reported nothing for has no number, and with no market or no Bing connection the read says which is missing.
  • Content gap. Topics your competitors publish pages about that you cover with neither a page nor a search query. Each topic lists the competitors and pages, with their sources and days. custdctl sites gap --site <site-uuid> --seed "trail running" keeps only pages about the seed. Until Custd knows words from your own pages or queries (a home page alone has none) the gap reads unavailable and lists nothing. partial means an inventory behind it is still running or stopped at its page limit.
  • Links. Who links to one of your own or competitor domains, from three sources, each total labelled with what it covers and when it was read. bing: the inbound links Bing Webmaster Tools lists for an own domain you connected to Bing, read weekly. commoncrawl: the domains linking to it in Common Crawl's quarterly domain graph, up to 1,000 best ranked, with its own harmonic centrality and PageRank positions. crawl: links Custd's audits of the Site's other domains saw. lost lists the domains that linked last quarter and no longer do; it needs two quarters read. A source that cannot read a domain shows unavailable, and one not read yet pending, never zero. custdctl sites links --site <site-uuid> --domain www.example.com.

The Admin API reads are GET /api/v1/site-domains/web-domains, web-pages, topic-trends, content-gap, keyword-clusters and referring-domains (?domain=<host>); lists use GET, PUT and DELETE /api/v1/site-domains/keyword-lists. All take ?siteUuid=<site-uuid>, and a site-bound credential reaches only its own Site.

Prepare a sourced brief​

  1. In Site Insights, open Settings → Project context and set your market. Google Trends currently covers South Africa, the UK, and the US. For a technical audience, select Include Hacker News topics for this Site’s technical audience.
  2. Save the phrases your readers use in a keyword list. Open Search → Emerging topics to see which public topics match those saved terms.
  3. Open the original source and check its date and relevance. Expand Prepare a brief to copy the topic, matched term, date, and source links into your outline.
  4. Choose one question your readers need answered. Add a dated observation from your own work and link to an existing page that helps the reader take the next step.

For example, if a saved term about web performance matches a browser release, read the release notes before proposing an article. Run a check on a page you can discuss publicly and record the date, device, and result. Use that observation to explain what readers can check on their own pages. A trend match shows recent attention. Search volume and rankings come from the separate keyword and search reports.

Export your data​

custdctl sites export --output www-example.json <domain-uuid> writes one domain's and its Site's Site Insights data into one JSON file: findings open and fixed with when each was first and last seen (up to 10,000, most severe first; the file says when there are more), outages, the 90-day uptime history, every measured page's Lighthouse runs for 90 days, keyword rankings for the sixteen months kept, URL inspections, IndexNow, saved reports with their content, campaign links (archived ones too), project context, keyword lists, competitor web domains, topic trends and every stored day of search clicks, impressions and positions. The file must not exist yet and is written readable only by you. Sealed Bing API keys and report share secrets are never exported. Access-log traffic, the latest audit's crawled pages and links, page facts, competitor page lists and network probes are not in the file; read them with their own commands. Run it once per domain to export a whole tenant (custdctl sites domains list).

The Admin API reads are GET /api/v1/site-domains/<domain-uuid>/export and GET /api/v1/site-domains/<domain-uuid>/export/search-metrics?offset=<n>&limit=<1-10000>, which pages the daily search metrics in a fixed order; the export's search.metricRows says how many there are.

Removing a domain removes everything kept for it, and deleting a Site removes its domains and its project context, keyword lists, web data, topic trends and alert destination. Offboarding a tenant removes all of it.

Limits​

  • An audit reads at most 200 pages, one request at a time, and skips paths your robots.txt disallows. When it stops at that limit it marks nothing fixed, because the pages it did not read may still have the problem.
  • Only public addresses are audited; a domain that resolves to a private address is refused.
  • Removing a domain removes its findings, check state and outages.
  • Custd keeps the latest check and the twenty most recent outages for each domain, not a history of every check.