Opening the site in your own browser does not prove Google can open it. Google’s robot arrives from outside, is not logged in, does not accept your cookie notice and can be stopped by a rule that never affects you. This article is the technical test: five checks, from the quickest to the most complete, to see the site the way Google does. For the list of possible causes when the site does not appear, see why your site does not show on Google.
The checks
| 1 |
Look at the page’s status code. In a terminal, curl -I followed by the address shows the first line of the response. You want a 200. A 301 or 302 is a redirect; a 403 is a refusal; a 404, a page that does not exist; a 5xx, a server error. See errors 500, 403, 404 and 508.
|
|
| 2 |
Ask for the page as if you were Googlebot. The same command with a Googlebot user agent (curl -I -A "Googlebot" address) shows whether some rule refuses robots. Note: this only imitates the name, it is not the real Google.
|
|
| 3 |
Use URL Inspection in Search Console. Paste the address, and Google tells you whether the page is indexed. “Test live URL” asks it to visit at that moment and shows whether it could, how it saw the page and whether anything was blocked. Search Console is a Google product; you need to confirm the site is yours, for example with a TXT record: proving the domain is yours.
|
|
| 4 |
Read the page indexing report, in the same place. It shows how many pages are indexed and, for those that are not, the reason.
|
|
| 5 |
Look for Googlebot in the server logs. In cPanel, the statistics and access logs show which visitors came and when. If the robot never appears, Google is not reaching you. See visitor statistics.
|
|
What stops Google seeing
| Cause |
Where to confirm |
| A firewall rule or security plugin that stops robots |
A 403 response at step 2. See firewall blocks. |
| A block in robots.txt |
The indexing report and the file itself. |
A noindex tag, or a header repeating it |
URL Inspection: the status mentions “noindex”. |
| Content that only shows after an action |
A cookie window, a compulsory sign-in or content loaded only by JavaScript: the robot sees an empty page. |
| An invalid certificate or a site slow to answer |
Live URL inspection and the speed-measuring tools. |
|
Do not open the door to every robot just to let Google in. If a rule stops it, fix it for the real Googlebot; do not switch the site’s security off. Robots claiming to be Googlebot and not being it are plentiful. A 403 in a test you ran yourself, with a faked name, does not prove the real Google is blocked.
|
|
Keep the result. When you fix a problem, repeat the test and keep the result from before and after. If it fails again later, you will know what changed. And once it is corrected, request indexing of the address in URL Inspection.
|
|
Does your test return 403, a 5xx or no answer? Send us the address, the command you ran and what it returned.
Open a support ticket
|
RECOMMENDED PRODUCT Web hosting with cPanel Domain and SSL included, daily backups and the panel you already know. from 321,75 MT/mo (3-year plan, with coupon) See plans |
Português
English
