Did you know that cPanel has a back end user–space API? I’ve yet to see a web front end user dashboard that’s reliable and consistent.
Some of you were curious. Text assistants and bots are booming, as shovelers seek the god oracle and magnum opus. Here’s the numbers for a naked robots.txt. Inferences, up to you.
In (30 day) levels of aggression, with the most aggresive/inefficient crawler at the bottom;
- Applebot/0.1 [20.1 ]
- heritrix/3.4.0-20200304 [21.8 MiB]
- YandexBot/3.0 [28 MiB]
- GPTBot/1.0 [30.1 MiB]
- IonCrawl [31.6 MiB]
- CCBot/2.0 [40.6 MiB]
- DataForSeoBot/1.0 [51.2 MiB]
- SeznamBot/4.0 [58.6 MiB]
- facebookexternalhit/1.1 [64.1 MiB]
- Google-Read-Aloud [86 MiB]
- PetalBot [91.3 MiB]
- Barkrowler/0.9 [98.6 MiB]
- SemrushBot/7~bl [141.1 MiB]
- BLEXBot/1.0 [152 MiB]
- AhrefsBot/7.0 [159.5 MiB]
- MJ12bot/v1.4.8 [285.2 MiB]
- bingbot/2.0 [373.4 MiB]
- Googlebot/2.1 [535.2 MiB]
- Bytespider [543 MiB]
- DotBot/1.2 [806 MiB]
- Amazonbot/0.1 [2.6 ]
Another wandering soul whispering into the void. If you are looking for my blog you are in the wrong place. The profile and header pictures are brought to you by cdd20.
Rule #1 of finding technical feeds; type ‘OPML’ into GitHub search. GitHub is like a social network for software engineers and I’m a conservationist of time.
Another wandering soul whispering into the void. If you are looking for my blog you are in the wrong place. The profile and header pictures are brought to you by cdd20.
Are you a safe kid? I’ve just had a memory of a past time in a library.