technave.com Review

 

# but allow only important bots
User-agent: Googlebot
Disallow:
User-agent: Googlebot-Image
Disallow:
User-agent: googlebot-mobile
Disallow:
User-agent: Mediapartners-Google
Disallow:
User-agent: bingbot
Disallow:
User-agent: msnbot
Disallow:
User-agent: msnbot-media
Disallow:
User-agent: Slurp
Disallow:
User-agent: Yahoo-Blogs
Disallow:
User-agent: Yahoo-MMCrawler
Disallow:
User-agent: Teoma
Disallow:
User-agent: Robozilla
Disallow:
User-agent: Yahoo-VerticalCrawler
Disallow:
User-agent: psbot
Disallow:
User-agent: online-webceo-bot
Disallow:
User-agent: Twitterbot
Disallow:
User-agent: Grapeshot
Disallow:


User-agent: Grapeshot
Allow: *
User-agent: Twitterbot
Allow: *

HTTP/1.1 403 Forbidden
Date: Sat, 23 Aug 2025 20:32:03 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: close
Accept-Ch: Sec-CH-UA-Bitness, Sec-CH-UA-Arch, Sec-CH-UA-Full-Version, Sec-CH-UA-Mobile, Sec-CH-UA-Model, Sec-CH-UA-Platform-Version, Sec-CH-UA-Full-Version-List, Sec-CH-UA-Platform, Sec-CH-UA, UA-Bitness, UA-Arch, UA-Full-Version, UA-Mobile, UA-Model, UA-Platform-Version, UA-Platform, UA
Cf-Mitigated: challenge
Critical-Ch: Sec-CH-UA-Bitness, Sec-CH-UA-Arch, Sec-CH-UA-Full-Version, Sec-CH-UA-Mobile, Sec-CH-UA-Model, Sec-CH-UA-Platform-Version, Sec-CH-UA-Full-Version-List, Sec-CH-UA-Platform, Sec-CH-UA, UA-Bitness, UA-Arch, UA-Full-Version, UA-Mobile, UA-Model, UA-Platform-Version, UA-Platform, UA
Cross-Origin-Embedder-Policy: require-corp
Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Resource-Policy: same-origin
Origin-Agent-Cluster: ?1
Permissions-Policy: accelerometer=(),autoplay=(),browsing-topics=(),camera=(),clipboard-read=(),clipboard-write=(),geolocation=(),gyroscope=(),hid=(),interest-cohort=(),magnetometer=(),microphone=(),payment=(),publickey-credentials-get=(),screen-wake-lock=(),serial=(),sync-xhr=(),usb=()
Referrer-Policy: same-origin
Server-Timing: chlray;desc="973d60830ae8d4d8"
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
Vary: accept-encoding
Report-To: {"group":"cf-nel","max_age":604800,"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=%2BTr1Rj3v%2ByRC1rwsYdbEjY0qbqw4mPqak63IGtzuazpOMu%2Bpk9vAN8N0D65yNaH9sTB5mOj2CijR6Ex8rXZkbkQ2%2BDN0HFN64vER"}]}
Nel: {"report_to":"cf-nel","success_fraction":0.0,"max_age":604800}
Content-Encoding: br
Server: cloudflare
CF-RAY: 973d60830ae8d4d8-NRT
alt-svc: h3=":443"; ma=86400