Hi, in addition to members-only web-page content, we need to now also add PDF files as members-only content. In our case, these are PDF versions of the members-only articles.
As I understand, MS uses Javascript to hide protected content – and I’m not completely clear whether this means that all protected content is also hidden from search engines. Similarly, will Google be able to find the link to the PDF (and the PDF itself) if it is within the protected part of the content; will it crawl and index any such PDF??
The site is hosted on Webflow, so we don’t have access to a .htaccess file.
A suggestion I’ve found online is to block access through robots.txt
User-agent: *
Disallow: *.pdf # Block pdf files. Non-standard but works for major search engines.
“The problem is that this only prevents Google from accessing the PDF files but does nothing to remove them from its index or being listed in search results.”
This is what I’ve read online.
So, what is the correct and 100% secure way to exclude protected PDFs? We just cannot have these PDFs appear in a search engine or be found any other way.
Thank you.