Let’s say you wish to block all URLs that have the PDF. extension. If you write in your robots.txt a line that looks like this: User-agent: Googlebot Disallow: /*.pdf$. The sign “$” from the end basically tells bots that only URLs ending in PDF shouldn’t be crawled while any other URL containing “PDF” should be. I know it might sound complicated, yet the moral of this story is that a simple misused symbol can break your marketing strategy along with your organic traffic. Below you can find a list with the correct robotx.txt wildcard matches and, as long as you keep count of it, you should be on the safe side of website’s traffic.
The term “organic traffic” is used for referring to the visitors that land on your website as a result of unpaid (“organic”) search results. Organic traffic is the opposite of paid traffic, which defines the visits generated by paid ads. Visitors who are considered organic find your website after using a search engine like Google or Bing, so they are not “referred” by any other website.
×