DoNotLink ‘Hides’ Websites From Search Engines, Gains Popularity In An Anti-Clickbait Age

A New York Post article by Doree Lewak arguing that women should be "flattered" by catcalls caused an online uproar Monday. The article, which defended the behavior of men who whistle, jeer and shout sexual comments at women passing by, drew the ire of many readers -- and the savviest among them have found a way to share their outrage without directly benefiting an outlet that publishes what some see as offensive clickbait.

DoNotLink is a link shortener that prevents an article from being boosted in search results. The service is designed to control for the likelihood that whenever someone links to a website – regardless of the reason, pro or con – it strengthens its position in search engines like Google.

Google’s algorithm for determining search engine placement is an always-evolving mystery, and historically, Google has not been particularly transparent about its mechanics, expect to say that it relies on more than 200 "unique signals" to determine how to display search results. But all observable evidence does point to the likelihood that an article with a lot of links and shares will get a higher ranking than one without. DoNotLink founder Menno van Ens is convinced that “links are the most important factor in how Google and other search engines rank the ‘value’ of websites. The more links, the higher the website will rank.”

Van Ens, a Web developer in Vancouver, Canada, launched the site in 2010 as a hobby project for skeptics to write and discuss homeopathy, psychics and pseudoscience “without giving the websites being discussed any of the search engine benefits that normally come with a link.”

“For example, if someone is writing about what a scam ‘Acme Chiropractic’ is, they can use donotlink to link to acmechiropractic.com without improving that website's search engine position,” van Ens said. “The link to acmechiropractic.com is essentially ‘hidden’ from search engines.”

DoNotLink uses JavaScript to reroute search engine robots through a server which prevents them from crawling to the site and indexing it. More specifically, the tool attaches a robots.txt file to a link telling Web crawlers to lay off. A “nofollow” attribute is also added to the link -- another command for robots to stay away. If for whatever reason, a search engine is able to index the link, it is directed to a blank page.

The page that is displayed after clicking a DoNotLink.com link is fully functional. Users can click on ads and navigate to other parts of the page. The sole difference is a DoNotLink banner is displayed at the top of the browser. DoNotLink is not a big money-maker, and makes just enough from Google's AdSense ads to keep the servers running, van Ens said. “This is more of a public service; As long as it pays for itself (and it does), I'm a happy camper.”

When he launched DoNotLink four years ago, it got “a few visitors every day," van Ens said. Earlier this year, the site picked up steam after Twitter users began to promote it. Now more than 50,000 people visit the site daily, with traffic doubling every three weeks, he said.

Niche online communities -- most notably feminists, men’s rights activists and Reddit users -- have become regular users, van Ens said. This was apparent after Lewak’s article went viral, and many who shared the story on Twitter doing so via donotlink.com, and asking others to do the same.

Despite the momentum gained for DoNotLink in the wake of Lewak’s article, the service is far from mainstream. The website has just 66 Twitter followers and 183 Facebook fans -- a sign the service may be far from making a dent in major news websites like the New York Post. Lewak’s article has generated nearly 2,000 tweets, 8,000 "likes" on Facebook and twice as many comments so far.

A general search on Twitter shows dozens of users sharing DoNotLink.com links to news related to the shooting of unarmed teenager Michael Brown in Ferguson, Missouri. The same goes for news related to the beheading of American journalist James Foley.

“A few links have been posted to articles about Foley's beheading, but those are mainly shared between friends and family on Facebook,” Van Ens said.

DoNotLink seems to be the only link shortener designed to block a link from being crawled. In 2011, Istyosty.com was established as a proxy site for the Daily Mail. The website allowed users to browse the news outlet’s website – known to produce sensationalist stories for clicks – without driving traffic to the site. IstyOsty was shut down later that year after the Daily Mail's owner issued a cease-and-desist order, threatening to demand payments of $150,000 for each cached story if it didn’t comply, Gigaom reported.