Why ISPs are hijacking your search traffic and how they prof
Source: Jolie O Dell
A handful of Internet service providers (ISPs) in the U.S. are redirecting search traffic around specific keywords to brands’ websites, presumably for affiliate marketing revenue.
A study released today by a UC Berkeley research group revealed that for some Internet users on some ISPs, using a search engine and typing in a word such as “apple” or “bloomingdales” would redirect the user to websites for Apple or Bloomingdale’s rather than to a page or search results about the keyword in question.
The Berkeley project, called Netalyzr, was created to measure DNS behavior. However, over the past few months, the Netalyzr team noticed some unexplained and unexpected redirections across at least 12 ISPs in the United States.
In a blog post on the findings, the team wrote, “The affected ISPs use services provided by a company called Paxfire to monetize certain web search requests. Paxfire’s main line of business is DNS-error traffic monetization, i.e., the practice of presenting advertisements and search results to users who mistyped a website’s address in their browser.
“In addition, some ISPs employ an optional, unadvertised Paxfire feature that redirects the entire stream of affected customers’ web search requests to Bing, Google and Yahoo via HTTP proxies operated by Paxfire.”
Following the money
The Electronic Frontier Foundation helped the Netalyzr team investigate the matter. As EFF senior staff technologist Peter Eckersley told VentureBeat, “They knew the general category of false DNS responses might be possible and worth checking for, while the details that emerged about Paxfire and what it was actually up to were a bit more surprising.”
The research team found that around 170 specific, brand-related keywords would trigger interference by the HTTP proxies, causing users to be redirected to affiliate marketing landing pages. “In the process, the ISPs and Paxfire presumably earn commission payments for the redirected flows,” the researchers wrote.
Some of the ISPs involved are, according to data presented by multiple organizations involved in the investigation, Cavalier, Cincinnati Bell, Cogent, DirecPC, Frontier, Fuse, Hughes, IBBS, Insight Broadband, Megapath, Paetec, RCN, Wide Open West and XO Communication. Charter and Iowa Telecom claim to have recently stopped doing DNS redirects.
While it’s likely that ISPs had at least some knowledge of at least some of the DNS redirection, if not search traffic redirection, it’s less likely that the brands themselves were involved in the scheme. “There is probably a chain of several intermediaries in these affiliate marketing programs between the brand itself and Paxfire,” said Eckersley.
In other words, it’s difficult to say at the outset where the buck stops in this scheme and whose hands are in the cookie jar. What we do know is that many of the ISPs involved are claiming a lack of knowledge about the search redirects and pointing to third-party vendors as the real villains in the scenario.
A Charter representative told VentureBeat today that when search traffic redirects were occurring across that ISP, “We were not aware of it. It was a third party, and in a sit-down with the vendor, we said, ‘You need to be more careful about putting us into this mix… Charter doesn’t think this practice is acceptable.”
Steven Crosby of Frontier Communications Corportation told VentureBeat, “In terms of Frontier’s practices, we do not hijack any search traffic. We have clear business rules in our legal agreement with Paxfire that allows them to monetize URL address bar errors (e.g., ‘www.abc.cmo’ instead of ‘www.abc.com’ or typing an actual word like ‘PC’ into the address bar). Paxfire is not allowed to touch any search traffic that originates directly from toolbars or search bars.”
While the Charter rep was not able to name the exact vendor involved, Paxfire is just one of many Internet marketing companies that are using technical architectures for commercial and marketing purposes. These firms, which include companies like Barefruit and Golog, engage in murky practices such as search redirects, practices that violate our expectations of how the web should work and that rob us of any trust we might have in our ISPs.
If you use one of the affected ISPs, the EFF recommends running a Netalyzr test and installing a browser plugin such as HTTPS Everywhere to use HTTPS for all your web browsing “With HTTPS, attempts by the ISP or a company like Paxfire to alter the results would cause a certificate warning,” said Eckersley.
Google has also recommended using Google Public DNS and is beta-testing encrypted web search for users who want to better protect their search traffic.
The problem with Paxfire
“I’m not an expert on affiliate marketing programs, so I can’t comment on whether anything that Paxfire is doing might be a violation of the rules or norms of that business sector,” said Eckersley. But he did say that the marketing company “has no business” granting itself access to the keywords people are using to navigate the Internet.
“If my search engine is untrustworthy or not returning the results I was actually looking for, I can go and pick a different search engine. But if Paxfire has snuck out onto the network and secretly replaced all my choices of search engine with itself, I no longer get to go elsewhere for my searches.”
And when Paxfire’s proxies malfunction, any search attempts return an error message. “Users will often blame the search engine for that, when in fact it’s the fault of the company that’s secretly hijacking them,” said Eckersley.
In the end, said the EFF spokesman, it all comes back to net neutrality and how the lack of neutrality fundamentally degrades the reliability of the Internet. “Programmers assume that when they send data from A to B over the network it will arrive as it was sent. But if in fact the data is transformed by a series of companies that are trying to find ways to make a quick buck, things become more complicated, unpredictable, and fragile.”
The Frontier fiasco
In the ongoing quest to put a stop to deceptive Internet marketing practices, it’s hard to tell exactly where to lay the blame for search redirection and the responsibility for ending it. But Google took the issue upon itself when users were complaining about redirects.
Google’s security teams had been aware of DNS-based traffic interference from ISPs for months, at the very least. Google security engineer Damian Menscher wrote in response to user issues with Frontier back in March, “At Google, we are following this very closely, and trying to get Frontier to fix the issue. The root of the problem is that Frontier is intercepting some traffic, so when you try to use Google your search actually goes through a Frontier server first.”
At that time, entrepreneur and investor Andrew Payne noticed the redirection happening in his own searches. He wrote, “ISPs have redirected DNS queries for a while, but have mostly focused on typos and misspellings. I’ve never seen an example of an ISP actually hijacking a user’s Google search and inserting their own results, and that seems pretty egregious to me.” Menscher recommends users contact Frontier directly about the practice.
With folks like Payne making waves online, Frontier responded directly. Maggie Wilderotter, the ISP’s CEO, told Payne a story similar to the one we heard from Charter today: “that this had been done by one of their vendors in violation of Frontier’s business rules and it’s been shut down,” as Payne wrote. Around May 2011, Payne said Frontier had stopped redirecting Google search traffic, as far as he could tell.
According to the EFF, Google has repeatedly put pressure on ISPs to stop DNS-based redirects and has been at least somewhat successful. However, the EFF notes that Yahoo and Bing search engines are still particularly susceptible to redirects.
“This is why the ISPs that were proxying Google stopped in the past couple of months,” wrote Berkeley researcher Nicholas Weaver in a Slashdot thread today. “Google’s abuse-detection threw up a CAPTCHA on the queries, and then Google posted about it.”
Evidently, the combined noise from the web and pressure from the search engine were enough to put a stop to search redirection in some cases. A Google spokesperson confirmed, “We aren’t aware of any DNS providers that are currently doing this hijacking for searches intended for Google.”
Hopefully, continued pressure and the watchful eyes of the media, Berkeley researchers and advocacy groups like the EFF will help to end the practice of search redirects.
| }
|