updated 05:47 pm EDT, Mon July 28, 2014
Dating site owns up to experiments, claims that's just 'how websites work'
Dating site OkCupid took to its blog today in a small defense of the outrage over Facebook's study involving manipulation of users' emotional states through data on its news feed for "psychological research." In a post titled "We Experiment on Human Beings," the dating company proceeded to make light of the data situations, while owning up to several of its own experiments.
In late June, information surfaced on a non-scientific study that was conducted by Facebook researchers that wanted to see if emotional states could be transferred online. Over a week in January 2012, 689,000 people had their news feeds manipulated to filter out content based on positive or negative words.
Facebook violated its own privacy policies (that were then altered after the fact to manufacture "consent") in conducting the study, which was handled by third-party "researchers." The "data scientists" who conducted the experiments did no pre-screening to ensure that underage children or mentally-ill people were excluded from the experiments, did not gain explicit consent for participation, nor did any follow-up studies to assess any harm or benefit that may have resulted from the manipulation.
The research exposure caused an Internet uproar, which would eventually lead to probes into the company being targeted by watchdog groups, a member of the United States Senate and the Federal Trade Commission.
OkCupid cofounder Christian Rudder authored the post, which points out that Internet experimentation is a common occurrence. Standing on the idea that the company "doesn't really know what it's doing" along with any other website, Rudder defended the idea that experimentation is how the good ideas are sorted from the bad.
"But guess what everybody: If you use the Internet, you're the subject of hundreds of experiments at any given time, on every site," said Rudder. "That's how websites work," he added, not distinguishing between marketing studies and psychological experiments.
Rudder detailed three of the "more interesting experiments" the company has run in the course of figuring out better experiences for users. One study, which was conducted January 2013, involved the removal of profile pictures to promote the launch of a blind date app. The company found that users were more willing to reply to the first message from someone if they weren't able to judge them based on their looks. After turning pictures back on, the site saw a decline in follow up conversation.
A second experiment involved the company replacing a rating system based on looks and personality with one based solely on the profile. The company then removed the profile text, and had users vote only on the profile picture in a small user sample. Results indicated that profile information is less than 10 percent of a user's opinion of someone.
In a third and more disturbing experiment, without a listed date of occurrence, OkCupid manipulated the compatibility percentage results for relationship research prediction. In an experiment that took users who were bad matches, and paired them together by telling them they were "exceptionally good for each other," users were found to send more "first messages" than if the results showed a low compatibility score.
However, the experiment went much deeper, as it was looking to see if the results would cause users to like each other based on the suggestion. From what the company was able to gather, it does have an effect. OkCupid then tried the reverse, showing good matches but saying they were actually bad for one another. The result confirmed conversations were less likely to occur when a pair of people were issued a suggestion that they weren't compatible. The company found that their algorithm for matching worked, showing the highest percent of conversations when the actual and stated compatibility was in the 90 percent range.
"OkCupid definitely works, but that's not the whole story," said Rudder. "And if you have to choose only one or the other, the mere myth of compatibility works just as well as the truth." Unaddressed in Rudder's remarks was whether the company had received any negative feedback from users.
While what Rudder says isn't far from the reality of the Internet, it seems that he missed the reason why people are upset over the Facebook study. The problem stemmed from how data was manipulated to actually cause an emotional shift in the states of the users. These users never gave an informed consent, as the company policy was changed after the fact to include using collected data for "research."
Looking into OkCupid's policy, it states that research is allowed with the information collected. It can be used to "perform research and analysis about your use of, or interests in, our products, services, or content, or products, services of content offered by others." Previous to the 2013 experiment, the same verbiage was in place.
The current policy also allows for sharing of "aggregated, non-personal, or personal information in hashed, non-human readable form, information with third parties, including advisors, writers, researchers, partners, advertisers and investors for the purpose of conduction general business analysis, studies, articles and essays media and entertainment or other business purposes."
While this policy statement allows for data to be given to researchers for a number of reasons, it wasn't in place as of October 2012. Another snapshot from June 2013 shows the current policy in place, with an updated date of February 2013. If the policy wasn't changed in the span of a four months, and data was provided to outside sources, the company could face similar backlash to Facebook.
While the experiments were done to collect data for improvement, it doesn't appear as problematic as Facebook's approach. However, the policy does appear to have an out or two, indicating that functions could be performed "as otherwise described to you at the time of collection," as well as be used to enforce anything outlined in the terms and conditions. The way Rudder talks about the research also makes it appear to have taken place in-house.