What does artificial intelligence see when it watches political ads?
Source: Kalev Leetaru
Two actors pose for stock footage that can be used in political ads. Karen O’Connell, left, and Leslie Luxemburg pretend to chat over coffee. In a political ad, this clip could be used to illustrate any number of topics. (Marvin Joseph/The Washington Post)
Two weeks ago, the Internet Archive launched its new Political TV Ad Archive, which monitors television stations in 20 markets in eight U.S. states to compile a list of 2016 primary-election advertisements and uses audio fingerprinting algorithms to automatically flag every airing of those spots.
The project will create a public database of political television ads in the 2016 race, where they are running and who is paying for them. Since the end of November, the archive has identified 267 distinct ads that, if broadcast end to end, would total 196 minutes. They have aired a collective 72,807 times on the stations it is monitoring.
To analyze the data, we can draw on artificial intelligence and especially “deep learning” neural network computer algorithms. These tools analyze the visual content of advertisements in the same way we are able to analyze text.
To explore this possibility, I converted all 267 ads into a sequence of images, one per second, and fed them through the new Google Cloud Vision API. The Vision API uses Google’s deep learning algorithms to tag each image with the major objects, activities and themes it depicts, extract any recognizable text, estimate the geographic location it captures, and identify the presence and emotional expression of any human faces.
At present the Vision API is still experimental and entirely automated. It is not perfect, and any errors represent algorithmic errors, not editorial statements. Moreover, the Internet Archive is monitoring only a subset of American television stations and does not always catch every advertisement, so the results here are not exhaustive. With those caveats, here are some initial findings.
Politics is about people and that certainly plays out in these advertisements, with 7,230 seconds (62 percent) of airtime featuring people. In all, 2,172 seconds (18 percent) featured at least two people and 1,152 (10 percent) featured three or more people.
[Donald Trump’s attacks on Muslims brought him more media coverage than ever]
Nearly a third of the airtime featuring people was of candidates talking with groups of potential voters, while the rest focused on a single person, either the candidate or someone supporting or criticizing the candidate. New Jersey Gov. Chris Christie (R) holds the record for the most people in a single shot, totaling 40 faces at one point.
The Vision API also estimated that about 0.4 percent of faces showed anger or sorrow, while nearly 14 percent showed joy, in keeping with the global prevalence of joyful expressions in news imagery.
Textual overlays are very common in political ads, with at least 7,000 seconds (60 percent) of air time containing a textual message of some form, ranging from large font bold overlays to the tiny font disclaimers at the end of ads.
[Politicians play the race card. This is what helps neutralize it.]
It is common for ads to run a brief message to frame the context of a video clip appearing immediately before or after. For example, an American Crossroads video opens with red text over a black background saying, “What is it with Hillary Clinton lying about terrorists and videos?” before playing a clip of Clinton speaking at the CNN Democratic Debate and then a CSPAN clip of the mother of one of the Americans killed in the attacks in Benghazi, Libya, in 2012.
Textual overlays also are frequently used to summarize or emphasize particular statements made by the candidate, such as the words “Strengthening the economy” that appear as Democratic presidential candidate Hillary Clinton says them in this Hillary for America ad.
Logos are somewhat common in political ads. The Vision API recognizes 538 distinct logos appearing across 986 seconds (8 percent) of airtime. The most frequently appearing logos tended to be those of major news organizations, including Fox News Channel, CSPAN, Fox Business, CNN and the New York Times. These tended to be clips of the candidate or an opponent from previous news interviews or speeches. Nationally recognizable locations are featured in 178 seconds of video (1.5 percent), with the top two locations being the U.S. Capitol and the White House.
The advantages of automation are numerous. It would require an army of volunteers watching nearly a hundred television stations 24/7 to identify every airing of every political ad. Similarly, although a single person could watch the current archive of three and a quarter hours of ads, creating second-by-second annotations would have required a very large team of human coders.
This is why the Internet Archive’s audio fingerprinting and Google’s algorithms are useful. This is the big data future of understanding political advertising on television.
To learn more, you can download the JSON data and run your own analyses.
Thank you to Google and the Google Cloud Vision API team for the use of their tools and the Internet Archive’s Television News Archive for making the ad archive available.
| }
|