Category Archives: case studies

Tomorrow’s election in the US

Yes: we, as many others, have followed the US elections in the social media. There are many measurements of social media mentions out there, some thorough, some others little more than simple counting. (The fundamentals of the actual issues, polls, and electoral mechanisms are best summarized by Peter Norvig.)


Ethersource has been reading social media posts on the main US presidential candidates for the past year or so. Based on this reading, our analysis is that

  • Obama will stay in the White House.

… which appears to be in agreement with what most bookies, pundits, and polls predict today.

As we have shown in previous posts on this blog, we have been thinking hard about which measures best capture political attitude in the social media, and what sort of attitude best translates to prediction of results. We already know that people do not usually waste bandwidth on plain simple endorsements or statements of personal voting intentions, but in general use their space for more or less thoughtful predictions of the candidates’ chances to carry the election. Aggregating these sentiments and opinions gives us a prediction market of sorts, composed on those representatives for the electorate who write in social media. We show here our PPI score – an intensity-normalised positivity index for the two main candidates – since mid-August, in a line graph.

Intensity-normalised positivity for the two main candidates since August

Intensity-normalised positivity for the two main candidates since August

As a visualisation experiment, we can show the same data in a quicktime clip, for the two main US presidential candidates since August, with the X-axis showing positive attitude, the Y-axis the intensity-normalised positive attitude and the size of the ball the frequency of mention for the candidates. (High and upper right corner and large ball: good.)



These data show that the candidates’ mentions appear to track each other well (indicative of a close election) and that the incumbent has the edge. Based on these and our other measurements, we believe Obama will stay in the White house.

What Ethersource has Learned About Al-Qaeda in the Past Few Days

  • This post gives examples of Ethersource’s learning capabilities.
  • It gives examples of automatically learned topics and senses of the use of the term Al-Qaeda in English social media.

Ethersource is continuously exposed to massive text streams. On a given day, it sees millions of blog posts, tweets, and forum posts. And it learns. It gobbles up information much the same way a human picks up new ways of using new language constructs. Ethersource learns how the terms it reads are related to each other. It learns about topicality, and it learns about the different senses of the terms.

As an example, let’s have a look at what Ethersource has learned regarding Al-Qaeda the past few days. Topicality-wise, the texts concerning Al-Qaeda are described by Ethersource using the following terms:

  • radicalization
  • LTTE
  • counterterrorism
  • ideology
  • tamils
  • jihad
  • terrorist
  • liberation
  • authoritarian

To us humans, possessing the background knowledge imposed on us in media over the past decade, these terms come as no surprise. They all make sense as describing Al-Qaeda. Ethersource, however, has learned these topics from scratch, without access to any prior knowledge.

Furthermore, Ethersource has discovered two distinct senses, or meanings, of the term Al-Qaeda, as it has been used in social media during the past couple of days.

  1. The first sense of Al-Qaeda was automatically labelled PKK. In this sense, Al-Qaeda is related to Turkish, terrorists, militants, and fighters.
  2. The second sense of Al-Qaeda was automatically labelled Syria. In this sense, Al-Qaeda is related to Iran, Libya, Turkey, Tunisia, and fighting.

Unsupervised topic detection and sense discovery are both inherent properties of the semantic representation at the core of Ethersource. This makes for a powerful tool for an analyst when forming an understanding of the use of target concepts, be it in brand management, Open Source Intelligence, or sudden swings in World Markets.

We conclude this post with the observation that Ethersource has recently learned a new synonym of “Obama”: Obameat.

A portrait of President Barack Obama made up by meat, by the artist Jason Mecier.

A portrait of President Barack Obama made up by meat, by the artist Jason Mecier.

Tiny Needle in Big Data

Weak signal emission, detection, retrieval and analysis

We are repeatedly asked about the predictive powers of Ethersource and we need to underline that Ethersource has no “predictive” power per se. The reason Ethersource can estimate – or forecast – the percentages of public votes in a television contest or the outcome of a national election with some accuracy is simply that Ethersource reads and understands massive amounts of data.

This post will focus on something slightly different, namely the ability to find, understand and analyse one or a few tiny pieces of crucial data in massive amounts of data. It is the needle in the haystack dilemma with the only difference that your proverbial haystack is the Internet, and that you have very limited time to detect the relevant blog posts, tweets or chat entries and to analyze them in time to take action. This is what we call weak signal detection.

The Yeonpyeong attack

On November 23, 2010, 1434 local time (0534 GMT), North Korea fired more than 200 artillery shells at the South Korean island of Yeonpyeong, killing at least two soldiers in the heaviest attack since the end of the Korean War in 1953. The attack, which was somewhat of a shock, had a substantial but short-lived market impact. A day after the firings, South Korea’s benchmark KOSPI index opened 2.33 percent lower and the KRW weakened against the USD.

However, the surprise attack was preceded – on November 22 – by an alert signalling a sharp increase in the weak signal violence propensity index (VPI) for Korea as a target, monitored by Ethersource:

The Domodedovo bombing

Two months later, on January 23, 1332 local Moscow time (1150 CET), the weak signal detection for Putin showed an extremely sharp increase in the violence propensity index (VPI), triggering an automatic alert:

 

 

Not even three hours later, 1632 local Moscow time (1432 CET), a suicide bombing at Moscow’s Domodedovo airport kills at least 35 people and injures more than 100. The market impact was short-lived: Russia’s rouble-denominated stock market MICEX fell by nearly two percent following the blast.

Just-in-time weak signal detection and analysis

The above examples are from an Ethersource prototype and quite dated. There have been many unexpected events since and the technology has been refined. Our ongoing analysis confirms that many unexpected events are preceded by leakage of weak signals. Such signals are very difficult – or even impossible – to systematically detect with other technologies. It might appear a big step from anticipating some kind of violent act to actually being able to take counter-measures, as this would require both the availability and the timely detection of more detailed information. Ethersource has addressed this challenge by allowing instantaneous and automated ranking, retrieval and analysis of any Internet post or document contributing to a weak signal alert on any sentiment concept, such as violence propensity, toward any given target concept.

Near real-time weak signal detection and analysis holds great promise for security and financial applications, in our view. 

Miserable Monday and the Effect of Vacation in Swedish Social Media

Recently, we found out that Miserable Monday might not be anything but a myth. As avid fans of the idea of a complete banishment of Mondays, it will take more than a couple of news articles to convince us. Luckily, Ethersource is more than ready to clear up any doubts.

For some time, we have been monitoring the Swedish domain of social media, and how people are feeling when talking about themselves. The curves have been steadily working their ups and downs. However, these past few months we have been noticing a very curious occurrence. First, let’s take a look at this graph.

What we are seeing is a curve representing the general happiness of people when speaking of themselves, for a period of time around March earlier this year, measured using an index we call Positivity Propensity Index (PPI). It’s not a particularly exciting graph, other than affirming what has already been stated: People do seem to speak more fondly of themselves when weekends are upon them. But other than that, there doesn’t seem to be any certain weekday that stands out among others. Our previous hard stance against the impartiality of research might have starten to soften up a bit.

Now, let’s continue on to the peculiarities.

This is a graph from the beginning of May until today. For this graph’s sudden change to make any sense, you might need to obtain some background info in Swedish culture, and especially in a holiday called Midsommar – a day full of culinary deliciousness and drinking. This is the peak of June 23 you see, and what happens thereafter seems to indicate that Swedes are no longer slaves of time. Suddenly, Tuesday no longer differs from Saturday, people are generally happier, and the regularities we clearly could see earlier in spring starts to become more clouded. Vacation has arrived.

Swedish social media has yet to return to its normal, moody self. But surely, it seems inevitable.

Winter is indeed coming.

The Severity of the Assange Affair Reaches Year High

Yesterday marked a year high in the number of people airing their concerns regarding Assange in terms of aggression, either toward Assange himself, the Swedish judicial system, or the possible intervention of the UK Government in order to extradict Assange to Sweden. The graph below illustrates that the steep rise in volume during the past 24 to 36 hours diminishes most of the previous on line activities. Although Assange is more or less inseparable from Wikileaks in that he is heavily associated with the organization, at the moment, the public’s subject matter of concern clearly lie with Assange himself.

The volume of aggression voiced in relation to Assange (blue line) and Wikileaks (green line) between January 1, 2012, and August 16, 2012.

The number of social media posts expressing aggression in relation to Assange (blue line) and Wikileaks (green line) between January 1, 2012, and August 16, 2012.

 

 

Greek Election Tomorrow!

The Euro and the European currency union is the major topic of the second Greek parliamentary elections of this Spring, to be carried out tomorrow, on Sunday, June 17.

Ethersource has been reading Greek-language social media for the past few weeks. Our prediction:

  • ND will be leader in number of votes
  • one needs more than tallying frequency of mention or simple assessment of positive vs negative sentiment to use social media for predicting electoral outcomes

We at Gavagai have been following party politics in the Greek social media for the past few weeks and have found – as have the major political commentary sites – that the major players are the conservative ND and the socialist coalition Syriza. After gauging frequency of mention in Greek social media one would be likely to conclude that the election is safely in the hands of Syriza. See the graph below. Syriza gains much more attention in the Greek social media sphere than do other parties. (The dramatic spike in attention given to the fascist party XA has to do with one of their representatives demonstrating practical violence in a TV-debate, punching and slapping a political opponent on camera).

But that is not the entire story. Mentions alone do not translate to votes. A further analysis gives pause to the first prediction. The pie chart shows what proportion party mentions are coloured by mistrust and skepticism.

One cannot predict election results by counting mentions alone – the type of mention is important as well. We have previously cut up attitude in many ways, beyond what is done by most. Here we will look at distrust and doubt as an attitude. Skeptical, worried, and doubtful mentions indicate not propensity to vote but concern about the outcome. The tweets, blogs, and forum posts by Greek voters we read are not simply rooting for the author’s favourite party – they are analyses, each in its own way, of the election outcome. By aggregating the sentiment given in each of them we find a clearer picture than we would by simply counting and tabulating mentions.

Our analysis is as follows: Syriza and ND are most frequently mentioned. Syriza mentions carry a considerable amount of concern and mistrust. We assess this to mean that the electorate will gravitate towards ND rather than Syriza at the polling station: the likely leader in votes will be ND.

How ND will be able to put together a governable majority of representatives is another matter!

Everyday racism in the Swedish blogosphere

  • We use Ethersource to monitor usage of racist terminology in the Swedish blogosphere.
  • We find that one of the largest demographic groups to use such terminology is young female bloggers.
  • We demonstrate how we are able to cluster and profile users of racist terminology.

One of the many benefits of Ethersource is that it is not limited to the standard positive/neutral/negative sentiment palette, but that it can be used to analyze and monitor any type of textually manifested phenomena. Previous examples in this blog include artist popularity, flu trend, aversive language, and positivity vs headache.

In this post, we report on some observations on using Ethersource to monitor racist expressions in the Swedish blogosphere.

The following image shows the frequency of occurrence of racist terminology in the Swedish blogosphere from late March to the end of May 2012. Obviously, racist terminology is a frequent everyday occurrence on Swedish blogs.

Frequency of occurrence of racist terminology in the Swedish blogosphere.

However, merely counting the frequency of occurrence of racist terminology is of limited usefulness for understanding what people say and mean, since there are many ways to use terminology. Some uses may signal ideological or political standpoints, but other uses may not (e.g. discussions about the terminology itself, such as the origin and appropriateness of various terms). Thus, only counting the frequency of occurrence of racist terminology in the blogosphere may lead to premature or misleading conclusions. We therefore also monitor negative or degrading usage of racist terminology, as well as aggressive or hateful usage. And there is a difference between counting frequencies and counting opinionated usage, as we can see in the image below, which shows frequency (in blue), degrading usage (in green), and aggressive usage (in red).

Racist terminology in the Swedish blogosphere. The blue line shows the frequency of racist terminology, the green line shows the frequency of degrading or negative usage of racist terminology, and the red line shows the frequency of aggressive usage of racist terminology.

It is obvious that the total frequency of occurrence of racist terminology is much larger than that of the frequencies of degrading use and aggressive use. As a rough estimate, approximately 10% of the total number of posts containing racist terminology are negative or degrading, while approximately 5% are aggressive or hateful.

The general trends in these graphs are not of lasting value, since the time span is relatively short. What is interesting – and surprising – is the demographic profile of bloggers found in the two bottom lines. Since Ethersource enables an analyst to retrieve individual blog posts which contain a given target (in this case, racist terminology), it is possible to further analyze the material. Looking at the blog posts that use racist terms in degrading ways, we find that roughly 25% are written by young female bloggers who write about their own lives. Perhaps even more surprising, around 10% of blog posts using racist terms in aggressive ways are written by these young females. This is a surprising discovery, considering that the topical content of these blogs revolve around everyday events, lifestyle, and fashion.

Demographic clustering and stylometric profiling

The noteworthy observation above suggests that it may be interesting to look more closely also at the non-opinionated usage of racist terminology (i.e. the occurrences that are neither aggressive nor degrading). We do so by automatically clustering all the blog posts containing racist terminology during 2012. Always keeping the obvious risk of overgeneralizing in mind, we infer from manual inspection of the material that the four main clusters represent following groups of bloggers:

For those not familiar with Swedish internet culture, Flashback is an infamous free-speech-oriented, no-holds-barred, frequently offensive and often provocative discussion forum.

Imagine that we for some reason could not inspect the material manually and therefore did not know the demographics of the clusters we found. In such cases, we can use stylometric profiling to characterize the stylistic differences between clusters, and based on these differences we can infer demographic information. As an example, consider the following comparison between the stylometric profile for the cluster containing the young female bloggers, and the stylometric profile for the cluster containing mainly political bloggers.

Stylometric profiles of two groups of bloggers (young females vs political bloggers) that use racist terminology.

The comparison between these two stylometric profiles shows that the main stylistic differences between these two groups of bloggers (let’s call them group F for the young female bloggers and group P for the political bloggers) can be found in the following variables:

Self
Group F is more self-oriented, which indicates that this group talks mainly about things that happen to the author, stuff the author thinks or worries about, or things that the author does.
Address
Group F refers directly to the reader more often than does group P.
Abstract vocabulary
Group P tends to use more abstract and complex vocabulary than group F.
Anchoring
Blog posts from group P contain more explicit temporal and spatial references than do posts from group F.

These differences suggest that authors in group F (the young female bloggers) write mainly from a subjective point of view, while authors in group P (the political bloggers) adopt a more factual perspective. Based on such differences, we may formulate hypotheses about the demographics of these two groups. This difference would allow us to propose that since the one group writes from a more personal and immediate perspective, they can be assumed to be younger and more personally engaged in their narration than the other group. This characterisation of author style is actually more salient than the objective notion of author age and gender since writing style and authoring background are more interesting for understanding blog posts than the age and gender or other demographich variables.

The analysis and discussion above serves as an illustrative example of how stylometric profiling correlates well with human intuition about demographic clustering, and that such profiles may serve as explanatory constructs for a demographic clustering solution. We conclude this blog post with the observation that the combination of attitude analysis, clustering, and profiling provides a very powerful framework for analysis of online content.

Measuring the popularity of the contestants in the Eurovision Song Contest using Twitter

In this post, we confirm that Loreen is well placed to win the popular vote in the Eurovision Song Contest final 2012.

We have previously shown in this blog that Ethersource monitoring of on-line sentiment can predict the popular vote in certain high-profile media events, such as the national Eurovision Song Contest. In this post, we report on some observations on using Ethersource to measure the popularity of the contestants in the international Eurovision Song Contest, based on analysis of expressions of popularity on Twitter. The following image shows the relative popularity scores of the participating countries.

Popularity of each country

It should be obvious to anyone following the pre-contest speculations about who will win the ESC 2012 that the proportions of popularity in this image do not correlate with current betting odds for the ESC final (the current odds can be found at any betting site). The image shows Ireland and the UK as the most popular contributions in the ESC final (they are ranked 11th and 5th in the current betting odds). One reason for this discrepancy can be that popularity and betting odds do not refer to the same type of measurement; popularity refers to population-wide opinion, while betting odds are estimates of who will win the actual contest (which is determined both by popular and jury votes). Another reason for this discrepancy is the issues identified in commentaries of other recent attempts to predict election votes based on sentiment analysis of the Tweet stream:

  • Twitter users (and users of other social media) do not constitute a perfect sample of the population, which means that measurements based on Twitter may not be representative for the population as a whole.
  • Twitter is a perfect medium for marketers and campaigns, which makes the analysis sensitive to ad-bots and automated Twitter campaigns.

These concerns are of course valid also for the present scenario. However, even more important when comparing measurements based on Twitter analysis across different countries are the following issues:

  • There is a huge difference in population size between the European countries: Russia has a European population of more than 100 million, while Iceland has a population of a mere 300 000 inhabitants.
  • The Twitter penetration (i.e. proportion of the population that use Twitter) is very different for different countries. In the present scenario, where we measure expressions of popularity on Twitter, it means that some countries may get high popularity scores merely because a comparatively large proportion of the population in that country uses Twitter (people tend to promote their own country’s entry in the ESC).

It is somewhat difficult to find recent and reliable estimates of the Twitter penetration per country, but not so recent studies show that the Netherlands, Turkey, UK, and Ireland top the list for Twitter penetration in Europe. Perhaps this explains the results we see in the image above? Scaling the popularity scores for each country by the estimated number of Twitter users in that country produces the following image:

Popularity of each country

When scaling with Twitter penetration, Sweden gets the highest relative popularity score. This is in line with current betting odds, which unanimously rank Sweden as the most likely winner. However, the other countries that receive high normalized popularity scores do not correlate with odds rankings: Greece has the second highest popularity score (ranked 14th place in the odds rankings), followed by Denmark (ranked 8th place), Ireland (11th), and Iceland (7th). These discrepancies may be due to the issues with non-representativeness and Twitter penetration discussed above. We may also add the following issues:

  • The activity level of the Twitter population in some countries may not correspond with the Twitter penetration; Twitter users may be more active in some countries than others.
  • The interest for the ESC may be higher in certain countries than others, thus leading to more Tweets about the contestants from that country.

We conclude this post with the observation that Loreen seems to be the likely winner of the popular vote in the ESC final 2012. We also conclude that attempting to model population-wide opinions based on Twitter analysis is a non-trivial task that requires more than merely counting word frequencies.

Weak signal synonym detection (in Swedish)

As we have previously discussed on this blog, Ethersource constantly and continuously learns new terminology by reading what is written on the Internet. As an example of how Ethersource picks up even weak linguistic signals, we noticed recently that Ethersource suggested the word “tutilurfräs” as a very positive Swedish term. None of us had ever encountered the term “tutilurfräs” before. We looked up the source of this linguistic invention, and found that it originates from a tweet by Swedish punk icon Kajsa Grytt, where she writes that:

A (somewhat creative) translation in English would be something like: “Oh Pelle! Oh Hives! What tutilurfräs!! I think they are genius. That band makes me absolutely happy.”

Quite obviously, Ethersource is correct in its understanding that “tutilurfräs” is a very positive word.

There are two lesson to be drawn from this example:

  1. If you do sentiment analysis in Swedish on Twitter and your model does not automatically learn new terminology, you should re-train or update your model to include the word “tutilurfräs“.
  2. If you invent a completely new word and start blogging or tweeting about it, Ethersource will learn it. It is true that in space, no one can hear you scream, but on the Internet, even if you whisper Ethersource will understand you.

A Minute-by-minute Popularity Contest – Loreen versus Danny

Despite the fact that the Swedish part of the Eurovision Song Contest final was broadcast live, as a TV viewer it was impossible to get a sense of just how popular the artists were at a given point in time. Having access to Ethersource made sifting out meaningful blog posts and Tweets in real-time a breeze! Below are two graphs outlining, minute-by-minute, the popularity of the two top contestants as expressed in Swedish on-line social media for the day of the final (click the image for a larger version). Note that the popularity score of Loreen’s reaches higher during her performance than does Danny’s. In fact, looking at the scale and the contents of the two graphs, it is clear that the expressions of popularity towards Loreen is consequently higher throughout the day.

The popularity of Loreen and Danny Saucedo, measured minute-by-minute during the day of the final of the Swedish Eurovision Song Contest.

The popularity of Loreen and Danny Saucedo, measured minute-by-minute during the day of the final of the Swedish part of the Eurovision Song Contest. The annotations in red denotes the appearance on stage by the two artists.

The timing information for the performances of the artists is available at the official web site of the contest.