Another potential confounding issue is that multiple hashtags may be applied to the same event. For instance, the #fashionweek tag is frequently joined by #model and #fashion.
So the development teams wrote an algorithm that clusters together hashtags that refer to the same event. It looks at how often hashtags are paired together, such as #equality and #lovewins. It looks at words that are very similar, in order to detect misspellings, so that #valentinesday and #valentineday are clumped together. It also runs an internal tool that classifies tags into a predefined set of topics.
"When we approached trending, we tried to break this project down into smaller problems that could be tackled separately by components with a very specific function. As a result, each individual in our team was able to focus on one problem at a time before moving onto the next one," the researchers wrote.
Sign up for CIO Asia eNewsletters.