Playing with Twitter streams
I’ve updated a POC I made two years ago, about mixing and streaming some Twitter searches with Play Framework.
The new version handles Twitter authentication through OAuth. The number of parallel Twitter search queries is dynamic and the results are pushed to the browser in real time through Server Sent Events. Instead of using the Play Web Service API to call the Twitter REST API, this version uses Twitter4J and the Twitter Streaming API (the connection stays open to retrieve new arriving Tweets).
MixedTweets github project
All the code is available in this mini project. The old version using Play WS API is available here.
Let’s see what the code looks like.
We define a stream method in our Controller :
For each query, we create a stream (an Enumerator in the Iteratee API). Then we can
reduce the streams into only one mixed stream using the
The last two lines will be explained later.
TwitterStreamListener class, we create an Enumerator from a Twitter search. We use Twitter4J to handle authentication and Twitter searches (via the Twitter streaming API) :
Concurrent.broadcast is useful to feed an Enumerator via an input channel.
When a new message arrives, we push the Twitter status into the channel (see
The resulting Enumerator contains tuples of search queries and Twitter status.
Adapt the content with an Enumeratee
An Enumeratee is a kind of adapter in the Iteratee API. We will use it to transform the results sent to the browser. The results will be converted into JSON values :
Finally we can stream the result over SSE using an
EventSource Enumeratee, with a
text/event-stream Content-Type :
Now we just have to make the stream alive on the browser (see
Note : To configure the Twitter API client you need to declare an application on the Twitter apps page. Then use your credentials as follows :