Showing posts with label NLP. Show all posts

Using OpenNLP for Named-Entity-Recognition in Scala

A common challenge in Natural Language Processing (NLP) is Named Entity Recognition (NER) - this is the process of extracting specific pieces of data from a body of text, commonly people, places and organisations (for example trying to extract the name of all people mentioned in a wikipedia article). NER is a problem that has been tackled many times over the evolution of NLP, from dictionary based, to rule based, to statistical models and more recently using Neural Nets to solve the problem.

Whilst there have been recent attempts to crack the problem without it, the crux of the issue is really that for approach to learn it needs a large corpus of marked up training data (there are some marked up corpora available, but the problem is still quite domain specific, so training on the WSJ data might not perform particularly well against your domain specific data) and finding a set of 100,000 marked up sentences is no easy feat.  There are some approaches that can be used to tackle this by generating training data - but it can be hard to generate truly representative data and so this approach always risks over-fitting to the generated data.

Having previously looked at Stanford's NLP library for some sentiment analysis, this time I am looking at using the OpenNLP library. Stanford's library is often referred to as the benchmark for several NLP problems, however, these benchmarks are always against the data it is trained for - so out of the box, we likely won't get amazing results against a custom dataset. Further to this, the Stanford library is licensed under GPL which makes it harder to use in any kind of commercial/startup setting. The OpenNLP library has been around for several years, but one of its strengths is its API - its pretty well documented to get up and running, and is all very extendable.


Training a custom NER

Once again, for this exercise we are going back to the BBC recipe archive for the source data - we are going to try and train an OpenNLP model that can identify ingredients.

To train the model we need some example sentences - they recommend at least 15,000 marked up sentences to train a model - so for this I annotated a bunch of the recipe steps and ended up with somewhere in the region of about 45,000 sentences.

As you can see in the above example, the marked up sentences are quite straight forward. We just wrap the ingredient in the tags as above (although note that if the word itself isn't padded by a space either side inside the tags, it will fail!).

Once we have our training data, we can just easily setup some code to feed it in and train our model:

This is a very simple example of how you can do it, and not always paying attention to engineering best practices, but you get the idea for whats going on. We are getting an input stream of our training data set, then we instantiate the Maximum Entropy name finder class and ask it to train a model, which we can then write to disk for future use.

When we want to use the model, we can simply load it back into the OpenNLP Name Finder class and use that to parse the input text we want to check:

So, once I had created some training data in the required format, and trained a model I wanted to see how well it had actually worked - obviously, I don't want to run it against one of the original recipes as they were used to train the model, so I selected this recipe for rosemary-caramel millionaire shortbread, to see how it performed, here are the ingredients it found:

  • butter
  • sugar
  • rosemary
  • caramel
  • shortbread


All in all, pretty good - it missed some ingredients, but given the training data was created in about 20 minutes just manipulating the original recipe set with some groovy, that's to be expected really, but it did well in not returning false positives.


In conclusion, if you have a decent training set, or have the means to generate some data with a decent range, you can get some pretty good results using the library. As usual, the code for the project is on GitHub (although it is little more than the code shown in this post).
Continue Reading

Sentiment analysis of stock tweets

Having previously wired up a simple spring app with Twitter to consume their tweet stream relating to last year's Rugby World Cup - mostly just to experiment with the event-driven programming model in Spring and Reactor - I thought on a whim, why not see if I can find some nice sentiment analysis tools to analyse the tweets, so rather than just consuming the number of tweets about a given topic, I could also analyse if they were positive or not.


Now, that probably sounded like a fairly glib comment. And to be honest, it was: sentiment analysis is very hard, and the last time I looked most efforts were not up to much. Added to that, to make it actually effective, you need some pretty specific training data - for example, if you had a model trained using this blog and then tried to apply that to another sort of text - say tweets - then it's most likely not going to perform well.  Tweets are particularly different as people use different language, grammar and colloquialisms on twitter (in part due to the 140 chars limit) compared to normal writing.

But still, I had my laptop on my commute home on the train, so I figured why not see if there are any simple sentiment analysis libraries that I could just drop in and run the tweets through.  Sure the resulting scores would likely be way off, but it would be an interesting experiment to see how easy it was (and if done, could we then find a decent training set to re-train our model so it was more accurate at analysing tweets).


A quick google later and I came across Stanford's Core NLP (Natural Language Processing) library, via the snappily titled "Twitter Sentiment Analysis in less than 100 lines of code!" (which seemed just as flippant as my original suggestion, so seemed like a good fit!).  Surprisingly, it was actually just as easy as I had hoped that it might of been! The libraries are nicely available in the maven repo, coming with a pre-trained model (albeit trained on film reviews) and are written in Java.  A lot of the code is taken from the approach outline in the above article and the Stanford Core NLP sample class, but its pretty simple and I managed to process a few thousand tweets last night having set it all up on my commute and analyse the sentiment (producing wildly in-accurate sentiment scores - but who's to know, right?!)

(I switched to streaming stock related tweets - mostly just so I could include references to Eddie Murphy in Trading Places)

Updating our dependencies

I will skip the normal app setup and Twitter connection stuff, as I was just building this on top of the app I had previously done for the RWC (which already connected to the Twitter streaming API and persisted info to Redis.

All we need to do here is add the two Stanford dependencies - you can see I also added a dependency for Twitter's open-source library - this provides tweet cleanup/processing stuff, and really just used to extract "cashtags" (like a hashtag, but starting with a $ used on Twitter to indicate stock symbols, e.g. $GOOGL etc).


Spring configuration

Next up, as we are using Spring its super easy to just add the configuration so we can let Spring manage our Stanford NLP objects and inject them into our service class that will have the code to analyse the sentiment

Now we have told Spring to manage the main Stanford class we need and the simple Twitter Extractor class. For the StanfordCoreNLP class we are passing in some properties for what text analysis we want to use (this can usually be done with a properties file, but I was feeling lazy so did it programatically - you can see details of which Annotators are available here: http://stanfordnlp.github.io/CoreNLP/annotators.html )


Next up, based on the code examples we have seen, we need a little bit of code to analyse a piece of text and return a score - So I created a simple Spring service called SentimentService that I later wire into my event listener.

That's mostly it really, in my event listener instead of just persisting the tweet along with its labels I also run the analysis and also save the score.


 (Analysis of a couple thousand tweets - an average score plus number of tweets for each symbol)

As always, all the code is available on GitHub, so feel free to fork it and play yourself (and if you manage to find a training set to accurately analyse tweets then let me know!)
Continue Reading
Subscribe to: Posts (Atom)