Personalised education & tech

As I have written about before, I am pretty interested in education.  There seem to be some pretty fundamental flaws in the modern education system, but it still seems unclear to me what exactly a good solution looks like.

The problem with modern education seems to be the whole standardised testing thing, and forcing all children down the standard curriculum path rather than recognising that intelligence and ability takes different forms etc.


In Seth Godin's TED talk, he explains these problems and describes the idea of having personalized education experiences, making a comparison to car manufacture:

Precise, focused education instead of mass batch stuff. That's the way we make almost everything we buy now, right?   It used to be you could have any colour of car you wanted as long as it was black - so we could keep the assembly line going. But now, we make ten thousand kinds of cars, because they can! So we should make ten thousand kind of education.


And that's absolutely right. In both car manufacturing and education, the tools and technology have moved on - we have the technology to cost effectively make loads of car, just like we have the technology, with things like Coursera, Khan Academy, iTunesU etc, to deliver lessons and lectures on pretty much any topic, covered by some of the experts in the field.


He also repeats some of the ideas that Sal Khan talked about, such as "homework at day, lessons at night" - the idea that watching lectures/reading etc should be done in the evenings independently, and then the daytime, classroom based stuff, where there are teachers and real human lead interaction focuses on solving problems, exploring ideas etc.  This idea is great for two reasons: 1) teachers engaging with students on problems, peers discussing ideas or concepts, interactive learning - this seems much more likely to get students enjoying learning new things (which is really a much better bi-product of the education system than just "knowing things")  2) Being able to study independently, reading and watching lectures is much closer to the real life world of work - one of the benefits that is sometimes preached from the home-school camp - having to learn independently makes it much easier to fit into the modern workforce.


Further more, standardised testing creates an environment of graded achievement - parents want to know what their children have achieved, and how they compare to other students or what they have achieved this week - which only serves to put more pressure on teachers to try and teach-to-test, and makes it hard to spend longer period on in-depth study and exploration over an extended period.

2015. Plans & resolutions

As it is a new year again, I have been toying with the idea of some sort of resolutions, but it's hard to come up with anything really meaningful and achievable.

I thought about (and am still thinking about) creating something every month - which has the nice fallback that even during a hectic month, I should have a chance to at least cook something, so even if it ends up being 12 months of just creating some meals, at least its something to aim for.  There are lots of projects that I have either started or keep meaning to get underway which would be good contenders for the monthly fun.

One of the stretch-goals, kind of in hand with the monthly objectives is towards the end of the year it would be nice to setup another company - doing what is still undecided!  Hopefully I suppose the plan is that this will be the result of one of the monthly projects, but let's see.


Possible things on the list..
  • Made of these - a family/private photo app to capture, annotate and privately share photos of your family
  • My lego Pi robot - a year later, I still haven't had a chance to sit down and concentrate on getting even my lego dc motor working with my Pi
  • CREAM - personal finance management app - previously mentioned before
  • a whole bunch of food ideas
  • unknown ideas - still interested in the enterprise, education or public services space


More generally, the aim is to be more pro-active and productive.. I have signed up for the Stanford Machine Learning course on Coursera, that has just started last week - but I'm not sure I will have the time to complete that on schedule (a good start to the year, right?!) so will maybe complete that over a longer time period - I completed the Stanford Algorithms course last year in my own time, and that was pretty good.


Anyway, it's approaching the end of January 2015, so what have I done:

Cooked a bunch, and started a food blog



So yeah. I cooked a bit, made up some recipes, and finally setup a food blog. I am really enjoying that so far, as the blog is helping motivate me to be creative in cooking.


Created/Updated a friends site


An aspiring singer friend of mine recently appeared on the BBC's The Voice (TV show in the UK) and was keen for his site to be updated.  It's just a basic Twitter Bootstrap template, but it was fun playing with js settings and general layout to get the stuff online. Check it out here.

Android: Building a cloud based quiz application

A long time ago, when Android was still in its infancy (1.5 I think..) I built and open sourced a basic quiz app.  The app was just a basic multiple choice question-answer app, driven from some questions in the database, but it did ok - it has had over 10k downloads on the app store, and the blog post tutorial here is still one of the most popular articles.

But here we are, Android 5.0 is released and the state of Android development is very different now. But I didn't really want to just re-skin/tweak the old app and push it out again - and I also wanted to write up some notes on using parse.com as a backend service - so this seemed like a good opportunity.

The source code for the app is all on GitHub.


The goal

So the aim is to create a an android app quiz game, but rather than using local storage, using the cloud to power the questions. This avoids the need for lots of boiler plate DB code and also makes it easier for us to update questions.  The tutorial will be broken into two parts - the first part will be basic quiz stuff with cloud questions, the second part will enhance the app to support user login and to track scores to allow users to compete against each other.


Parse

Before you start the tutorial, you need to get an account setup at parse.com - its a cloud DB/backend as a service that was recently bought by Facebook. They allow real easy setup of DBs and provide a load of nice libraries/APIs to really easily interact with their endpoints across lots of platforms (its also incredibly well priced -the free tiew is really good and if you find your mobile app is going beyond that then you can probably think about monetising the app!).  All you need to do is head over there, sign-up and then create a new app - all you need to do is give it a name and hey presto!  You can either make a note of the keys then, or come back and grab them later. There are no other changes you need to make now, as that will get handled from our source code. The only other thing to do is to download the parse android library to make use of their sdk in android (if you have grabbed the source code from GitHub then you will not have to worry about these)



OK, lets get started!

Again, I am not going to spend a lot of time covering the real basics of Android and only really mention the things of note or that are different from standard application development - hopefully the code & general explanation will be clear enough to get an understanding of what is going on.


Android manifest
First, lets get our AndroidManifest.xml file configured.  The only things to note here are the permissions we are setting - we will request internet access and network state permissions. Also worth noting that I have set the min sdk for my sample app at version 16.


Our Application class
We will have to create a custom implementation of the Android Application class. This class is instantiated on application startup, and hopefully if you are looking at Android development you should be familiar with this class.  We will do a couple of things in this class:

  1. Register our parse.com application with out secret keys
  2. Initialise the Parse library and our domain objects
  3. Try to fetch all the questions for the quiz and store them for offline usage 
  4. Create a GamePlay object, that will keep track of the state of the current game in progress
First lets look at the Parse setup - this is standard parse boilerplate and is covered in parse docs and sample apps - you just need to add your ID/Key here (also note, we have just registered the Parse object class Question - this is our domain object - like a JPA entity etc - if we add more domain objects they need to be added here too)

Next we will make a call to parse.com to fetch the questions from our cloud API - we will save this in the background (make an asynchronous call) and "pin it" to make it available for offline usage. Also note that we do not un-pin existing questions until we have successfully found new ones - that way users should always have questions once they have successfully loaded them the first time.
Hopefully the above is quite clear - the parse libraries are quite straight forward to understand - we create a query (typed Question) and then call findInBackground and implement an on success handler.


Domain objects: Question
Parse library provides a nice interface to create POJOs to model your domain model, if you are familiar with JPA/Hibernate/etc and the approach of POJOs representing a domain model its much like this. Using these classes you can easily query/load/save data from the cloud by just using these objects. You will have spotted that in the query we use in the application class to load all our questions we just run a plain query with the Question class - this, as you should expect, will just retrieve all Question objects from your cloud table (parse). The domain models are just an annotated POJO, and you just define appropriate getter/setters for the fields you want to include.


Welcome activity
Now we are into Android basic stuff really - we have setup parse and fetched the questions for local usage, now we just need a basic menu to start the quiz and some activities to display the questions and the end results.

We will just apply the layout and then implement a method to handle the button clicks. For this tutorial we are skipping the high score stuff and just playing the quiz.
All we need to do is reset the current GamePlay object and grab the questions from the local store (by this point they should be updated from the cloud so no problems, then kick off the quiz!


Question activity
There is nothing really interesting to see here - it's all on github if you want to take a look in detail (or have already downloaded and working along) - This is just a standard Android activity that pulls out the question and possible answers and presents them.




This just progresses along fairly simply, until it gets to the end of the quiz and then it presents a simple screen saying the score - all this stuff can be tweaked/styled etc - but there are the basics for a cloud powered multiple choice quiz app!


Creating the questions

All the questions will be stored in the cloud in parse.com - once you have a look at the interface, it should be pretty clear - you can easily create the data for questions either manually or by importing a csv/json file.



You will need to login to your parse account and the quiz app and create a Question class. This will just match the domain POJO we have created. Just login, go to "CORE" then "DATA" Then select "+ Add Class", adda a custom class called "Question" (this must be exactly the same as the name provided in the POJO annotation for the Question class).. Then select "Add Col" and add the fields to match the POJO (question[string], option1[string], etc).  Once you have the class/table added on parse, you can simply add data by selecting "Add Row" and just manually entering the data, or using the import function.



Source code

Tech: Functional programming in Groovy

I have recently started the Coursera Functional Programming with Scala course (taught by Martin Odersky - the creator of Scala) - which is actually serving as an introduction to both FP and Scala at the same time having done neither before. The course itself is great, however, trying to watch the videos and take in both the new Scala syntax and the FP concepts at the same time can take a bit of effort.

I wanted to work through some of the core FP concepts in a more familiar context, so am going to apply some of the lessons/principles/exercises in Groovy.


Functions:

If you have done any Groovy programming then you will have come across Groovy functions/closures. As Groovy is dynamically typed (compared to Scala's static typing), you can play it fairly fast and loose.

For example, if we take the square root function that is demonstrated in the Scala course, it is defined as follows:



As you can see, Scala expects the values to be typed (aside, you don't actually always need to provide a return type in Scala). But in the Groovy function it is:



A groovy function can be defined and assigned to any variable, thereby allowing it to be passed around as a first class object. If we look at the complete solution for calculating the square root of a number (using Newton's method - To compute the square root of "x" we start with an estimate of the square root, "y" and continue to improve the the guess by taking the mean of x and x/y )


So that's in Scala, if we try Groovy we will see we can achieve pretty much the same thing easily:


Recursion (and tail-recursion):

As FP avoids having mutable state, the most common approach to solve problems is to break the problem down in to simple functions and call them recursively - This avoids having to maintain state whilst iterating through loops, and each function call is given its input and produces an output.

If we again consider an example from the Scala course, with the example of a simple function that calculates the factorial for a given number.

This simple function recursively calculates the factorial, continuing to call itself until all numbers to zero have been considered. As you can see, there is no mutable state - every call to the factorial function simply takes the input and returns an output (the value of n is never changed or re-assigned, n is simply used to calculate output values)

There is a problem here, and that is as soon as you attempt to calculate the factorial of a significantly large enough number you will encounter a StackOverflow exception - this is because in the JVM every time a function is called, a frame is added to the stack, so working recursively its pretty easy to hit upon the limit of the stack and encounter this problem.  The common way to solve this is by using Tail-Call recursion. This trick is simply to have the last code that is evaluated in the function to be the recursive call - normally in FP languages the compiler/interpreter will recognise this pattern and under the hood, it will really just run the code as a loop (e.g. if we know the very last piece of code in the block of code is calling itself, its really not that different to just having the block of code/function inside a loop construct)

In the previous factorial example, it might look like the last code to be executed is the recursive call factorial(n-1) - however, the value of that call is actually returned to the function and THEN multiplied by n - so actually the last piece of code to be evaluated in the function call is actually n * return value of factorial(n-1).
Let's have a look at re-writing the function so it is tail-recursive.


Now, using an accumulator, the last code to be evaluated in the function is our recursive function call. In most FP languages, including Scala, this is enough - however, the JVM doesn't automatically support tail-call recursion, so you actually need to use a rather clunkier approach in Groovy:

The use of the trampoline() method means that the function will now be called using tail-call recursion, so there should never be a StackOverflow exception. It's not as nice as in Scala or other languages, but the support is there so we can continue.


Currying:

This is like function composition - the idea being you take a generic function, and then you curry it with some value to make a more specific application of the function. For example, if we look at a function that given values x and y, it returns z which is the value x percent of y (e.g. given x=10, y=100, it returns the 10 percent of 100,  z=10)

The above simple function is a generic mechanism to get a percentage value of another, but if we consider that we wanted a common application of this function was to always calculate 10% of a given value - rather than write a slightly modified version of the function we can simply curry the function as follows:

Now, if the function tenPercent(x) is called, it uses the original percentage() function, but curries the value 10 as the first argument. (If you need to curry other argument positions you can also use the rcurry() function to curry the right most argument, or ncurry() which also takes an argument position - check the Groovy docs on currying for more info)


Immutability:

Immutability is partially supported in Java normally with use of the final keyword (meaning variables can't be changed after being initially set on object instantiation). Groovy also provides a quick and easy @Immutable annotation that can be added to a class to easily make it immutable.  But really, there is more to avoiding immutable state than just having classes as immutable - As we have functions as first class objects, we can easily assign variables and mutate them within a function - so this is more of a mindset or philosophy that you have to get used to. For example:

The first example is probably more like the Groovy/Java code we are used to writing, but that is mutating the state of the list - where as the second approach leaves the original list unchanged.


Map Reduce:

As a final note, there are some functions in FP that are pretty common techniques - the most famous of which these days (in part thanks to Google) is Map-Reduce, but the trio of functions are actually Map, Reduce(also known as Fold) & Filter - you can read more about the functions here (or just google them!), but these functions actually correlate pretty nicely to core Groovy functions that you probably use a lot of (assuming you are groovy programmers).

Map

map is the easiest to understand of the three. It takes in two inputs - a function, and a list. It then applies this function to every element in the list. You can basically do the same thing with a list comprehension however. 
Sound familiar? This is basically the .collect{} function in Groovy

Reduce/Fold

This one is a bit more complicated to descibe, but is the same as the .inject{} function in groovy

Filter

And another simple one - filtering out a list for desired elements, this is Groovy's .findAll{} function



As I said at the start, I am new to FP and coming from an OO background, but hopefully the above isn't too far from the truth!  As I get further through the Coursera course I will try to post again, maybe with some of the assignments attempted in Groovy to see how it really stands up.


Some useful references:

Tech: Building an RSS reader for Android (RIP Google Reader)

This tutorial will walk through building an RSS reader on the Android platform (focusing on 3.0 + as it will be using Fragments). All the code is available as a complete, working Android app that you can fork/download and fire up straight away on a compatible Android device/emulator. So feel free to go grab that from GitHub before continuing.

It is not an unusual requirement for mobile apps these days to be able to consume an RSS feed from another site (or many) to aggregate the content -  Or maybe you just want to build your own Android app now that Google has announced it will be retiring Reader.

Those of you who have worked with RSS in other JVM languages will know that there are plenty of libraries available that can handle parsing RSS - however, because the android platform doesn't actually contain all the core java classes, almost all of the RSS libraries have not been supported.

Fear not though, as Java's SAX parser is available, so with a bit of code we can get a custom RSS parser up and running in no time!

This walk through will cover off the basics of getting an RSS reader app up and running quickly and will also cover some details of Android's fragment system for tablet optimization as well as some things to watch out for (such as changes in the platform that mean network operations cannot be run in the main thread, which requires some tweaking if you have worked on earlier versions).

All the code for the app is also available on our GitHub so feel free to fork that and try loading it up on your Android device/emulator.

Parsing an RSS Feed:


So to get started we will look at parsing the feed - if you have any experience parsing XML using SAX in Java then you will know how easy this is. All we need to do is to tell the parser which XML nodes we care about, and what to do with them.

If  you have never implemented a SAX parser before, there are three primary methods that we will override: 
  • startElement() - This is called by the parser every time a new XML node is found
  • endElement() - This is called by the parser every time an XML node is closed (e.g. </.. )
  • chars() - this is called when characters are found between nodes



Because we only really care about capturing data from the leaf nodes, our startElement() method is left empty. The chars() element has to be watched, as there is no guarantee when it will be called (e.g. in a node like hello world  this method might be called several times between the start and end) so every time we will just append the contents to a StringBuffer - that way we can be sure that we will have captured all the data in the node.  By the time the endElement() method is called, we know that we have the contents of the node itself, and we just have to store the data.  At this point, we just quickly knocked up a POJO with the attributes that we wanted to capture - the Strings that we match on are the node names from the ATOM RSS feed (that Blogger uses) - if you are using another feed, just have a quick look at the feed and update the node names appropriately.

Using our Feed in an Android App

So, that was easy right? Once that parser has run through (and you could use that code standalone in any java app really) then you will have a list of Java objects that have the core details about the latest blog posts on the feed (title, author, datecreated, content, etc) - So now lets look at using it in an Android app.

We will assume a basic understanding of the Android SDK and the concept of Fragments, so won't go back to basics with that stuff.

What we will do, is create a basic ListFragment and an RSSService class that we will use to populate the list. In our ListFragment we will simply tell our RSS service to populate the list:



Simple right?

Let's take a look at what our helpful RSS service is doing for us.



The first thing to note is that this class is extending Android's AsyncTask- The reason for this is that since Android 3.0, you are no longer able to perform network operations in the main application thread, so being as our class is going to have to fetch some RSS feeds we are going to have to spin off a new thread.

As you can see, the constructor just sets some context info that we will use later on, and then builds a progress dialogue - this is then displayed in the onPreExecute() method - This lets us show a "loading" spinning disk whilst we fetch the data.

Android's AsyncTask's primary method that handles the actual work that you want to do asynchronously is called "doInBackground()" - In our case, this is simple - we just invoke our SAX RSS parser and fetch our feed data:



Finally, we will override the "onPostExecute()" method of the async class to use our newly fetched list to populate our ListFragment.  You note how when we overrode the doInBackground() method earlier we set the return to List of Articles (where Article is my simple POJO containing my RSS blog post info) - well this must correspond to the argument of the "onPostExecute()" method, which looks like this:



Actually, all we really needed to do in this method would be pass our new List or articles to the ListFragment and notify it of the change as below:



However, in our application we have added a bit more sugar on the app - and we have actually backed the app with a simple DB that records the unique IDs of the posts and tracks whether or not they have been read to provide a nicer highlighting of listed blog posts.

So that's it - there's plenty more you can add to your RSS reader app, such as storing posts for reading later, and supporting multiple feeds/feed types - but feel free to fork the code on GitHub, or just download on to your android device to enjoy all our NerdAbility updates!

Application running in Andriod emulator
RSS application running in an Andriod  tablet emulator




Spring security & subdomains

As previously mentioned, I have been working with a Spring MVC app that has had to deal with multiple subdomains for the one app (in other words, the subdomain really needs to just be considered as part of the normal URL path in all routing/security configuration and concerns).

Having gone through the details on how to make the @Controller and @RequestMapping routing to play nicely with subdomains, here is a quick overview of how to handle subdomains in Spring security.


A custom matcher

The main thing we really need to handle with security, is how to configure Spring-security so we can define permissions for URLs that include the subdomain.

Normally, Spring MVC permissions looks something like this:

As you can see, this just specifies a URL path to authenticate.


The specific details of how you implement the matcher exactly will be dependent on your applications approach to identifying and extracting the subdomain (maybe from http request, maybe just use a regex on the request etc)

As you can see above, the matcher we have created is just a convenient wrapper around another two spring matchers to let you match easily on both the full URL and subdomain.

Now, with a little convenience method, we can make some pretty nice Spring security configuration:

As you can see, the subdomain makes a difference to the permissions and who should access the two /dashboard/ URLs in the different contexts, but with the above simple code, we can make some pretty convenient & readable configuration to take subdomains into account.