Innovating towards the traditional - Facebook & Amazon

Just a short post now - and really just two observations/trends I have commented on before that are continuing their trajectory.

Amazon bring grocery delivery to the UK - Announced today, Amazon will be bringing their "Fresh" service to the UK, starting with London and Leeds - continuing with my previous observations on innovating towards the traditional.

Facebook messenger no longer needs a Facebook account - this is older news, and I have been meaning to comment on this for a while. Basically, customers can now sign up for and start using Messenger without an account, just using a phone number, which is an un-surprising move towards phone as identity and recognising that the phone has the potential to be a node in any social connected network on the platform (much like I have preached about before..).  It is also another step towards Facebook moving further down the stack and more directly competing with native messaging apps as a system level.

Hello Facebook..

I have been away on a mini-break for the last few days, and whilst relaxing I was mentally planning out a blog post about mobile phones as identity and the social platform they provide (again!), then, I woke up this morning to Facebook's announcement of their Hello app.  In short the app will attach FB user details to phone numbers, so if you get a call from a mobile number you don't know, but have the app installed on your phone, it will attempt display FB info to identify the caller.

For my money, this is a very smart move.

Moving down the stack

One of the questions for the future mobile landscape is whether players like Facebook and Amazon can move "further down the stack" - that is, to start being more core to devices, or making their own. Amazon have tried with their Kindle Fire phone, and Facebook had Home - both bombed out, however, and there hasn't been much to suggest that there is more promise for them.

However, Hello seems to me like it could be a small step to bridging that gap.  The app is simple in itself, and done well I could see it being very popular - after installing, it doesn't need any active engagement from the user, but (hopefully - without having tried it myself) just sits there and enhances the users phone experience (as long as they don't try and do anything stupid, crazy annoying notifications, battery hogging etc) - As a simple enhancement to the incoming call screen, I can imagine pretty high retention/install rate - because why would you go back to unknown numbers?

One reason this could be a huge win for Facebook is that this step could make Facebook more core to the mobile device (and seemingly appear more and more like just part of the OS).  If you have the app installed, and miss a call, I would expect the missed call notification screen to be Facebook's Hello - with the normal stuff you would expect - so and so called you, return call, send message etc - which will inevitably give FB the hook to drive users to FB messenger, WhatsApp etc - This is a chance for FB to hijack the core functionality of the phone and drive traffic to its other properties, quickly FB messenger could become the default messaging app.

Your phone as your identity & network

This is something that I have ranted about before - there are two powerful and fairly unique traits of mobile phones that have the power to really disrupt the social/technology/online space:

Your phone is your identity - Whilst people continue to struggle to fix online identification/authentication issues, with a host of different providers offering Login-as-a-Service (Twitter, Facebook etc) and lots of people complaining about the proliferation of user logins/passwords that need to be remembered, the phone offers an interesting solution to that - Granted, phone numbers are not for life, but they offer simple, unique identification of individuals, plus, with two-factor authentication, it can be simple to verify that the person attempting to login is in physical possession of the device.  And with more moves to ApplePay and NFC payment with your device, this identity/device becomes even more tightly coupled.  

By Facebook tying your phone number with your profile it is an impressive move to help future proof their platform: so your phone is your identity? Well now your FB profile is intrinsically linked to that too.

Your phone as a social network - For a long time, Facebook have been the dominant force in social networking. Despite Google's effort with Google+, they haven't been able to displace the social network king. Do you think they struggled because they couldn't match Facebook's superior product? You think people stay at Facebook because they just love the features so much? Of course not. They stay because of user inertia - the feature that keeps users at Facebook is the network effect, no one would move to Google+ without the rest of the network, and how to you move your network?  As if to prove this, there are fairly regular outcries from the Facebook masses when FB changes their layout, or people find out about some terrible clause in the privacy policy - but do people leave? Nope. They just get on with it and keep using Facebook.  After all, the users are the real product, not the software.

Now, mobile phones change that a bit.  Traditional online social-networking basically just models the real world network of friends/colleagues/etc - In your real life social network, you are the node in the graph, and your mobile phone basically represents your node - If you think of Facebook as a centralised social network, mobile phones are a distributed social network (in other words, Facebook knows about the entire graph, and keeps that information centralised, but with your phone you know about your immediate neighbours in the graph - e.g. the people you have contact details for, but not the rest of the graph).  This is a pretty powerful idea - and this is what has made it possible for apps like SnapChat and WhatsApp to enjoy sudden and un-precedented user growth where non-mobile only apps (Google+) haven't.

If we add to that the fact that modern smart phones aren't just an address book - they have your photos, they have your videos, they have your messaging, they have your friends - is there much on your Facebook profile that you don't have (the ability) to have locally on your mobile phone?

All of this makes the smart phone and the real world mobile connections a very powerful platform for anyone to come and disrupt Facebook's crown - So this move by Facebook again just helps establish them as leading the mobile-social platform, not just an old internet company trying to compete.

An aside - Google needs to catch up

My original thinking for the blog was going to be about how Google should, in my opinion, be capitalising on the Android platform as a social network, and rather than trying to convince users to sign up to Google+ (or Wave, or Buzz etc) focus their efforts on making Android a more social OS and make more of the Android as your identity.  They would have to do it carefully with regards to stuff like "private by default" so not to cause uproar, but if they could effectively make all their Android users a node in the social network, then all of a sudden they could find themselves with a pretty good social network, and can start thinking about how to add value to the product.  Furthermore, with Android being the OS (as low down the stack as possible) then they would have a lot more options with regards bringing together all the aspects above - Your smart phone knows instantly when you take a photo or record a video (because you are doing it with the device!), it knows your location, it knows when/who you are messaging, it knows who your friends are - some nice usability around that and it starts to sound like a very strong value proposition!

Nerdability: A retrospective

In 2011, I came up with the idea to build a web based platform that lets developers and technologists build better CVs.  The idea was that as a technologist, we have a prety big online footprint that represents our skills/interests/experiences much more than a traditional CV that just rattles off academic achievements and work experience.

As a technologist applying for jobs, I wanted to make my CV stand out, and figured I have a StackOverflow profile that could demonstrate my knowledge & communication skills, I had Google-Code/GitHub/BitBucket projects which showed examples of my code, OSS contributions, technologies I have used plus then LinkedIn, blogs, Coursera courses, Geeklist.. the list goes on - and more importantly, these were things that were all changing much faster than my traditional CV was.

On the flipside of that, as an employer trying to recruit technologists is really hard - CV screening doesn't rule out many candidates, and beyond that you are left to navigate technical tests & interviews which are fraught with difficulties in trying to really assess whether someone is actually fir for a job.

Nerdability started life as an idea that was entered in a cloud competition to build a webapp (some details here originally called NerdStar) - which it won.  I then recruited two co-founders and we re-wrote a bunch of it (originally all data persistence was mongo, but switched that to relational) and launched it to the public!

In January this year (2015) we retired the application, we had somewhere in the region of 800 users signed up (with no active/paid marketing).  We have currently parked the page and a basic user profile tour on and are in the process of moving this holding page back to

Things that went well

There was a lot of great things we learnt/that came out of the project, here are some highlights:

We contributed to OSS - During development I created a Spring-Social implementation for Khan Academy API and for the GeekList API - both of which are now listed on the Spring site as community projects.

The content driven marketing - As mentioned, we didn't do any marketing, just a few tweets, etc - but the big win was the blog. We setup a basic blog over at (still up and running!) using blogger for hosting and just posted tech articles - tutorial type stuff plus more link-bait-y type stuff like new years resolutions for developers, or getting hackathon projects into production - and the blog drove lots of our traffic.  The beauty was the people who visited/discovered our blog were a perfect match for our target demographic, so we just had plenty of placements for nerdability and the users just trickled in!  The link-bait stuff would give us spikes of traffic, but the tutorial/how-to stuff still drives a few thousand unique views a week.

User response - The user response was good, we had a fair few early adopters who liked what we were doing and blogged/tweeted positively about the project.

Things that were hard

Competition - The weekend we were due to launch, I got an early beta-invite to StackCareers - with an almost identical value proposition, but they already have money and a massive developer community.

Two-sided marketplace problem - so we started getting users signed up, but there was little engagement as users couldn't really do much with their profiles - we had not companies/jobs listed, so beyond just sharing their profile links there was not much they could do.  We were working on the company integration when we finished, but didn't get into it.


In the end, the three of us weren't at the right place/point in time to go into the project full time (family stuff, wrong career stage etc). However, I still believe there is a good business model in the idea, and still think that even if it was setup today it could be successful.  Despite the community and backing of StackCareers, I don't think they have really nailed it, and seem to have stagnated at a slightly improved jobs board - but not really using the intelligence/information to really improve the recruitment process and take any burden of employers/interviewers (which is where I think one of the big wins is for the model)

The joyless world of a data-less startup

I recently read "The Joyless World of Data-Driven Startups" over on medium.

It's a nice article, and there are plenty of good points made - but my concern with the article is that it champions data-less decisions without covering the times when data-less or gut-instinct decisions are not appropriate.  In many ways it mirrors some conversations I have had over the past year or two regarding the benefits of data and using the Build-Measure-Learn approach from the Lean methodology.

I actually think the article hits the nail on the head, but really this post is to address the fact that some people may read what they want from it (and use it as an excuse not to measure).  I think really, the main take away point from the article is this:

Data-driven innovation sucks.

That is really what is at the crux of the article, using data to drive innovation is as close to paint-by-numbers as you can get.  Think about it, innovation is by its very nature a creative process - its the act of creating something new, or something that tackles a particular domain or problem in a new and original way - obviously not something that can be done formulaic-ally by numbers.  Creating a new and exciting product has to be creative by nature. Creating a clone of an existing product/service is not (that's not to say that it wouldn't still be a viable business though!).

The problem is, in the past I have ended up in lots of conversations as a proponent of Lean and the B-M-L iterative approach to attempting to better understand the solution and problem domain and have faced this very argument - that its OK that we aren't measuring result correctly(if at all!), and its OK that we are releasing multiple changes, and pulling lots of levers, all at once (thereby making data we do measure difficult to tie back to single changes), because just look at Steve Jobs! He didn't bother about asking what users wanted! he just gave them his vision!

And that is the main distinction I would like to highlight - there is a big difference between using data to drive your creative/innovative decisions and using data to learn from your innovations.  Yes its OK to be creative and out-of-the-box with innovation and new product features - that's how we got so many amazing products that we have today - but you need to be ruthless in your learning and understanding of the data.

To the Steve Jobs example - yes, he was a visionary and he ignored data/user research when designing products - Jobs/Apple are famous for their approach of just giving customers what they decide they want, and with great success - but they still use the data.  You think if the iPhone was a dismal failure they would have ignored that data and not changed it? You think Apple haven't killed off products that haven't performed well?  These are fairly extreme examples, as in both cases the data is very visible (e.g. sales figures) that people might not think of as post-product learning, but really its the same thing.

Essentially, the bottom line is whilst data-driven innovation sucks, data-driven learning is essential.

That's my opinion, anyway.

Spring MVC - Page caching and If-Modified-Since

Spring (and in general Java/groovy etc) support a range of caching layers for your web application - with options for Hibernate first/second level cache, Spring's @Cacheable and simple integration with lots of providers (EHCache, Redis, etc), but aside from server side caching, you can also implement page level caching using the standard If-Modified-Since cache headers.

If your web application is sitting behind an apache web server, then using this mechanism, you can set it up such that Apache will do a lot of the work, and a lot of requests never even trouble our web application.  This will of course only work if you have fairly static pages that don't update to frequently, and aren't user specific (e.g. so its actually possible to cache at a web-page level - user account pages obviously are harder to cache at this level as they are very user specific).  Even without Apache in the mix, most modern browsers are built to handle If-Modified-Since headers and 304 response codes, so using the approach can mean that browsers are less eager to even request the page from your server whilst a user is browsing the site if we have said its cache-able.

Thankfully, the machinery for this stuff is all baked into Spring MVC.

Updating the RequestMappingHandlerAdapter

This walkthrough is assuming that your app is using standard @Controller annotations and a standard RequestMappingHandlerAdapter to route the requests - although there are still relatively easy mechanisms to do this if you are using other Controller/HandlerAdapter patterns.

The RequestMappingHandlerAdapter class has a method that can be overridden called getLastModifiedInternal() - This method simply returns a long value (the epoch time) that the requested resource was last modified. All we need to do is extend the RequestMappingHandlerAdapter and implement this method to return a timestamp.  For example:

The above assumes we have initialised a timestamp at startup (easy to do using Java config) and assumes that if you have visited the site since last app startup, then there is no change (in reality, we will likely need something more complicated to calculate this)

And that's really all we need, as Spring will handle the rest for us - from this, if a brand new request comes in with no If-Modified-Since headers, then Spring will take this date/time stamp and return it with our response as the Last-Modified date (HTTP standard header - this will inform the browsers/web servers action next time).  If however, a user has already visited your site and the browser has received a valid Last-Modified timestamp, then on the next request it will include this value in the request If-Modified-Since header, when Spring recieves this request, it compares the timestamp against the value that is returned from our getLastModifiedInternal() method, and if there has been no change then Spring will automatically return the response to the client with a 304 response - so none of the Controller code will be executed.

As you can probably guess, this can provide huge efficiencies and improvement on latency, server overhead etc.

More complex Last Modified Dates

As mentioned, in all likelyhood you will need a more sophisticated mechanism than just checking the application start time - So we have plenty of options here: we could query the DB for changes, we could have other flags/properties set for when particular resources are set (bare in mind that if static resources like CSS are changed, these need to be considered)

If you need to consider a fairly unique LastModified date for every endpoint, then this solution isn't a good fit, as this is the more generic approach - you can alternatively implement a similar last modified method on your (every) controllers.

Another option that I have considered is a halfway compromise - where I need endpoints to have specific considerations, but I only have a set of 4-5 (or relatively manageable) specific queries/checks that need to happen, and every endpoint will just need to check some subset of these conditions - this solution involves custom annotations and marking up Controller methods with this annotation to signal to our HandlerAdapter what checks should be considered for the Last Modified timestamp.  This has the advantage of relatively little intrusion in all my controller endpoints, but granular enough to provide enough control for effective page caching.

A custom annotation

First we define a simple annotation that can be used to mark up Controller methods to indicate which conditions are important to the endpoint:

The above code shows the annotation code, a sample enum (just to provide some type safety whilst using the annotation - this could be a string or anything else) and an example of the annotation on a controller endpoint.

Updating our Last Modified method

Now, in our last modified method, we have access to the Handler (e.g. the controller method being invoked) so we can quite simply check the controller method for our annotation, and if found we can just check the values said and perform the appropriate checks to work out what the last modified date needs to consider:

As mentioned, this won't work if there are lots of pages with lots of different requirements, but if you have a manageable of changing entities in your application this can strike a nice balance between clean code and flexibility.

Spring Boot - Tomcat error handling

The default (maybe even recommended?) behaviour for Spring Boot is to package your web application as a JAR with a bundled Tomcat - following the increasingly popular pattern of "Application Servers are Dead" - a more extreme version of one-app-per-app-server pattern I guess.  However, by and large I am still using Tomcat instances (although usually still one-app-per-server) so I am building WAR files for deployment.

So error handling.  Normally, in our web.xml we would define errorPages for response codes, exception types etc, that Tomcat would be able to forward any response code/exception that comes from your application back into an end-point within your application so you can display a custom error page and handle any additional logging etc.

Now for Spring Boot (and Spring 4 generally) there is a shift away from XML and a lot of effort has been put in to make it possible to create web apps with out a single XML file in sight.  This includes the error page mappings - so if you want to define custom mappings you can simply create an error configuration file as so:

Simple right? Now that all makes sense if we are running tomcat bundled with our app - we can use this configuration to configure Tomcat and everything would work as expected - but what about if we are deploying a built WAR file to a Tomcat instance? How can Spring be configuring Tomcat's error page mapping?  To me it just seemed too magical, but as you might expect, there is a simple answer:  ErrorPageFilter - This is just a default Filter that is added that intercepts all requests with error codes/exceptions and forwards it to the correct endpoint.  From the comment at the top of the Filter:

* A special {@link AbstractConfigurableEmbeddedServletContainer} for non-embedded

* applications (i.e. deployed WAR files). It registers error pages and handles

* application errors by filtering requests and forwarding to the error pages instead of

* letting the container handle them. Error pages are a feature of the servlet spec but

* there is no Java API for registering them in the spec. This filter works around that by

* accepting error page registrations from Spring Boot's

* {@link EmbeddedServletContainerCustomizer} (any beans of that type in the context will

* be applied to this container).

A simple solution that will work for standalone JARs or deployed WARs.