Showing posts with label opinion. Show all posts

We need to talk about AI

Ethical and Regulatory questions facing AI





Regardless of area of expertise, most of us are probably already aware of the momentum around Artificial Intelligence (AI). Between self driving cars, home assistants (Alexa, Google Home, et al) and the growing capabilities of our mobile devices there is no escaping the ever looming presence of AI in our lives.

Furthermore, it seems unlikely that this will slow down anytime soon. A recent Narrative Science study found that AI adoption grew by 60% in the last year with 61% of organisations having reported to have implemented AI within their business, and a Gartner report predicted that by 2020 85% of customer interactions will be managed without human intervention.

But despite this growth, there is still a question mark over whether, and if so, how, the field should be regulated. Having been brought up on decades of sci-fi about AI going rogue and robots enslaving the human race, it feels like there is both the fear of this possible future, whilst also scepticism that these fears are only the stuff of movies. Elon Musk has famously warned of the future risks of AI: “I think we should be very careful about artificial intelligence. If I had to guess at what our biggest existential threat is, it’s probably that” whilst others, including Mark Zuckerberg, have downplayed the claims of doomsday scenarios as irresponsible.

So what's the big deal? AI already permeates so many aspects of life and business, but considering for a moment that these technologies could be being used to control autonomous cars on public roads, determine people’s credit score or suitability for a job, to detect illness or even in policing and judicial decision making - it is pretty clear that we should have a good understanding of these technologies and clear systems of accountability and control in place. In all these examples getting a decision wrong has the potential to ruin lives, yet there is still limited regulation, control or even understanding of the algorithms, the data and their usage.

A common analogy is with other heavily regulated industries: big pharma companies can’t release drugs without thorough testing and approval, yet several big tech companies have already started testing autonomous vehicles on public roads with limited regulatory controls (that’s not to say that they have had a completely free pass, there are varying levels of regulation, depending on the region. Arizona has long been promoting itself as an AI friendly state to try to attract business from big tech, making it as easy as possible for companies to test self driving cars with minimal regulatory friction, and they recently saw the first fatality from a self-driving car).

In its 2017 report, the AI Now Institute recommended that AI be outright ban from use in any high risk areas, such as criminal justice, healthcare, welfare and education and further measures for other domains - which given the potential impact of errors in these domains, seems like a fairly sensible starting point.



Uncertainty and the unknown


One key aspect that is especially troubling is the lack of understanding of both the data and the underlying technology. This isn’t necessarily a surprise - we have computers being trained on millions of data points, to the point of being able to outperform humans at their tasks, so it should come as no surprise that both the inner workings and the end results could be beyond easy comprehension.

This problem has been demonstrated by several high profile mishaps from large tech companies, showing that even companies that have a wealth of resources and technical expertise in the domain can be caught out - such as Microsoft’s AI chatbot Tay, who quickly became racist when released into the wild. Clearly Microsoft had neither intended nor envisaged that end result. Similarly, when Google translate revealed gender bias in pairing “he” with “hardworking” and “she” with “lazy” - it clearly wasn’t an intentional or foreseen behaviour, but eventually revealed itself with wider usage.



Understanding where bias in AI comes from


To get a better understanding of where these biases and blind spots come from, let’s take a look at how AI learns. Broadly speaking, there are three primary approaches to training AI: Supervised, Unsupervised and Reinforcement.

Unsupervised learning is where the AI is fed very large amounts of raw data - for example an entire corpus of fictional texts - and it is left to work out patterns or groupings. That is, it doesn’t know a right or wrong answer, but can identify related things from the dataset and group them together (for example, AI reading popular fiction might group together terms such as “batman” and “wonder woman”, but it would have no knowledge of what these terms actually mean).

Supervised learning is where the AI is fed very large amounts of marked up data - that is, for each input, it also gets passed the expected output. An example of this is if you had a large set of photos (say Google Photos) which are pre-tagged with descriptions of what is in the photo, the dataset could be used to train an AI to identify contents of a photo.

Reinforcement learning is similar to supervised in as much as the algorithm gets information as to whether or not it is performing well (like knowing the answer for a given input) but is in the form of a feedback loop and works more like a trial-and-error approach to learning (it might have a general fitness score function that can be used by the algorithm to determine whether or not its response to given input has been successful or not and adjust its response for the next cycle). The simplest example of this is something like AlphaGo/AlphaZero, where an algorithm learns to play a game like Go or chess by trial and error and gets feedback on its attempted response from the game itself.

Both Supervised and Unsupervised learning cases require vast amounts of data to accurately train AI, which really leads us to one of the primary challenges for building fair and ethical AI: sourcing the data to train on. AI is dependent on these huge datasets, and finely tuned to all the details and subtle underlying patterns, regardless of whether we are aware of them or not, and as we will see, getting objective, raw data sets of sufficient magnitude is rife with challenges.



Institutional bias


Similar to the concept of Conway’s Law, which states “any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure”, the data we naturally generate in action, conversation and interactions as a society or organisation will naturally reflect the values, beliefs and structure of the society (or organisation). There is an intrinsic and inescapable subjectivity in all big data, best described by Lisa Gitelman in her book Raw Data is an Oxymoron:

“Objectivity is situated and historically specific; it comes from somewhere and is the result of ongoing changes to the conditions of inquiry, conditions that are at once material, social, and ethical

A simple example of this could be in criminal statistics: if a police force stop-and-search a particular demographic more heavily than others, then that will be reflected in the numbers and therefore that cultural subjectivity influences the data set - this subjectivity will then naturally carry over to, and likely be amplified by, the trained AI as it becomes finely tuned to the data (an example of this was seen where some software used to inform sentencing decisions relied on data that had institutional bias, which resulted in a racial bias in the risk assessment - strengthening the AI Now report’s proposal of banning AI use in these areas).



Finding complete & representative data


Compounding this problem is the fact that researchers working in AI face the challenge of finding datasets that are big enough and permitted for such use, which can be hard to come by, meaning they often make-do with incomplete or skewed datasets. For example, the popular community discussion web site Reddit makes its vast historic dataset publicly available, which is a rich source of natural text and conversation, and makes for a very tempting dataset for engineers and researchers to take advantage of - however, Reddit is a very specific subset of the internet, and the real world demographic, meaning that whilst there is undoubtedly a lot that can be learnt from that wealth of data, any AI trained on it will be heavily subjective.

There have been several reports finding that these incomplete or skewed data sets just further add to the bias. The 2017 AI Now report said:

“data can easily privilege socioeconomically advantaged populations, those with greater access to connected devices and online services

Which is to be expected when you think about it really - always connected people with mobile devices will naturally be generating a lot more date than those without easy access to computers. On a very simple level, the core regular users of reddit, for example, will likely have access to mobile devices or in the very least have available access to computers and the internet - which rules out large parts of the population - not to mention the inclination to partake in the online community.

There are also other challenges that are intrinsic to the way AI currently works: if we have a dataset where a particular demographic is only reflected by 1% of the data, then the AI could claim to achieve 99% accuracy whilst being completely inaccurate for all of that 1% minority. Furthermore, we know that there is a strong relationship between the amount of training data and the accuracy of AI, so in the scenario we have a perfect representation of the population, by definition, all minority groups will have a smaller selection of data points to train on so inevitably the performance of the AI for minority groups will fare worse.

Finally let’s consider again that we have a huge, rich dataset (the idea scenario), and we try to intentionally exclude sensitive features that might explicitly encode bias: race, gender, age, etc. There are still loads of data points that may still act as a indirect proxy to these features, so even without including gender, age and sex in the input data, it is easy to see how these features can get encoded in other data points such as names, location, interests, communication style. This makes it even harder to detect and prevent bias in our datasets.

There is no objectivity in big data.



How can we address the problem?


Some of these examples might have clearer cases of existing bias that we need to be address in training our AI, but a tougher challenge is how can we address the more subtle biases hidden in the cultural objectivity that we might not even be aware of? We all carry our own opinions and biases that subconsciously affect our opinions and attitudes toward things - but if we are not consciously aware of those, we need to think about how we can ensure that developers training AI can have the foresight to engineer around these biases?

This issue highlights one often recommended  approach to tackling the problem of having a greater emphasis on the need for diversity in the teams building AI. Both diversity in terms of individual identities but also cross-functional teams. Statistically and broadly speaking, AI is often developed by teams of engineers with limited diversity, which results in a limited range of views when thinking about the dataset and in what goals are optimised for in the training process. The 2017 AI Now report recommended:

“​stakeholders​ ​in​ ​the​ ​AI​ ​field​ ​should release​ ​data​ ​on​ ​the​ ​participation​ ​of​ ​women,​ ​minorities​ ​and​ ​other​ ​marginalised​ ​groups within​ ​AI​ ​research​ ​and​ ​development.

Aside from trying to recognise subtle bias in the data, we also need to consider that the objective norm, and what we consider to be ok at the moment is changing. Going back to Lisa Gitelman’s quote: “Objectivity is situated and historically specific”. If you could get a dataset from even just two decades ago, it’s not hard to imagine that AI trained on that would have un-acceptable biases because the societal norm and general attitudes to race, gender and identity, etc have changed significantly since then.

As a simple example, take the motor insurance industry. For decades, insurance companies identified young male drivers as a particularly high risk of accident so traditionally charged much higher premiums for that demographic - previously a widely accepted approach, and one based in statistics: young male drivers were statistically more likely to have an accident behind the wheel. But then, in 2012 EU gender discrimination regulation came into effect that prevented companies charging men more than women, so now the insurers have stopped that categorisation for pricing despite the data being available. If that was AI it would need to be re-trained with a modified dataset, with gender probably removed from the data and thought put into other data points that would also need to be removed (names, for example, might very easily be a broad proxy to gender). Whilst this is a simpler example, as its a binary change in legislation with clear requirements, there are also the more gradual shifts in attitude where it becomes a lot fuzzier - like the changes in attitudes on race, gender and secuality over the last thirty years.

We previously discussed the idea that even if we exclude socially salient data points, such as gender, those features can still get encoded via other proxies in the data, and this example of the change in EU regulation and its effect on the insurance industry provides an interesting case study in exactly that phenomenon. There was an article written in the Guardian following the EU ruling, explaining that, despite the ruling meaning insurers couldn’t charge more because a driver was male, male premiums have actually increased in comparison to female premiums since. The reasoning they provide, is that rather than classifying on the crude, data point of gender, the system instead places greater importance on a wider set of data points, and it turns out that these other data points are really just acting as encoded proxies (they list car size, occupation, vehicle modifications). The article makes the observation that MoneySupermarket released a study showing that 8 out of the worst 10 occupations for drink/drug drive incidents were the building trade, with midwives being the least likely to have a drink/drug drive offence, the suggestion being that building trade is predominantly male, and midwives, predominantly female.



It certainly seems to me like there are still lots of challenges as to how we can foresee potential problems and how to tackle them. A key starting point will be ensuring teams working in the area have a good understanding of the dataset they are working with: where it comes from, any inherent bias or blind spots and which of the data points might need modifying or weighting due to their contextual/social salience. This will need to be driven through agreed best practices and AI development standards from organisations like AI Now and from academia, as well as a need for appropriate regulatory controls (although these face their own challenges, which I will discuss in a later article).

I also believe that these challenges mean an even greater need for for diversity of the teams -  both in terms of the race, background, gender etc of the team, and also cross-functional members, not just engineers but also working closely with the specific domain experts for the field.




Photo credits:

Heading Photo by Alex Knight on Unsplash
Anonymous person Photo by Andrew Worley on Unsplash
Continue Reading

On Education: Let us play

Education is something that has been an interest to me for some time, and with a child currently going through primary school in the UK, it's something that I think about quite a lot, so this is something of an open letter on the state of the education system in the UK.


First off, I should say that I am a strong believer in the importance of both the role of free-play in learning, and also of instilling a curiosity in children as an approach to ensure future success, rather than more structured didactic teacher and test driven approach. This belief is largely grounded in various things I have read on the topic, but appreciate there is undoubtedly a lot more depth to the subject matter than I know about.


Positive Examples

If we are thinking about the education system, it seems prudent to look elsewhere for success stories to see what we can learn from and improve on in the UK system, and a glaringly obvious example would be the Finnish education system. Finland's often cited education system has consistently been the top ranked system in Europe for the last 16 years, so what are they doing right?

The first notable difference is the age at which children start school: children will attend preschool from an early age, but primary school doesn't start until 7 years old - in other words, formal teacher-lead instruction on what we would consider core topics: maths, reading and writing, do not start until the age of seven. Before that, the education system is entirely focused on free, creative play.

This model also ties into research from some neuroscientists who believe that before the age of seven or eight, "[children] are better suited for active exploration than didactic explanation" - claiming that "the trouble with over-structuring is that it discourages exploration". Having witnessed this behaviour first hand, this very much supports my anecdotal data on the subject: trying to explain to a 6 year old a moderately complex process can be a challenge, but, let them watch you perform it a few time and they will often pick it up with much greater ease (case in point: using a tech device, playing a video game etc). Furthermore, it seems to me that encouraging exploration and independent discovery should surely be a key part of any process aiming to instil curiosity in children.

These results were also mirrored in the research by the Lego Foundation, who claimed children should learn through play until at least the age of eight (despite possible cynicism based on a report from a toy manufacturer recommending more play, that article is really spot on).

To put this schooling approach into perspective, in the UK, children will already have had up to three years of five days a week, full day, classroom based teaching by the age of 7. When my eldest son turns 7 he will be finishing his third year in school and will already faced the prospect of national standardised testing in the form if the SATs (thankfully the government have decided to scrap these, but they will only stop being compulsory in 2023). Pretty heartbreaking.

Thinking of this child, so full of joy and enthusiasm for playing, whether it be running around outside lost in an imaginative world of play or sitting down playing with lego, having to spend something like 25 hours a week in a classroom seems unthinkable.




But it's not just starting late, either, even once more formal education starts, they make sure to keep play an integral part of the school day, and children are required to have 15 minute play breaks every hour. Aside from the potential educational benefits of regular playing, research has also found that outdoor play is linked to healthier and happier children (aside: have you ever tried to get a 6 year old to concentrate for 45 minutes? If so, you will probably see the futility in anything other than regular play breaks)

Once again, for some perspective, whilst visiting UK primary schools for my eldest son, one school head mistress casually boasted that the traditional afternoon playtime break had been dropped in favour of more classroom time. Unsurprisingly, we didn’t apply for a place at that school.


But why?

The common thinking behind starting formal schooling earlier is that the earlier they start learning, the better prepared they'll be, and the greater the head start they'll have. But even if the Finnish school system didn't appear to disprove this theory, it's worth considering the difference in benefits of learning by rote/testing Vs independent learning (via play or other means) and the independent curiosity needed for the latter. I'd suggest that, in the modern knowledge economy in which we live, and with the quickening rate at which information and understanding is being changed by advancing technology, the most beneficial skill that someone can leave school with is curiosity and the ability along with the desire to learn independently. That is, to leave school as lifelong learners. I think it says a lot that a key topic on the Finnish national curriculum is simply “learning to learn”.


Beyond looking at success stories of other education systems, we can look at history. Current incarnations of the education and school systems are a relatively modern thing, so what did we do to learn before then? Of course, families and communities have long recognised the importance of amassing and passing on information to younger generations, if not through formal education, but in many cultures, children learn through imitation and experimentation (which, as I mentioned previously, is easy to believe if you have a young child that has grown up around adults using mobile devices and have witnessed the speed at which they become proficient through imitation and experimentation).

It might be tempting to think that whilst humankind were able to learn through such basic play techniques in time gone by purely because what we needed to learn was simpler, and that the as we have progressed as a society it has also demanded people have a greater understanding and depth of knowledge in order to keep up with industrial and technological advances, so the education system has evolved out of necessity.

However, I would argue that the opposite is true - firstly, as pointed out earlier, the speed at which science and technology is advancing means whatever level you leave the schooling system, within a couple of years understanding and techniques will likely have moved on, and your ability to learn independently and keep up with the fast paced changes is going to be essential to success.

Secondly, as I have written before, I would argue that play provides the essential understanding and building blocks for going on to study and understand computer science, engineering and maths.  As a computer scientist, I personally think the education that stood me in best stead for going on to learn - and be successful professionally - was playing with toys like Lego and puzzle solving games.

For example, let’s take a look at one of the Key Stage One goals for computing in the current UK National Curriculum:

“use logical reasoning to predict the behaviour of simple programs”

To be clear, Key Stage 1 is 5 to 7 years old - this is children who, some neuroscientists think, are of an age that is too young to be in formal taught education, who, in Finland, would still be enjoying creative play, and who may not all be capable of reading un-aided. I don’t know about you, but I wouldn’t like to have to teach that in any medium other than play.



However, if we just re-frame the problem and consider this goal in the context of playing with train sets, it becomes simpler: “use logical reasoning to predict the behaviour of trains” - give the kids train sets, let them build tracks and think about what happens when a train is added: what happens if we add two trains? What happens if we change the direction of trains? Or change the behaviour of a junction piece? Being able to reason logically about such behaviours and changes is a very transferable skill that is useful for thinking about a range of problem solving disciplines, including computer science.

And its not just computing - there is a growing group of mathematicians who posit that preschoolers are actually capable of understanding calculus and algebra. But not just that, but  by actually attempting to teach them maths the way we currently do, it is crushing almost all appetite or future interest in the subject, that is actually an amazing world of wonder and surprise, by taking all the playful fun out of maths and making it a boring case of memorising numbers and patterns - which obviously has the end result of killing their curiosity or interest in going out and independently learning.


Ultimately, I would love it if UK schools embraced free play more, if they embraced teaching STEM subjects through play, but I understand that it’s a huge shift that needs to come from the government. Recognising that the SATs test is not a positive thing for 6 and 7 year olds is a good first step, but the UK primary schools are still so governed by the national curriculum and expectations around performance that it seems impossible for any individual school to start to move the dial.


References


  1. The Atlantic: The underrated gift of curiosity
  2. The Guardian: The secret of Europe's top education system
  3. The New York Times: Let the kids learn through play
  4. The Guardian: Children should learn mainly though play until the age of 8
  5. Gov.UK: SATs practice material
  6. The Atlantic: How Finland keeps kids focused through free play
  7. The Play Return: An assessment of play initiatives
  8. Wikipedia: Knowledge Economy
  9. Gov.UK: Computing National Curriculum
  10. Fostering mathematical thinking through playful learning (paper)
  11. The New York Times: What babies know about physics
  12. The Atlantic: Five year olds can learn calculus
Continue Reading

Building a RESTful API like a Product Manager

Something I have been thinking about for a while is the idea of APIs as your Product - the idea that you should design, build and test your API like it's a public facing product (as it might actually be if its a public API), and what lessons we should be learning from Product Managers about how we achieve this.

In the latest instalment of their "Technology Radar", ThoughtWorks recommended the concept "API as a Product" as ready to trial (although I think it's far beyond trial stage - I can see no real risk of adopting the lessons from Product Management for your API).



I have generally been interested in Product Management for several years - I think it's hard not to get interested in the science of Product Management when you are building side projects, as you naturally have to think about the product you are building - sure, with side projects they can easily turn into vanity projects, where you just build what you think is cool, but I think it's still fun to think about real users and good Product Management techniques that can be applied even to side-projects.

Having tried to gain some understanding, I have turned to some classic material on the subject of Product Management: The Lean Startup, Don't Make Me Think, How to Build a Billion Dollar App. I have also been lucky enough to attend Mind The Product Conference twice (though volunteering - but managed to watch lots of the talks), and to have worked with several great Product Managers.

So what lessons can we start applying? I think, if one statement sums it up, its an idea from Kathy Sierra:

Does your product make your users awesome?
This is a great way to think about APIs (I'm mostly talking about REST/GraphQL type APIs here - but this thinking is a great way of approaching design for normal class/interface/library APIs too).

As engineers, I'm sure we have all used APIs that don't achieve this - APIs that make us have to think, that make us feel stupid, that seem overly complicated (for example, the old Java Date library - the number of times I had to google just to find out how to simply create a specific date for something simple like a unit test is ridiculous). And hopefully, we have all used APIs that make us feel awesome - APIs that make us productive, that let us do what we expect to be able to do, and can (with the help of the IDE's auto-complete) let us guess what the method calls are.

Another idea Kathy Sierra mentions in that talk, and is also made famous from the title of the Product Management classic, is:

Don't make me think

I have written before about the StackOverflow and GitHub APIs, but they deserve a mention again, if you have ever written an integration with them, you will know.  The APIs are designed in such a way that sensibly models the domain model.

For example, take StackOverflow, if we were going to describe their domain model, what first class objects would we have? I guess Questions, Answers, Comments etc - and sure enough, their API models these:

/questions - will get you all the questions on the site

/answers - will get you all the answers,

Now, how do you think you might get all answers for a given question? Well, a list of answers should probably be a child of a question, so something that represents that, something like:

/questions/{Question ID}/answers

Sure enough, thats the URL - and the behaviour is consistent through the design, I'm sure you can guess how you might retrieve all comments for a given answer.

I'll be honest, developing against an API where I can guess the endpoints makes me feel pretty awesome.

MVP and the Lean Product

The idea of a Minimum Viable Product is a core concept in Lean Startup - the idea that you build the minimum version of the product that can get you feedback as to whether you are heading in the right direction before investing more effort in building out a fuller fledged product.

Whilst I think an appreciation of an MVP is a good awareness for any engineer - I have generally found it is useful in facilitating conversation when building a new product - if as an engineer you recognise some new feature is going to be considerable effort, its helpful to think about what hypothesis the feature is trying to test, and is there a simpler engineering solution to test the hypothesis.

However, the principal of not over engineering the API is also a good one - once you have a well designed domain model, its quite tempting to get carried away with REST design best practices and start building endpoints for all different variations of the model - however, unless you need them, it's better to start with less (if you are building a public REST API from the start then it might be harder to know what isn't needed, but you can still certainly build an MVP version of the API to test the hypothesis - e.g. whether people actually want to use your API)

User Testing

Another important part of building a product is collecting user feedback - Whilst a REST API may not be well suited to A/B testing (your consumers won't be very happy if switch the API design on them after they have implemented against a specific variation!), listening to user feedback is still important - because the consumers will likely be engineers (writing the client code), then hopefully they will be expecting REST standards and good API design principles, but another way to think about this is the importance of not making breaking changes to your API.

A technical way of thinking about this is Consumer Driven Contracts Testing - As RESTful APIs are de-coupled from the client code, the API doesn't have any idea what clients are using or doing, so this approach allows consumers to define what they expect from the API, and then the API can always test against these contracts, so be sure that any change to the API is still in keeping with existing consumers, or users, of the API.

Conclusion

I think, in general, all engineers can benefit from thinking about how good products are built, what concepts are important, and be able to think about the products or services they are building in this way - both in terms of thinking about the actual implementation as a product that someone else will be the user of (either externally if a RESTful API, but also internally if its cross-team or even just someone coming to take over your code once you have moved on).
Continue Reading

RESTful API Design: An opinionated guide

This is very much an opinionated rant about APIs, so it's fine if you have a different opinion. These are just my opinions. Most of the examples I talk through are from the Stack Exchange or GitHub API - this is mostly just because I consider them to be well designed APIs that are well documented, have non-authenticated public endpoints and should be familiar domains to a lot of developers.

URL formats

Resources

Ok, lets get straight to one of the key aspects. Your API is a collection of URLs that represent resources in your system that you want to expose. The API should expose these as simply as possible - to the point that if someone was just reading the top level URLs they would get a good idea of the primary resources that exist in your data model (e.g. any object that you consider a first-class entity in itself). The Stack Exchange API is a great example of this. If you read through the top level URLs exposed you will probably find they match the kind of domain model you would have guessed:

/users
/questions
/answers
/tags
/comments
etc

And whilst there is no expectation that there will be anyone attempting to guess your URLs, I would say these are pretty obvious. What’s more, if I was a client using the API I could probably have a fair shot and understanding these URLs without any further documentation of any kind.

Identifying resources

To select a specific resource based on a unique identifier (an ID, a username etc) then the identifier should be part of the URL. Here we are not attempting to search or query for something, rather we are attempting to access a specific resource that we believe should exist. For example, if I were to attempt to access the GitHub API for my username: https://api.github.com/users/robhinds I am expecting the concrete resource to exist.

The pattern is as follows (elements in square braces are optional):
/RESOURCE/[RESOURCE IDENTIFIER]

Where including an identifier will return just the identified resource, assuming one exists, else returning a 404 Not Found (so this differs from filtering or searching where we might return a 200 OK and an empty list) - although this can be flexible, if you prefer to return an empty list also for identified resources that don’t exist, this is also a reasonable approach, once again, as long as it is consistent across the API (the reason I go for a 404 if the ID is not found is that normally, if our system is making a request with an ID, it believes that the ID is valid and if it isn't then its an unexpected exception, compared to if our system was querying filtering user by sign-up dates then its perfectly reasonable to expect the scenario where no user is found).

Subresources

A lot of the time our data model will have natural hierarchies - for example StackOverflow Questions might have several child Answers etc. These nested hierarchies should be reflected in the URL hierarchy, for example, if we look at the Stack Exchange API for the previous example:
/questions/{ids}/answers

Again, the URL is (hopefully) clear without further documentation what the resource is: it is all answers that belong to the identified questions.

This approach naturally allows many levels of nesting as necessary using the same approach, but as many resources are top level entities as well, then this prevents you needing to go much further than the second level. To illustrate, let’s consider we wanted to extend the query for all answers to a given question, to instead query all comments for an identified answer - we could naturally extend the previous URL pattern as follows
/questions/{ids}/answers/{ids}/comments

But as you have probably recognised, we have /answers as a top level URL, so the additional prefixing of /questions/{ids} is surplus to our identification of the resource (and actually, supporting the unnecessary nesting would also mean additional code and validation to ensure that the identified answers are actually children of the identified questions)

There is one scenario where you may need this additional nesting, and that is when a child resource’s identifier is only unique in the context of its parent. A good example of this is Github’s user & repository pairing. My Github username is a global, unique identifier, but the name of my repositories are only unique to me (someone else could have a repository the same name as one of mine - as is frequently the case when a repository is forked by someone). There are two good options for representing these resources:

  1. The nested approach described above, so for the Github example the URL would look like:
    /users/{username}/repos/{reponame}

    I like this as it consistent with the recursive pattern defined previously and it is clear what each of the variable identifiers is relating to.

  2. Another viable option, the approach that Github actually uses is as follows:
    /repos/{username}/{reponame}

    This changes the repeating pattern of {RESOURCE}/{IDENTIFIER} (unless you just consider the two URL sections as the combined identifier), however the advantage is that the top level entity is what you are actually fetching - in other words, the URL is serving a repository, so that is the top level entity.

Both are reasonable options and really come down to preference, as long as it's consistent across your API then either is ok.

Filtering & additional parameters

Hopefully the above is fairly clear and provides a high level pattern for defining resource URLs. Sometimes we want to go beyond this and filter our resources - for example we might want to filter StackOverflow questions by a given tag. As hinted at earlier, we are not sure of any resources existence here, we are simply filtering - so unlike with an incorrect identifier we don’t want to 404 Not Found the response, rather return an empty list.
Filtering controls should be entered as part of the URL query parameters (e.g. after the first ? in the URL). Parameter names should be specific and understandable and lower case. For example:
/questions?tagged=java&site=stackoverflow

All the parameters are clear and make it easy for the client to understand what is going on (also worth noting that https://api.stackexchange.com/2.2/questions?tagged=awesomeness&site=stackoverflow for example returns an empty list, not a 404 Not Found). You should also keep your parameter names consistent across the API - for example if you support common functions such as sorting or paging on multiple endpoints, make sure the parameter names are the same.

Verbs

As should be obvious in the previous sections, we don’t want verbs in our URLs, so you shouldn’t have URLs like /getUsers or /users/list etc. The reason for this is the URL defines a resource not an action. Instead, we use the HTTP methods to describe the action: GET, POST, PUT, HEAD, DELETE etc.

Versioning

Like many of the RESTful topics, this is hotly debated and pretty divisive. Very broadly speaking, two approaches to define API versioning is:
  • Part of the URL
  • Not part of the URL
Including the version in the URL will largely make it easier for developers to map their endpoints to versions etc, but for clients consuming the API it can make it harder (often they will have to go and find-and-replace API URLs to upgrade to a new version). It can also make HTTP caching harder - if a client POSTs to /v2/users then the underlying data will change, so the cache for GET-ting users from /v2/users is now invalid, however, the API versioning doesn’t affect the underlying data so that same POST has also invalidated the cache for /v1/users etc. The Stack Exchange API uses this approach (as of writing their API us based at https://api.stackexchange.com/2.2/)

If you choose to not include the version in your API then two possible approaches are HTTP request headers or using content-negotiation. This can be trickier for the API developers (depending on framework support etc), and can also have the side affect of clients being upgraded without knowing it (e.g. if they don’t realise they can specify the version in the header, they will default to the latest).  The GitHub API uses this approach https://developer.github.com/v3/media/#request-specific-version

I think this sums it up quite nicely:


Response format

JSON is the RESTful standard response format. If required you can also provide other formats (XML/YAML etc), which would normally be managed using content negotiation.

I always aim to return a consistent response message structure across an API. This is for ease of consumption and understanding across calling clients.

Normally when I build an API, my standard response structure looks something like this::

[ code: "200", response: [ /** some response data **/ ] ]

This does mean that any client always needs to navigate down one layer to access the payload, but I prefer the consistency this provides, and also leaves room for other metadata to be provided at the top level (for example, if you have rate limiting and want to provide information regarding remaining requests etc, this is not part of the payload but can consistently sit at the top level without polluting the resource data).

This consistent approach also applies to error messages - the code (mapping to HTTP Status codes) reflects the error, and the response in this case is the error message returned.

Error handling

Make use of the HTTP status codes appropriately for errors. 2XX status codes for successful requests, 3XX status codes for redirecting, 4xx codes for client errors and 5xx codes for server errors (you should avoid ever intentionally returning a 500 error code - these should be used for when unexpected things go wrong within your application).

I combine the status code with the consistent JSON format described above.
Continue Reading

Mobile: Web vs Native (again)

It seems like by now, pretty much everyone has weighed in on the native vs web conversation for mobile development, and being as I haven't yet, I thought why not? The age-old question being, should you make a native mobile app or just go with a standard responsive website? (or just use some common technology to wrap up your mobile website in an app).

The reason that I started thinking about this again was because an article as circulated at work, which seemed to be saying that you should do both native and web, as they have different strategic values - which, whilst I largely disagree with that, I think the underlying point being made was that there are different reasons for going down either path.

A quick caveat:

I will get this out the way up front - I'm not talking about any scenarios where you are in a very mobile-centric business - so if you want to make use of phone hardware like camera etc, or you are specifically a mobile-first type app, or one that is intrinsically linked with the mobile as an identity, then yes. Native is the only option.  This discussion is more for normal existing businesses that might (or not) already have a website and is at the point of inflection of whether to  build a native app or not11.

--- 

So, native or responsive web?

My general rule of thumb is as follows: Don't go native.

You would be right in thinking that's a fairly sweeping rule. But I think probably fair - and whys that? I think there are two primary concerns that make the cost and effort with building a native app not worthwhile:

1. Discoverability

The web is a great place for discovering things - just the word "web" portrays it quite nicely, from any given point there are undoubtedly loads of little threads(I'm talking links) that could be followed at no real cost - assuming it's not a dodgy looking link, the barrier to prevent someone following a link is practically non-existent - So you read a nice featured article about a new company/product on a blog, at the end of the article they link to their site, you click it. I mean, why not right? if it ends up looking crappy the back button saves you and what have you lost? a few seconds? Easy.

This is something that is clearly missing in the mobile app eco-system - assuming you aren't some behemoth of a company that millions of people want (or need) to interact with, a mobile app isn't going to help increase your customer base. Sure it might give you a richer experience for the handful (I'm talking in web scale here) of customers - but its not going to help grow customers, it will cost you time and money to produce/maintain and that's before you start having to work on directing existing customers to your mobile app:

So again, if you are a bank, who has millions of customers who have specific, regular needs to transact with you, or if you are a hot new social-local-sharing company then sure, go for that enhanced UX.  But an app is only going to be any good if the users already have the intent to transact with you. If a user is browsing the web on a mobile, your conversion rate is going to be a lot higher with a link through to a responsive website than to an app download.


2. Scalability

For me, this is the deal breaker. I'm not talking technical scalability - will your servers be able to withstand the un-doubted, soon to be approaching, mass of people who will rush out and download your app as soon as its released (see my previous point), I'm talking customer-to-app scalability.

Imagine your a regular business - you have thousands of happy customers, maybe your website even gets tens or hundreds of thousands of uniques a day - so lets build an app right?

The problem is, there is a limit to how many apps that a user will have on their phone - on a standard android phone you will likely have two screens worth of app icons when you un-box it, so there is a limit to how many more apps they will install - Again, this is how it is not like the web: visiting a website is free, but there is a much greater barrier to installing and keeping an app (let alone using it) - and when the phone is getting full, or sluggish, or the user wants a bit more space to download a show from BBC iPlayer, then the apps that aren't regularly used are going to get the chop.

When you are competing with limited resources and the big players - Facebook, messaging, banks, geo-based stuff (maps, uber etc) - then it's hard to make a compelling case for the phone user to keep or even install the app.  This demand makes it even harder to convert your loyal web customers to mobile - and weighed up against the fact that you could make an awesome(and consistent) responsive web experience, the choice looks more clear for me.


Benedict Evans says you should build an app if people are going to put your app on their homescreen, which seems like pretty sound advice - and given the size of a a phone homescreen, this makes for fairly few companies.

Final caveat:

If you are building the app because it came out of a side-project organically, or a hackathon or something similar, then by all means - there's lots of fun to be had and lessons to be learnt in building, testing and launching a mobile app - so if you don't mind the potential cost then definitely go for it!
Continue Reading

Innovating towards the traditional - Facebook & Amazon

Just a short post now - and really just two observations/trends I have commented on before that are continuing their trajectory.

 Amazon bring grocery delivery to the UK - Announced today, Amazon will be bringing their "Fresh" service to the UK, starting with London and Leeds - continuing with my previous observations on innovating towards the traditional.

Facebook messenger no longer needs a Facebook account - this is older news, and I have been meaning to comment on this for a while. Basically, customers can now sign up for and start using Messenger without an account, just using a phone number, which is an un-surprising move towards phone as identity and recognising that the phone has the potential to be a node in any social connected network on the platform (much like I have preached about before..).  It is also another step towards Facebook moving further down the stack and more directly competing with native messaging apps as a system level.
Continue Reading

Technology, innovation and 2015

There has already been several people, much smarter than me, offering predictions for the next year (my favourite is Fred Wilson's look back at last year and look forward to the next) which you should probably read rather than mine, but here we are anyway, so lets go with a few thoughts on 2015 and onward..


Television

I have written about this for a while, and the battle for the living room will continue to rumble on. I'm not sure if we will start to see one or two winners fighting it out this year, but I can only see this area getting more interesting.  NetFlix, Amazon Instant, NowTV have established themselves as the more dominant on demand providers but that is only a part of the battle - there is still the problem of platform fragmentation (no common platform across devices/TVs/etc to run consistent experience apps - some devices not being able to support updating OD apps etc).  Android has a real opportunity to tackle this problem, given that pretty much all content providers already support the Android platform in smaller sizes, but let's see what they do. If they can become the defacto television/set-top box OS it would be a big win for them, but would also mean that device manufacturers & content providers don't need to worry about building & supporting their apps for a range of platforms and can focus on building the best experience on a single platform.

There is also the question as to the role of the actual TV device - will it be a smart device with a dedicated platform, or will it become dumber that just streams content from any other mobile device.


Enterprise

There has been some hype around Enterprise startups growing over the last 12 months, and this will also slowly increase - as has always been the way with enterprise, things have been very slow in terms of adoption and businesses and executive teams generally very wary about adopting new technologies until they are really widely adopted and proven - but this is starting to change. As more and more areas of business are starting to be disrupted by upcoming startups, more established businesses are starting to adopt technology more quickly to try and stay in the game, and more and more starting to spin off departments/teams from the business to operate as micro-startups to help drive innovation.  I think with this trend for bigger businesses becoming more willing to adopt new tech continuing, we will see some big noise in the enterprise/SaaS areas.


FinTech

Similar to wider enterprise trends, the banks and financial institutions are starting to become more aware of the risk and trends in their business with more and more people starting to expect better/different ways to do business.  From the larger tech of Bitcoin through to ApplePay, Square etc I think we will see continued innovation and surprises from the bigger players as well as a whole host of startups snapping at their heels.  With London currently the world FinTech capital it could be an interesting year here.


Education

This is an area I am interested in and have written about on a few occassions. I'm not sure this will really be a big area in 2015, there certainly aren'y any startups I am aware of that look like they might make really big waves in the year, but here's hoping.  Either way, there are still interesting things going on - there is the continued change in the UK curriculum with things like the "hour of code" and the continued drive to put more tech on the curriculum. Plus, I have recently spoken to two startups recently that are doing interesting things in the education mobile app space: Zzish and kahoot  - they are both working on mobile app/web platforms for the classroom with a bunch of tools to help teachers and children learn and work together.


The rest..

I agree with many of the other observations on Fred Wilson et al's writing - continued growth, acquisitions and big IPOs from the current crop of big startups will continue (AirBnB, Uber etc).

Companies like Xiaomi will continue to grow and hopefully come and start to be real contenders in the western markets (even if the Xiaomi laptops have been leaks, I'm still interested to see if they do come with a laptop anything like the leaked specs.. could get interesting!)

As Fred Wilson mentions, and as I have previously written, I don't think wearables are really ready to be a thing. Still more work needed on how these will fit into the market, and I don't think that will be happening in 2015.
Continue Reading

Twitter API & Smartphone as your identity

At Twitter's Flight conference this week, they announced a couple of new toys for developers: Fabric and Digits.

Fabric is a new platform for building mobile apps

The Fabric platform is made of three modular kits that address some of the most common and pervasive challenges that all app developers face: stability, distribution, revenue and identity.


It seems everyone has to be building a platform these days, but really it sounds like Twitter, having previously shut down developers on their API/platform a while ago, realised that actually letting devs on their platform (now that it includes advertising APIs - thanks to their MoPub acquisiting a while ago) is going to be profitable for them.

Maybe it will be well received, and the way the market changes maybe people will go for it - but I wouldn't be building an app based around a Twitter platform given their track record.  The company are a lot older, and maturer now (and they probably released their API too early last time around), but it still seems like quite a risk to have something as central as login/revenue tied to Twitter (I dont like third party logins anyway..).



Anyway, the much more interesting product they released was Digits - which allows sign-in with your mobile number - which as I have ranted before, is an idea that I love, and seems like a really smart move by Twitter. Both in as much that it will move twitter to that space of smartphone is your social-identity/network and also offers a lot more value than just another platform that serves ads and supports login etc.
Continue Reading

Amazon & the circle of life

Last week, Amazon announced the planned opening of their first bricks & mortar store - planned to be opened in central New York, not far from Macy's department store. The move is an attempt to provide some of their customers with a traditional face-to-face customer service.

If you have been following other recent Amazon announcements and expansions, you will be familiar with their other recent moves:

Same day delivery - Amazon has been a dominant power in e-commerce for a long time, and given the small margins they operate, and being loss-leaders in some products to drive business elsewhere, it's going to be difficult for any newcomer to genuinely compete, and as Amazon continue to spread more and more into all forms of goods, its also slowly making it harder for shops to operate in a specialist vertical.  However, there is always going to be times when convenience and the ability to have something immediately trumps price, so there was always going to be a market share that Amazon will loose. A lot of the time people will pay a small percentage increase in price for being able to have the product in their hands in a few hours.  Same day delivery will reduce that - again their will be the premium cost of the delivery service, but customers can have products instantly (almost).  There are some significant logistical challenges to do this, but being in a position to do this also opens up another opportunity..

Fresh groceries (having been doing non-perishable groceries for some time). One of the biggest challenges to Amazon rolling out fresh grocery deliveries more widely (currently only available to parts of North America) is building the capacity to enable fast, same day delivery of fresh goods; so the goods can leave a chilled warehouse and be with the customer in a short time - what is a short time exactly? It seems from existing supermarket delivery services that most people are happy with same day delivery (or at least fixed day delivery - e.g. the goods leave the warehouse on the same day they arrive to the customer), using refrigerator delivery vehicles, which seems simple enough for an individual case - in the UK Amazon could ship from their large Swansea warehouse in the morning and deliver the goods to most parts of the UK the same day. The inevitable problem becomes when this starts to scale up, As soon as even a small percentage of the UK want to order their weekly fresh groceries from Amazon the problem gets a lot tougher, and Amazon would need to have a large, distributed warehouse/delivery infrastructure to enable this kind of efficient, same day grocery deliveries.  The kind of infrastructure that existing supermarkets like Tesco or Sainsburys already have, having long built the infrastructure to provide a similar level of stock control and service to their supermarkets.


It's no secret that Amazon returns very little profit to their shareholders, and continues to plough the majority of their revenue into new business and further expansion - a large part of the investment going into expanding their delivery and warehouse capacity and distribution.

Scaling same day delivery is a hard problem to solve, if only because the solution is really just more distributed warehouses. Amazon have tried to ease the scaling problem with Amazon lockers and local partners/shops that can take delivery and can then be picked up from by the customer at a convenient time (a local shop that a customer can pick up their goods from on the way home from work for example), but really their move to bricks and mortar is really just another step to a long established industry.

Is Amazon really disrupting groceries/shopping? Or are they just competing against the usual retail giants?  Neither same/fixed day delivery or fresh grocery delivery is a new feature. Providing both online and physical stores is also not a new model, so really all we are seeing is another retail powerhouse slowly marching onwards and upwards to a fairly conventional business plan**, maybe the only thing of interest is that it is doing it in a different order (Tesco scaled from bricks and mortar, to online shopping, to same day delivery; Amazon are simply starting from online and expanding to the others).


So I guess we will need to wait for drone-delivery to see any real innovation in the retail space.



** Conventional business plan for the retail/groceries aspect of the business - there is still lots of innovation and interesting things that Amazon are doing, with mobile devices, could services, video delivery/production etc.
Continue Reading

iWatch & wearable tech

Much has been made lately of Apple's latest announcements of the iPhone 6 and their new Apple watch, and really, I'm pretty late to the party on this one.  I don't normally comment much on Apple announcements, but this time I fancied a rant.


Personally, it's not to my tastes. I know the strap is inter-changeable, and you can change the watch face screen, but I'm not a fan.  In part, because I am more of a fan of classic watch design (so really, some of the photos of the Moto-360 look closer to what I would want a smartwatch to look like), but generally, it feels a little garish, and a bit like something designed 5 years ago.  The curved, almost bubble like glass shape of the device, the square watch face.



I'm not sure what it is.

Maybe its because it feels at odds with current web design trends on flat design. Maybe its because it feels like the original iPhone matured/evolved from it's original curved shape to the current sharper, flatter designed shape and this still seems to hark back to the original iPhone design.

Anyway, as an aside, I think if I was going to spend hundreds on a flash-y digital watch, I would quite like this one:



Sure, I can't check emails on it, but it looks nice.  But then I'm not really someone who should be commenting on style and fashion, so will stick to tech trends..


Is wearable tech the next big thing?


Honestly, I think probably not. Not for the time being at least. I'm sure some smart people will work the market out eventually, but for the time being, and with the current incarnations of smartwatches, I don't think the market is really there.


So, here's the thing - I'm not really sure what the point of the apple watch is (and I guess that is what needs to be cracked before the market can take off). I think, it's going to face the same challenges that tablets have faced - it needs to grow up and work out what it is. It needs to understand what its purpose is and find its niche - it's not going to be good enough to be the same as a smartphone but a different form.


Let's have a look at the markets:

Smartphones:
  • Defined and fairly standard "upgrade-cycles" - the expectation and standard of upgrading devices every 12-24 months is fairly established in the west, and this is both driving existing customers to newer, better devices, but is also slowly migrating existing feature phone users to smartphones.
  • Everyone has one - at its core, as a phone/communication device it provides that roaming communication functionality and is at least one per-person (not shared)
  • Convenience - easy to carry and roam.

Tablets:
  • No real defined upgrade-cycle (in terms of device contracts) and too early to see patterns being established - optimistically will fall back to traditional PC cycles of approximately every 5 years
  • Not per-person devices - You might be safe to typically expect everyone in an average household to have a (smart)phone, but probably only one/two tablets per house.
  • It doesn't solve any real problem - sure, its a little better for watching movies than a smartphone, but apps, browsing, emails and other comms are not noticably better. Further, tasks that might need greater control, or screen real estate - like creating spreadsheets, or presentation slides for example - people still fall back on using regular PCs. There needs to be a purpose/task/app that is made for a tablet sized device - where tablets solve a problem that only they can. As of yet, we're still waiting.

Smartwatches:
  • Still yet to see upgrade cycles - will be interesting if carriers/manufacturers try to tie these into existing mobile contract structuring. It will make adoption even harder if they try to sell it with a 1-2 year lifecycle for sure.
  • Aimed to be per-person, as a sidekick to your smartphone - but if all it offers is your smartphone in a slightly different form, its again going to be a tough sell - Sure, its slightly more convenient than getting your phone out of your pocket, but that seems like a dubious USP to base an entire market on.
  • It doesn't solve a problem - Again, there needs to be a task/area/job where the smart watch is the answer. Where it fills a need that simply can't be filled by a smartphone (or can be, but is a pain in the ass)



For a change, let's have a look at some smartphone/tablet data:


Tablet sales struggle: Apple iPad growth projections by quarter

(Source: Computer World: As tablet growth slows, Apple may face a year-long iPad sales contraction )

Just focusing on Apple for the moment, Benedict Evans presents some interesting data analysis of their recent sales/revenue numbers. Firstly, we see that iPad sales have flattened out and basically settled where they are for the last two years, whilst iPhones have continued to see growth year on year:

http://ben-evans.com/benedictevans/2014/4/25/ipad-growth
(Source: Benedict Evans - iPad growth - Apple's trailing 12months sales)


More generally, if we look at the comparison of sales across PC vs Android/Apple smartphones, we see that PCs have levelled out, but the smartphone continues to see huge growth:

http://ben-evans.com/benedictevans/2014/4/25/ipad-growth
(Source: Benedict Evans - iPad growth - General shipping - PCs vs Smartphones)




It really looks like the smartphone market is continuing to surge. With standard upgrade-cycles, and low end Android smartphones becoming more widely accessible, this trend seems set to continue.

On the other hand, until the tablet market works out its purpose and finds a niche, I think it will continue to stagnate with fairly flat growth. 

It feels to me like a similar fate awaits smartwatches, until someone comes up with a compelling problem that the watch form factor solves, I think it will struggle to see big growth in the market.
Continue Reading

Yo: One reason it doesn't suck, completely

There has been a lot of talk about Yo recently. Initially it seems like the talk was all driven by the fact that this seemingly pointless app had raised $1million in angel funding (a good way to generate publicity and hype I guess if you happen to have a milli lying around). A lot of people speculated this as further proof of the tech bubble they had been long predicted. Others just talked about how crappy it was (albeit indirectly)

At a glance, neither the app nor the funding round seem particularly interesting - The app sounds distinctly like a novelty app that has only generated interest (and therefore users) on account of the funding, and seems destined to be a blip in the tech-history books much like Chat Roulette etc (although I can't really see it hitting he heights of Chat Roulette - at least people still make an occasional joke about Chat Roulette - I think give it three months and no one will be talking about Yo). The funding is less interesting when you hear the full story: according to Forbes, the app was created by a chap called Or Abel, after his former boss, Moshe Hogeg apparently asked him to make the app so he could buzz his PA without having to call, when Or Abel then switched up the idea a little bit Moshe lead the funding round (so a guy invested in what was basically his own idea, probably wasn't the world's hardest pitch).

Context based messaging

One reason cited for the app not being that lame, is apparently that it is actually trying to fill in a gap - providing context-based-messaging e.g. when you ping someone a "yo" they know the context of the message so you don't need to know anything else.

I don't agree. At least with the examples cited so far, it still seems pretty lame. The main example that has been used is the World Cup - you could subscribe to WORLDCUP by yo-ing them and then you will get a "yo" back everytime a goal is scored. Which honestly doesn't sound that great. I have been following the WC pretty avidly, but getting intermittent messages saying someone scored doesn't sound useful, and the main problem is that I would then have to launch another app to actually get the context (e.g. which team scored).  There are lots of other solutions to this that provide simple notifications including the context.


Some other examples:

Wanna say "good morning"? just Yo. 
Wanna say "Baby I'm thinking about you"? -- Yo. 
"I've finished my meeting, come by my office" -- Yo. 
"Are you up?" -- Yo.

These all make sense, but what next? How do you continue the conversation? how do you confirm/agree that all important context of the message? All these simple "yo"s all drive users to other apps - which means more effort/taps so why bother starting the conversation in Yo at all? why not just use WhatsApp/SMS/Facebook/GChat/etc from the start?


Ok, so clearly Context-Free messaging is a bad idea..

However, in my opinion, there is a glimmer of hope for the app, but the question for me is really just does it have enough runway to execute it before it just becomes another novelty app in the history books.

A few years back, just before Twitter went all dick-ish with their API and started locking out the entire developer community and third party eco-system there was a few good articles discussing an alternative business model for Twitter, and rather than becoming a media company (that it has become, rich content including pics/videos & trying to drive all eyeballs onto twitter.com or official apps - not via third party clients) it should become a global messaging system - it had the infrastructure made to be a massive pub-sub/notification system (I think at the time there was a better article that I read, but can't find the link now, so that one will have to do - if anyone thinks they know the one I mean then please add to the comments!).


This is a space that I think Yo could step up and fill. And its possible that they are thinking the same - having already announced their API, they already suggest some simple notification systems that could utilise it - In which case, it could become a really interesting platform.


So who knows, there is potential for it to become something pretty neat. Odds are on it will just end up a passing gimmick though.
Continue Reading

The Idiot Box: Disrupting TV

There has been a lot written in recent months about the changing role of content on the web and devices, particularly recently with Apple's acquisition of Beats (things just aint the same for gangsters) - and Benedict Evans recently wrote questioning whether content is actually still king. And I think we can agree he makes a good point, when it comes to music, it is no longer a USP, its just expected. Between YouTube, SoundCloud, Spotify, Google's Play services etc there is no reason people can't just stream any music they like, fairly seamlessly, and switch between providers/apps just as easily.  Apple had invested massively in iTunes, but the iTunes buy-download model isn't what people want any more (hence buying Beats, actually for their streaming service maybe).


A more interesting area is Television - content is still a key factor, NetFlix, Amazon Instant(formerly LoveFilm) etc are largely compared entirely based on their content - as a service there is little between them, other than content.  So really it's no suprise to see all the normal big players getting involved in the market Google (ChromeCast etc), Amazon Instant, Apple TV.

Within 5years I think we will see a massive shift in viewing patterns, with pretty much all new TVs sold today being web-enabled I think its an inevitability that people will move to all on-demand services rather than being dependent on scheduled programming. Within a further 5years I wouldn't be surprised if scheduled programming was all but dead and gone.

We recently bought a NowTV box - at just £10 its a pretty low barrier to web-enabling your TV - and we have pretty much switched over to entirely on-demand service - despite having a PVR we still go for the on demand options.


I think there will be some interesting things that come from this:

Platform Fragmentation

I think this is the biggest problem facing the market at the moment, and to me looks like a massive opportunity. At the moment there is so much fragmentation across the platform: Playstations, Xbox, Android, iOS, TVs (Sony, Samsung, etc) - all have their own platforms, and any on-demand service that wants to offer an app on every platform needs work from either the platform owners or the service providers - if its the platform owner then the provider looses control of UI/UX app features and consistency across platforms, and if its the service provider then they have a lot of work to do to support the different platforms, and they will inevitably have to make decisions whether or not to bother with each platform.

At the moment, in the UK, if you buy a web-enabled device then you can't guarantee that it will have the basic UK free-to-air OD services (BBC, ITV, Channel4, Channel5) - When I bought the NowTV it didn't have Channel4 apps and still doesn't have ITV (and that's a platform backed by BSkyB, which is a fairly large organisation and platform). There are already some Sony TVs that are no longer supported and provider apps are no longer being developed/maintained/supported. 

Android and iOS aside, all platforms will suffer this problem - that is until someone comes with a platform standard/OS that is open and can be re-used across devices. And given Android's prevalence, and Google's investment in TV it would seem like the best placed candidate to tackle that, but let's see!


What is driving creation production?

With scheduled TV there are quiet times, early hours of the morning, working week daytime - and content is created for that specifically. Providers have to each fill their schedules for these hours, so these are commissioned/bought/run.  However, if scheduling was to end, and it was all on-demand then would people still make this content? I'm sure there will still be a demand for some of this content, but people who watch it just because its on will obviously diminish. Students in the UK have had a tradition of watching daytime TV, whether it be Countdown, Diagnosis Murder or Quincy - but in the era of on-demand why would they search out this content? 


Continue Reading

Ideas

So this is just a quick note on some things that I am personally finding really interesting right now

Education 

as mentioned, I think this is going to be cracked soon, and maybe Khan academy will do it. Either way, I think whoever does it would need to be a "full stack" startup. I have thought about trying some things - like a GitHub type system for open-sourcing education materials and resources, creating open-standards for curriculum and educational texts etc - but they have always just been tools or things around the periphery. I don't think I have the resources right now to be thinking full stack!


Android first

Android is the most pervasive mobile OS (yes, there are some caveats about the stats, such as numbers coming from China, and the value of the customer compared to other iOS), and Google continue to widen their reach (recently purchasing Nest etc) I think we are going to seeing our first truly Android first apps. If Instagram was built today, would it be iOS first? Probably, but I think that landscape is changing. I think coupled with the following areas this could be a big one.  If I'm building a mobile app it's going to be Android first. Silicon valley is under-invested in Android - There is also growing appetite for it:



Smartphone as your social graph

I have mentioned this one a few times here. Smartphone usage is continuing to increase (even whilst tablet sales falter) and more and more existing mobile users across the world upgrade. Further to this, your smartphone is really where you social graph is. Your address book has your contacts in it, your phone number gives you a unique identifier, and as WhatsApp proved, it gives great power to mobile first startups to disrupt the social network incumbents. It could be argued that Google+ was never going to succeed to usurp Facebook as king of the social networks because people are lazy, and essentially creatures of habbit - if all your friends are on Facebook, why try to convince your entire network to switch to g+? But the smartphone takes away that power, as your network is on your device, not a particular platform.  Couple that with the fact that so many use their phones to double up as cameras it now means most of our photos/videos are also on the device.

Television

I was recently talking with some colleagues about the future of TV and I speculated that in 5-10 years we may see the end of scheduled television, and everything will be only on demand. This would leave some interesting questions/problems:

  • One big problem as I see it is the fragmentation of device software - if you are creating an on demand app, you need to think about Android, iOS, browsers, Xbox, PS, not to mention all the TV manufacturers that have their own software running on their web enabled TV. This means at the moment, if you buy a new web enabled TV or set top box, the on demand apps may not be available, and may not be consistent. I think this could be a good market for Android and wouldn't be surprised if they do make some bold moves in this area (yes, bolder than Chrome Cast) - I would think it would make sense for them to be linking up with TV manufacturers to as the de-facto TV OS (would benefit from android app eco system - even if it would mean a LOT more work for app developers to fix up for another range of screen sizes
  • Another interesting implication of such a switch would be would we see a decline in produced content. At the moment there is a lot of content created specifically for quieter times of the viewing schedule (mon-fri afternoons, early morning, etc) - what might be considered as "filler", but we might see a decline in this as consumers will have complete control of what they want to watch, and there won't be any watch it because its on mentality.
I think its an interesting areas where battles are really going strong, with big players in the content production/distribution space (Netflix/Amazon/YouTube) as well as Google/Apple etc taking on the incumbents in hardware.  I just today saw a link for a site called Glass that is dedicated to ongoing conversation about this topic.
Continue Reading
Subscribe to: Comments (Atom)