Showing posts with label development. Show all posts

Building a RESTful API like a Product Manager

Something I have been thinking about for a while is the idea of APIs as your Product - the idea that you should design, build and test your API like it's a public facing product (as it might actually be if its a public API), and what lessons we should be learning from Product Managers about how we achieve this.

In the latest instalment of their "Technology Radar", ThoughtWorks recommended the concept "API as a Product" as ready to trial (although I think it's far beyond trial stage - I can see no real risk of adopting the lessons from Product Management for your API).



I have generally been interested in Product Management for several years - I think it's hard not to get interested in the science of Product Management when you are building side projects, as you naturally have to think about the product you are building - sure, with side projects they can easily turn into vanity projects, where you just build what you think is cool, but I think it's still fun to think about real users and good Product Management techniques that can be applied even to side-projects.

Having tried to gain some understanding, I have turned to some classic material on the subject of Product Management: The Lean StartupDon't Make Me ThinkHow to Build a Billion Dollar App. I have also been lucky enough to attend Mind The Product Conference twice (though volunteering - but managed to watch lots of the talks), and to have worked with several great Product Managers.

So what lessons can we start applying? I think, if one statement sums it up, its an idea from Kathy Sierra:

Does your product make your users awesome?
This is a great way to think about APIs (I'm mostly talking about REST/GraphQL type APIs here - but this thinking is a great way of approaching design for normal class/interface/library APIs too).

As engineers, I'm sure we have all used APIs that don't achieve this - APIs that make us have to think, that make us feel stupid, that seem overly complicated (for example, the old Java Date library - the number of times I had to google just to find out how to simply create a specific date for something simple like a unit test is ridiculous). And hopefully, we have all used APIs that make us feel awesome - APIs that make us productive, that let us do what we expect to be able to do, and can (with the help of the IDE's auto-complete) let us guess what the method calls are.

Another idea Kathy Sierra mentions in that talk, and is also made famous from the title of the Product Management classic, is:

Don't make me think

I have written before about the StackOverflow and GitHub APIs, but they deserve a mention again, if you have ever written an integration with them, you will know.  The APIs are designed in such a way that sensibly models the domain model.

For example, take StackOverflow, if we were going to describe their domain model, what first class objects would we have? I guess Questions, Answers, Comments etc - and sure enough, their API models these:

/questions - will get you all the questions on the site

/answers - will get you all the answers,

Now, how do you think you might get all answers for a given question? Well, a list of answers should probably be a child of a question, so something that represents that, something like:

/questions/{Question ID}/answers

Sure enough, thats the URL - and the behaviour is consistent through the design, I'm sure you can guess how you might retrieve all comments for a given answer.

I'll be honest, developing against an API where I can guess the endpoints makes me feel pretty awesome.

MVP and the Lean Product

The idea of a Minimum Viable Product is a core concept in Lean Startup - the idea that you build the minimum version of the product that can get you feedback as to whether you are heading in the right direction before investing more effort in building out a fuller fledged product.

Whilst I think an appreciation of an MVP is a good awareness for any engineer - I have generally found it is useful in facilitating conversation when building a new product - if as an engineer you recognise some new feature is going to be considerable effort, its helpful to think about what hypothesis the feature is trying to test, and is there a simpler engineering solution to test the hypothesis.

However, the principal of not over engineering the API is also a good one - once you have a well designed domain model, its quite tempting to get carried away with REST design best practices and start building endpoints for all different variations of the model - however, unless you need them, it's better to start with less (if you are building a public REST API from the start then it might be harder to know what isn't needed, but you can still certainly build an MVP version of the API to test the hypothesis - e.g. whether people actually want to use your API)

User Testing

Another important part of building a product is collecting user feedback - Whilst a REST API may not be well suited to A/B testing (your consumers won't be very happy if switch the API design on them after they have implemented against a specific variation!), listening to user feedback is still important - because the consumers will likely be engineers (writing the client code), then hopefully they will be expecting REST standards and good API design principles, but another way to think about this is the importance of not making breaking changes to your API.

A technical way of thinking about this is Consumer Driven Contracts Testing - As RESTful APIs are de-coupled from the client code, the API doesn't have any idea what clients are using or doing, so this approach allows consumers to define what they expect from the API, and then the API can always test against these contracts, so be sure that any change to the API is still in keeping with existing consumers, or users, of the API.

Conclusion

I think, in general, all engineers can benefit from thinking about how good products are built, what concepts are important, and be able to think about the products or services they are building in this way - both in terms of thinking about the actual implementation as a product that someone else will be the user of (either externally if a RESTful API, but also internally if its cross-team or even just someone coming to take over your code once you have moved on).

Eclipse & LESS - Better Development time with Incremental Builds

All source code for this app is on GitHub at the moment - feel free to fork/download/etc

So one of my recent experiments I decided to start playing with LESS (CSS pre-processing, letting you use variables etc in CSS code, which has been long needed) - It's all actually relatively simple (although you can do some nice, powerful stuff with it) to code with if you are coming from an development background, so I found myself quickly getting to grips with basics like defining variables for colour schemes (very basic stuff, but sooooo much better than having to maintain masses of CSS and every time you want to tweak the colour scheme having to search for hex codes..).

However, I found myself wanting to start using LESS on new/existing web app projects which were largely Spring MVC/Java based apps, which was more problematic. LESS requires pre-processing/compilation to generate CSS, which is kind of a pain in the ass when trying to prototype web app projects - being used to just altering some CSS and then hitting refresh in the browser (at most, having to re-load the tomcat server inside  eclipse), having to try and generate the full LESS/CSS files everytime anything changes was not something I was going to be doing.

Thankfully, having done some digging, I managed to get it all setup and working like a charm - Every change I made to the LESS, Eclipse performed an incremental build (like it does for other compiled code like Java etc) and voila! I could just change the LESS and press F5 in the browser.  Here's what I did:

Software

  • Eclipse
  • Maven
  • LESS
  • WRO4J Maven plugin

Directory Structure

As I am using maven, I am of course following the standard /src/main/webapp convention - so that is assumed here.
In the webapp root, I created a new "less" folder, alongside the "css" - This holds all my *.less files - I won't go into detail about LESS here, but assume you will only include valid, compiling LESS files here (non-compiling LESS files will cause you a headache here)



As you can see, in my normal "css" folder I have only included vendor CSS files - I will not be placing any custom CSS here (All valid CSS is also valid LESS, so even if I wanted to just write CSS, I can do in my "less" directory).

Maven Configuration

The incremental build will be performed by the use of a Maven plugin. This will also mean that any formal full build/packaging that you do will also include the LESS compile step.

The below configuration simply defines a "group" that will attempt to compile and the target folders we will generate the artefacts to (more on that later).



We also define a little more confuration regarding what we want WRO (Web Resource Optimizer) to do:

/WEB-INF/wro.properties defines exactly what optimization we want to perform - in this case I have selected some pre & post processing steps:


/WEB-INF/wro.xml defines the location of the source files:


The above config will make Eclipse perform the incremental builds on your LESS files as well as generating a CSS file if you perform a regular maven build.  As can be seen below, Eclipse incremental build has generated a web-all CSS file in the target "css" directory.


Including Generated CSS

The simple part of this is of course to include it in the webpage header as an import - Despite it not being in your /src/main/webapp/css folder, you can still reference it the same as the static vendor stuff we have included as we know it will be generated there for us (either as part of full app build/deploy or incrementally in Eclipse).


There is also another gotcha at this point - You have to check that your Eclipse will be deploying the contents of your m2e-wtp directory to your webapp. This should be happening by default, but you can check (and fix if necessary) by right-clicking on your project in Eclipse and going to "properties".  In the left hand menu, select "Deployment Assembly" - this will show the directories/libraries that will be deployed as part of your application (obviously this only takes effect if you are deploying the app to a server inside of Eclipse).

As you can see below, my Eclipse defaults to deploy /target/m2e-wto/web-resources to the root of the webapp. If yours doesn't just add an entry for this and everything should work!






Footnote:

If you are struggling to generate the LESS files in the m2e directory, then there is a chance you have an compilation problem with your LESS files.  With the incremental build, if LESS compilation fails then it will fail silently - for full details you should run the full maven build to see errors logged.

Essential Resources to Become a Life Long Learner (in tech)

Is one of your New Year Resolutions to re-skill? Thinking about re-training for a new career (or even just a new hobby) in tech? Then you're in luck! 

Today, more than ever, the barrier to entry for starting to learn a new technology or programming language is all but nonexistent, all you really need is a computer (or even a mobile device) and a web connection and you are pretty much good to go - just choose your preferred technology, an IDE and get started.  Almost everything is open source or at least free to use for a single developer just out to learn and there is a wealth of blogs, articles and Q'n'A sites ready to help you with tutorials, walk-through's and helpful advice - many of these backed with ready and running code bases on GitHub free for you to play with and generally work out what is going on.

However, with all these resources it can sometimes be a bit daunting with so much content. Once you have chosen a language how do you know where to start? Here are some of our favourite sites and resources that we have discovered and found useful in learning new skills:

  • iTunes U - a lesser known category on iTunes is their academic section, iTunes U(niversity) featuring loads of podcasts and lectures from a range of academic organisations, and some of this stuff is serious! Several large universities have uploaded full lecture series there, and by and large they are free to download (yes, you have to install iTunes, which sucks, we know).  Want to take the full term of Stanford university's iOS course? Its up there. Want to learn AI for chess playing from Cambridge uni? Yep, got that too. And for free.
  • MIT OpenWare - MIT have been one of the strongest advocates of open sourced education. A lot of there lecture series are online (can also be found on iTunes, but can be avoided).  Is it just us who thinks its amazing that anyone around the world with a web connection can get educated by the most prestigious academic organisations around?
  • Khan Academy - there is a lot of hype around this one, well funded with some pretty big names supporting it (jQuery creator John Resig is a Dean there), a not-for-profit aiming at providing free education for everyone. The academy provides lots of video based courses as well as interactive challenges and detailed stats on how you are doing.
  • Udacity - this is another recent, well-funded startup trying to tackle free higher education for all. Founded my three robotocists it is slowly building a very respectable catalogue of uni level courses ranging from CS101 to AI for robotics. As with the Khan academy, the lectures are purely for the web so the videos are clear and designed for remote learning (different from the filmed university lectures which are targeting classroom based learning).  We have recently created and Open Sourced the Spring-Social implementation of the Khan Academy API - so if you are working with the JVM and want to have a play with the Khan Academy API then check it out on GitHub
  • CodeAcademy - we have mentioned before we are fans of code academy, code academy is an in-browser development environment that walks you through programming exercises to help you learn with your hands - currently supporting JavaScript, HTML, Ruby and Python
  • Free eBooks - there are loads of great free eBooks available online, so many there is no point listing them, instead I will just point you here. Which leads nicely on to the next point..
  • StackOverflow - what really needs to be said about SO? It is the definitive q'n'a site for tech. If you are just starting learning head over and sign up, the help from the incredibly active community over there will be invaluable (although be sure to read the posting guides, they can be a little unforgiving at times!).
  • Coursera - Another massively popular online learning resource, this one recently generated a lot of interest with its recent Scala course taught by the original creator of the language!  We are currently working on some secret integration with Coursera at NerdAbility, and you will soon be able to integrate your Coursera account and show off which courses you have completed!

Hopefully the above resources help on your path to re-skilling.  In reality, getting your hands dirty with code and trying to solve problems and fix errors is the best way to learn, so don't forget to get stuck in - and maybe when you are more confident try answering questions on StackOverflow!

Of course, with these new found skills you will want to show them off, so we'd recommend heading to NerdAbility and registering (if you haven't as already) and update your skills, add your StackOverflow profile and even add a custom section talking about what you are learning (employers always love to know that candidates are proactive and motivated when it comes to learning new things and keeping up with technology). 

Leave your comments with any other tools and resources you have found useful in your journey of becoming a life long learner.