Android ORM - GreenDAO - Simplifying DB access in your app

I am currently working on a new Android app, and as usual, it requires a considerable amount of DB interaction.  I would say DB interaction isn't that difficult in Android - you can create a few simple classes to setup your schema and then DAO functions for the various domain objects that you want to map to the DB.

Simple.

Just a pain in the ass.

Coming from working on a variety of Java type ORMs (Hibernate et al), this is really just all boiler plate that is painful to write, so this time around I decided to look for an Android ORM. The first result in google was greenDAO, which claims to be being used on over 10million installed apps (AppBrain lists it as being used on Pinterest and Zynga apps), which seemed like pretty good reason to take a look.


So I started taking a look at the code on GitHub - the first thing I noticed was that it wasn't a traditional ORM that I expected, really it's more of a code generator rather than an ORM like Hibernate - I was expecting to just define some POJO entities, maybe marked up with annotations or config  that did all the magic - but what is actually going on is you have to create a "generator" project that defines your domain entities and then that produces several DAO and domain object classes that can be generated directly into your normal Android app.

Does generated code always suck?

I saw the words generator and alarm bells started to ring - I have worked with and on projects that generate code and it so often seems to have descended in to pain - often needing to slightly tweak the generated code, which then results in painful re-generation or just walking away from the generator altogether and being left with having to manually maintain generated code.

However, I had a look at the generated code and it was actually quite nice - not that different from if I had written it, and what swung it for me was the performance. I had briefly pondered building a lightweight ORM for Android more in the style of Hibernate (I love an annotation), but then I thought on the performance impact - reading annotations on POJOs at runtime - even if cached - the performance footprint was likely to be pretty big would suck for people with older/less powerful devices.


So actually, removing the pain of having to create all the DAO objects etc, whilst not having a prohibitive performance impact made this a pretty strong choice.


Let's get down to business, don't got no time to play around, what is this..

So lets have a look at how we generate the code - its actually relatively easily to programmatic-ly configure your entities for generation.





The above is an example of how you can configure two simple domain entities (which will map to a table - this is like defining a Hibernate @Entity POJO). In the above simplified, hypothetical design, we have two real domain entities "Email" and "People" - emails can be sent to multiple people, and people can recieve multiple emails.

As you can see, we define a very simple People entity. and then programatically add three columns - an ID, a name and an email address. Simple - and as you can see(or imagine) the API provides programatic constructs to configure not null, column types, etc.


All in all, so far it has been a good experience - It has removed a certain amount of the pain from developing the DAO/DB layer boiler plate stuff and let me concentrate on the important stuff about the app. However, one gripe I do have is that it doesn't currently support modelling of many-to-many relationships - the best you can do is create an entity to model what is effectively a join table, then have a one-to-many relationship from that to either side of your "real" many-to-many relationship.

Again, in the above example, we try manage the many-to-many relationship between people and emails (emails can have may people recipients, and people can be recipients to many emails). As mentioned - we have had to model this join explicitly with an EmailPeople entity, and then add the OneToMany relationship between People/Email to our new Join entity.  This is no real headache to do, but is undoubtedly going to be a performance hit - The underlying code will undoubtedly be being cleverly efficient in retrieving all the EmailPeople entities for a given email (assuming attempting to get all recipients to an email) - but from that list we then need an efficient way to load all the People enties linked to that list of EmailPeople entites.
 
Continue Reading

Raspberry Pi - Getting Started (& ready for hardware hacking)

So, it was all plain sailing getting the Pi up and running with NOOBS following the instructions there, and quickly got it all setup with wifi connectivity and ssh access (which is essential for me, as I don't have a HDMI monitor other than my TV in the living room - so until I could get ssh access from my normal workstation it meant having to take over the living room to configure the Pi on the TV, which my wife was not best pleased about! Also a much cheaper option than buying a new HDMI monitor just for the Pi).

I had been looking around for how to get started with some hardware hacking with the Pi, and must admit the last time I did anything vaguely electronics was when I was in school at 14yrs old (actually I did a robotics module in uni, but that was still mostly programming stuff rather than actual chips/resistors/wires etc).  Some of blogs were writing about how great/essential the Adafruit Pi Cobbler was, and again, I had to go and google and read up exactly what a breakout "cobbler" would be useful for etc but decided that enough people were going for it to invest


(So yeah, that's the Amazon affiliates link - it means I get some money from amazon if you buy this thing from here - it doesn't cost you anything, but is nice for me, so by all means click & buy!)

That lead me on to looking more extensivley at the adafruit site, which has some nice tutorials and all recommend using the Occidentalis OS if you are planning some hardware hacking.



Setting up Occidentalis OS

Occidentalis doesn't come in as nice a package as NOOBS - You basically need to get the image, flash it to an SD card, then boot up your Pi.  I'm on Windows 7 setting up the SD card, so here are the steps/software I used:
  • SD Formatter (from the SD association - this is one of the standard tools recommended in normal Pi setup tutorials and is a pretty convenient way to format SD cards) - https://www.sdcard.org/downloads/formatter_4/
  • Fedora ARM installer - This can be used to install OS images on to an SD card - you can't just copy & paste downloaded images on to an SD card and this just makes it pretty easy - https://fedoraproject.org/wiki/Fedora_ARM_Installer
  • Occidentalis distro image - This is Adafruit's mod of the Raspian Wheezy OS and is modified specifically with hardware hacking in mind - The latest version is currently at v0.2 - http://learn.adafruit.com/adafruit-raspberry-pi-educational-linux-distro/occidentalis-v0-dot-2


 Getting the Occidentalis image and flashing it to an SD card is pretty straight forward if you have the above software - the steps are as follows:
  1. Format an SD card - it really needs to be 4gb - I am using an 8gb samsung card that is working fine (disclaimer: as with any formatting or writing images to drives - make sure you are targeting the correct drive for your appropriate SD card - as all data will be wiped!)
  2. Assuming you have downloaded the Occidentalis image from the above link, extract the zip file locally
  3. Right-click and select "Run as administrator" on the Fedora ARM installer executable.  For the source option, select "Browse" and navigate to the .img file you have just extracted from the zip. For the destination triple-check that you have your SD card drive selected (again, this drive will be wiped and all data overwritten when you hit the install button). Then press install. This will correctly setup the SD card with the OS image.
  4. Jam your newly setup SD card in your Raspberry Pi and startup as usual. For most of the boiler plate setup, this will work just like Wheezy/NOOBS setup.

Uh.. Wait.. My Pi isn't Booting..

So yeah. This happened to me. I setup the SD card and jammed it into my Pi and nada. The red power light flickered on, but no signs of activity and no output on the HDMI.  I did some research, and it turns out that there is a problem with some OS booting if they are using the Hyinx/Samsung RAM chip on the newer models of the Pi. However, my Pi used neither of those RAM chips, mine was fitted with the Micron RAM chip, but still suffered the same problem.

Thankfully with a little tinkering, the same solution generally seemed to resolve this problem for me. If you are experiencing this problem, here is what fixed it for me:
  1. Download and flash and standard OS distrubution to the SD card - I just grabbed the standard Wheezy distro from the Pi download site and flashed that to my SD card as per above instructions 
  2. Once on the SD card, I backed up the files written (only ~35mb) locally
  3. I then installed the Occidentalis image to my SD card (the image that wasn't booting previously, again using the steps above)
  4. I then copied the bootloader files from Wheezy backup on to my SD card - this was just overwriting the bootcode.bin and start.elf files from my SD card with those I had backed up from the standard Wheezy distro.
  5. Jam the SD card in and power up..

So.. I'm off. I will now get the normal wifi stuff setup, plus set up the ssh keys for my github account so I can stick all my code on there (I'm not relying on some flaky SD card for my code backups), the next Pi post will likely be about the Cobbler and basic stuff connecting the Pi to a bread board and a few LEDs, maybe a motor or two.. but the first step for that is soldering my Adafruit Cobbler.

Yep. I'm gonna have to use my hands..
Continue Reading

A Brief Update

So, once again I am on paternity leave, and whilst that may seem like a most unlikely time for me to have more spare time on my hands (I really don't.. this parenting stuff is non-stop), it does afford me more time where my job is just watching a sleeping baby to make sure he's ok, which gives me the opportunity for some quieter time.

Last time around I experimented with and wrote up some work with Spring & MongoDB (see previous blog posts here and here) and this time I have a whole host more planned.  Firstly, I am getting back on some Android development and am working on a new app - part of which is using greenDAO ORM for Android (apparently used by Pinterest.com, which is pretty good - although I would think describing themseleves as an ORM might be a little stretch, at first glance it certainly seems to make DB/DAO Android work a lot less of a pain in the arse) - I will write up some notes on using that soon.



The other part is a Raspberry Pi project. I held back from getting a Pi for sometime, as I didn't really have any projects in mind, and as tempting as it was, I didn't want to get one and just have it sitting in a box unused for ages (or alternatively just using it as a mechanism to web-enable my TV) - However, having been inspired by this project I decided that I wanted to build a lego robot powered by the Pi, and long term, having a robot that can do clever things (map out rooms, shortest path algo stuff etc) but also have a code framework that makes it easy for kids to be able to program rules about how the robot behaves in different circumstances etc.

I have a Pi, I have dug out all my old Technic Lego (including motors) and have discovered the joy of buying lots of incredibly low-priced electronics kit (resistors, chips, breadboards, leds, etc) and am almost ready to start - just have to solder a few pieces together and we are off (well, that and learn some basic electronics stuff!)


Expect post on both topics shortly..
Continue Reading

Traversing data structures - A Groovy Visitor Implementation

I have been using Groovy for a while now, having come from a solid Java/J2EE/Spring/ORM background where patterns and solid OO is a mainstay.  Although it took me a bit to get in to the swing of the Groovy stuff, I have now really taken to it (in no small part, encouraged having taken the Coursera FP class) - The simplicity and ease to use the built in functional stuff like .each{}, .findAll{}, .collect{} is really neat, and once you get started with them there really is no stopping!

As Groovy is dynamically typed (not weakly typed though!) you do end up using a lot less POJO classes and a lot more Maps, Lists of Maps, etc (also Groovy makes it really easy to work with these guys) and I find myself fairly often needing to traverse complex, dynamically typed data structures to do some kind of data processing.  Normally, in my Java OO background, when frequently processing tree type structures (a nested Map of Lists/Maps/Simple elements can really be thought of taking this form) I would fall back to the Visitor patter (if you aren't familiar with the GoF Visitor pattern, see here, here, etc for details), but if you have a dynamically typed complex data structures, and you don't really want to have each potential node in your structure to have to implement a set interface with a visit() method on it, then I use the following approach.

(disclaimer: yes, it feels as though it is a hacky, dodgy approach of the pattern - but it works well with Groovy and has been working well for me. If you have ideas on how to improve etc I would love to hear thoughts in the comments!  As such, I like to refer to it as "The Unwelcome Visitor Pattern").

First, I create an iterator class - this basically has the code to iterate through a nested, complex structure - I see this as boiler plate code that is a pain to re-write and will be used by all code that wants to traverse a complex data structure (Map with n-level deep nested Lists/Maps)



As you can see, its just a basic re-cursive piece of code to traverse List/Maps - you will note that it expects a visitor class to be passed in to it as an argument on first calling that has a visitMap() and visitList() methods.  In normal Java, this would need to be a class that implements a particular Interface/Abstract class that has implementations of the required methods. However, as Groovy is a little more dynamic we can do some pretty nice on the fly stuff (yes, I know, if you are performing some really common stuff, you may still want to have the traditional Java interface/explicit class approach as well, but that's not why we are here!).  The code below is an example of doing some on-the-fly processing of a dynamically typed complex data structure (in this case, we are just converting all Date objects to Strings, but this is just an example for funsies)



As you can see, in the above we are using Groovy's ability to create Interface implementations as Maps and just defining a simple closure for each of the visitMap() visitList() methods.


It may not be the most graceful solution, but it works simply and allows easy definition of closures that can process Maps/Lists easily (could also be used in the same way fo rtraversing JSON structures etc)
Continue Reading

Login with Friends

Social logins seem to be quite a divisive topic. Every now and then an article will pop up, and regardless of which way the author leans, the comments always seem to be almost always opposing the argument.


I think, by and large, we all agree that the login model on the web is kinda broken, and is continuing to become more and more fragmented with every new site gaining traction and offering their authentication API as a service.


To put it in fairly broad strokes, I think the main points for either side are:

Pro
  •     Simplifies login for your users — by offering one less username/password combo to remember it reduces the barrier to entry/conversion for your potential lead
  •     Outsources security — There are a lot of very smart people at Google/Facebook/etc that are working hard on authentication and security. Outsourcing your authentication process to them is surely better than trying to roll your own?
  •     Viral promotional channels — Authenticating with the likes of Facebook/Twitter will also offer you the possibility of easy integration with their social-sharing API

Cons
  •     Outsources security — the flipside of this is you are dependent on the third party getting security right. LinkedIn have been one of the high-profile victims of user account details hacking. If you were authenticating with LinkedIn, then your user accounts have also been compromised
  •     Reliant on third parties — should you ever try to build something that is reliant on the eco-system/infrastructure of another organisation? Will they always be around? Will they always be popular?
  •     Brand association — Especially with the recent Prism/NSA fallout, do you want to be associated with all these brands? Do you know that your users still trust these brands?
  •     Viral promotional channels — Many users are wary of apps that connect to third parties, and fear that if they sign up with Facebook then the application may try to share things with out their approval



Personally, I don’t like them. I generally don’t use them, but have done on occasion (yes, I know, medium is linked to my Twitter account).


I don’t like them for two reasons:

As a user it frustrates the hell out of me when I go to a site and I have to try and remember which registration mechanism I used — it normally goes something like this: 


“Did I sign up with email? Let me try my usual password”
[Username or password incorrect]
“Hmmm, maybe I did and have just put the wrong password in, let me try another password..”
[Username or password incorrect]
“Hmm, still no joy, I know, I will try password reset — that should send me an email if I have a password setup”
[Wait for email.. no show]
“Ok, must have signed in with Twitter”
[Click sign in with Twitter.. “This Twitter account is not linked to an account, please enter account details to link now”]
[close the tab]
[Later, receive a password reset email, confirming password is now a randomly generated password]
[sob quietly to myself]
 
From a product point of view, I don’t like them because I think they are ugly. I think littering your site with a myriad of other companies logos and brands is something of an eyesore, and provides suitable noise and clutter to distract from the key conversion goals of a page. Really, the only time I will sign up using these mechanisms is when they are the only option and I am already suitably sold on the product that I know I want to sign up (Medium being a good example — I would have signed up using a traditional mechanism given the choice, but I was already sold on the product so put up with the fact that I had to sign-up with Twitter).


By and large, I agree with MailChimp CEO, Ben Chestnut, and I am glad that in this case the evidence ended up being in his favor. However, I do appreciate that in other times, there is probably evidence showing them more favorably, and at times, I’m sure they can be pivotal in increasing a product’s viral coefficient or lead-conversion, rate — and in those cases, you have to go with the data.



But if I’m building something, until I see the evidence, I’m most likely going to avoid them.
Continue Reading

Essential Resources to Become a Life Long Learner (in tech)

Is one of your New Year Resolutions to re-skill? Thinking about re-training for a new career (or even just a new hobby) in tech? Then you're in luck! 

Today, more than ever, the barrier to entry for starting to learn a new technology or programming language is all but nonexistent, all you really need is a computer (or even a mobile device) and a web connection and you are pretty much good to go - just choose your preferred technology, an IDE and get started.  Almost everything is open source or at least free to use for a single developer just out to learn and there is a wealth of blogs, articles and Q'n'A sites ready to help you with tutorials, walk-through's and helpful advice - many of these backed with ready and running code bases on GitHub free for you to play with and generally work out what is going on.

However, with all these resources it can sometimes be a bit daunting with so much content. Once you have chosen a language how do you know where to start? Here are some of our favourite sites and resources that we have discovered and found useful in learning new skills:

  • iTunes U - a lesser known category on iTunes is their academic section, iTunes U(niversity) featuring loads of podcasts and lectures from a range of academic organisations, and some of this stuff is serious! Several large universities have uploaded full lecture series there, and by and large they are free to download (yes, you have to install iTunes, which sucks, we know).  Want to take the full term of Stanford university's iOS course? Its up there. Want to learn AI for chess playing from Cambridge uni? Yep, got that too. And for free.
  • MIT OpenWare - MIT have been one of the strongest advocates of open sourced education. A lot of there lecture series are online (can also be found on iTunes, but can be avoided).  Is it just us who thinks its amazing that anyone around the world with a web connection can get educated by the most prestigious academic organisations around?
  • Khan Academy - there is a lot of hype around this one, well funded with some pretty big names supporting it (jQuery creator John Resig is a Dean there), a not-for-profit aiming at providing free education for everyone. The academy provides lots of video based courses as well as interactive challenges and detailed stats on how you are doing.
  • Udacity - this is another recent, well-funded startup trying to tackle free higher education for all. Founded my three robotocists it is slowly building a very respectable catalogue of uni level courses ranging from CS101 to AI for robotics. As with the Khan academy, the lectures are purely for the web so the videos are clear and designed for remote learning (different from the filmed university lectures which are targeting classroom based learning).  We have recently created and Open Sourced the Spring-Social implementation of the Khan Academy API - so if you are working with the JVM and want to have a play with the Khan Academy API then check it out on GitHub
  • CodeAcademy - we have mentioned before we are fans of code academy, code academy is an in-browser development environment that walks you through programming exercises to help you learn with your hands - currently supporting JavaScript, HTML, Ruby and Python
  • Free eBooks - there are loads of great free eBooks available online, so many there is no point listing them, instead I will just point you here. Which leads nicely on to the next point..
  • StackOverflow - what really needs to be said about SO? It is the definitive q'n'a site for tech. If you are just starting learning head over and sign up, the help from the incredibly active community over there will be invaluable (although be sure to read the posting guides, they can be a little unforgiving at times!).
  • Coursera - Another massively popular online learning resource, this one recently generated a lot of interest with its recent Scala course taught by the original creator of the language!  We are currently working on some secret integration with Coursera at NerdAbility, and you will soon be able to integrate your Coursera account and show off which courses you have completed!

Hopefully the above resources help on your path to re-skilling.  In reality, getting your hands dirty with code and trying to solve problems and fix errors is the best way to learn, so don't forget to get stuck in - and maybe when you are more confident try answering questions on StackOverflow!

Of course, with these new found skills you will want to show them off, so we'd recommend heading to NerdAbility and registering (if you haven't as already) and update your skills, add your StackOverflow profile and even add a custom section talking about what you are learning (employers always love to know that candidates are proactive and motivated when it comes to learning new things and keeping up with technology). 

 Leave your comments with any other tools and resources you have found useful in your journey of becoming a life long learner.
Continue Reading
Subscribe to: Posts (Atom)