Showing posts with label webapp. Show all posts

Managing your personal finance (a Java webapp)

"Cash Rules Everything Around Me" - Method Man

A few years ago I was fed up of how useless all the banks were at helping me understand my transactions/spending patterns with their online banking. Basically, they all just seemed to let you log on, and then see a list of transactions (often just referencing codes) of payments in and out by date. Nothing else. It seemed like there were lots of simple things missing that could make it a lot more useful, to name a few:


  • Smarter tagging/categorisation - where some direct debit was just listed by the code of the recipient rather than meaningful info, or being able to group all the spending in coffee shops
  • Alerting to changes/unexpected behaviour - for example when a fixed term-price contract comes to an end and the price changes - detecting a change in a regular fixed aount/recipient should be easy

So this is what I started building - a lot of the functionality has since been built into banks such as Monzo, the aim of this webapp was simply to allow you to import transaction histories (most online banks I use provide the ability to export transactions in csv) so you could view/filter/tag any existing bank data. It only supports Santander at the moment, and the alerting stuff never actually got built in, but I thought I would chuck it on Github rather than have it just sit around gathering dust in a private repo.

I have written about the project a while ago, when it was first underway, but as it has been not worked on for a while, I thought I would move it to GitHub - hence the post here today. The app and all the code can be downloaded here: https://github.com/robhinds/cream


Building & Running

Its a simple java, maven app (it was several years ago, so not migrated to Spring Boot, Gradle, Groovy etc) - and it builds a WAR file. It also uses LESS for style stuff, which is also built by maven (although watch out for silent failures if you make changes!). If you are using an eclipse based IDE you can follow the guide here: to get incremental build support for LESS building (e.g. just change the LESS source and go refresh in the browser and it will reload).

You can run the WAR file in any tomcat/IDE server, and the backend is currently expecting a MySql DB (but can easily be switched for a differed DB driver if required)


Roadmap


  • Migrate to Spring Boot, Groovy, Gradle
  • Add further bank transaction importers
  • Add alerting framework



Screenshots

(I loaded in a generated transaction history - which is why some of the charts look a little weird)




Cream money management - A Side project

Cash rules everything around me, it's all about the money, dollar dollar bill yo



I have for some time been bemoaning the state of online banking in the UK offered by the major banks I have banked with. It feels like they just can't be bothered - they are confident that no-one is going to disrupt them so they just don't make any effort.

My current bank makes it almost impossible to manually pay off your own credit card online, and all it really offers is a list of transactions against an account - the only good thing it offers is the ability to download your transactions.

It's frustrating. They have so much information, but provide so little.  So I told my wife I would make us an app to make this better.

Features of the app are:

  • create multiple accounts to manage together
  • upload statement/transaction lists as exported from online bank providers
  • automatically categorise and tag as many imported transactions as possible based on a set of rules
  • using sensible full text search, attempt to categorise and tag any remaining transactions based on other similar transactions that have been tagged
  • allow manual tagging/categorisation of transactions (that will then feed back into later imports of transactions etc)
  • link together transaction groups - identify recurring transactions and group them so they can be automatically categorised and tagged and also provide alerts/warnings on changes in payments (e.g. if a recurring transaction suddenly increases in cost, then it likely suggests that a fixed price deal has come to an end etc, so the system identifies this and alerts the user to the change
  • easy filtering and cutting up of data based on category, tags, date, price, description etc.. basically anything
  • tonnes of charts to show the different cuts of data and interesting points


Spring Social Integration

So, you may notice that I am something of a Spring fan, and generally like to try out different Spring projects where I get the chance. I have been aware of the Spring Social project for a little while (was looking at integrating LinkedIn with flutterby but decided against it), so recently, when a competition was announced in work to create a Java web app in the cloud for a chance to win a new Android tablet I thought this would be a good chance to put together something of a showcase of technology.

As always, the web app is a straight forward Spring MVC project built with the current latest Spring libraries (3.1 - also making use of declarative caching with ehcache, which was nice) and also includes the Spring Data MongoDB integration (again - primarily because it seems to be the most commonly supported NoSQL db in the cloud.. well CloudFoundry and OpenShift anyways). I also took the opportunity to integrate some Spring Social stuff, so I will go through that, as its pretty easy (will post again on some thoughts about Redhat's OpenShift platform and further SpringSocial stuff).



Maven Configuration

First of all, there is obviously some Maven configuration needed in the pom. It took me a little while to find the correct repositories to get all the different Spring Social libraries from (some implementations are less mature than others so not all have full proper releases yet)


	 spring-snapshot
	 Spring Maven Snapshot Repository
	 http://maven.springframework.org/snapshot



	 spring-milestone
	 Spring Maven Milestone Repository
	 http://maven.springframework.org/milestone




 Next we need to add the libraries we need:


	org.springframework.social
	spring-social-web
	${spring-social.version}
	
		
			spring-core
			org.springframework
		
		
			spring-web
			org.springframework
		
		
			spring-webmvc
			org.springframework
		
	



	org.springframework.social
	spring-social-github
	${spring-social-github.version}



	org.springframework.social
	spring-social-twitter
	${spring-social-twitter.version}


As you can see, we take the core Spring Social library, and for my project I also used Twitter and GitHub integration (I also had LinkedIn early on, but removed that as the LinkedIn terms of use on the API are pretty limiting). I added the exclusions to ensure that the latest Spring libraries were being used, as these were declared elsewhere in the Pom.



Storing User Connection Details

Spring Social supports both OAuth1 and OAuth2, and provides the underlying mechanisms to perform the "OAuth Dance" which enables your application (once approived by a user) to capture the access token required to enable your application to access the third party API. Usually your application would capture the user details including access token and store them (encrypted of course!) so the user does not need to approve the application every time they use it. There are examples in the Spring Social showcase on GitHub of doing this with a standard JDBC connection implementation, but as I was capturing my user details in Hibernate I decided to implement a Hibernate version.

To implement your own storage mechanism you need two classes, a ConnectionRepository and a UsersConnectionRepository and must implement the same interfaces:

public class HibernateUsersConnectionRepository implements UsersConnectionRepository { 

public class HibernateConnectionRepository implements ConnectionRepository {


You can see my hibernate implementations here. Obviously I also needed a Hibernate entity to represent this connection details, but that was a straight forward entity as follows:

@Entity
@Table(name = "CV_CONNECTION")
public class SocialConnection {

       @Id
       private Long id;
       private String providerId;
       private String providerUserId;
       private String displayName;
       private String profileUrl;
       private String imageUrl;
       private String accessToken;
       private String secret;
       private String refreshToken;
       private Long expireTime;
       private Integer rank;


Configuring the Beans

The next step is to define the  Spring Social beans to use (such as Twitter, GitHub etc) so they can be used throughout the application.

For this I used a Configuration class rather than adding the config directly to my application context xml (this just involves creating a POJO and annotating it with the @Configuration annotation - this then allows you to create code equivalents of xml config that would normally exist in app context xml - we will see more in the details below).

@Configuration
public class SocialConfiguration {


The beans we need to define are the ConnectionRepository, UsersConnectionRepository, ConnectionFactoryLocator and then any Social interface that we are using, in our case Twitter and GitHub.

ConnectionFactoryLocator:


@Bean
public ConnectionFactoryLocator connectionFactoryLocator() {
	  ConnectionFactoryRegistry registry = new ConnectionFactoryRegistry();
	  registry.addConnectionFactory(new GitHubConnectionFactory(gitHubKey, gitHubSecret));
	  registry.addConnectionFactory(new TwitterConnectionFactory(twitterKey, twitterSecret));
	  return registry;
}

The ConnectionFactoryLocator is simply used to initialise the UsersConnectionRepository, for every SpringSocial third party you want to use, you need to add a ConnectionFactory instance as per above - all Spring Social implementations will provide a ConnectionFactory implementation.

The @Bean annotation is like defining a <bean> in the application context xml.

UsersConnectionRepository:


@Bean
@Scope(value = "singleton", proxyMode = ScopedProxyMode.INTERFACES)
public UsersConnectionRepository usersConnectionRepository() {
	  HibernateUsersConnectionRepository repository = new HibernateUsersConnectionRepository(connectionFactoryLocator(), Encryptors.noOpText());
	  return repository;
}

This is to configure the UsersConnectionRepository - as you can see, in this case we are telling Spring to instantiate a HibernateUsersConnectionRepository.


ConnectionRepository:

This one is straight forward as well - again, using the Hibernate implementation of the interface and

@Bean
@Scope(value = "request", proxyMode = ScopedProxyMode.INTERFACES)
public ConnectionRepository connectionRepository() {
	  Authentication authentication = SecurityContextHolder.getContext().getAuthentication();
	  if (authentication == null) {
			 throw new IllegalStateException("Unable to get a ConnectionRepository: no user signed in");
	  }
	  ApplicationUser user = (ApplicationUser) authentication.getPrincipal();
	  return usersConnectionRepository().createConnectionRepository(String.valueOf(user.getAccountId()));
}

You will note that this method also includes an @Scope annotation the value is set to "request" - this is because for every individual user that accesses the applciation, we want them to have access to their own connection (so the application connects to their twitter/github accounts etc, so this bean is scoped to each individual user request (but for user session life). 

ThirdParty Interfaces:

Finally we need a config method for every third party interface that we want to use - they all take a common form as follows:

@Bean
@Scope(value = "request", proxyMode = ScopedProxyMode.INTERFACES)
public Twitter twitter() {
	if (connectionRepository().findPrimaryConnection(Twitter.class) != null) {
		return connectionRepository().findPrimaryConnection(Twitter.class).getApi();
	}
	return null;
}

You will note once again these are scoped to  a user's request and simply gets the connection from the ConnectionRepository for the given interface.



Using the Interfaces

Every implementation exposes the third party API, so each will have a specific API available to it to allow you to perform common tasks (depending on the maturity of the implementation, it may only offer a limited subset of the overall API) .

Below is an example of using the Twitter interface in the Spring-Social-Twitter library to post an update:

@Autowired
private Twitter twitter;
public void postTweet(String content) {
	  twitter.timelineOperations().updateStatus(content);
}

As you can see, once the Twitter interface has been configured and injected, it becomes very simple to utilise the API.



On the whole, it is a relatively simple set of libraries to use and offers (an ever growing) set of integrations with third party sites so worth having a look at.


As always, the entire code base is on my GitHub account (if this link fails, then check the GitHub link on the right) and for the time being the application is being hosted on the CloudFoundry platform (although this is still in development, so it may come up and down, and data may be cleared out intermittently - it is not intended as a live production site and you should not put any data in there that you want to keep!) - The application is a dynamic online resume/CV application, that allows users to create a CV as well as pull in data from their GitHub accounts to show off actual examples of code and skills etc.