• Maven-based (no custom schell scripting )
• Minimal configuration
• Multiple servers (test, qa, production)
• Application server independent

After some research, I found Cargo that seemed to meet my requirements and I decided to try it out. I used the following setup to test Cargo:

• Apache Tomcat 7.0.26
• Maven 3.0.3
• example.war deployed in my local maven repository

1. Create maven deployer project

First of all I created a new empty project called example-deployer, the idea was to use this project to deploy my example.war to various servers:

mvn archetype:generate -Dversion=1.0-SNAPSHOT -DgroupId=org.example -DartifactId=example-deployer -Dpackaging=pom

2. Create maven profiles

Next, I wanted to be able to deploy to two different servers: qa, and production. Since I used different ports, different protocols and different context paths for these two environment I created two Maven profiles that can be activated using the env-property. In these two profiles I set property values for each environment.

<profiles>
  	<profile>
  		<id>prod</id>
  		<activation>
  			<property>
  				<name>env</name>
  				<value>prod</value>
  			</property>
  		</activation>
  		<properties>
  			<remote.protocol>https</remote.protocol>
  			<remote.port>443</remote.port>
  			<remote.hostname>prod.example.org</remote.hostname>
  			<remote.context>/</remote.context>
  		</properties>
  	</profile>
  	<profile>
  		<id>qa</id>
  		<activation>
  			<property>
  				<name>env</name>
  				<value>qa</value>
  			</property>
  		</activation>
  		<properties>
  			<remote.protocol>http</remote.protocol>
  			<remote.hostname>qa.example.org</remote.hostname>
  			<remote.port>8080</remote.port>
  			<remote.context>/qa</remote.context>
  		</properties>
  	</profile>
  </profiles>

It is now possible to activate the qa or prod profile by adding -Denv=prod or -Denv=qa

3. Configure Cargo

I was surprised how easy it was to configure Cargo’s maven plugin. There are basically three blocks of configuration 1) The container to use, 2) Configuration how to access the container and 3) the war-artifact to deploy to the remote container.

Since I do not want to exploit the username and password to the application server, I enter these manually during deploy time. Here is the configuration used:

<build>
  	<plugins>
  		<plugin>
  			<groupId>org.codehaus.cargo</groupId>
  			<artifactId>cargo-maven2-plugin</artifactId>
  			<version>1.2.0</version>
  			<configuration>

  				<container>
					<containerId>tomcat7x</containerId>
					<type>remote</type>
				</container>

				<configuration>
					<type>runtime</type>
					<properties>
						<cargo.hostname>${remote.hostname}</cargo.hostname>
						<cargo.protocol>${remote.protocol}</cargo.protocol>
						<cargo.servlet.port>${remote.port}</cargo.servlet.port>
						<cargo.remote.username>${remote.user}</cargo.remote.username>
						<cargo.remote.password>${remote.pass}</cargo.remote.password>
					</properties>
				</configuration>

				<deployer>
					<type>remote</type>
					<deployables>
						<deployable>
							<groupId>${project.groupId}</groupId>
							<artifactId>example-web</artifactId>
							<type>war</type>
							<properties>
								<context>${remote.context}</context>
							</properties>
						</deployable>
					</deployables>
				</deployer>

  			</configuration>
  		</plugin>
  	</plugins>
  </build>

The last thing we need to add to the configuration is to include a dependency to the war-artifcact that we want to deploy.

<dependencies>
  	<dependency>
  		<groupId>org.example</groupId>
  		<artifactId>example-web</artifactId>
  		<version>${project.version}</version>
  		<type>war</type>
  	</dependency>
  </dependencies>

It is now possible to deploy to out qa or prod server. Note that if we don’t want to expose the username and password in the script (that probably is in our source code repo) we can force the user to specify them on the command line. We are now ready to deploy, execute the following command:

mvn cargo:deploy -Denv=qa -Dremote.user= -Dremote.pass=

The example above does not work out of the box without some configuration of the application server. For example, in the Apache Tomcat case, you will need the manager web application to be deployed and configured to allow username/password access of a certain user. You can find all the details in the reference documentation as well as in the Apache Tomcat container documentation:

• Cargo Reference Documentation
• Cargo Tomcat Reference

The new version of the Firefox is called Firefox Extended Support Release or simply Firefox ESR. The intention is to provide security updates continously but let the web and add-ons platform be stable throughout the 1 year release.

I have experience from developing and maintaining both web frameworks and web applications at big companies and have noticed that Firefox’s recent 6 weeks release cycle have created problems for developers keeping up with certifying the quality of their software with the latest browser release. Firefox ESR might be a welcome change for framework and application developer’s in enterprises since it, in my humble opinion, might ease the the task of choosing Firefox as a tactical appointed enterprise platform for internal web-applications.

For more info, see the post at the Mozilla blog:

http://blog.mozilla.com/blog/2012/01/10/delivering-a-mozilla-firefox-extended-support-release/

Fort Lauderdale was greeting us with a comfortable wintery weather with temperatures at around 25°C so it was very refreshing to take a walk outside during the lunch breaks.

The conference was hosted by No Fluff Just Stuff whose conferences aim to be stuffed with as much information as possible without the usual exhibition and advertisements seen at many other conferences. I think they have a really good concept there, which makes it easier to connect with fellow software engineers without worrying about what they are selling.

The Rich Web Experience is a conference aimed at web client development in all it forms; browser web applications, mobile web applications, native mobile apps etc etc. It was this year colocated with the Project Automation Experience where the focus was more on build systems, CI, cloud computing etc.

Since I have a big interest in client development, most of the sessions I attended were from the Rich Web Experience.

One of the most interesting things that I noticed was that almost noone at all was talking about server generated HTML anymore. Almost everyone talked about self contained JavaScript/HTML5 clients consuming services published by their respective back end systems. I attended a presentation hosted by Dan Allen from JBoss who was presenting his idea of future mobile web applications and even he (with some past influence in the JSF 2.0 project) was also leaning towards abandoning server rendered HTML. Instead he says that JBoss are (amongst other solutions I suppose) looking at a solution based on GWT, Errai and POH5, Plain Old HTML 5. When I asked him a direct question about the future for JSF he was a bit slippery but he said it is definitely time to start looking around on what’s out there and that it wasn’t impossible that JSF would transform to something else in the future, unknown what.

I also had a quick after session chat with Dylan Schiemann, co-founder of Dojo Toolkit which was really inspiring. He said they are working on a new Grid component for Dojo called dgrid which sounded very interesting. It is designed to be more light-weight and way faster than previous grids to fit even into mobile devices.

Other highlights from the conference was Venkat Subramaniam’s iOS and JavaScript sessions, Tim Berglunds poetic coding examples, Gabriel Dayley’s and Tom Valletta’s excellent talks on HTML5, node.js and WebSockets and of course Eric Wendelin’s JavaScript test automation presentation.

I can really recommend the RWX 2012 if you are interested in client development.

Some links for the interested reader to look more at:

http://mustache.github.com/ – Logic less templating system, amongst others for JavaScript

http://requirejs.org/ – JavaScript file and module loader

http://documentcloud.github.com/underscore/ – Utility-belt library for JavaScript

http://nodejs.org/ – Server side JavaScript, a must have

http://maqetta.org/ – WYSIWYG editor for HTML5, looks interesting at first glance

http://coffeescript.org/ – A language that compiles to JavaScript and adds syntactic sugar. CoffeeScript is almost to JavaScript what Groovy or Scala is for Java

http://www.dartlang.org/ – Google’s new language for the web

http://www.html5rocks.com/en/tutorials/device/orientation/ – How to code mobile applications that make use of device motion tracking (accelerometer etc).

https://github.com/SitePen/dgrid – Home of the Dojo dgrid project

http://spinejs.com/ – A minimalistic MVC utility for JavaScript

http://code.google.com/p/js-test-driver/ – Drives your JavaScript tests from the browser

http://pivotal.github.com/jasmine/ – BDD style tests for your JavaScript application

http://www.jboss.org/errai – A framework for building GWT applications.

In this article, I will cover both how to trust a server certificate for secure communication with the server, as well as providing a client certificate to the server for mutual authentication. The client certificate is bundled with the app and of course, the server needs to trust this certificate.

Setup

Before we can start, keys and certificates need to be in place. My article “Creating self-signed certificates for use on Android” covers how to create keys and certificates as well as importing them into supported key stores. If you don’t have any keys or certificates, create the required files by reading that article.

Android and self-signed certificates

I assume that you already figured out how to create a simple Android app. However, I do not assume that you have put clienttruststore.bks or client.bks in your Android project’s res/raw directory. Before you continue, make sure the two files are there. These files were created in “Creating self-signed certificates for use on Android” but you can replace them with your own, just make sure that your keystores uses the Bouncy Castle Provider.

Implementation

This section covers how you can implement a custom HttpClient by registering a scheme for https communication and load a keystore and a truststore into a SSLSocketFactory.

Extend HttpClient

Android uses Apache Commons HttpClient but since we want communicate securely we must extend the DefaultHttpClient with some custom behavior (load our keystores). Therefore, we start by creating SecureHttpClient.java and extend the HttpClient.

public class SecureHttpClient extends DefaultHttpClient {
    private int securePort;

    public SecureHttpClient(final int port) {
        this.securePort = port;
    }
}

Simply put, we need to do two things to get our mutual authentication to work. 1) Create an SSLSocketFactory where we load our keystores and 2) Register a scheme for https communication that uses our custom SSLSocketFactory. This method will load our keystores.

Create SSLSocketFactory and load the keystores

Extend the SecureHttpClient with the two new methods found below. If you do not want to use mutual authentication you can just load the trust store in which the server’s certificate is. The resulting SSLSocketFactory will later be used when creating a scheme for https-communication.

private SSLSocketFactory createSSLSocketFactory(final Context context) {
    Log.d(TAG, "Creating SSL socket factory");

    final KeyStore truststore = this.loadStore(context.getResources().openRawResource(R.raw.clienttruststore, "password", "BKS");
    final KeyStore keystore = this.loadStore(context.getResources().openRawResource(R.raw.client, "password", "BKS");

    return this.createFactory(keystore, this.keystorePassword, truststore);
}

private SSLSocketFactory createFactory(final KeyStore keystore, final String keystorePassword, final KeyStore truststore) {

    SSLSocketFactory factory;
    try {
        factory = new SSLSocketFactory(keystore, this.getKeystorePassword(), truststore);
        factory.setHostnameVerifier((X509HostnameVerifier) SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
    } catch (Exception e) {
        Log.e(TAG, "Caught exception when trying to create ssl socket factory. Reason: " + e.getMessage());
        throw new RuntimeException(e);
    }

    return factory;
}

We now have a method that loads our keystores and return an SSLSocketFactory.

Register a scheme for https

Before we can communicate with our server, we must register a scheme for https communication that uses our SSLSocketFactory. The createConnectionManager() in HttpClient allows us to register our scheme. Override the createConnectionManager() method and provide the following implementation:

@Override
protected ClientConnectionManager createClientConnectionManager() {

    Log.d(TAG, "Creating client connection manager");

    final SchemeRegistry registry = new SchemeRegistry();

    Log.d(TAG, "Adding https scheme for port " + securePort);
    registry.register(new Scheme("https", this.createSSLSocketFactory(), this.securePort));

    return new SingleClientConnManager(getParams(), registry);
}

That’s about it. We now have the SecureHttpClient class that we can use from our Android app to communicate over Https.

Using the Secure client

It is time to use our SecureHttpClient and make a call to a web server. Since this is just a test, we can basically make the call from anywhere in the app and I choose to make it in an Activity’s onCreate()-method. The test server I used is a simple Jetty server (jetty-maven-plugin) where I added configuration for SSL containing a trust store and the server’s certificate. You can see how a simple server with SSL can be setup in the article: “Quick Start – Jetty, Maven and SSL”.

To make a https call to the server, simply provide an implementation like the one below:

@Override
public void onCreate(Bundle savedInstanceState) {
    final HttpClient client = new SecureHttpClient(443);

    // Provide ip or address to where your test server is runnning
    final HttpGet request = new HttpGet("https://192.168.0.10:8443/example-server")
    final HttpResponse response = client.execute(request);

    Log.d("ExampleActivity", "Response code: " + response.getStatusLine().getStatusCode());
}

Create a web application project:

mvn archetype:generate -DgroupId=org.example -DartifactId=example-server -DarchetypeArtifactId=maven-archetype-webapp -Dversion=1.0

This gives us a new directory (example-server) which is a Maven web application project. To run the web application, we configure the maven-jetty-plugin. Add the following configuration to project’s pom.xml

<build>
 <finalName>example-server</finalName>
 <plugins>
   <plugin>
     <groupId>org.mortbay.jetty</groupId>
     <artifactId>maven-jetty-plugin</artifactId>
     <version>6.1.26</version>
   </plugin>
 </plugins>
</build>

Enter the example-server directory and do:

mvn jetty:run

As soon as the server is started you can enter the following url in your browser.

http://localhost:8080/example-server

As you can see, the server is started and listens on port 8080 by default. If you want to change this, it can easily be configured. Just extend the plugin with a configuration element and add a connector.

<plugin>
 <groupId>org.mortbay.jetty</groupId>
 <artifactId>maven-jetty-plugin</artifactId>
 <version>6.1.26</version>
 <configuration>
   <connectors>
     <connector implementation="org.mortbay.jetty.nio.SelectChannelConnector">
       <port>9090</port>
     </connector>
   </connectors>
 </configuration>
</plugin>

Adding TLS/SSL support

Assume you want to communicate in a secure way. The only thing you need to do is to add another connector element and specify a keystore containing the server’s certificate. If you don’t know how to create a certificate for your server, you can read my other blog post “Creating self-signed certificates for use on Android”. Simply add the following connector element and make sure the server.jks is located in your example-server directory:

<connector implementation="org.mortbay.jetty.security.SslSocketConnector">
 <port>9443</port>
 <keystore>${basedir}/server.jks</keystore>
 <password>password</password>
 <keyPassword>password</keyPassword>
</connector>

You can test this in a nice way using openssl to see what the server returns when you try to access it on port 9443.

openssl s_client -connect localhost:9443

Finally, if you for some reason want mutual authentication, you also need to specify a trust store in which the server keeps certificates of trusted clients. Extend the previous connector with the following information:

<connector implementation="org.mortbay.jetty.security.SslSocketConnector">
 <port>9443</port>
 <keystore>${basedir}/server.jks</keystore>
 <password>password</password>
 <keyPassword>password</keyPassword>
 <truststore>${basedir}/serverTruststore.jks</truststore>
 <trustPassword>password</trustPassword>
 <needClientAuth>true</needClientAuth>
</connector>

Now you have a web server up and running your web application with mutual authentication. The clients must provide a valid certificate in order to communicate with the server. At last I just want to add a final element to our configuration. Since TLS/SSL can be quite horrible to troubleshoot, I add the following configuration which gives a lot of nice output

<systemProperties>
  <systemProperty>
    <name>javax.net.debug</name>
    <value>ssl</value>
  </systemProperty>
<systemProperties>

Have fun!

References

• Maven Jetty Plugin
• Maven Webapp Archetype

For this article to be useful, the required tools are: openssl, Java’s Keytool and the BouncyCastle-provider (Click here to download). There are also some resources that I strongly recommend and has been very useful:

• SSL Certificates Howto
• OpenSSL Keys Howto
• OpenSSL Certificates Howto
• Java Keytool

One might argue why I don’t use keytool to generate the keys and certificates and use them right away. Well, I was very curious about learning more about openssl and how to deal with various formats of keys and certificates.

1. Create private keys

Let’s start from scratch. First of all we need private keys. We use openssl to create these:

openssl genrsa -des3 -out client_key.pem 2048

openssl genrsa -des3 -out server_key.pem 2048

This will create the two keys; client.pem and server.pem. We will use these in the next step to sign our certificates with. In normal cases we would create a CA-signing request, that is sent to a CA who will issue your certificates. But since we want to self-sign our certificates this step is redundant.

2. Create self-signed certificates

openssl req -new -x509 -key client_key.pem -out client.pem -days 365

openssl req -new -x509 -key server_key.pem -out server.pem -days 365

Additionally, instead of being prompted for the certificate’s subject line you can use the <pre>-subj</pre> parameter and pass it to the <pre>openssl req</pre> command. What we just did was basically creating a CA signing request using our private keys to sign the outgoing x509-certificates. The certificates will be coded in pem-format and valid for 365 days.

3. Create trust stores

In order to use our keys and certificates in Java applications we need to import them into keystores. First of all, we want the client to trust the server certificate. To do this we must create a client trust store and import the server’s certificate.

keytool –importcert -trustcacerts –keystore clienttruststore.bks –storetype bks –storepass <truststore_password> -file server.pem -provider org.bouncycastle.jce.provider.BouncyCastleProvider –providerpath <path_to_bcprov_jar>

Note: On the client side, which in our case will be an Android app we use Bouncy Castle as our provider since it is supported on the Android platform.

Create a trust store for the server and import the client’s certificate into it.

keytool –importcert -trustcacerts –keystore  servertruststore.jks –storetype jks –storepass <server_truststore_password> -file client.pem

Currently, we have two trust stores one for the server in which we imported the client’s certificate and one for the client in which we imported the server’s certificate.

4. Combine keys and certificates

A problem with Java’s keytool application is that it won’t let us do such a simple thing as importing an existing private key into a keystore. The workaround to this problem is to combine the private key with the certificate into a pkcs12-file (which is understood by Java’s keytool) and then import this pkcs12 keystore into a regular keystore.

Combine the certificate and the private key for the server and client respectively:

openssl pkcs12 –export –inkey  client_key.pem –in client.pem –out  client.p12

openssl pkcs12 –export –inkey server_key.pem –in server.pem –out server.p12

5. Convert from pkcs12

Import the created keystores to new ones with common formats:

keytool –importkeystore –srckeystore client.p12 –srcstoretype pkcs12 –destkeystore client.bks –deststoretype bks –provider org.bouncycastle.jce.provider.BouncyCastleProvider –providerpath <path_to_bcprov_jar>

keytool –importkeystore –srckeystore server.p12 –srcstoretype pkcs12 –destkeystore server.jks –deststoretype jks

We should now have all files we need for a successful TLS/SSL mutual authentication. The files we move to our Android project will be: clienttruststore.bks and client.bks. The files we move to our server will be: servertruststore.jks and server.jks.

<context:property-placeholder location="classpath:config.properties" />

By injecting the configurations this way they will be packaged inside the application in and you will have to rebuild and deploy it if you need to make any changes. Considered you have environment specific configurations you will also have to build an artifact for each and every environment.

A better way to handle the configuration files is to put them outside of the application in a separate location on the server. Changing configuration using this setup will at most force you to restart your application to make Spring reload them. I normally put the configuration files in a separate folder under my home catalog, i.e. ~/.myapp/config.properties. In the spring config file you will inject the configurations using:

<context:property-placeholder location="file:${user.home}/.myapp/config.properties" />

user.home is a standard JVM system property which is resolved by Spring using the ${…} placeholder to the home catalog of the user running the server. You can even make the configuration location configurable through an environment variable:

<context:property-placeholder location="file:${MY_APP_CONFIG}" />

But what if you want to package some default configurations within you application but still give the opportunity to manipulate them without the need to rebuild and make it possible to choose your own config location? No problem use:

<context:property-placeholder location="classpath:config.properties, file:${user.home}/config.properties, file:${MY_APP_CONFIG}" />

Spring will read and merge the configuration files in the order they appear and will override duplicated configurations. This means that you can override the default configurations by adding them in ${user.home}/config.properties or ${MY_APP_CONFIG}.

Ok, now I showed how to configure the property files but what if you want to configure the Spring application context? What if you want to configure the way your application is put together? Lets take an example: If an application uses JNDI to retrieve the database connection you will first need to configure your server with a database connection and a JNDI-name. Secondly you will need to create a datasource bean in you application context which is injected into your DAO:s, i.e:

The dao:
public class OrderDao {
private JdbcTemplate jdbcTemplate;
public void setDataSource(DataSource dataSource) {
this.jdbcTemplate = new JdbcTemplate(dataSource);
}
}


The spring config file dao-config.xml:
<bean id="myDao" class="com.example.OrderDao">
<property name="dataSource" ref="dataSource"/>
</bean>


Using JNDI will force you to configure the server to before you can run the application. But as a developer of you don’t change the JNDI configuration on your server for every application you work on. Instead it would be great if you could create a direct connection using Spring, this would remove the need of server configuration. Here is two example showing the datasource bean configured to use JNDI in the first case and a direct connection in the second case:

JNDI – datasource-jndi-config.xml:
<bean id="dataSource" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="${datasource.jndiName}" />
</bean>


Direct – datasource-direct-config.xml:
<bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="${datasource.driverClassName}" />
<property name="url" value="${datasource.url}" />
<property name="username" value="${datasource.username}" />
<property name="password" value="${datasource.password}" />
</bean>


In the dao-config.xml it is now possible choose which one of the two datasources I would like to use by importing the corresponding file. When deploying on the dev. environment use:

<import resource="datasource-direct-config.xml" />

and when deploying to production use:

<import resource="datasource-jndi-config.xml" />

It could be quite annoying having to switch like this, but luckily Spring has a solution for this. Using the ${…} placeholder we can inject which type of datasource connector we would like to use, we could even give it a default value incase we don’t inject a connector type. This is done by adding a colon) in the placeholder like this:

<import resource="datasource-${datasource.connector:jndi}-config.xml" />

The ${…} placeholder picks up parameters from System properties, environment variables and the property files we injected earlier. So this mean that we Now have an application context that is externally configurable. By setting a property named datasource.connector=direct we could easily change the application to use a direct connection to the database instead of retrieve it from JNDI.

Great we are done? No, unfortunately the ${…} placeholder is unable to pick up an property from a property file injected to a property-placeholder bean if it is used in an import-tag, why I will explain in the next blog post. This will limit us to set the datasource connector using either a System property (i.e. use -Ddatasource.connector=direct when starting you server) or an environment variable. To be really happy want to keep all configuration parameters in config.properties but in the way Spring works this is not possible out of the box. In my next blog post I will explain why this isn’t possible and I will also show a solution to the problem.

Kim Nordlander (SLL) och Åke Rosander (Cehis) presenterade projektet Mina Hälsotjänster under nationella eHälsodagen här om veckan: http://www.nationellehalsa.se/program.html. Ett sätt att dela upp tjänster (i betydelsen IT-stöd) för invånaren är att utgå från avsändaren. Om vård och omsorg är avsändaren kan marknaden leverera e-tjänster till invånaren, men det måste förmodligen ske genom att sälja tjänsten till landstingen. Det betyder (som jag uppfattat saken) att e-tjänsten måste ligga inom ramen för landstingets uppdrag – dvs ha direkt koppling till vårdprocessen. Några exempel kan vara tidbokningstjänster för invånaren och visualisering av mina remissers status i remissprocessen.

Men det talas allt mer om tjänster invånaren själv väljer för att samla, hantera och utbyta information kring sin hälsa. Dessa tjänster går under många namn:
- Mitt Hälsokonto
- Min hälsodagbok
- PHR – Personal Health Record

Dessa kan användas för att samla in och sammanställa information från träningsdagböcker (t.ex. Runkeeper), från egen privat hälsotillståndsmätning (en växande marknad av gadgets för såväl hypokondriker som olika former av kroniker) eller helt enkelt egna observationer. Informationen kan användas i sociala medier riktade mot olika sjukdomstillstånd eller kanske som underlag för en “second opinion” för den som känner osäkerhet kring kvalitén i hanteringen av ett hälsoärende.

Denna typ av invånar-tjänster söker man själv upp. Det är ett privat beslut att “hoppa på” en specifik “app”. Man byter kanske till en annan om det dyker upp en bättre, eller om en annan tjänst fått en större community och därför över tiden visat sig vara ett bättre val. Så vad har det då med Mina Hälsotjänster att göra? En tanke som sysselsatt mig ett tag är vad det skulle innebära att öppna upp dagens vård-data-api:er till denna typ av applikationer för privat hälsodata.

Idag kan en app anslutas till nationella api:er mot landstingens vårdsystem så länge appen har en vårdhuvudman som avsändare. Den bakomliggande säkerhetsmodellen är okomplicerad och beprövad: så kallad organisationstillit. Det betyder lite förenklar att man kan “öppna informationskanalerna” mellan parternas system så länge parterna är betrodda organisationer (läs vårdgivare). Tekniskt sett upprättas tilliten med enkla medel så som klient-authenitiserade (funktionscertifikat) krypterade kanaler (TLS).

Men hur gör man för att en app som marknaden byggt och som invånaren själv beslutat använda för sin hälsodata ska kunna hämta in data från vårdens verksamhetssystem? Jag kanske vill integrera information från mina labbsvar i min hälsodatabas? Ett annat scenaro är att min vårdgivare vill få in mina egna mätningar som bilaga till journalen. Eller en bokningsfunktion som liksom hotell-och flygbokningsagenter aggregerar en mängd underliggande tjänster för min bekvämlighet? Det behövs något sätt att låta invånaren ta ansvar för vilka appar som får läsa och påverka infromation hos vårdgivarna under invånarens identitet.

Det juridiska perspektivet måste förstås belysas. Patientdata-juristen Andreas Hager har gjort en genomlysning som indikerar att det kan finnas stöd i lagar och författningar. Jag tänker mig att det vore ung. samma frågeställning som om jag kunde logga in i Mina vårdkontakter, se mina labbsvar och sedan gavs möjlighet att ladda ner dem (“spara som …”) till min dator för att sedan importera filen till min valda “hälsodagboksapp”. Men att appar jag väljer istället kan göra jobbet åt mig – dvs utan manuell filhantering.

I presentationen nedan har jag försökt skissa en arkitektur för invånarstyrd informationsåtkomst. Det är mina funderingar och representerar alltså inte någon av mina uppdragsgivares strategier eller beslut.

Vad tror du? Finns behovet? Är OAuth2 i så fall rätt väg? Skulle du låta Google Health läsa labbsvar från dina hälsoärenden i vård och omsorg? Skulle du låta en facebookapp som jag utvecklat läsa din medicinlista från journalen hos en vårdenhet för att ge dig påminnelser när du ska ta dina mediciner? Är detta den gyllene länken mellan patienten, dagens journalsystem och marknadens innovatörer?

For instance the possibility to separate commit from delivery seems like a nice option.
If you work on something that takes a couple of days to do, you want to commit often even when only parts of the work is ready without disturbing your colleagues. You want to commit even if all tests are not green. You want to be able to experiment and throw some stuff away by going back to previous committed versions. With a DVCS (Distributed Versioning Control System) you do all of this in your local repository and only when you are finished you deliver to the project repository where other people then can use it.

Which product should you use ? Git and Mercurial seems to be the popular choices.
Based on the post “Git is McGywer and Mercurial is James Bond” http://importantshock.wordpress.com/2008/08/07/git-vs-mercurial/ i choose git, although
I’m not a command-line wizard.

Installation on OS-X is very easy as expected. I used http://git-scm.com/download
In windows it is a bit more complicated because you have to choose between 2 different setups. I choose the option without cyg-win and running bash in command windows. I wonder who has the balls to choose the third option in this question ?

Anyway creating a repository and checking files in is very easy

The init command will create the repository. The only bit of magic is the git add command. It will add one or many files to something called the index. Only changes added to the index will we affected by the next git commit.

Now you can start playing around with creating branches, make changes and merging branches. Working with branches is much less expensive than you thing and a recommended way of using git.

But having only a single repository probably is not want you want to do. Collaboration between different team members is of course the next thing to try out. In order to understand how that work I did like this.

Now you have three complete repositories. You can imagine that clone1 and clone2 are local repositories at different users and that repo the a project common repository. I used three different command windows, one for each repo. Then it was easy to change something in clone1, push it to repo an pull the changes to clone2. Also simulating conflicts and how to resolve them is very easy since you have full control of all three repos.

In conclusion getting started with git is easy. Mastering all (or much) of the git functionality and different distributed repository topologies will probably take much longer.

Martin Fowler makes an interesting comparison between different products

From this you can see that git and mercurial are useful but more complex to learn.

When it comes to topologies I think that Scott Chacon has some nice pictures, how about this one :

Some useful resorces

http://help.github.com/git-cheat-sheets/

http://book.git-scm.com/index.html

http://ndpsoftware.com/git-cheatsheet.html#loc=remote_repo;

http://whygitisbetterthanx.com/#

http://martinfowler.com/bliki/VersionControlTools.html

And .. this video is a must

While Web Services (SOAP or REST based) gets all the attention in the press, there are many companies still struggling with file transfers and transformation of very large files (for the scope of this blog defined as files of size > 1 GB). Typically, these companies, can’t get a file of that size through their integration platform and messaging backbone without having to do all sort of tricks such as splitting files into smaller one and later on both re-sequence and aggregate them back again and/or setting up huge memory heaps on their servers and so on…

In this blog I will describe how I have been working with some of our customers to help them find inexpensive and simple (removing unnecessary complexity) but still robust solutions, using open source products such as Mule ESB and the transformation framework Smooks, to achieve large file transfer and transformation.

It turned out during this work that the current status of some of the open source components we decided to use did not meet all our requirements. But since we are talking about open source there is a very good solution to that, add the missing pieces yourself (in the way you want it) and bring it back to the open source project!

The open source projects mentioned below lives in the MuleForge, a hosting site and community for development and sharing of open source extensions to Mule ESB and that is where my contributions ended up.

SFTP

In late 2009, I was involved in a customer project using Mule ESB. In this project we needed to transfer large files but we didn’t want to use FTP. In fact we strived to replace FTP with SFTP.

We started to evaluate the SFPT transport in Mule ESB (that lived in the MuleForge at that time). We found out that the SFTP transport, thanks to its streaming capabilities, could handle large files very well but there were a number of other features required by the customer that were missing in the transport, such as archive functionality, handling of duplicate file names, file-size checks to determine if a file is completely created before consuming it and so on.

To address these shortcomings I decided to join the SFTP-transport project. A couple months later the project released a new version of the transport that solved the mentioned concerns. Since then, our customer has been using the SFTP-transport in production.

Below is a very basic example of a file transfer from one SFTP-server to another SFTP-server applying a Java based transformation of the content. If you download the source code of the transport and take a look in the integration tests and their mule-config-files, you can find many other examples of its use.

Note #1: The example assumes usage of PKI-keys and not old style username/password.

Note #2: In a real-world case attributes such as address would not be hardcoded but instead be provided as a configurable property.

Over time the SFTP-transport has been used more and more widely and in early 2011, MuleSoft decided to add the SFTP-transport as a core transport of the Mule ESB product. Since Mule ESB v3.1.2 the SFTP-transport is part of the Mule ESB distribution.

…and my work as a MuleForge committer was over for the time being…

Smooks

Earlier this year (2011) I started to look at the Smooks project, specifically regarding its capabilities to transform between many different formats using a declarative and template driven approach. Added to that Smooks also have support for stream based transformation opening up for transforming very large files without any hassle (as described above). Very compelling when combined with the streaming capabilities of Mule ESB’s file transports (file, FTP and of course SFTP)!

The way to integrate Smooks in Mule ESB is to use the Smooks Module for Mule at the MuleForge, i.e. Smooks support in Mule is not a core part of the Mule ESB product.

Using the Smooks Module you can for example easily replace the Java based transformer in the SFTP-transport example above with a much more versatile Smooks based transformer by replacing the line:

    <custom-transformer class="...MyTransformer"/>

with:

    <smooks:transformer configFile="my-smooks-transformer.xml"/>

A problem I found out working with Smooks and Mule was that the Smooks Module for Mule was not yet released for Mule 3, only for Mule 1.x and Mule 2.x. However, all changes required for Mule 3 was already developed and committed but no one have had time to do the final testing and release management for a Mule 3 compatible release.

…so I decided to join in again, this time with the goal to release a Mule 3 compatible version of the Smooks Module!

Last weekend we released a release candidate for the next version, v1.3-RC1, which brings in compatibility with Mule 3.1.x. See Smooks for Mule v1.3-RC1 released for details!

A very good example of what the Smooks framework is capable of doing for you is the example in the Smooks distribution called “freemarker-huge-transform”. It demonstrates a setup where Smooks transforms very large XML-files containing order heads and order lines. An outer streaming SAX-parser is setup to stream through the order head elements and delegates the processing of the order lines to a DOM-parser. This means that the memory intensive DOM models only exist for one order line at the time resulting in a very small memory footprint even for transforming very large files. Freemarker is used as the templating language to declare what the actual transformation should do with each order head and order line.

Copy of the smooks-config – file from the example, comments removed to keep the size down:

Going through the example in detail is out of the scope for this blog (see Smooks v1.4 Examples for details) but some interesting aspects are worth mentioning:

• The outer sax-parser that read order lines with a streaming approach:
  <core:filterSettings type=”SAX” defaultSerialization=”false” />
• The two Freemarker based templates that work together.
  The outer template handling transformation of order-heads (using the streaming SAX-parser) and the inner template transforming order-lines one-by-one using a traditional DOM-parser.

• The Freemarker templates declare the outline of the transformed outgoing message format and the variables ${…order…} refers to elements in the incoming message and declares where their corresponding values should be placed in the outgoing transformed message.

To summarize

By combining a couple of open source products and making some contributions of your own, if required, you can construct very cost-effective, low-complex but still very robust solutions to problems that traditionally are concerned to be very hard, complex and expensive to solve.

Feel the power of open source!