Template Based Reporting for Sparx Enterprise Architect with Java

Generating EA Model reports

As a regular user of Enterprise Architect who has encountered it accross various projects and organisations I have come to appreciate it as a capable and versatile modelling tool in the software industry. I have been using EA for years now and was always quite happy with it. 

However, recently I have been working on a project that had some reporting needs not supported in the built in reporting tooling. For instance, I could not deviate from the structure of my EA model in my reports. I also needed to be able to create tabular overviews of objects in specific diagrams and could not find an easy solution. This all made it impossible to meet my project's documentation standards using out of the box EA features.

These shortcomings led me to create a standalone reporting tool, which I have named EART (short for Enterprise Architect Reporting Tool).  EART enables one to write templates in the documents themselves, similar to EA's built in generator and therefore adjust the formatting in a WYSIWYG manner. From the template directives, it can query into the EA object model in a more flexible way so that documents can be structured independently from the model. It is also possible to embed the diagrams created by EA in the documents as well.

Having used it only for myself until now, I decided it would be a good idea to share it with others so I put it on github. In this blog post I will explain how it works.

The templating

First, I needed some sort of templating technology for documents that I could customize to act on the EA data. After some looking around I found XdocReport. XdocReport itself uses Apache Freemarker, a very capable and widely used general purpose templating language. I recommend going through the their website to understand the full scope of the Freemarker capabilities.

XdocReport works with the ODT format as well as Word docx and Powerpoint pptx. Note that such documents are nothing more than zipped XML. The XML can therefore be processed by text based templating technologies such as Freemarker. XdocReport takes care of unpacking the zipped documents, then finding and feeding the inline Freemarker directives to the Freemarker engine so it can do the actual merging with the data. After that, it just packs this all up in a proper document again.

XdocReport is well documented but just to get an idea here is a brief exampel of how it works. First create a new docx template document using Word. Insert a field (CTRL-F9), then right click on it and choose 'edit field':

Next choose 'MergeField' from 'Field Names' list. In the 'Field name' property type in the Freemarker directive. For example, let's say we have an Author class with a getName() method. To access the author's name, type in the Freemarker directive ${author.name}.

After entering, you should see the document like below:

From JAVA, one should then invoke XdocReport's XDocReportRegistry and IXDocReport classes to generate a document based on the template. IDocReport should be supplied the context data for the template via a Map as shown below:

After the code above runs, the output document should contain the line 'JK Rowling'.

Accessing the EA model

Now we need to make the EA object model available to Freemarker. Fortunately for Java developers Sparx provides a .jar with a JAVA API. The key is the Repository class, which opens the .eap file which contains all the EA data in the model which I show below. Note this involves actually starting an instance of EA based on that particular .eap file from your JAVA code and then talking to that instance through the JAVA API. Hence I added a shutdownhook to ensure proper cleanup: we just started an EA instance and if we don't end it, it will just continue running even after the JVM exits.

After the model has been opened, you can start browsing through it programmatically by invoking the getModels() method, which will give all root level Packages in the EA model. From there you can get diagrams in a package or elements in a package. This sort of thing we would like to do from Freemarker though, so at this point in the JAVA code we want to just hand Package instances to Freemarker, similar to how we gave it an Author instance earlier. I found this blog from Sparx with some useful info and a picture of the EA object model:

However, there is a mismatch between Sparx's JAVA API and Freemarker. That is because Freemarker templates work only based on JavaBeans convention. Which the Java API supplied by Sparx does not follow for some reason. Meaning that if in your template file you want to access an element's name with ${element.name}, it won't work since Freemarker then expects to find a getName() method on the Element class which is not the case. There is a GetName() method, but it is not accessible since it does not follow JavaBeans convention.

Fortunately Freemarker can also work purely on a Map interface. The solution then, was to create a proxy object to wrap EA's awkward object model in a neat Map interface. Java proxies fit nicely here:

• we have an interface as proxy target (Java proxies require that) 
• we can then easily only implement that part of the interface that needs it
• due to the generic way we can intercept any method call, the proxy logic becomes more compact

Here is a listing of the code: As you can see the proxy only exposes the properties of the proxied object as read-only Map key-value pairs. It does not do all the work itself though. Actually extracting the properties is done by PropertyCollector, which applies a more lenient version of JavaBeans convention for determining what is a property getter. In particular, it does not care for the case of the first letter. Public fields and boolean properties will also be exposed.

Actual wrapping is done by ObjectWrapper, who decides how to wrap an object based on it's type: primitives, Strings and Enums are not wrapped, while Arrays and Iterables will be converted to Lists (more convenient to Freemarker) whose elements are themselves wrapped. A big help here was org.apache.commons.lang3.ClassUtils, which tells if a class is a primitive type or not.

We are not done though. Another inconvinience is that the EA object model is broken up in pieces at some places. By that I mean that sometimes, getting to related elements involve not a 'getXxx' but rather invoking the Repository instance to find that element based on id (see also the picture of the EA objectmodel shown previously). Now in ordinary code I would find this pretty annoying already; but in a templating language this is a burden.

To get over that I used Aspect Oriented Programming to extend the original Sparx Java classes with convinience getters for accessing related objects. For instance, in the code snipped below, some getters are added to the original Connector class from Sparx to retrieve the client and supplier Element of a Connector:

In this way a seamless object model is exposed allowing for clean, readable templates. In the example above, we can just say ${connector.client} in the template document and not be bothered with lookups there. Note the EA object model extensions are a work in progress and do not yet cover the entire object model.

Inserting diagrams from EA

Finally, we want to include images of the model's diagrams into the reports. XdocReport has the basic capbabilities in place: it provides the AbstractImageProvider class for making images available in template. Here is how it works: first we need to go to the template document and add a picture from file (any file will do). Then click on the picture and choose 'Insert' then 'Bookmark':

We must provide a bookmark name, e.g. 'Diagram_1'. XdocReport will then look into the provided context Map (see the first example) and search for a key named 'Diagram_1'. The associated value must be an implementation of AbstractImageProvider.

The AbstractImageProvider works by writing the image's bytes to an OutputStream. But how to get hold of a EA diagram's image as bytes? It would be nice to be able to simply ask the EA Repository for a stream of bytes representing some diagram image but that is not supported. What is possible though is to programmatically ask EA to generate an image file of a particular diagram. This file can then be converted to an InputStream implementation and given to XdocReport which makes it addressable from templates. This is how the code should look like broadly speaking:

Upon startup EART recursively scans the model for diagrams. Then it creates entries on the XdocReport context for each diagram found based on it's name as defined in Enterprise Architect. Since the modeller types in these names himself in EA, they can be easily remembered for use in the template: just type in name_of_diagram as the bookmark name.

Diagram names must as a result be unique otherwise only the last one found will be retained. In addition to that a context package is to be supplied via configuration to EART, which is then used as the starting point for template data and diagram scanning. Uniqueness of diagram names should hold (recursively) within that context package.

Note the actual generation and loading of the image is done only when that particular diagram is used in the template document, and not up front. After that, it is kept in memory.

Executing report generation

In order to create reports, first ensure you have the following:

• Windows (tested with Windows 7 Professional)
• a git client (any should do, else just download as zip from github)
• Word for docx editing (tested with version 2007)
• Apache Maven (tested with version 3.3.9)
• JAVA 32 bit jdk (tested with jdk1.8.0_77). 64 bit JAVA won't work because EA is a 32 bit runtime.
• A local installation of Enterprise Architect (tested with EA version 11)

Use your git client to checkout the code from the git repository or download the zip from github. In the project root directory (where pom.xml is located) run mvn_install_jars.bat and  mvn clean install then wait for the build to complete. In the target subdirectory a zip file should have been created. You can copy this zip to a suitable directory on your computer and unpack it.

Next, go into the unzipped directory and edit the text file named eart.properties (all properties must be specified):

Define the template file in a document editor, and place it in the location referenced by eart.properties. Then open a command prompt in the directory of the jar file and issue this command:

run_report.bat

This will start the generation process and will create an output document with the merged data from EA.

Note the assumption during these steps is that you installed EA in the directory  C:\Program Files (x86)\Sparx Systems\EA\ . Otherwise you will need to edit mvn_install_jars.bat and run_report.bat to reflect that. Also it is assumed that the 32 bit JAVA installation is being referenced in the Windows JAVA_PATH variable.

Closing thoughts

With some glueying together of libraries, it is possible to leverage the XdocReport engine and have template based reporting for Word docx. Besides that pptx and odt files should be supported but that was not tested. EART itself brings seamless object model on top of EA and some convenience for inserting diagram images.

Right now the full EA object model has not yet been extended, so there will be gaps in what is accessible from templates. Additions to the AOP aspects should be easy to make and share via a pull request to the git repository though. Or you could use EART as a library and apply AOP in your own project. Any getter you create should be accessible from Freemarker afterwards automatically.

Meanwhile I am also quite enthusiastic about Freemarker and pleasantly surprised at what it can do. It has all sorts of things to keep templates clean and easier to maintain. It is one of those little gems that makes your work more enjoyable.

Though maybe not a very exciting subject, creating good  documentation from the EA Model can be crucial in effectively communicating the model or complying to standards. Being able to do that without pain, limitations and in an automated fashion is in my opinion a great help to the many Enterprise Architect users worldwide.

The full code including example document templates have been on a git repository. If you have any trouble or have some remark or suggestion, please leave a comment below.

Een race spel maken in Scratch

In deze post ga ik uitleggen hoe je een race spel met Scratch kunt maken.Scratch is een grafische programmeertaal waar men verhalen, spellen en animaties kan maken. Het is bedoeld voor de leeftijden 8 tot 16, maar kan voor iedereen een aardige introductie zijn in het maken van programma's. Toen ik van Scratch hoorde werd ik benieuwd. Een race spel maken in Scratch leek me een leuke manier om het te leren. 

Ik had vooraf de volgende wensen voor mijn race spel:

- je moest kunnen racen tegen een ander mens

- je moest kunnen racen tegen computergestuurde tegenstanders

- je ziet alles van bovenaf

Nu ik daar een tijdje mee aan de slag ben geweest, wilde ik delen hoe ik bepaalde dingen had gedaan. Het is een best lange post geworden, en om het niet nog langer te maken ga ik er van uit dat je al wat ervaring met de Scratch basisbegrippen hebt.

Het programma in volgevlucht

Speelveld 

Het speelveld van Scratch is als het ware een achtergrond. Je kunt het speelveld meerdere uiterlijken geven. In dit geval is er maar een uiterlijk: de racebaan inclusief startfinish lijn bevat. De rand van de racebaan mag niet gepasseerd worden, als je die raakt wordt je snelheid op nul gezet.

Auto sprite 

Binnen Scratch is een sprite als een poppetje die je kunt laten bewegen maar ook kunt programmeren om bepaald gedrag te vertonen als er dingen gebeuren. Iedere sprite heeft een of meerdere uiterlijken, dat bepaalt hoe hij er uit komt te zien in het spel. Voor de auto sprite kun je een plaatje van een bovenaanzicht van een auto op internet opzoeken en inladen.

De auto sprite is voor zowel menselijke als computer gestuurde auto's. De auto sprite bevat niet alleen een afbeelding van een auto, maar ook gekleurde vlakken voor botsing detectie. De auto sprite heeft 3 uiterlijken:

computer gestuurde auto	menselijk gestuurde auto	garage

Dit sprites zijn erg simpel, maar je kunt het zo mooi maken als je wilt. De 'garage' sprite heb ik gemaakt als zijnde de plek waar auto's vandaan komen en voorlopig maar als een paars vlak getekend.

Computer navigatie zones 

Dit zijn sprites die de computergestuurde tegenstanders helpen hun weg te vinden op de racebaan. De racebaan tekening alleen volstaat niet: de computer tegenstanders zouden dan aan de hand van botsingen met de rand sturen hetgeen ze zou vertragen. Bovendien ziet het er niet erg realistisch uit: menselijke tegenstanders volgen niet zozeer de racebaan rand maar de zogenaamde 'ideale lijn'.

Door de auto op deze lijn te houden bereiken echte coureurs de optimale snelheid en de snelste rondetijden. Menselijke coureurs vinden deze lijn door ervaring van anderen en zelf te experimenteren. De gevonden lijn onthouden ze voor tijdens de race en passen die zonodig ter plekke aan al naar gelang de omstandigheden, bijvoorbeeld regen.

Om dat gedrag deels na te bootsen geven we de computer tegenstanders kennis van de ideale lijn. Dat doen we door twee sprites te introduceren, die zo groot zijn als het speelveld zelf:

• zone_links_van_ideale_lijn: hiermee kunnen computer tegenstanders weten wanneer ze zich links van de ideal lijn bevinden (en dus naar rechts moeten):

• zone_rechts_van_ideale_lijn: hiermee kunnen computer tegenstanders weten wanneer ze zich rechts van de ideal lijn bevinden (en dus naar links moeten)

Daarnaast rijden coureurs scherpe bochten met een lagere snelheid dan de rechte stukken of flauwe bochten. Om de computer tegenstanders dit ook enigszins te laten doen, moeten ze op de een of andere manier weten dat ze zich in een bocht bevinden. Een simpele manier om dat te bereiken is door een derde navigatie zone sprite te maken:

Als je de racebaan en navigatie sprites combineert krijg je dit plaatje:

De bedoeling si dat de computer auto's langs een nauwe corridor rijden die de ideale lijn voorstelt, en op de roze stukken vaart zullen minderen.

De navigatie sprites zullen overigens onzichtbaar worden gemaakt door middels van het 'geest effect'. Hiermee kun je een sprite geheel of gedeeltelijk doorzichtig maken. We zullen de waarde op 100 zetten, hetgeen de navigatie zones geheel doorzichtig zal maken. Ze kunnen echter wel degelijk worden 'gezien' door Scratch, ze zijn enkel onzichtbaar voor het menselijk oog.

Autos programmeren

Om autos te programmeren zullen we ze scripts toekennen. De scripts beschrijven het gedrag van de autos en worden door Scratch ingelezen om ze de instructies te laten uitvoeren. Om de scripts goed te doorgronden is het handig om eerst de structuur van de scripts van de Auto sprite op hoofdlijnen te begrijpen. 

Autos klonen

Voor het maken van iedere auto, ongeacht of het door mens of computer wordt bestuurd, maken we gebruik van sprite klonen. De 'Auto' sprite zelf wordt dan eigenlijk een soort van sjabloon waarmee we andere sprites zoals hem kunnen maken.

We hadden ook de Auto sprite kunnen kopieren vooraf met de functie 'kopie maken'. We zouden dan een kopie krijgen van de Auto sprite op dat moment, die we bijvoorbeeld 'Tegenstander' zouden kunnen noemen. Dit kent wel wat nadelen: ten eerste moet je voor iedere extra tegenstander deze handeling opnieuw doen. Maar er is nog een probleem: stel dat je een verandering aanbrengt in 'Auto' die ook in 'Tegenstander' moet komen? Dan moet je opnieuw een kopie maken. Maar als je 'Tegenstander' al aangepast had om hem een specifiek gedrag te geven, dan zul je die aanpassingen opnieuw moeten doorvoeren.

Daarom werken we met enkel de 'Auto' sprite. Het zijn allemaal auto's per slot van rekening. We willen ook de spelers visueel van elkaar kunnen onderscheiden. Scratch geeft de mogelijkheid iedere Sprite een aantal uiterlijken toe te kennen. We geven zo de auto's verschillende kleuren. De sjabloon wordt zo met andere parameters toegepast, waardoor we controle hebben over het gemeenschappelijke en afwijkende deel van hun gedrag.

Als het programma wordt gestart willen we de klonen aanmaken. Dat doen we door te reageren op het klikken van de groene vlag (start van het programma):

Hier worden een aantal variabelen van begin waardes voorzien. Ik zal een aantal nu uitleggen, de rest komt later aan de orde:

• speler_auto: als 0, dan is de auto computer gestuurd, anders door een speler. Momenteel ondersteunt het programma maar 1 speler
• draai_snelheid: de snelheid waarmee auto's van richting veranderen.
• aantal_ronden: aantal ronden die de race telt
• clone_id: unieke identificatie getal van een auto kloon. Om die reden wordt clone_id met 1 opgehoogd binnen de herhaal blok, anders krijgt iedere kloon dezelfde identificatie en is die niet uniek

In het herhaal blok daarna worden de klonen aangemaakt. Iedere keer dat de 'maak kloon van' blok wordt aangeroepen, wordt een kloon gemaakt.

Hoe de auto op hoofdlijnen werkt

Om de kloon te gaan programmeren en niet de Sprite zelf, moeten we reageren op de gebeurtenis 'wanneer ik als kloon start':

Wederom is er een begin stuk en een herhaal blok. Ik leg eerst kort uit een aantal dingen die in het stuk voor het herhaal blok gebeuren:

• my_id: de kloon kopieert de waarde van clone_id op dat moment, en houdt hiermee zijn eigen id bij
• nieuwe_richting: met deze variabele bepalen we welke richting de kloon moet gaan rijden. In het begin krijgt hij dezelfde richting als de Sprite (het sjabloon dus!). We kunnen de beginrichting bepalen door de Sprite zelf te slepen en te draaien en dat is wel zo makkelijk
• topsnelheid: klonen met een hogere id hebben een hogere topsnelheid. Dit is puur gedaan om het spel interessanter te maken

Het belangrijkste is nu het 'herhaal' blok. Alles wat binnen deze blok gebeurt wordt continu herhaald, dus gebeurt niet alleen op het moment van kloon start maar zolang de kloon in het spel is.

Het 'herhaal' blok heeft een hoofdstructuur, waarbinnen andere blokken worden aangeroepen:

• in het begin verplaatst de auto zich naar de nieuwe richting met de nieuwe snelheid
• in het eerste 'als' blok worden deze eigenschappen bepaald voor de computer auto's door het aanroepen van de paarse blokken
• in het tweede 'als' blok gebeurt dat voor de mens gestuurde auto's
• de paarse blokken die daarvoor of daarna zitten, worden in beide gevallen aangeroepen

De paarse blokken zijn eigen gemaakte script blokken die bij elkaar worden gevoegd onder een specifieke naam. Hiermee zijn ze makkelijk herkenbaar en herbruikbaar. Zo zie je dat sommige paarse blokken voorkomen bij beide soorten auto's, andere weer niet.  En als je kijkt naar hoe die paarse blokken genoemd zijn, begrijp je dat hier het echte werk gebeurt: botsen met muren, snelheid aanpassen, rondes registreren. Met andere woorden, alle beslissingen die nodig zijn om een race te kunnen rijden. Een blik op deze blok is zo voldoende om te zien wat een auto kan. Laten we de blok daarom het 'gedrag blok' noemen.

Een mens gestuurde auto

We willen de auto zo maken dat hij reageert op onze invoer. Dat bepalen we in de 'stuur_mens' blok. We gebruiken het toetsenbord. Scratch heeft handige scripts die je daarvoor kunt gebruiken: 

De auto moet nog verkleind worden. Racebanen zijn meestal een aantal autobreedtes breed, tenzij het straatcircuits zijn. Laten we de auto tot ongeveer een vijfde van de breedte van de weg verkleinen.  Gebruik daarvoor de 'kleiner maken' knop in de grijze bovenbalk van Scratch (dit is een screenshot van de offline editor):



We kunnen nu de auto sturen, sneller laten gaan, stoppen en achteruit laten gaan. Het probleem nu nog is dat we door de muren van het circuit kunnen rijden. Dat zullen we bijstellen met het 'detecteer_muur' scriptblok:

Dit zorgt ervoor dat je snelheid nul is als je met de muur in aanraking komt. De auto weet dat dit zo is omdat een signaal stuur naar de auto sprite wordt gestuurd als hij een bepaalde kleur raakt. Daar zorgt Scratch voor. Je kunt het dus zien als een soort omroepmechanisme op kleur gebaseerd. Wij maken die kleur gelijk aan de kleur van de muur.

Dit blok wordt bewust aan het eind van het gedrag blok aangeroepen, zodat ongeacht wat de andere blokken hebben besloten, we nooit door muren heen kunnen rijden. Andere blokken kunnen zich dan concentreren op een ding en blijven op deze manier zo simpel mogelijk. Het maakt het ook makkelijker om ze te programmeren als je je niet hoeft te bekommeren met muur detectie in andere blokken.

Dit principe wordt consisten toegepast: ieder blok heeft zijn eigen taak, onafhankelijk van de andere. Door ze te combineren krijgt de auto zijn gedrag. Zo heeft het programma als geheel een duidelijke opbouw en is het tevens makkelijk om nieuw gedrag toe te voegen of specifieke delen aan te passen.

Rondjes tellen

Als we de rondjes niet kunnen tellen, loopt de race nooit af. We hebben daarvoor een groene lijn getekend, die als start-finish zal dienen. Iedere keer dat een auto start-finish passeert, wordt een ronde genoteerd voor die auto. Dat wordt bereikt met het volgende script blok:

Wederom maken we gebruik van kleuren om te weten wat er gebeurt: als de kleur van start-finish (groen) door de auto sprite wordt geraakt, wordt het signaal doorgegeven aan dit script blok en die registreert de ronde.

We moeten hier wel voorzichtig zijn: omdat de lijn dikte heeft, wordt dit signaal afgevuurd niet alleen als de auto voor het eerst te lijn raakt, maar ook tijdens het passeren van de lijn. Dat zou voor een heleboel overbodige ronde registraties zorgen.

Wij ondervangen dat door bij te houden of we al eerder start-finish gezien hadden met de variabele finish_lijn_gepaseerd. De variabele start met de waarde '0', hetgeen betekent 'niet gepasseerd'. Bij iedere aanraking met groen wordt  geverifieerd dat de waarde nog steeds '0' is en zo ja dan:
- wordt de variabele op '1' gezet
- de ronde wordt geregistreerd.
- als het maximum aantal rondes is gepaseerd, eindigt het spel (stop alle)

Bij de volgende aanraking met groen, dat tijdens het passeren van de start-finish lijn gebeurt, zullen we dit blok niet ingaan. Als we groen niet aanraken, wordt 'finish_lijn_gepaseerd' weer op '0' gezet voor de volgende ronde.

Computer gestuurde tegenstanders

Nu de computer tegenstanders. Murendetectie en ronde registratie is hetzelfde als bij de menselijk bestuurde auto's. Wat nu dus rest is logica voor het volgen van de ideale lijn, het afremmen in de bocht, en het vermijden van botsingen met andere auto's.

De ideale lijn volgen bereiken we met het volgende code blok. Doordat we de zones hebben gemaakt, is het een hele eenvoudige blok omdat Scratch ook voor Sprite detectie handige scripts aanlevert:

Afremmen in de bocht volgt hetzelfde principe van sprite detectie. We moeten nu er wel op letten dat auto's na de bocht weer gas geven, anders blijven ze langzaam rijden. De auto's hebben een topsnelheid variabele dat aangeeft hoe snel ze kunnen rijden. De topsnelheid is buiten de bocht afhankelijk van de auto_id, zodat er auto's van verschillende snelheden zijn. In de bocht mag die topsnelheid niet groter dan 2 zijn, dus als dat zo is dan wordt de topsnelheid bijgesteld.

De laatste twee 'als' blokken zorgen ervoor dat de snelheid stapsgewijs aan de topsnelheid wordt aangepast: als de huidige snelheid te hoog is wordt er afgeremd, als te laag wordt er gas gegeven.

We hadden ook de snelheid gelijk kunnen zetten op 2, en daarna op 'topsnelheid'. Dat is echter wel erg onnatuurlijk en lijkt niet op hoe echte coureurs het doen.

Computergestuurde auto's moeten ook elkaar kunnen ontwijken. Om dat te kunnen programmeren moeten we de het proces van botsen wat beter bekijken. Bij botsingen zijn er twee klonen van dezelfde Sprite er mee gemoeid, allebei auto's. Alle auto's werken volgens dezelfde basisprincipes, maar hun gedrag kan verschillen afhankelijk van de situatie. De truc is om de gebeurtenis te bekijken van uit de auto zelf, alsof je de bestuurder bent van die auto.

Om een auto te laten weten dat er een botsing dreigt maken we weer gebruik van kleurgebaseerde detectie.De botsing detectie vlakken zijn onderverdeeld in links (paars), rechts (geel) en achter (blauw). Voor het ontwijken van een botsing kijken we allen naar botsingen op de voorkant, dus als paars of geel wordt geraakt. De regels die de auto's moeten volgen zijn dan:

• als paars wordt geraakt door een andere auto, stuurt de bestuurder de auto naar rechts
• als geel wordt geraakt door een andere auto, stuurt de bestuurder de auto naar links

De bestuurder reageert dus niet als blauw wordt geraakt, omdat het niet van voren is (vanaf de bestuurder gezien). De blauwe kleur is bedoeld om te weten of we een andere auto van achteren raken en dan te reageren. Dus niet om zelf te reageren als een andere auto ons van achteren dreigt te raken, want in dat geval doen we niets. De 'ontwijk_autos' blok wordt dan:

Let op dat dit niet botsing detectie tussen auto's is, dit is juist een methode om botsingen te voorkomen. In sommige situaties kan het echter voorkomen dat auto's niet snel genoeg kunnen draaien, en dan zullen ze 'botsen'. Je ziet dan een overlap in de auto afbeeldingen van de klonen. Dit is dus nog niet helemaal klaar maar is een aardig begin.

Het nadeel aan deze kleur gebaseerde detectie is ook dat de detectie vlakken zichtbaar zijn terwijl het eigenlijk beter was geweest om ze onzichtbaar te maken zoals de navigatie zones. Ik heb nog niet goed kunnen nadenken hoe dat voor elkaar te krijgen. We kunnen op deze manier wel ervaring opdoen en de logica goed krijgen.

Conclusie

Ik heb besloten dit te maken omdat ik met Scratch bekend wilde raken en een racespel leek me wel aardig. En ik heb een hoop van Scratch en diens (on)mogelijkheden geleerd: klonen, detectie op kleur en op klonen, eigen gemaakt script blokken, het was een hoop gegoogle en gepuzzel af en toe.

Wat ik met name krachtig en leuk aan Scratch vond is het combineren van visuele elementen en het programmeren ervan. Zo kan dit spel makkelijk met extra circuits worden uitgebreid door ze eenvoudigweg te gaan tekenen, terwijl het programma er verder voor  zorgt dat het spel gaat werken. Het was ook verbazend makkelijk om de computer auto's autonoom te laten rijden (al gaat het niet altijd vlekkeloos). Het reageren op gebeurtenissen geeft een makkelijk en flexibele manier van indelen van je programma's, heel geschikt voor spelletjes.

Het spel is ook nog niet af: het was de bedoeling om tegen een ander mens te kunnen racen. Dat zal ik later moeten toevoegen, en misschien nog wat andere verbeteringen. Als je meer wilt weten post een vraag of beklijk dit project is online op https://scratch.mit.edu/projects/99557843/

Dual Booting Sophos

Introduction

In this article I will explain how I got Fedora 15 KDE installed alongside Windows 7 with Sophos Bootloader and fully encrypted Windows 7 partitions. Bypassing Sophos still does not seem possible, the reason is because the Windows partitions only get decrypted through Sophos which resides in the MBR. I have therefore chosen to use the Windows 7 boot manager in order to boot Fedora.

Beware that this is not a step by step tutorial, and should not be taken as such. This is more like a collection of my personal notes taken during this project. Please be careful en make sure you understand the risks of a certain technique before using it, as missteps could very well lead to complete data loss and not being able to boot your machine (from the harddisk at least). Some machines have recovery options which rely on an image on a separate partition from which Windows can be re-installed. This could also get compromised if things go wrong, make sure you understand if this applies to you and the possible consequences of a mistake.

My setup looks like this right now:

• Sophos boot loader in the MBR
• Sophos Encrypted Windows 7 with custom boot manager and Grub4Dos
• Fedora 15 KDE (Linux) Grub on the same partition

Backup

It is wise to have a complete backup of the hard disk. Because the partitions are encrypted, Windows does not boot without the Sophos bootloader in the MBR, which easily could be caused by making a mistake while installing Fedora. There are more disaster scenarios conceivable, so backup is something worth thinking about it. The backup method must be able to restore the MBR, work with encrypted partitions, and work even if the Windows installation on the machine is unusable.

I have done the backup from the Fedora liveusb using the dd-command, which does bytewise raw data copying. Dd will copy entire harddisks or partitions with complete disregard for its contents, which makes it suitable to restoring your system to a previous state byte for byte. This is especially important since the windows partitions are encrypted, and because Sophos resides on the MBR. For me it was useful because I often was experimenting and taking corrective actions, but I can not really recommend it to everyone because you can quickly make a mistake and loose everything.

Partitioning

The first condition is that non-partitioned space on the machine must exist to install Fedora on. This is achieved by reducing existing Windows partitions, which can easily be done from a Fedora liveusb as long as they are not yet encrypted (sophos usually encrypts the drives on first boot).

Creating the Fedora partition had to be done from Windows, because when I tried it with a Linux partition manager (gparted) Windows would not boot afterwards. I used the Partition Wizard tool instead, which can format EXT4. Assigning a drive letter in Windows was not necessary, so I skipped that. My partitions look like this now:

Windows Modifications

In order to start the Fedora partition from the Windows 7 boot manager I installed Grub4Dos, a port of Grub for the DOS/Windows platform. I have simply followed this guide. Remember to copy grldr.mbr, grldr and menu.lst to the root of the C drive.

Additionally, you  must edit the menu.lst to boot Fedora. This is my menu.lst:

# This is a sample menu.lst file. You should make some changes to it.

# The old install method of booting via the stage-files has been removed.

# Please install GRLDR boot strap code to MBR with the bootlace.com

# utility under DOS/Win9x or Linux.

color blue/green yellow/red white/magenta white/magenta

timeout 0

default /default 

title Fedora

root (hd0,2)

chainloader +1 


The last section tells Grub4Dos to boot whatever is on the third partition of the hard disk. In this partition I have a (Linux) Grub installation, which in turn starts Fedora. This is useful for kernel updates or when you change Linux distributions later, such changes would have no impact on the Windows part.

Sophos launches Windows 7 and, after a moment, this screen follows:

The entry 'Grub' brings me in Fedora and is set as default ;-). One thing I have ensured is that the timeout for choosing an OS not zero, because this would probably lead to a situation where booting into Windows is impossible.

Fedora Installation

Installing Fedora from a liveusb is largely automatic up to the point when a type of installation must be selected:

You must select 'Create Custom Layout' here. In the next screen select the partition created with Partition Wizard and click Edit. Use as a mount point '/'. The following warning regarding swap can be ignored.

You could also make an additional partition (from Windows!) and set it as swap, but I have not tried that. I have currently a swap file, which I have setup by following these instructions.

Follow the installer until the time comes to choose where to install Grub. Here it is important to make sure that Grub is installed on the Fedora partition by choosing 'Change Device'. This is because by default Grub is installed on the MBR which will remove Sophos and render Windows unusable.

Conclusion

Fedora is now alongside Windows 7 in its own partition so that both operating systems can have the entire system resources. The Fedora partition is not encrypted though, but that should be possible without much trouble.

One little drawback is that you must login into Fedora in addition to Sophos, for which I have unfortunately no solution.

Fedora hibernate does not work because I have no swap partition, but here is explained how to do hibernating with a swap file. I myself have not tried it and use sleep instead. On wake up, you do not need to login into Sophos again.

If you have any further questions, tips or improvements please leave a comment. And if you are going to try it yourself: get a good backup solution.

Testing More by Testing Less

Introduction

Recently I have been thinking about unittests an their utility. I've had the opportunity to try a different approach, which I want to talk about in this post.

We are all familiar with unittesting by now. For most of us, that means that for each class we write, a corresponding test is produced, preferably beforehand. However, I find this method brings a drawback: if you refactor that code, you must refactor your unittests as well. But as we all know, refactoring is good and should be encouraged.

Refactoring is needed because getting the design of the code right the first time is notoriously difficult. Sometimes this is due to poor judgement or time constraints. But the fact is that in practice it's hard to solve complex problems and have good design at the same time. And by 'good' I mean that the various classes and their assigned responsibilities are such that the code is easily understood and easily extensible.

The latter is particularly hard to achieve because, while we often think otherwise, we usually don't know what changes will be brought upon our application. That means spending more time refining the design will not help us, what would help is finding ways to change the code quickly and confidently when the need actually arises.

I feel this is not being addressed by the unitest-per-class method, because these kind of changes typically break the original APIs and/or introduce new ones in the form of new classes. That means your unittests are hit as well so they need to be updated, slowing you down and introducing more scope for errors. You are also less confident in the modified unittests than you would be if they could be reused unaltered.

That creates a barrier to refactoring code. Not only is there more code to refactor, but also you cannot rely on your unittests to protect you from errors. Which is ironic really, because this is a big reason why people write them in the first place. Meanwhile the bad code lingers, and when it must be touched because of defects, you would rather only fix that and get out as soon as possible. And each time you do that, the bad code grows. Wouldn't this be easier if the refactoring wasn't so risky or required such effort? 

There is another disadvantage: waste. Writing tests for all classes in your codebase is, in my opinion, not a good way to use resources. Because unittests are a lot like boilerplate: they don't really add any real value. They are only there to help us developers do our job, but they do not fulfil any business needs. In other words it is an activity which destracts us from achieving business value, and should therefore be limited.

Testing more by testing less

I propose that to effectively address these issues instead of writing a unittest per class, we test multiple classes collectively. This would give freedom in refactoring them, because your testcode would not use all classes directly, only a few necessary to input testdata and collect output.

I have pictured this below. The squares are classes, and the arrows denote the direction of inter class dependencies. The blue squares inside the dashed lines are the classes under test, or 'the system'. Squares outside the dashed lines are left out of the test, they effectively define the test boundary. 

The yellow classes need to be mocked, so that the tested classes have someone to talk to at runtime and won't crash. But also so that we can provide test data, and capture data sent from the system in order to assert their correctness and by extension the correctness of the system itself. There are many frameworks designed to help with these tasks, which we can reuse.

This is all quite similar to ordinary unittesting. You could even pretend that the classes being tested are just one class. Actually, we can do more than just pretend: we can introduce an intermediate class, which we use in writing tests instead of the real classes. I call this a TestProxy. It should be responsible for wiring the objects under test to the mock objects, and provide easy access methods for invoking real functionality on the objects under test, on a coarse grained level.

Let's consider a simple example based on the Model-View-Presenter (MVP) pattern. In this example the Presenter uses a a BusinessService for accessing business functions, this service in turn   accesses a resource service that fetches data from a remote location, like a webservice.

The View and the Resource class are good candidates for keeping out of the test: the Resource class needs a webservice running to work properly which makes testing cumbersome. And the View is usually not programmatically testable, often the reason for using the MVP pattern.

The picture below shows the classes involved in the test:

Note that I have drawn mocks for the View and Resource. These mocks need to be instantiated and wired to the actual Presenter and BusinessService, which happens in the TestProxy. It also makes these mockservices accessible to the TesterClass, because that is where the actual test logic is. The TesterClass selects mockdata, invokes functions through the TestProxy and does assertions.

That means that instead of talking to the Presenter or BusinessService in your tests, you only talk to mocks. This totally insulates the testlogic from the actual object hierarchy making this logic reusable should the hierarchy change. You should only have to change your test logic if there is some change in the mocks, which would only happen if the data shown on the view or fetched from the webservice changed.

Another advantage is that the TestProxy can provide an API that makes testlogic more readable and compact. We can achieve this in this particular case by mirroring the actions a user does in our API, because we mock views that interact with users directly. For instance, this is what some test pseudo code might look like:


function TestDeleteEntry(){
 
    staringEntriesList = { ... }
    endEntriesList = { ..... }
 
    startOverviewData = { ... }
    endOverviewData = { ... }

 
    TestProxy.ResourceMock.setEntries( startingEntriesList ); 

    TestProxy.start();
    TestProxy.MainMenuMock.assertHasFocus();

    TestProxy.MainMenuMock.chooseOverview();
    TestProxy.OverviewMock.assertHasFocus();
    TestProxy.OverviewMock.assertData( startOverviewData );

    TestProxy.OverviewMock.selectEntry(2);
    TestProxy.DetailViewMock.assertHasFocus();
    TestProxy.DetailViewMock.assertData( startingEntriesList.get(2) );

    TestProxy.DetailViewMock.chooseDelete();
    TestProxy.OverviewMock.assertHasFocus();

    TestProxy.OverviewMock.assertData( endOverviewData );

    TestProxy ResourceMock.assertEntries( endEntriesList );     

}

The exact API doesn't matter, and may well depend on the type of mocking libraries at your disposal. What matters is that the test logic does not depend on the internal wiring of the classes being actually tested. So maybe at first you implemented this using three views but the same presenter. Later you switch to three presenters. Than you decide to refactor the BusinessService, or the Model. Your test remains the same, and can therefore be trusted to catch any errors.

What matters also is that methods like chooseOverview and selectEntry relate directly to user actions. This makes it easy to write tests based on functional specifications, and working with testers in order to validate and design good tests.

We could further tweak this API, for instance turn the groupings of actions and assertions into methods themselves. But more importantly is reuse on another level. Suppose instead of deleting an entry, we would like to edit it? In order to get to the edit page, we would need to repeat some steps, may reuse some mock data. By creating methods which accomplish a reusable part of a flow, we can avoid any copy-and-paste. We can likewise structure our mock data into extensible sets.

Conclusion

I have argued that conventional unittesting methods have some flaws, and described a method for testing groups of classes instead of individual classes in order to address them. While there are some areas to work on, particularly reuse, after actually trying this method certain advantages are already clear to me.

As we have seen in the example, this method makes it possible to write tests on a level much closer to functionality observable to users, business level if you will. You can then work more closely with testers. Besides saving time in thinking of test data and test cases, this should also result in a higher quality of tests. This also means recreating defects found by testers is easier.

You can confidently refactor code with extensive freedom and more speed. Freedom and speed are good, but the confidence is also important. The tests are untouched, so they can be trusted to protect you in case of regression. If you had modified the original tests, you would have to verify that they work first.

I have also found that it gets relatively easier to reach a higher code coverage. This is due to the fact that a given input can trigger a cascade of objects calling each other, instead of you manually writing tests to call methods. But it is also because thinks like constructurs, getters/setters are tested automatically, and you would normally not test those.

This seams somewhat distorting and I besides the point. And to met testing constructors and getters/setters does not add much value. But I can think of one case where it might be usefull: dynamic languages. With dynamic languages you can't rely on the compiler to check for typing errors, so sloppy programming could easily cause a bug. The way to protect yourself is through a high unittest code coverage, where more of the code is exercised and such errors are exposed in the unitest.

The threshold to start testing does get higher, however, because you need to write more code in order to 'get testing' since we need to abstract the class hierarchy. While I think in the long run you need less code to achieve the same coverage, this does require some extra discipline. And as we all know this is often not easy given tight schedules and limited resources.

Finally, I think this method works best when you already have a modular architecture. That creates natural islands of related classes, and introduces loose coupling with other such islands. That means the amount of mocking gets minimized in relation to code tested, and you can truly 'test more by testing less'. You should ideally mock views, or a webservice call: these are more or less stable boundaries that won't change if the internal design of the code is altered.

I believe the testapi is not quite ready yet and warrants some more thought, in order to fully exploit the similarities with UC scenarios and promote reuse. Separating the test logic from the test data enables you to play different scenarios by changing data sets, this seems like a good strategy to increase test productivity so it's worth exploring further also.

High Availability with JBoss SOA Platform

High Availability and why we need it

When employing ESBs, one of the most import aspects is that the ESB be highly available. This requirement will arise naturally from the fact that the ESB is an intermediate between other applications which rely on it to supply them with data. When the intermediate is offline, then the entire chain comes offline. To counter this, the ESB is deployed with some sort of failover mechanism in order to make it more resilient to failures.

JBoss SOA Platform is Red Hat's solution for enterprise SOA. It contains, among other things, the Jboss Application Server plus two products that we are interested in: JBoss ESB and JBoss Messaging. JBoss ESB is at it's core a mediation engine, it contains the logic for getting messages from one endpoint to another.  The endpoints are the sources and destinations for messages. They can be implemented with various technologies, like JMS queues, webservices, ftp sites, etc. JBoss Messaging is a JMS implementation, it enables you to setup and manage queues.

Combined this two technologies enable you to move data accross your enterprise: by letting an application put messages on one JMS queue, JBoss ESB can then transport it to another queue where the messages are picked up by another application. One of the advantages of such an approach is that the two applications that are actually sharing data in this way don't need to know about each other. They don't even need to be running at the same time. The downside, as previously noted, is that the ESB is a single point of failure.

HA strategies and concepts

We mentioned the need to make the ESB resilient to failures. But how do we go about this? Remember that the root of the problem is that the ESB is a single point of failure. That is the case because it is running in a process, and that this process could cease for whatever reason. 

But if we could have multiple ESBs running in their own processes, then the failure of one would not affect the other. The better the different processes are insulated from each other, the more robust the solution. For this to really work however, a mechanism would be needed so that in case of failure of one instance, the remaining ones take up the work it was doing.

But making the ESB resilient is not enough. Because the endpoints are the interface between the ESB and other systems, they too need to be HA for without them no data can be shared. The HA strategy of the endpoints is strongly correlated with the endpoint transport mode. For instance: JBoss Messaging can cluster queues, webservices can be made HA via a dedicated http loadbalancer. As it turns out JBoss Messaging has it's own out-of-the box clustering support, so we just use that.

The diagram below illustrates the ideas. The dashed lines represent process boundaries. The two ESBs are insulated from each other, so the failure of one does not influence the other. For simplicity we consider the case where an external agent (sender) puts messages into the messaging system via an endpoint which is by itself HA.
  

Note that the process boundary in which the HA endpoint is running arises because of failover. There are actually multiple instances on different processes, but they can work together and take over each other's work. So as far as the other systems are concerned, it is as if only one very resilient instance was running. Moreover, adding more ESB instances will increase performance, as long as there are enough messages to consume.

One question comes to mind: what happens when a ESB instance takes a message from the endpoint and then fails? If no precaution is taken, then the message is lost. A crucial feature of our HA implementation should be that each message processing is done inside a transaction, so that should it fail the message becomes available on the endpoint again for processing by still active instances.

JBoss Clustering and Messaging

Since we are using JBoss Messaging for our endpoint implementation, we should briefly explain how it's clustering works. JBoss Messaging can be made HA via JBoss Clustering, which is built into the JBoss Application Server. It uses the JGroups protocol to enable different JBoss instances on the same network to find each other and keep each instance aware of the others. Should one instance fail, then the others become aware of this and take over it's workload. JBoss Messaging uses this facility to detect a queue failure, and migrate it's messages to another running queue. 

In order to have our cluster we will deploy JBoss AS on different nodes on the same network. A node in this case would be a different (virtual) machine, which would do a good job insulating the different processes from each other. 

We consider a case with just one queue, Queue1, for simplicity. Queue1 is deployed on separate nodes, and clustered. As you can see JBoss ESB is deployed along with Queue1 on the same node, therefore that node's failure would affect both. Still we have a backup on the other node, so HA is achieved.

One thing to note is that the consumers of Queue1, including JBoss ESB, can directly communicate with any instance of Queue1 on any node in the cluster. This is because JBM does loadbalancing and failover on its JMS client implementation. This is actually the key of JBoss Messaging's HA, because without it clients would simply fail along with the failing node.

JBM clients are configured to contact one single node, from which they receive a list of all available nodes in the cluster which the node knows via JBoss Clustering. The client then can access the Queue1 instance on any of these nodes. Should it happen that the node which the client is currently connected to fails, then it will automatically switch to another node on the list, and access its Queue1 instance. The client can also loadbalance over the Queue1 instances, improving performance. This all happens transparently to the code that uses JBM, as far as that is concerned there is only one Queue1, and it never went down.

But what happens in the remote event that the complete cluster goes down? We still do not want to lose our messages. Therefore they are persisted by the queues in a separate storage system, shared by all nodes in this case. This database must of course be made HA also. The best strategy and implementation depends on the database vendor, and is beyond the scope of this article. We will assume a mysql database because it is a popular, proven product which can be made HA in a way transparent to clients.

Setting up an actual test  

Now that we have discussed some basic concepts, we can consider an actual case. For this example  the ESB will take messages from  a JMS queue named InBoundQueue and place it in the OutBoundQueue. Inside, we will configure the ESB to pause for a while, thereby simulating some heavy processing.

The nodes are virtualized using Virtualbox, or your favorite virtualization product. We will setup JBoss ESB and JBoss Messaging in HA configuration. Then we will consider a simple failover test and how to verify the results.

Virtualbox

Setup three virtual machines using your favorite OS, two for JBoss, one for Mysql. Make sure the virtual machines can resolve each other over the network by choosing for instance host-only networking in Virtualbox. Also, make sure the ip address of the mysql server is static, this depends of the OS you choose. It also makes things easier if the other nodes also have a static ip for deployment, sending messages, etc.

Install and setup the necessary software on the nodes, like mysql server and Java JDK. Copy JBoss soa platform on the JBoss nodes. I recommend setting up one JBoss node completely, then cloning it to create the other. The clone will need some customizing, see below.

JBoss Server Profile

First, create a new profile by creating a new directory under {soa.platform.install.dir}/jboss-as/server and copy into it the contents of the 'all' directory. I have chosen this one mainly because it contains the facilities necessary for clustering. Let's assume this profile is called 'my_cluster'.

JBoss setup

Then edit the file{soa.platform.install.dir}/jboss-as/tools/schema/build.properties according to your needs such as database vendor and connection info.

Take special care to set org.jboss.esb.server.config to the one created earlier, 'my_cluster'. Also make sure to set org.jboss.esb.clustered to 'true'. Also, copy the JDBC driver jar for your database vendor (for instance, mysql) to the lib directory of  'my_cluster' to avoid any errors further on.

Then, just run ant. The script will update the 'my_cluster' profile to work with your database and support clustering. 

JBoss will automatically use the credentials supplied to create the necessary tables for JMS.

JBoss startup

JBoss must be bound to the ip of the machine it is running on. Also, do not forget to start with the
jboss.messaging.ServerPeerID  option to a value unique inside your cluster, and to use the 'my_cluster' profile. There is also an option to set the cluster partition name jboss.partition.name. Nodes with the same name form a cluster when on the same network. This is useful for running multiple clusters on the same network, but does not concern us now.

JBoss ESB setup 

I will assume you are already familiar with developing and deploying .esb files. Edit jboss-esb.xml as follows:

<?xml version="1.0"?>

<jbossesb parameterReloadSecs="5"

 xmlns="http://anonsvn.labs.jboss.com/labs/jbossesb/trunk/product/etc/schemas/xml/jbossesb-1.2.0.xsd"

 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://anonsvn.labs.jboss.com/labs/jbossesb/trunk/product/etc/schemas/xml/jbossesb-1.2.0.xsd http://anonsvn.jboss.org/repos/labs/labs/jbossesb/trunk/product/etc/schemas/xml/jbossesb-1.2.0.xsd">

 <providers> 

  <jms-jca-provider connection-factory="ClusteredConnectionFactory" name="JBossMQ">

   <jms-bus busid="quickstartEsbChannel">

    <jms-message-filter dest-name="queue/EsbQueue"

     dest-type="QUEUE" persistent="true" transacted="true"/>

   </jms-bus>

   <jms-bus busid="GwChannel">

    <jms-message-filter dest-name="queue/InBoundQueue"

     dest-type="QUEUE" persistent="true" transacted="true"/>

   </jms-bus>

  </jms-jca-provider>

 </providers>

 <services>

  <service category="myCategory"

   description="Hello World File Action (esb listener)" name="myFileListener">

   <listeners>

    <jms-listener busidref="GwChannel" is-gateway="true" name="gwlistener"/>

    <jms-listener busidref="quickstartEsbChannel" name="helloWorldFileAction"/>

   </listeners>

   <actions mep="OneWay">

    <action class="org.jboss.soa.esb.actions.SystemPrintln" name="PrintBodyConsole">

     <property name="message" value="== Printing body =="/>

     <property name="printfull" value="false"/>

    </action>

    <action

     class="org.company.WaitAction" name="wait">

     <property name="timeSecs" value="60"/>

    </action>

    <action class="org.jboss.soa.esb.actions.SystemPrintln" name="PrintSendConsole">

     <property name="message" value="== Sending to Queue =="/>

     <property name="printfull" value="false"/>

    </action>

    <action class="org.jboss.soa.esb.actions.routing.JMSRouter" name="PutInOutQueue">

     <property name="jndiName" value="queue/OutBoundQueue"/>

     <property name="unwrap" value="true"/>

    </action>

   </actions>

  </service>

 </services>

</jbossesb>

In the provider configuration, note that we are using 'ClusteredConnectionFactory'. This connection factory makes sure we will access the clustered queues as one, and profit from failover and loadbalancing. Also note that we have set the transacted flag to 'true', so that messages will reappear on the queue if processing in the service pipeline fails. 

We also set the persistent flag to 'true', so that messagese will be preserved even when the whole cluster goes down. This amounts to letting the queue know we want persistent messages, it is up to the queue to actually provide this which it will since JBoss Messaging has been configured to work with Mysql.

Furthermore, note the WaitAction. This is a custom class which causes processing to pause for a configurable amount of time. This gives us time to kill the node during the pause in a controlled manner:

public class WaitAction extends AbstractActionLifecycle
{   
    private Long _timeSecs ;   
    private Logger _log;

    public WaitAction(ConfigTree tree) throws ConfigurationException {
        _timeSecs = Long.parseLong( tree.getRequiredAttribute("timeSecs") );
        _log = Logger.getLogger( this.getClass() );       
    }

    public Message process(Message message) throws InterruptedException {   
        _log.info("Going to sleep for " + _timeSecs +" secs."    );       
        Thread.sleep(_timeSecs.longValue()*1000);       
        _log.info("Awake, resume processing.");       
        return message;
    }
}

Don't forget to use the correct FQN for the class in jboss-esb.xml.

JBoss Messaging

We need to setup the required queues. Edit jbm-queue-service.xml als follows:

<?xml version="1.0" encoding="UTF-8"?>

<server>

      <mbean code="org.jboss.jms.server.destination.QueueService"

            name="jboss.esb.quickstart.destination:service=Queue,name=EsbQueue"

            xmbean-dd="xmdesc/Queue-xmbean.xml">

            <depends optional-attribute-name="ServerPeer">
                jboss.messaging:service=ServerPeer

            </depends>

            <depends>jboss.messaging:service=PostOffice</depends>            

            <attribute name="Clustered">true</attribute>

      </mbean>
      <mbean code="org.jboss.jms.server.destination.QueueService"

            name="mycluster.destination:service=Queue,name=OutBoundQueue"

            xmbean-dd="xmdesc/Queue-xmbean.xml">

            <depends optional-attribute-name="ServerPeer">
                 jboss.messaging:service=ServerPeer

            </depends>

            <depends>jboss.messaging:service=PostOffice</depends>           

            <attribute name="Clustered">true</attribute>

      </mbean>       

      <mbean code="org.jboss.jms.server.destination.QueueService"

            name="mycluster.destination:service=Queue,name=InBoundQueue"

            xmbean-dd="xmdesc/Queue-xmbean.xml">

            <depends optional-attribute-name="ServerPeer">
                 jboss.messaging:service=ServerPeer

            </depends>

            <depends>jboss.messaging:service=PostOffice</depends>

            <attribute name="Clustered">true</attribute>

      </mbean>

</server>

We mark the queues as clustered by setting the 'Clustered' attribute.

Starting the cluster and deployment

To start the cluster, first start one JBoss instance on one of the Virtualbox machines and wait for it to be fully up and running. This will be what JBoss calls the 'primary node'. You should be able to see some logging similar to this:

2011-10-21 04:17:55,252 INFO  [org.jboss.ha.framework.interfaces.HAPartition.DefaultPartition] (main) Initializing partition DefaultPartition

2011-10-21 04:17:55,402 INFO  [STDOUT] (JBoss System Threads(1)-3)

---------------------------------------------------------

GMS: address is 192.168.56.101:55200 (cluster=DefaultPartition)

---------------------------------------------------------

2011-10-21 04:17:55,712 INFO  [org.jboss.cache.jmx.PlatformMBeanServerRegistration] (main) JBossCache MBeans were successfully registered to the platform mbean server.

2011-10-21 04:17:55,864 INFO  [STDOUT] (main)

---------------------------------------------------------

GMS: address is 192.168.56.101:55200 (cluster=DefaultPartition-HAPartitionCache)

---------------------------------------------------------

2011-10-21 04:17:58,410 INFO  [org.jboss.ha.framework.interfaces.HAPartition.DefaultPartition] (JBoss System Threads(1)-3) Number of cluster members: 1

2011-10-21 04:17:58,410 INFO  [org.jboss.ha.framework.interfaces.HAPartition.DefaultPartition] (JBoss System Threads(1)-3) Other members: 0

2011-10-21 04:17:58,414 INFO  [org.jboss.cache.RPCManagerImpl] (main) Received new cluster view: [192.168.56.101:55200|0] [192.168.56.101:55200]

2011-10-21 04:17:58,416 INFO  [org.jboss.cache.RPCManagerImpl] (main) Cache local address is 192.168.56.101:55200

2011-10-21 04:17:58,429 INFO  [org.jboss.cache.RPCManagerImpl] (main) state was retrieved successfully (in 2.57 seconds)

Then start the second node. You can see in the logging that they will find each other and form the cluster:

2011-10-21 04:20:55,432 INFO  [org.jboss.messaging.core.impl.postoffice.GroupMember] (Incoming-13,192.168.56.101:55200) org.jboss.messaging.core.impl.postoffice.GroupMember$ControlMembershipListener@32af3289 got new view [192.168.56.101:55200|1] [192.168.56.101:55200, 192.168.56.102:55200], old view is [192.168.56.101:55200|0] [192.168.56.101:55200]

2011-10-21 04:20:55,433 INFO  [org.jboss.messaging.core.impl.postoffice.GroupMember] (Incoming-13,192.168.56.101:55200) I am (192.168.56.101:55200)

2011-10-21 04:20:55,434 INFO  [org.jboss.messaging.core.impl.postoffice.GroupMember] (Incoming-13,192.168.56.101:55200) New Members : 1 ([192.168.56.102:55200])

2011-10-21 04:20:55,434 INFO  [org.jboss.messaging.core.impl.postoffice.GroupMember] (Incoming-13,192.168.56.101:55200) All Members : 2 ([192.168.56.101:55200, 192.168.56.102:55200])

After you have built your .esb file, deploy it in the farm directory of the primary node under my_cluster\farm. Watch in the logs as the queues are deployed on the primary and then secondary nodes.

Running some tests

In order to test failover have your cluster fully running, then send some messages to the inbound queue using your favorite tool, like Hermes JMS. Make sure the body of the messages are unique, so that they are easily identified in the logging. Then take one node offline and watch the logging on the other node. You should see the message appear there. 

When all processing is done, you should see all messages safe and sound in the outgoing queue. I prefer to inspect the database in order to validate this:
  

[root@jboss ~]# mysql jboss -e 'select HEADERS from JBM_MSG'
+-------------------------------------------------------------------------------+
|HEADERS |                  
+-------------------------------------------------------------------------------+
 OutBoundQueue  |ID:JBM-f36d6d66-9c2c-413e-a879-0510af996601 H.CORRELATIONID uickstartId[1321627299207] H.DEST
 OutBoundQueue  |ID:JBM-9a448807-1911-475f-aa7f-661dfefa3164 H.CORRELATIONID uickstartId[1321627300266] H.DEST
 OutBoundQueue  |ID:JBM-f83a3c80-31a7-484e-ba9a-e15eef316cc1 H.CORRELATIONID uickstartId[1321627302355] H.DEST

Conclusion

Clustering with JBoss is actually not that hard to setup because it is built into the JBoss Application Server, but the complexity of the solution can make the task seem daunting. I hope you now have some insight in HA concepts and implementation with JBoss SOA Platform, as well as how to verify that you have a working solution.