One of the hardest parts of writing an UI is to keep track of updates and changes to whatever we're showing inside the UI. Ideally the UI is just a "wrapper" around some stateful data model, and changes to the model are automatically propagated in the correct way to the UI (the view). This is what is (mostly) done with DomUI data binding - but there are lots of things that still need manual labor. I would like to find ways to minimize that labor.

This document is meant as something to order my thoughts.


Changes inside a model

Simple stuff: property data binding

DomUI data binding works by connecting some property on a model to the value of a component. The component or model is updated depending on whether the value changed, which is usually done by Objects.equals() between both values - which in turn calls equals on either object. Once the comparison returns false the whole component is invalidated and redrawn.

Binding like this works OK except for the following cases:

All in all data binding works well if the objects obey either of the following:

DomUI properly implements most of this without problems.

Binding to lists

Binding to lists is harder, as stated before. Components that accept a list (either as a value property but also for other bindable properties, like the "data" property on ComboXxxxx combo boxes) have a few choices to handle them:

Instrumenting?

Not having proper list events is a real problem, sadly enough, and I see no other feasible solutions. It would be great if we could somehow instrument the list implementations so that they would start collecting change information- but that would mean having to add instrumentation in a separate module which should then change the byte code for the common collection classes we want to be able to check.

This, however, is too much like black magic to interest me.

Updates of database (ORM) data

When we edit data from an ORM we have a specific set of trouble:

These are the basic issues, but many more can be found that are related. What solutions can we find?

The elephant in the house: write models, damnit

There is actually a proper solution to most of these issues which is this: do not use the ORM data objects directly but always create a model which is a editable copy of that model. All operations and all business logic must then be defined in terms of this model; the only time the actual ORM objects are used are when retrieving data and when saving data.

This works all the time because we fully control and know what is being done, and it is easy (through careful coding) to know what data changed or was needed.

We however dislike this because making models is a lot of work and involves a lot of copying of concepts - especially for the most common (CRUD) screens. For these we want a solution that works quickly without extra typing or copying of code.

Any complex data manipulation should however always be done using handwritten models.

Let's start with issues and possible solutions.

Common ORM related issues

TODO Discuss:

Edit dialogs and cancel/rollback

Another class of update trouble has to do with edit dialogs and what happens when we cancel them 8-/. When we use data binding all changes made in the dialog will be propagated to the model. This is all fine and dandy when we finally decide to save those changes: this would persist them and the changed values is what we want to have.

But if we change data and decide to cancel we're in trouble: we do not want those changes to be persisted and all changes must be undone.

There are several basic solutions to handle this.

Separate edit screen with separate QDataContext (session)

One way (often used in DomUI 1.0 code) is to separate the edit data by opening a new screen (new UrlPage, new URL). This new page uses a new QDataContext and loads a copy of the data inside the screen. If the screen is cancelled then the QDataContext is rolled back (ensuring that no updates enter the database) and the code goes back to the previous page. This previous page reloads (by devious means like the onShelve event) itself from the (unchanged) database and so no data changes where on rollback.

This works like a charm as long as we have a solution for updating the other pages when save is pressed, and when it is acceptable to have full page updates.

FIXME More details on this method will be written in a separate document later which I will link to here once done.

Using SubPage and notifications

A SubPage (DomUI 2.0) is a page like structure that can be used on an UrlPage. An UrlPage can contain multiple SubPage objects and indeed SubPage objects themselves can be nested too.

A SubPage has its own QDataContext which is inherited by all nodes added to the SubPage, and so objects loaded by a subpage are copies inside that SubPage.

With this it is possible to create a single page interface which acts mostly as the 1.0 "Separate edit screen" solution:

As long as the data model is not too complex (CRUD++) this works technically very well: cancelling data is no more work than just destroying the SubPage; the underlying QDataContext will then rollback and all changes go to that big bitbucket in the sky.

In practice however this is very hard to work with: a single object tree (the page tree) contains objects loaded in a lot of different QDataContext instances and it is depressingly easy to make a mistake in matching object with the QDataContext it belongs to. A simple case is for instance adding a SubPage which has a Listener which is listened to by the parent of that SubPage. If an object is passed in that listener it will most probably be connected to the child subpage's QDataContext. Any attempt to do something (like saving) in the parent page will cause no end of exception fireworks. This is already bad, but tracking what went wrong is often hours of work because of the many QDataContext instances and the difficulty of knowing which object attaches to which QDataContext - if at all; problems with detached objects come from a special place in Hell reserved for the worst of people (like politicians).



Possible alternative solutions

Copying models

Instead of letting the database handle the problem we could try to do it ourselves - by making a copy of the data we want to edit. We would edit the copy inside the dialog or subscreen and only when we press some save option would we copy back the copy inside the real model- and save that (we would however still be in deep sheet if the save failed - this would destroy Hibernate's session and hence next saves become unreliable).

Copying is technically not too complex, but it quickly breaks when we consider some common cases.

Problems with object trees

If we are not just editing a single instance but also data branching from it (either parent relation objects or child list objects) then it becomes a challenge: which objects do we copy and which not? It is not easy to find this out automatically, and so the obvious solution would be to specify, at copy time, which branches from the root object(s) need to be copied. The idea is more or less to specify all properties and property paths which need to be "deep copied"; all data outside of that path would have shallow copies.

While this works it is a maintenance and debugging nightmare: any change to some code inside the screen which poops outside the copy will change data that it is not meant to change, and this leads to very hard to find bugs.

Problems with Business Logic

The models edited in screens use business logic (like controllers) to do complex changes. For instance a booking screen might move an amount by calling business logic to effect the actual move. Business logic should be a black box (well, considering the code found in them they might be brown boxes, but well) and we should not (need to) know what data objects are changed by that logic. But as we need to somehow specify which objects in the data tree need to be copied this does not work- we can only know what to copy if we know exactly how each logic part operates. And that again makes the code very fragile.

Copying through a "Subsession model"

The fundamental theorem of Software Engineering states: All problems in computer science can be solved by another level of indirection. So let's throw that on this problem.

A non exhaustive list of things we would like to have is:

One way that we can probably do this is by using the existing QDataContext abstraction. This is already used for all data access so it is a central point where all data obtained from the database comes from. It does not even need to be ORM data: we have QDataContext implementations that can use JDBC too, and it is not too hard to write other implementations.

Wrapping QDataContext

We could create a "SubSession" QDataContext which "wraps" another QDataContext. This is the first of the "indirection" layers we need. The subsession context would mostly act as a delegate for the wrapped QDataContext, but it would have some tricks up its sleeve.

All queries (or other methods that would return new data classes) will register the originals inside a "source map". Each object will then be copied and wrapped into a dynamic proxy. The dynamic proxy "extends" the original data class but intercepts all getters and setters (at the very least those for relation properties). As all objects are proxied it is easy to expose specific helper interfaces on those proxies so it becomes easier to control (and show) what data came from where, and where it belongs to. This means we can give way more meaningful error messages if objects are mixed up in data contexts.

The proxies intercept getters so that they can properly register the result of the get, i.e. if a getter returns a lazily loaded instance we can wrap that instance too and register it, so that we can save it when needed.





Stream of conscience - endless, endless delta's

An alternative to keeping before images is to actually change the processing model itself. Instead of collecting values into the objects we use the initial state of these objects as read-only data; from there on all changes made are not made to the objects but to a supermodel on top of them, and this supermodel collects the changes as a train of delta's. While a screen is edited the delta's are collected, and to show values the code rolls forward all delta's for a given field starting from the base value.

This has the advantage of being easily able to rollback to any state in the past, so it can be used to things like undo and rollback easily.