Service components in this zone provide secured access to the Data at Rest across multiple channels which serves as a basis for an omni-channel user experience.
Data presentation components would typically include:
- Native Mobile applications
- Desktop client applications
The role of the Data Presentation components is to display the records held in the Data at Rest components.
If the user experience requires business logic to be executed, then the functionality will be delegated to a Data in Action micro-service.
The ideas discussed below replace our traditional approach to developing user experience and eliminate most of the code associated with the standard Create, Read, Update and Delete (CRUD) processing. For our example Insurance Policy with 100k+ lines of code, this could eliminate 30-40k lines of user experience code and another 30-40k lines of test code. This should cut our development costs by a further 30-40%. Again, we will improve the quality of the solution we are producing – smaller and simpler solutions are inherently faster and more stable! We can also significantly reduce the time to market for the solution.
Dynamic Page Generation
A consequence of storing business information in a data agnostic Data at Rest service is running our CRUD+ model on XML. This means that we don’t have to tie the user experience to a table/attribute-based data model. The obvious next question is, do we still need to hard code the user experience forms against a fixed data model, or can we leverage the XML representation to create user experience on demand?
Traditional Development Approach
When you think about our traditional application development approach, the fact that our user experience overlays a table/attribute data model constrains our implementation choices. Our database CRUD functionality extends beyond the database and into our presentation layer. We will have forms that let us search/filter, display and edit each of our concepts. Each of these forms builds on an associated set of business objects to abstract the database. If our data model encompasses 20-30 concepts, we are likely to have 40-60 forms in our user experience – most of the code behind these forms will simply be rendering the underlying data and then writing back updates as the user makes changes. In many applications, the actual business logic can also end up in the form’s code, intermingled with the presentation. This increases the complexity of the solution and increases dependencies between the data, its representation, and its usage.
User Experience On Demand
Our Data at Rest service is serving XML, so we are no longer bound to a fixed table/attribute data model. We can instead think about user experience in terms of the forms we wish to present. We can create a set of Form Definitions which determine how the XML is represented to the user. Our user experience then becomes a simple matter of matching the XML data with the appropriate Form Definition and rendering a user experience. We can do this by associating the ConceptGUID from the instance metadata with the Form Definition used to represent it - the two become synonymous.
We can now take maximum advantage of the CRUD+ functionality exposed by our Data at Rest service to provide additional functionality in our user experience. Since our Form Definitions will also be stored in our Data at Rest service they can be exposed to provide the same user experience across client, web and native mobile applications – all at the same time. This gets around a major versioning issue with release solutions that target multiple platforms and present an omni-channel user experience.
Data Presentation Model
We will use a simple Data Presentation Model to create a user experience that can be dynamically rendered across multiple platforms with minimal code and maximum flexibility. This approach is based on many years experience building customised user experience across many project types and platforms. We have abstracted a lot of the common features that demand the creation of custom code and converted them into a data representation of how the user experience should render any XML Instance record from the Data at Rest zone.
Let’s start by addressing the definition of functionality.
For the Presentation Data Model, we will define functionality as the provision of a user experience to a person that will let them move data securely between screen and database.
Our functionality needs to track the Roles associated with the person and be able to render the user experience accordingly.
Our functionality needs to render one or more Forms with which the person will interact. TRELLIS has a standardised set of form templates to simplify the creation process based on the type of form. Most forms will be a concept of some type - a simple concept for forms which describe a single class of objects, or a subform concept for concepts that cover a set of subclasses. We have colour coded our diagrams to use blue for concepts and green for concepts that need to be sub-classed. We have sub-classed the Form Definition concept itself because we will need to be able to provide forms for different purposes: Applications, Portals, Data Entry, etc.
At its most basic level, a Form Definition comprises a set of Tab Definitions and Field Definitions.
A Tab Definition lets us separate out the fields presented to the person into logical Tabs.
A Field Definition defines what data will be displayed or captured by the Form.
Our Data Presentation service now allows us to render a basic user experience by taking an XML instance record from the Data at Rest service and combining it with a Form Definition XML. For data entry forms (which typically account for 90+% of a business solution) we simply associate the Form Definition GUID with the Concept GUID of the XML instance record. For other form types, we need a more complex mapping mechanism that we will discuss later.
When we examine our instance record, we can map the XML elements to our Field Definitions. This allows us to determine where on the user experience we wish to display or collect data. The type of data, display format, and validation is determined by the Field Definition. Our Field Definition can also be extended to provide online help, tooltips and other descriptive information to help the person correctly collect or understand the data.
We can enrich the basic user experience by considering the security model associated with the instance XML record. We can verify that the person’s user account has been given access to the record, and prevent display of any data that the user doesn’t have access to. Here we are enforcing the Instance Access table rules established by the Data at Rest service. At the Form Definition level, we also determine who can save changes to the instance record based on user role and instance status.
Our Tab Definitions enable us to selectively hide tabs of data from the user experience based on the instance status and roles assigned to the person’s user account. This provides a coarse-grained approach to providing different functionality - for example, on a customer order, the Fulfillment tab may not be visible until the order has been approved.
Within our Field Definition, we could also specify fine-grained permissions that prevent certain fields of the record from being displayed to the person, based on the state of the instance record or the roles assigned to the user. We can extend this selective Field Permission to limit which fields the person can edit, modify or interact with.
Our Field Definitions are in no way limited to just collecting or reading data; they can also invoke services in the Data in Action zone against data that already resides in the Data at Rest zone.
State Transition Model
Already our Data Presentation Model is providing a complex baseline functionality that can render adaptive user experience based on the roles and permissions assigned to the person’s user account and the state of the data being presented. Now let’s extend that to include a State Transition Model for the instance data being displayed by the form.
For each Form Definition being used to render a Concept, we can identify a set of Allowed States for the Concept. We can extend the definition of the Instances table to include an extra field called Instance Status which will represent the current state of an Instance and be restricted to the Allowed States listed by the Form Definition.
Our Form Definition can also maintain a set of Events that can be triggered against any instance of the Concept. We can now combine our Allowed State and Event sets to create a set of State Transitions that can determine how any instance of this concept can move through its lifecycle. For example, an Order may go through the following states: Draft, Waiting for Approval, Fulfillment, Completed.
A typical State Transition will define the Initial and Final States, the Event that triggers the transitions, as well as gateway tests to confirm that the Event fits the instance and user role. It can also trigger a service in the Data in Action zone either before of after the state change.
In this section we have been colour coding our diagram boxes to identify Concepts that can be sub-classed (green boxes) and those which aren’t (blue boxes).
The idea of being able to sub-class a Concept using a Subform Definition provides a very powerful extension to our user experience.
The essence of a Subform Definition is the ability to add extra Field Definitions to a form (and hence the underlying instance XML) depending on a subtype of the concept.
For example, if the Concept is an Insurance Policy it will have a general user experience with an associated state transition model that will be shared by all Insurance Policies. However, the specific information required for a Life Insurance Policy will require a few extra fields, whilst a Vehicle Insurance Policy will need a few extra different fields.
Using the Form and Subform Definition model allows us to extract the common user experience to the Form Definition and abstract the differences into as many distinct Subform Definitions as we need. It also makes is simple to add more Subform Definitions over time as the needs of the business change.
Documents and Extracts
Once we have collected data from our users, we will need to be able to manipulate it to achieve business objectives. Two common requirements are the ability to produce formatted documents and extract data for other systems (e.g. Excel Reports)
We can extend our Data Presentation Model to enable the creation of formatted documents by combining the underlying instance XML with an XSLT file. Again, we can restrict the ability to create documents by the user roles and instant state.
We can also add the ability to pull pre-defined record sets from Data at Rest services based on the currently displayed instance record, and again restrict the ability to create documents by the user roles and instance state.
Code Generation for CQRS
In the Data at Rest section we discussed optimisation of the database for search and general query. We can now use the Form and Field Definitions to provide baseline data to a code generation process, and create the underlying database elements required to optimise the user experience.
At the Form Definition level, we can specify whether we should create read only lookup tables for any specific Concept. If we decide to create a read-only lookup table, we can specify whether to include all elements or a subset of elements based on Field Definitions in the lookup table.
We can then use a code generation service to create any required read-only lookup tables and link them with appropriate population trigger T-SQL to maintain their contents.
Use of code generation based on simple templates and the Form/Field Definitions allows us to create highly optimised query/search functionality without handwriting or testing lots of code. We simply craft and test the templates in detail and then we can re-use them whenever we create or modify functionality. The important point here is NOT to hand-modify any code that was generated by code generation. As soon as you start making manual modifications to generated objects you lose the ability to quickly regenerate them as the Data Presentation Model changes.
Localization and Translation
Increasingly, we need to be able to provide our user experiences in multiple languages. In some cases, we may need to provide translations based on the person’s language (e.g. English, French, Spanish) but we may also need to provide “translation” based on type of person who is accessing the application (e.g. medical terms for Doctors and Nurses, common body part terms for Patients). We can include these translations within the Form, Tab, Field Definitions and pull out the required textual form as the user experience is rendered.
Not all platforms have the same capability when it comes to creating a user experience. One dimension of our Form and Field Definitions will provide a mechanism for selecting an appropriate representation for each possible platform. In some cases, it may be a reduced capability; in others it may be to suppress the representation completely.
User Role and Domain Driven Design (DDD)
Consider again our Insurance Policy example: we might want to provide a micro-service that permits “Policy Rating.” Our standard DDD approach is to define a Policy concept strictly in terms of the requirements for Policy Rating. Our business may need to use Policy data in other micro-services such as Claims Management, Renewal, Accounts, Risk Assessment, etc. In each of these cases, the definition of Policy would be slightly different, and each micro-service would store its own representation of Policy. Each micro-service would have to implement its own security model to protect its internal data and synchronise its Policy store with the other micro-services. There are several approaches to data synchronisation, but they can negatively impact performance, and increase solution complexity and fragility.
A simpler solution is to adopt the DAS approach where the Policy records would be converted to XML and stored in the Data at Rest service instead of the Policy Rating service. Each micro-service could add the XML fields it needs to the Policy record, and they could also share the “common” fields that would be required by most (if not all) services – for example: Policy Number, Effective Start and End Dates, Policy Type, etc.
The advantages of storing the Policy data once in a centralised System of Record include:
- There is only one copy of the Policy data with CRUD+ functionality
- The latest version of the Policy data is always available to all micro-services in real-time
- Policy data does not have to be copied and cached across multiple micro-services creating consistency issues and service dependencies
- Policy data can be secured with a single authorisation/authentication model
- We can still use DDD, the micro-service just maps the XML it retrieves from the DAS into its own representation of a Policy and leaves the fields it doesn’t know/care about unchanged
When constructing the Form Definition, we need to be able to consider the role of the person who will be using it. Let’s consider our example Insurance Policy - if the person is a broker rating the Policy, then they will need to see a different representation of the Policy than a person in the Claims department processing a claim.
In a DDD solution we might have separate Domains and ultimately services for Rating and Claims. The brokers will be using the Rating service and the claims adjusters will be using the Claims service. The two different groups of people will have two different roles, but will be accessing the same set of common Policy records. To provide this functionality, we just assign each user to one or more roles, and then set up our Form Definition to provide an appropriate representation of a Policy to each user role.
Versioning and Environments
We always recommend maintaining separate environments for development, testing, quality assurance, staging and production. A key take-away for the Data Presentation Model is how we version it and migrate it across environments.
Since the Data Presentation Model is constructed within a single XML hierarchy, we can extract it with a single command and place the entire definition of the functionality within a single XML file. We can now simply version that XML file in any manner we chose. We could use a version numbering scheme, date-based formatting – whatever works for your business.
Loading a specified version of functionality from an XML file will automatically replace the existing definition within that Data at Rest service. When we load a new version of functionality into a Data at Rest service we should also re-generate the CQRS objects that support it.
Migrating functionality across environments then simply becomes:
- Extract the required functionality from Development
- Version it
- Add it to a repository or component library
- Extract the required functionality version from the repository
- Load it into the selected environment
- Re-generate the CQRS objects
- Run any required testing
When rendering an instance, we should also be considering its state. In our Insurance Policy example, a Policy that is still being rated shouldn’t have the ability to be claimed against.
This means we need to extend the Form Definition to take into account the Instance State. When we load the instance XML, we can use its state to determine its representation. As the state of the instance changes over time, so does its representation.
Having a single Form Definition rendering platform means that we no longer need to create many custom-built forms for all possible combinations of platform, user role, and instance state.
This means that it is easy to change Data Presentation. We sometimes do this in real-time during a requirement gathering meeting - this ensures that the users are getting exactly the presentation they need to do their jobs effectively.
Having a single Data Presentation service means that we are also deploying fewer components, with a shared overall user experience. This makes it easier to move employees between roles – they are essentially using the same application, in the same way, with the same data – just seeing different representations and functionality based on their new role.
It also makes it easier to onboard new resources. There is less software to learn, and most of the user experience changes they will need to make can be done in data, within impacting any code.