Merging under the hood

Merging is the process of including data changes from one service transaction into the data of one or more other service transactions.

Merge base, source and targets

Service transactions involved in merges share a common service transaction ancestor on which their data is both based – the merge base. The merge process is triggered by an action occurring on a service transaction – the merge source. When this happens, any other active service transactions that are based on the same merge base are merge targets.

• Merge base– the common ancestor service transaction
• Merge source– the service transaction being acted upon that instigated the merge process
• Merge targets– any active service transactions that have the same common ancestor

Three-way diffs

So for each merge we have three service transactions to consider:

1. the merge base
2. the merge source
3. the merge target

This is where the term three-way-diff comes from – the fact there are three sets of data to consider.

Merge process vs. merge attempts

For a single merge process, there are only ever one merge base and merge source, but there can be multiple merge targets. The number of individual merge attempts made during a merge process is the same as the number of merge targets. So if there are five merge targets, then there will be five merge attempts – one per merge target, using the same merge base and merge source in each instance.

Merging data changes

The data changes to merge are defined as the difference in data between the merge base and the merge source – the source changes. For each merge target, these data changes are compared with it’s own data changes – the difference in data between the merge base and itself, the merge target – the target changes. This means for each merge attempt there are two sets of data – the source changes and the target changes. If these two sets of changes don’t relate to the same data at all, then they can be happily merged together. If some of the changes involve the same domains but in non-conflicting ways, then they can still be merged without issue. In some cases however the changes cannot be applied together without risk of causing data integrity issues – in these cases it is deemed a merge conflict and the merge process is aborted as unsuccessful.

Merge triggers

As mentioned above, a merge process is triggered by an action on a service transaction that makes it a merge source. These actions that trigger merge processes are:

• Activation – when a maintain service transaction is activated, then a merge occurs to all other non-correction service transaction tips that are based on the same version as the maintain service being activated
• Save-commit – when a service transaction is save-committed, then a merge occurs to all active service transactions tips for that same service instance

Merging on activation

A versioned root domain (Entity) in Verne can only have a single current, active version. When an authorised user performs a maintenance service on the entity a service is created with a pointer to the version of the entity that it has started from. These maintenance services may be short lived or have an associated workflow that means that the service may not be applied directly to the register. Additionally it is possible that a service may be started and abandoned by the user.

While the maintenance service is in progress it is possible for another maintenance service for the same entity to be created. These services both start from the same version of the entity as there is only ever a single current version.

When the first of these services is completed it will become the new current version for the entity. The inflight services that started from the same version are no longer valid and need to be updated to use the current version the entity.

When the first maintenance service is applied the following occurs:

• All active service transactions relating to services that have started from the same version of the entity are found
• A three way diff is performed on the domain trees of the original version, newly current version and each pending service transaction
• A list of merge instructions are evaluated to ensure that the data from the new current version can be merged into the pending service
• If there is no conflicting data:
• The original pending service will be cloned and have its status set to archived with a reason code of ‘merged’. It will also have its view tree removed and have its nextSiblingId field updated to point to the clone.
• The clone becomes the active instance of the service. It’s prevVersionId field is set to the newly applied active version and the prevSiblingId field to its, now archived, originator. The view tree is removed as it may contain data no longer current. A merge audit field is added to the new service transaction indicating which version has caused this merge, when and that the merge was successful.
• The rule scope transaction-merge-complete is executed against the active instance of the service. This provides the application a chance to react to the newly created instance of the service.
• A system log message is recorded of the merge to ensure visibility.
• If a user is actively working on the now archived service transaction the next time they interact with the server they will be redirected to the merged service transaction and they will recieve a message indicating what has happened. They should be able to continue on with their transaction.
• If there is a conflict and the pending version can not be merged:
• A system log message is recorded of the conflict to ensure visibility.
• The pending transaction has a merge audit field added with a status of conflict.
• The rule scope service-conflict is executed against the conflicted transaction.
• If a user is actively working on the service their next interaction with the server will cause their transaction to become read only and they will be notified of the conflict.

sequenceDiagram
    participant ActivateVersion
    participant MergeEngine
    participant OriginalPendingTransaction
    participant NewPendingTransaction
    activate MergeEngine
    ActivateVersion->>MergeEngine: Merge any pending versions with same previous version
    loop For each pending version
      MergeEngine ->>OriginalPendingTransaction: Retrieve
      MergeEngine ->MergeEngine: Create 3 way diff
      loop For each merge instruction
        MergeEngine ->> MergeEngine: Check conflict
      end  
      alt Conflict detected
         activate OriginalPendingTransaction    
         MergeEngine ->> OriginalPendingTransaction: Set merge audit to 'conflicted'
         MergeEngine ->> OriginalPendingTransaction: Execute 'service-conflict' rule scope
         deactivate OriginalPendingTransaction
      else No conflict
         activate OriginalPendingTransaction  
         MergeEngine ->> OriginalPendingTransaction: Clone
         activate NewPendingTransaction
         OriginalPendingTransaction ->> NewPendingTransaction: Clone
         NewPendingTransaction  -->> MergeEngine: Return clone
         MergeEngine ->> OriginalPendingTransaction: Update next sibling field
         MergeEngine ->> OriginalPendingTransaction: Archive
         deactivate OriginalPendingTransaction
         MergeEngine ->> NewPendingTransaction: Update previous version id field and previous sibling id
         MergeEngine ->> NewPendingTransaction: Remove View Tree
         MergeEngine ->> NewPendingTransaction: Merge domain tree
         MergeEngine ->> NewPendingTransaction: Add merge audit information
         MergeEngine ->> NewPendingTransaction: Execute 'transaction-merge-complete' rule scope
         deactivate NewPendingTransaction
      end 
    end
    deactivate MergeEngine

Merging on save-commit

The merge process is the same as that defined in the merging on activation section above, except that the target service transactions in the first step are calculated differently. In this case they are all active service transactions that relate to the same service instance as the service transaction that was just save-committed.

Merge Instructions

Merge instructions are configured based on the entity’s data set. There are several types of instruction and where in the domain tree their target resides also plays a part in determining whether it constitutes a conflict. For a detailed configuration guide see merge instructions.

• Root domain attribute
• Impact attribute
• Attribute conflict
• Attribute parcel
• Record array parcel
• Mixed parcel

Root domain attribute

This type of conflict is implicit and does not require any additional configuration. If the same attribute on the root domain has changed in both the newly activated version and the pending version there is a conflict.

Impact attribute

Defines attributes that if changed in the new activated version will always place any pending transactions in conflict. For example a companies register may have a StruckOffDate on the company domain. If the company is struck off any pending services would become invalid.

Attribute conflict

This type of conflict is implicit and does not require any additional configuration. If the same attribute on the same domain has changed in both the newly activated version and the pending version there is a conflict.

Attribute parcel

Used to identify combinations of attributes on the same domain that should be treated as an atomic unit. For example the first, middle and last names of person may be defined as a unit. If the newly acttivated version has for example changed the last name of the person and a pending version has changed the first name there is a conflict.

Record array parcel

A record array parcel is used to define conflict scenarios where adding additional items into an array may cause conflict. For example the registered office address of a company may be configured as an record array parcel to keep track of changes to the office address with start dates recorded for each address. If the newly activated version has added an address any pending versions that have added records into the same array would be conflicted.

Mixed parcel

A mixed parcel defines potential conflict between attributes on a parent domain and children of that domain. A common example of a mixed parcel may be a flag on a parent domain indicating if the company has a constitution and a child domain for the consitution document upload.