Overview
Verne business services introduce changes to the data of a specific entity on the register. The term "merge changes" refers to managing the situation whereby two concurrent change processes are in action simultaneously on the same baseline set of data. Once one of the change processes is applied to the baseline set of data, the second process needs to be refreshed with those changes to ensure that the changes that are being made are still appropriate.
In Verne concurrency and therefore merging happens on two levels:
| Business service instance | Concurrent business service instances are the ones that intend to maintain the same entity. Once one of the business services is applied to the entity, the second business service that is still in-progress needs to be refreshed with those changes to ensure that its content is up-to-date and appropriate. Example: Director John submitted an application to appoint a secretary for company ABC Limited. Secretary appointment is being reviewed by the Registrar. While review is underway, another director Mary started a new application for the same company ABC Limited to update her residential address, she had issues with payment and saved the application for later. Now both applications – secretary appointment and director address change – are considered concurrent, as they have not yet been applied or rejected and both relate to the same entity. When John’s application is approved, Mary’s application gets updated to incorporate John’s changes. |
| Service transaction | Concurrent service transactions are the ones that intend to maintain the same service instance. Once one of the service transactions is saved by the user (committed), the second service transaction that is still not committed needs to be refreshed with those changes to ensure that its content is up-to-date and appropriate. Example: Organisation PWC started a new entity registration and saved it for later. The two members of PWC did not coordinate their effort, so they picked it up and started editing this application simultaneously. Each one has in fact started a new service transaction and since they have not saved their changes yet editing those service transactions are competing to change the details of the same service instance. |
Merging (refreshing) is essential even if services do not have any overlaps in data that is being changed, as long as they attempt to maintain the same entity / service instance, they need to be merged as each of them holds the full snapshot of the entity data / service instance data (refer to versioning section to learn more about this concept).
For convenience content below described business service level merging, however it works in the same way for the service transaction level merging.
“First come – first served”
Example above only covers one possible scenario of concurrency, in fact any business service type, whether external or internal, in any in-progress status will be competing with another service on the same entity. There could be more than two concurrent services for the same entity at any point in time. Concurrent services are called so for a reason, as they are all competing between each other to change data on the same entity and can potentially have conflicting changes. So whichever one gets approved first will change the entity and can cause issues to all other remaining unfinished services on the same entity, making them invalid should the issues arise.
There are two main areas of issues:
Concurrent merge issues
When one of the concurrent business services is approved and applied to the Register, all other unfinished business services for the same entity will be refreshed to incorporate those changes. In Verne language this is called "merge", as system attempts to merge changed data into other business services. When Verne detects data conflicts, unfinished business service cannot be refreshed, so it becomes invalid. Invalid service cannot be progressed further to submission or approval, user will need to delete it and start again.
Example: Two different users with authority request to maintain the address details of the same secretary.
Note that conflicts would not occur very often and in majority of cases merge will happen successfully without user knowing about this.
Concurrent lock down issues
For some business service types it is incorrect from business perspective to have more than one in-progress service of the same type on the same entity at any given point in time or during the entire life-time of the entity. For example, entity re-registration business service can only be applied to the entity once in its life-time. Refer to Concurrency lock down page.
Example: Director John decided to re-register company ABC Limited, he started the application and then went to a meeting, so he saved it for later. He forgot about it for a few days. Director James in the meantime filled in the re-registration application for company ABC Limited and has sent it for review. From that moment John is not able to submit his application anymore.
Change Detection and Merging
Verne invokes change detection and merging upon business service activation, so called "push-out" principal. When a business service is activated Verne performs the following steps:
- Step 1: Verne first detects if there are any in-progress services that exist on the same entity that the service is activated for
- Step 2: If such services found , Verne performs change detection and merging procedure for each in-progress service separately – one at a time.
Example: Mary started a name change application for the Blue Sky Limited and saved it to the dashboard. Rob has just started an annual return application for the same entity and is filling in the details. The Registrar has corrected a typo in the postal address of the Blue Sky Limited using Correct the Register (CTR). In this case upon activation of the CTR service, Verne detects the two in-progress applications and does the merging procedure twice: (1) first time it compares changes made in CTR with Mary’s changes and updates her application so that typo is corrected there, (2) and the second time it compares CTR changes with changes made by Rob which results into his application containing corrected postal address too.
Verne uses Three Way Merge protocol to detect and compare changes for the purpose of merging:
In Verne terms:
- A is one of the concurrent business services that has been activated
- B is one of the concurrent business services that is still unfinished / in-progress
- C is the previous entity version that was used as a starting point by both concurrent business services (i.e. the data held by the business service activated before A)
- D is the next version of the in-progress business service B that was refreshed with the changes introduced by the activated business service A
Verne performs the three tests for each piece of data from the entity domain tree and depending on the outcome of those tests arrives at the decision for each piece of data:
- Nothing to merge
- Will merge
- Can’t merge
The following table summaries how Verne leverages the Three Way Merge concept to detect change to the data, and further, where any attempt to merge the changes will raise a conflict.
| Test 1 Compare values for the same piece of data in A an C. Are they different? | Test 2 Compare values for the same piece of data in B an C. Are they different? | Test 3 Compare values for the same piece of data in A an B. Are they different? | Outcome |
|---|---|---|---|
| Yes | Yes | Yes | Conflict! Can’t merge. Both applications requested to modify the “same data” and made conflicting changes OR one requested to modify and another to remove the “same data“. |
| Yes | Yes | No | Nothing to merge. Both applications requested to modify the “same data” and made identical changes OR “same data” is removed in both applications. |
| Yes | No | [N/A as will always be Yes] | Will merge. The “same data” that is changed in the approved application and is NOT changed in the in-progress one. |
| No | Yes | [N/A as will always be Yes] | Nothing to merge. There is a change in the in-progress service only. |
| No | No | [N/A as will always be Yes] | Nothing to merge. There is no change to this piece of data in both applications. |
Invalid Application
An in-progress application that has not been refreshed with activated changes is known as invalid and can never be activated (note, that in subsequent product releases it can be refreshed and activated if its owner requests so). Verne disables the options that would normally be used to progress the service towards activation and displays the messaging to inform users about the restrictions and available options.
Merging and Conflict Example
For example, registration business service 1 was approved and then the three concurrent services 2, 3 and 4 were started by different users for the same entity. All three service are "in-progress" and they are all based on the entity data version that was effective after registration (version 1):
Then business service 2 got applied, this made transaction 2 current and all changes introduced in transaction 2 were successfully merged in the two remaining in-progress applications 3 and 4. So now the remaining two in-progress applications are based on entity data version 2. They are both "alive" because they are based on the current version. Also transaction 5 was started and saved for later.
Then another business service 5 was started for the same entity and got applied, this made transaction 5 current and all changes introduced in transaction 5 were successfully merged in the in-progress application 3, however service 4 failed to merge as it had conflicting changes with service 5 – they both requested to change the name of the same director. As a result service 3 is "alive" because it is based on the current version, whereas service 4 is now "invalid" as it was not refreshed and is based on the outdated entity version 2.
Conflict Levels
Conflict (collision) happens when "same data" is changed in both approved and in-progress applications and the changes are different. "Same data" or "same piece of data" is a logical grouping of the attributes and records contained in the entity domain tree. They are always assessed as a group, e.g. it is enough for one attribute in the group to be changed for the whole group to be considered changed for the purpose of three way merge. Same data is a data parcel that can contain an individual attribute, or a group of attributes, or a record, or a group of records, or combination of attributes and records.
Level 1. Data Parcel Conflict
Individual attributes and/or records are logically grouped to form one parcel of data where data integrity must be enforced. When both approved and in-progress applications change at least one piece of data within the boundaries of the parcel, then in-progress application is in collision with the approved application, hence merging will NOT happen. Note, change does NOT have to be the same attribute or same record.
There are five parcel types:
| a. Single Attribute Parcel | Parcel consists of one attribute. Example: Entity email address attribute. When both approved and in-progress applications changed value for an entity email address attribute, then in-progress application will be in collision with approved and merge will NOT happen. Default: This is a default setting for any data element that belongs within the entity domain tree root. If a new attribute is added into the domain tree root by a developer and no conflict level is specified, then this will kick in. |
| b. Multi-Attribute Parcel | Parcel consists of multiple attributes. Example: When shareholder is a registered company, user is able to search the Register and pull through company details. Under the hood system records associated company id, company current version, company name and company business number. All of these attributes belong to one data parcel. |
| c. Single Record Parcel | Parcel consists of all attributes that belong to a record. One record is a one parcel, regardless of how many record there are in the parcel. Example: Beneficial owner details are captured for shareholders. Beneficial owner address is stored as a child record of the shareholder record, address record consists of multiple attributes Care of, Address Line 1, Address Line 2, Address Line 3, Country Code. Default: This as the default setting for any data element that belongs within any child record in the entity domain tree, in which case the whole record is a parcel. If a new record is added into the domain by a developer and no conflict level is specified, then this will kick in. |
| d. Records Array Parcel | Parcel consists of all records that belong to the same records array. Example: Current Registered Office Address and all historical Registered Office Addresses are in one data parcel because they represent history of changes of one address. More specific scenario could be that Approved Application 1 has changed current Registered Office Address from A to B (under the hood system updated End Date attribute on the record for the current address A, and created a new record for address B). While there is another in-progress Application 2 where current Registered Office Address was changed from A to C (under the hood system updated End Date attribute on the record for the current address A, and created a new record for address C). There is a direct record and attribute level conflict on record A for End Date field, however since records B and C are both new to the entity, system will not deem them to be conflicting with each other unless we bundle up all records in array as a parcel. This will prevent records B and C being merged into one entity data set. Note: Consider setting this as the default for – any records array that represents history of changes of a certain object, e.g. Entity Name History, Registered Office Address History. – any document uploads. Document records are never updated, even when a single document is replaced, the old document record is not updated, is is removed from the new version and replaced with a new document record. Each individual document upload is in fact an array. |
| e. Mixed Parcel | Parcel content is a combination of attributes and records. Example: “Have your own constitution” checkbox and “Constitution upload” and “Translation constitution upload” are logically grouped as one data parcel called “Constitution”. Checkbox in this case is a single attribute and uploads are the two document records. |
Level 2. Impact Attribute Conflict
Some key attributes can define conditional business logic that is applied to other attributes and records. Therefore when value of such attribute is modified, other impacted attributes can be "in trouble" if there is another in-progress application that requested changes to those impacted attributes. That another in-progress application cannot be refreshed and must be blocked. Such attribute is called "impact attribute" and the attributes they affect are member of an "impact group". If data for the impact attribute has changed, then all members in the impact group are in collision.
Example: Company Type value defines what entity fields are applicable and what aren’t. Company Type in this case is an Impact Attribute with Impact Group including all entity domain attributes and records. Approved Application 1: company type converted from private to close company. In-progress Application 2: change of director address is requested. Application 2 requested change to the Company Type Impact Group attribute, therefore Application 1 and Application 2 are in conflict and merge will NOT happen.
Example: Director Cease Date being empty or populated defines whether director is active or not. Director Cease Date in this case is an Impact Attribute with Impact Group including all details of this particular Director. Approved Application 1: Director A is ceased – Cease Date is populated. In-progress Application 2: Director A Name is modified. Application 2 requested change to the Director Cease Date Impact Group attribute, therefore Application 1 and Application 2 are in conflict and merge will NOT happen.
Data Conflict Definition
Consultants must complete data conflict definition for each project. This involves defining data parcels and conflict levels for all attributes and records that constitute entity domain tree of all entity types that the project covers.
The best place for this information to be collected is the data elements specification, for example:
