Home

• explain
• command-line
• schemas
  • b: ms: base
  • c: msc: concept
• contact

Message parts

• Contracts
• Services
• Transport
• Payload
• Maintenance

Data maintenance

Computer systems traditionally take little care on removing data. Collecting is challenging enough, let alone removing aged data-structures. This has to change: interfaces must contain ways to correct or cancel previous deliveries.

Reasons to implement removal are plenty-fold:

• Privacy law (like GDPR in the EU), copyright law (take downs) and other regulations require removal procedures.
• When the total data managed is too large to store indefinitely, the processors of that data need to be informed the resource is disappearing.
• Data which is accidentally incomplete, over-complete or incorrectly delivered MUST get corrected in the whole processing chain.
• Changes in licensing and regulations must progress through the chain of resulting products.
• Expiration of a license MAY require withdrawal of data provided via the contract.

An obligatory part of any contract will require clients which have stored or processed data collected from a service to regularly poll the service to get updates and removals. When the contract ends, all related data SHALL be removed (unless explicitly agreed on).

Referencing data items

Processing removals and changes is quite hard, but crucial. The generalization is based on the unique identifier which each exchanged data item carries. Protocols come close to features of database log replay.

The service can instruct the clients to remove or update the item based on

• explicitly listed identifiers
• pattern matched identifiers
• collected before/since a certain moment of time
• created before/since a certain moment of time
• originating from a specific data source

Processing changed and removed items

When you are caching data elements (units), you have to monitor their version. You MUST upgrade your data to a newer version when the server informs you about it, or remove it from the cache.

A service MAY decide to report changes in a few steps: for instance when it can be expected itself or its clients require considerable resources to process the facts.