From ee46216093a743db0c051fa210fdfc9bbb6192d9 Mon Sep 17 00:00:00 2001 From: William Brown <william@blackhats.net.au> Date: Sat, 9 Nov 2024 16:18:10 +1000 Subject: [PATCH] Brain Dumps --- book/src/SUMMARY.md | 3 + .../developers/designs/ephemeral_entries.md | 24 ++++ book/src/developers/designs/schema.md | 47 +++++++ book/src/developers/designs/subentries.md | 131 ++++++++++++++++++ 4 files changed, 205 insertions(+) create mode 100644 book/src/developers/designs/ephemeral_entries.md create mode 100644 book/src/developers/designs/schema.md create mode 100644 book/src/developers/designs/subentries.md diff --git a/book/src/SUMMARY.md b/book/src/SUMMARY.md index 8fe5299dc..f66b69dea 100644 --- a/book/src/SUMMARY.md +++ b/book/src/SUMMARY.md @@ -84,8 +84,11 @@ - [Cryptography Key Domains (2024)](developers/designs/cryptography_key_domains.md) - [Domain Join - Machine Accounts](developers/designs/domain_join_machine_accounts.md) - [Elevated Priv Mode](developers/designs/elevated_priv_mode.md) + - [Ephemeral Entries](developers/designs/ephemeral_entries.md) - [OAuth2 Device Flow](developers/designs/oauth2_device_flow.md) - [OAuth2 Refresh Tokens](developers/designs/oauth2_refresh_tokens.md) + - [SubEntries (2024)](developers/designs/subentries.md) + - [Schema (2024)](developers/designs/schema.md) - [Replication Coordinator](developers/designs/replication_coordinator.md) - [Replication Design and Notes](developers/designs/replication_design_and_notes.md) - [REST Interface](developers/designs/rest_interface.md) diff --git a/book/src/developers/designs/ephemeral_entries.md b/book/src/developers/designs/ephemeral_entries.md new file mode 100644 index 000000000..be2d5c736 --- /dev/null +++ b/book/src/developers/designs/ephemeral_entries.md @@ -0,0 +1,24 @@ +# Ephemeral Entries + +We have a number of data types and entries that may need to be automatically deleted +after some time window has past. This could be an event notification, a group for a +temporary group membership, a session token, or more. + +To achieve this we need a way to mark entries as ephemeral. After a set time has past +the entry will be automatically deleted. + +## Class + +A new class `EphemeralObject` will be added. It will have a must attribute of `removedAt` +which will contain a time at which the entry will be deleted. + +## Automatic Deletion + +A new interval task similar to the recycle/tombstone tasks will be added that checks for +and deletes ephemeral objects once removedAt has past. + +## Ordering Index + +To make this effecient we should consider addition of an "ordering" index on the `removedAt` +attribute to improve searching for these. Initially this won't be needed as there will be +very few of these, but it should be added in future. diff --git a/book/src/developers/designs/schema.md b/book/src/developers/designs/schema.md new file mode 100644 index 000000000..17423baa6 --- /dev/null +++ b/book/src/developers/designs/schema.md @@ -0,0 +1,47 @@ +# Schema Changes 2024 / 2025 + +Our current schema structure has served us very well, remaining almost unchanged since nearl 2018. + +The current design is a heavily adapted LDAP/AD style structure with classes that define a set +of may and must attributes, and attributes that define properties like single value, multivalue, +the types of indexes to apply, and the syntax of the attribute. + +However, after 6 years we are starting to finally run into some limits. + +## Proposed Changes + +### Removal of Multivalue + +We currently have many types that have to be multivalue capable out of syntax compliance but are never +actually made to be multivalue types. This creates overheads in the server but also in how we code +the valuesets themself. + +The multivalue type should be removed. The syntax should imply if the type is single or multivalue. +For example, bool is always single value. utf8 is single value. utf8strings is multivalue. + +This allows consistent handling with SCIM which has separate handling of multi/single value types. + +### Indexing + +Currently we have a number of indexing flags like equality, substring, presence. In the future we +would like to add ordering. However, these don't make sense on all types. How do you "order" certificates? +How do you "substring" an integer? How do you perform equality on two passkeys? + +To resolve this schema should indicate a boolean for "indexed" or not based on if the value will be +queried. The syntax will then imply the class of indexes that will be emitted for the type. + +### Migration Behaviour + +Certain attributes for internal server migrations need to have their content asserted, merged, or +ignored. This behaviour should be flagged in the schema to make it more consistent and visible how +these types will be affected during a migration, and to prevent human error. + +### SubAttributes and SubAttribute Syntax + +SCIM allows complex structure types to exist. We could consider a schema syntax to allow generic +structures of these based on a set of limited and restricted SubAttributes. For example we might +have a SubAttribute of "Mail" and it allows two SubAttributeValues of "value": email, and "primary": bool. + +We would need more thought here about this, and it's likely it's own whole separate topic including +how to handle it with access controls. + diff --git a/book/src/developers/designs/subentries.md b/book/src/developers/designs/subentries.md new file mode 100644 index 000000000..00a2c2fd8 --- /dev/null +++ b/book/src/developers/designs/subentries.md @@ -0,0 +1,131 @@ +# Sub-Entries + +As Kanidm has grown we have encountered issues with growing complexity of values and valuesets. These +can be hard to create and add, they touch a lot of the codebase, and they add complexity to new +features or changes. + +These complex valueset types (such as authsession, oauth2session, application passwords) arose out +of a need to have data associated to an account, but that data required structure and nesting +of certain components. + +Rather than continue to add more complex and unwieldy valuesets, we need a way to create entries +that refer to others. + +## Existing Referential Code + +The existing referential integrity code is designed to ensure that values from one entry are removed +cleanly if the referenced entry is deleted. As an example, a group with a member "ellie" should have +the reference deleted when the entry "ellie" is deleted. + +If the group were deleted, this has no impact on ellie, since the reference is defining a weak +relationship - the user is a member of a group. + +## What Is Required + +What we need in a new reference type are the following properties. + +* A sub-entry references an owning entry +* A sub-entry is deleted when the owning entry is deleted (aka recycled) +* Sub-entries can not exist without a related owning entry +* Deletion of the sub-entry does not delete the entry +* When an entry is searched, specific types of sub-entries can be fetched at the same time +* The owning entry can imply access controls to related sub-entries +* Conditional creation of sub-entries and adherence to certain rules (such as, "identity X can create sub-entry Y only if the owning entry is itself/X") +* Subentries may have a minimal / flattened representation that can inline to the owning entry via a phantomAttribute + +Properties we can not maintain are + +* An entry has a `must` relationship for a sub-entry to exist +* SubEntries may not have SubEntries + +## Example SubEntry + +Auth Sessions, OAuth2 Sessions, ApiTokens, Application Passwords, are examples of candidates to become SubEntries. + +``` +class: person +name: ellie +uuid: A + +class: subentry +class: authsession +SubEntryOf: A +sessionStartTime: ... +sessionEntTime: ... +sessionId: ... +``` + +Good candidates are structured data that are logically indendent from the owning entry and may not +always need presentation with the owning entry. Displaying a person does not always require it's +subentries to be displayed. + +## Non-Examples + +Some attributes should not become subentries, generally things with minimal or small structures +that benefit from being present on the owning entry for human consumption. + +* Mail +* Address +* Certificates +* Passkeys + +## AccessControls + +Access Controls need to be able to express a relationship between an owner and the subEntry. For +example we want rules that can express: + +* Identity X can create an AuthSession where the AuthSession must reference Identity X +* `idm_admins` can delete/modify ApiTokens where the owning entries are persons and not members of `idm_high_priv` + +We need to extend the `filter` type to support a `SubEntryOfSelf`. This +is similar to the `SelfUUID` type, but rather than expanding to `Uuid(...)` it would expand to +`SubEntryOf(...)`. As `create` access controls define that the resultant entry *must* match +the target filter, this achieves the goal. + +We also need a new ACP Target Type. This new target type needs two filters - one +to express the relationship to the SubEntry, and the other to the relationship of the SubEntryOwner. This +would constitute two filters + +``` +SubEntryTarget: class eq apitokens +EntryTarget: person and not memberOf idm_high_priv +``` + +Both conditions must be met for the access control to apply. In the case of a `create`, the SubEntryTarget +is used for assertion of the SubEntry adherence to the filter. SubEntryTarget implies "class eq SubEntry". EntryTarget +implies `and not class eq SubEntry`. + +## Search / Access + +How to handle where we need to check the entryTarget if we don't have the entry? Do SubEntries need +to auto-dereference and link to their owning entry for filter application? + +If we deref, we need to be careful to avoid ref-count loops, since we would need to embed Arc or Weak +references into the results. + + +Alternately, is this where we need pre-extraction of access controls? + +Could SubEntries only be accessed via their Parent Entry via embedding? + + + +## Deletion + +During a deletion, all deleted entries will also imply the deletion of their SubEntries. These SubEntries +will be marked with a flag to distinguish them as an indirect delete. + +## Reviving + +During a revive, a revived entry implies the revival of it's SubEntries that are marked as indirect +deleted. + +## Replication / Consistency + +If a SubEntry is created with out an owner, or becomes a orphaned due to a replication conflict of +it's owning entry, the SubEntries are deleted. + + + + +