Brain Dumps

book/src/SUMMARY.md

@@ -84,8 +84,11 @@
   - [Cryptography Key Domains (2024)](developers/designs/cryptography_key_domains.md)
   - [Domain Join - Machine Accounts](developers/designs/domain_join_machine_accounts.md)
   - [Elevated Priv Mode](developers/designs/elevated_priv_mode.md)
+  - [Ephemeral Entries](developers/designs/ephemeral_entries.md)
   - [OAuth2 Device Flow](developers/designs/oauth2_device_flow.md)
   - [OAuth2 Refresh Tokens](developers/designs/oauth2_refresh_tokens.md)
+  - [SubEntries (2024)](developers/designs/subentries.md)
+  - [Schema (2024)](developers/designs/schema.md)
   - [Replication Coordinator](developers/designs/replication_coordinator.md)
   - [Replication Design and Notes](developers/designs/replication_design_and_notes.md)
   - [REST Interface](developers/designs/rest_interface.md)

book/src/developers/designs/ephemeral_entries.md

@@ -0,0 +1,24 @@
+# Ephemeral Entries
+
+We have a number of data types and entries that may need to be automatically deleted
+after some time window has past. This could be an event notification, a group for a
+temporary group membership, a session token, or more.
+
+To achieve this we need a way to mark entries as ephemeral. After a set time has past
+the entry will be automatically deleted.
+
+## Class
+
+A new class `EphemeralObject` will be added. It will have a must attribute of `removedAt`
+which will contain a time at which the entry will be deleted.
+
+## Automatic Deletion
+
+A new interval task similar to the recycle/tombstone tasks will be added that checks for
+and deletes ephemeral objects once removedAt has past.
+
+## Ordering Index
+
+To make this effecient we should consider addition of an "ordering" index on the `removedAt`
+attribute to improve searching for these. Initially this won't be needed as there will be
+very few of these, but it should be added in future.

book/src/developers/designs/schema.md

@@ -0,0 +1,47 @@
+# Schema Changes 2024 / 2025
+
+Our current schema structure has served us very well, remaining almost unchanged since nearl 2018.
+
+The current design is a heavily adapted LDAP/AD style structure with classes that define a set
+of may and must attributes, and attributes that define properties like single value, multivalue,
+the types of indexes to apply, and the syntax of the attribute.
+
+However, after 6 years we are starting to finally run into some limits.
+
+## Proposed Changes
+
+### Removal of Multivalue
+
+We currently have many types that have to be multivalue capable out of syntax compliance but are never
+actually made to be multivalue types. This creates overheads in the server but also in how we code
+the valuesets themself.
+
+The multivalue type should be removed. The syntax should imply if the type is single or multivalue.
+For example, bool is always single value. utf8 is single value. utf8strings is multivalue.
+
+This allows consistent handling with SCIM which has separate handling of multi/single value types.
+
+### Indexing
+
+Currently we have a number of indexing flags like equality, substring, presence. In the future we
+would like to add ordering. However, these don't make sense on all types. How do you "order" certificates?
+How do you "substring" an integer? How do you perform equality on two passkeys?
+
+To resolve this schema should indicate a boolean for "indexed" or not based on if the value will be
+queried. The syntax will then imply the class of indexes that will be emitted for the type.
+
+### Migration Behaviour
+
+Certain attributes for internal server migrations need to have their content asserted, merged, or
+ignored. This behaviour should be flagged in the schema to make it more consistent and visible how
+these types will be affected during a migration, and to prevent human error.
+
+### SubAttributes and SubAttribute Syntax
+
+SCIM allows complex structure types to exist. We could consider a schema syntax to allow generic
+structures of these based on a set of limited and restricted SubAttributes. For example we might
+have a SubAttribute of "Mail" and it allows two SubAttributeValues of "value": email, and "primary": bool.
+
+We would need more thought here about this, and it's likely it's own whole separate topic including
+how to handle it with access controls.
+

book/src/developers/designs/subentries.md

@@ -0,0 +1,131 @@
+# Sub-Entries
+
+As Kanidm has grown we have encountered issues with growing complexity of values and valuesets. These
+can be hard to create and add, they touch a lot of the codebase, and they add complexity to new
+features or changes.
+
+These complex valueset types (such as authsession, oauth2session, application passwords) arose out
+of a need to have data associated to an account, but that data required structure and nesting
+of certain components.
+
+Rather than continue to add more complex and unwieldy valuesets, we need a way to create entries
+that refer to others.
+
+## Existing Referential Code
+
+The existing referential integrity code is designed to ensure that values from one entry are removed
+cleanly if the referenced entry is deleted. As an example, a group with a member "ellie" should have
+the reference deleted when the entry "ellie" is deleted.
+
+If the group were deleted, this has no impact on ellie, since the reference is defining a weak
+relationship - the user is a member of a group.
+
+## What Is Required
+
+What we need in a new reference type are the following properties.
+
+* A sub-entry references an owning entry
+* A sub-entry is deleted when the owning entry is deleted (aka recycled)
+* Sub-entries can not exist without a related owning entry
+* Deletion of the sub-entry does not delete the entry
+* When an entry is searched, specific types of sub-entries can be fetched at the same time
+* The owning entry can imply access controls to related sub-entries
+* Conditional creation of sub-entries and adherence to certain rules (such as, "identity X can create sub-entry Y only if the owning entry is itself/X")
+* Subentries may have a minimal / flattened representation that can inline to the owning entry via a phantomAttribute
+
+Properties we can not maintain are
+
+* An entry has a `must` relationship for a sub-entry to exist
+* SubEntries may not have SubEntries
+
+## Example SubEntry
+
+Auth Sessions, OAuth2 Sessions, ApiTokens, Application Passwords, are examples of candidates to become SubEntries.
+
+```
+class: person
+name: ellie
+uuid: A
+
+class: subentry
+class: authsession
+SubEntryOf: A
+sessionStartTime: ...
+sessionEntTime: ...
+sessionId: ...
+```
+
+Good candidates are structured data that are logically indendent from the owning entry and may not
+always need presentation with the owning entry. Displaying a person does not always require it's
+subentries to be displayed.
+
+## Non-Examples
+
+Some attributes should not become subentries, generally things with minimal or small structures
+that benefit from being present on the owning entry for human consumption.
+
+* Mail
+* Address
+* Certificates
+* Passkeys
+
+## AccessControls
+
+Access Controls need to be able to express a relationship between an owner and the subEntry. For
+example we want rules that can express:
+
+* Identity X can create an AuthSession where the AuthSession must reference Identity X
+* `idm_admins` can delete/modify ApiTokens where the owning entries are persons and not members of `idm_high_priv`
+
+We need to extend the `filter` type to support a `SubEntryOfSelf`. This
+is similar to the `SelfUUID` type, but rather than expanding to `Uuid(...)` it would expand to
+`SubEntryOf(...)`. As `create` access controls define that the resultant entry *must* match
+the target filter, this achieves the goal.
+
+We also need a new ACP Target Type. This new target type needs two filters - one
+to express the relationship to the SubEntry, and the other to the relationship of the SubEntryOwner. This
+would constitute two filters
+
+```
+SubEntryTarget: class eq apitokens
+EntryTarget: person and not memberOf idm_high_priv
+```
+
+Both conditions must be met for the access control to apply. In the case of a `create`, the SubEntryTarget
+is used for assertion of the SubEntry adherence to the filter. SubEntryTarget implies "class eq SubEntry". EntryTarget
+implies `and not class eq SubEntry`.
+
+## Search / Access
+
+How to handle where we need to check the entryTarget if we don't have the entry? Do SubEntries need
+to auto-dereference and link to their owning entry for filter application?
+
+If we deref, we need to be careful to avoid ref-count loops, since we would need to embed Arc or Weak
+references into the results.
+
+
+Alternately, is this where we need pre-extraction of access controls?
+
+Could SubEntries only be accessed via their Parent Entry via embedding?
+
+
+
+## Deletion
+
+During a deletion, all deleted entries will also imply the deletion of their SubEntries. These SubEntries
+will be marked with a flag to distinguish them as an indirect delete.
+
+## Reviving
+
+During a revive, a revived entry implies the revival of it's SubEntries that are marked as indirect
+deleted.
+
+## Replication / Consistency
+
+If a SubEntry is created with out an owner, or becomes a orphaned due to a replication conflict of
+it's owning entry, the SubEntries are deleted.
+
+
+
+
+