* feat: Rebuild the deb packaging flow
fix: Add more sudo, GHA likes sudo
fix: Give build_debs.sh only the triplet argument
fix: Work around more GHA weirdness in apt sources
Drop crossbuild as it was only used by debian packaging
docs: Update book and other docs for packaging flow
feat: package kanidm_tools aka kanidm cli
docs: Update packaging docs for latest process and clarity
fix: use full triple in sdynlib variants
fix: Correct kanidm.pam asset placement
fix: Give pam & nss modules a description so the debs get it
fix: Work around wonky libssl3 naming in Ubuntu 24.04
fix: Place kanidm bin correctly :3
feat: Pin all blame on @yaleman :3
WIP: Swap out the submodule reference. Still not the final one though.
refactor: Switch kanidm-pam & kanidm-nss to mandatory deps
While in theory unixd will start and run without them, it also won't do
anything useful.
fix: explicit depends for nss & pam libs without versions
We build the debs on the ubuntu24.04 GHA runner so automatic pins
versions that are too new for 22.04. Ideally we'd run cargo-deb also on
the target images but that'll have to be a future improvement.
* refactor: Switch nss_kanidm & pam_kanidm package naming closer to debian guidance
* feat: Attempt enabling unixd by default with secure defaults
* fix: Relax config permissions so the kanidm user can read
Also, update postinst config instructions
* fix(scim_proto): fixing an issue with building due to dependencies
* feat(cli): more error message detail when things go wrong with images on the CLI
Incorrect logic in cred update meant that valid transactions would not be allowed to commit as a mistake in the UI flow.
This is a skill issue on my part.
* Orca is happy, everyone is happy
* added model selection option for Orca and properly included opt.rs inside the file tree instead of using `include!` macro
* added variable login delay to both read and write models!
* Clippy and William are happier this way
* fixed toml syntax for member count and removed old person_count_by_group inside ProfileBuilder
* added strong typing for group names, in the future this should help adding CLI support
* added the `latency_measurer` model and improved the ActorModel trait
* Fixed lots of bugs and made clippy happier
* updated all models to use random backoff time for the login transition
* fix typos and misspellings
* use proper capitalization
* Apply suggestions from code review
---------
Co-authored-by: James Hodgkinson <james@terminaloutcomes.com>
* added `thread_count` configuration for the server
* added `thread_count` to orca
---------
Co-authored-by: Sebastiano Tocci <sebastiano.tocci@proton.me>
Thanks to @Seba-T's work with Orca, we were able to identify a number of performance issues in certain high load conditions.
This commit contains fixes for the following issues
* Unbounded Memory Growth - due to how ARCache works, to maintain temporal consistency it must retain copies of keys (not values) in a special data set for tracking. The Filter Resolve Cache was using unresolved filters as keys. This caused memory explosions when refint or memberof were updating a group with a large number of members because they would emit a query with hundreds of filter terms that would only be used once and never again, causing the ARCache haunted set to grow without bound. To limit this, we no longer cache large/complex queries for resolution, and in future we may implement some other methods to reduce this like sha256/hmac of the queries.
* When creating a new account, dyngroups would be engaged to add the account as a member due to the matching scope. However the change to the dyngroup was triggering an update of all the dyngroups *members* related memberof attributes. This would mean that adding an account would trigger every other account to be loaded an updated.
* When memberof would iterate over leaf entries and update them one at a time. This mean a large number of small fragmented queries in the case of a lot of leaf entries being updated. Now leaf entries are updated in a single stripe once groups are stabilised.
* Member of would always trigger it's members to always update. Instead, we should only update members where a difference is observed, or all members if the group's memberof itself has changed since this needs to propogate to all leaf entries. This significantly reduces the amount of writes and operations to examine the changed member of set.
* Referential integrity would examine all reference uuids on entries for validity rather than just the reference uuids that were altered within the transaction. This change means that only uuids that were *added* are validated during an operation.
* During async write backs (delayed actions) these were performed one at a time. Instead, when possible this should be done in a single transaction as the write transaction caches all writes in memory until the commit meaning that by batching we reduce overall latency.
* In the server there can only be one write transaction and many readers. These are guarded by tokio semaphores that act as fair queues - first in gets the lock next. Due to the design of the server readers would be blocked on the *database* semaphore, and writers would block on the write semaphore and THEN the database semaphore. This arrangement was creating a situation which unfairly advantaged readers over writers, as any write would first have to become the head of it's queue, and then compete with all readers to access a db transaction. Instead, we now have a reader semaphore with size threads minus 1, clamped at a minimum of 1. This means that provided there are two or more threads, then a writer will *always* have a database handle available, and readers will pre-queue with each other before queueing on the db ticket. If there is only one thread, then writes and reads will alternate between each other fairly.
* disable mimalloc on illumos, in part because it immediately segfaults,
but also because we prefer libumem and link it into all Rust binaries
* switch from fs2 (unmaintained crate) to fs4 which provides the same
interface and has wider platform support
While basking under the shade of the coolabah tree, I was overcome by an intense desire to improve the performance and memory usage of Kanidm.
This pr reduces a major source of repeated small clones, lowers default log level in testing, removes some trace fields that are both large and probably shouldn't be traced, and also changes some lto settings for release builds.
* kanidm cli logs on debug level - Fixes#2745
* such clippy like wow
* It's important for a wordsmith to know when to get its fixes in.
* updootin' wasms
Improve the models and what can be performed in the orca benchmarks.
---------
Co-authored-by: Sebastiano Tocci <seba.tocci@gmail.com>
Co-authored-by: Sebastiano Tocci <sebastiano.tocci@proton.me>
This completely reworks how we approach and handle cryptographic keys in Kanidm. This is needed as a foundation for replication coordination which will require handling and rotation of cryptographic keys in automated ways.
This change influences many other parts of the code base in it's implementation.
The primary influences are:
* Modification of how domain user signing keys are revoked or rotated.
* Merging of all existing service-account token keys are retired (retained) keys into the domain to simplify token signing and validation
* Allowing multiple configurations of local command line tools to swap between instances using disparate signing keys.
* Modification of key retrieval to be key id based (KID), removing the need to embed the JWK into tokens
A side effect of this change is that most user authentication sessions and oauth2 sessions will have to be re-established after upgrade. However we feel that session renewal after upgrade is an expected side effect of an upgrade.
In the future this lays the ground work to remove a large number of legacy key handling processes that have evolved, which will allow large parts of code to be removed.
