Previously on schema definitions for attributes, the list of index
types was manually set on attributes. The issue with this approach is
that not all index types apply to all attribute syntaxes. This made it
error prone not just to Kanidm developers, but to future users who
want to define custom attributes and may incorrectly index those
attributes.
Instead, this changes the index value to be a boolean to indicate
if this attribute should or should not be indexed. Internally Kanidm
has a list of appropriate indexes to apply to these syntax types.
As part of this change, the tests were reviewed to find missing index
types for syntaxes, and other causes of unindexed searches which led
to some changes around the dyngroup plugin (which pushes the boundaries
of a lot of things in Kani due to how it works).
* Add missing lld dependency and fix syntax typo in devcontainer_postcreate.sh
* replace pushd/popd with shell agnostic solution and do not throw away std out/err
---------
Co-authored-by: foobar <foobar>
The existing shell.nix uses whatever versions of rustc and cargo are in
the system nixpkgs. In the current stable nixpkgs version (24.11), this
is rustc 1.82.0. Unfortunately, we depend on the `strict_provenance`
feature, which was unstable before 1.84.0. (See: kanidm/concread#132)
This patch makes minimal changes to shell.nix to overlay nixpkgs with
the rustc version defined in rust-toolchain.toml, enabling Kanidm to
build locally on stable versions of NixOS.
Co-authored-by: Firstyear <william@blackhats.net.au>
* packaging: Use cargo-deb multiarch support
This allows building all platforms from one definition,
assuming the --multiarch=foreign flag is used.
* packaging: Use correct path naming for unixd service files
While cargo-deb works around the mistake, better to name them as per the
rules: https://github.com/kornelski/cargo-deb/blob/main/systemd.md#systemd-unit-file-naming
* docs: Update book chapter on Debian packaging
* packaging: Shift Debian builds to a separate build profile
* packaging: Add deb for kanidmd
The implementation of the unixd cache relies on inotify to detect changes to files in /etc so that we know when to reload the data for nss/passwd. However, the way that groupadd/del and other tools work is they copy the file, change it, and then move it into place. It turns out that william of the past didn't realise that inotify works on inodes not paths like other tools do (auditctl for example).
As a result, when something modified /etc/group or another related file, the removal was seen, but this breaks notifications on any future change until you reload unixd.
To resolve this we need to recursively watch /etc with inotify - yep, that's correct. We have to watch everything in /etc for changes because it's the only way to pick up on the add/remove of files. But because we have to watch everything, we need permissions to watch everything.
This forces us to move the parsing of the etc passwd/group/shadow files to the unixd tasks daemon - arguably, this is the correct place to read these anyway since that is a high priv (and locked down) daemon. Because of this, we actually end up solving the missing "shadow" group on debian issue, and probably similar on the BSD's in future.
In order to make my life easier while testing I also threw in a makefile that symlinks the files to needed locations for testing. It has plenty of warnings as it should.
Fixes#3499Fixes#3407Fixes#3249
A lot of legacy directory servers will use rfc2307 schema where
members of groups are stored as the uid instead of a dn. Within
kani, we absolutely need this to be a dn, else we risk accidentally
adding kanidm entries into ldap synced groups which isn't what we
want.
If we have an rfc2307 schema, then we pre-resolve the uid to the
member dn so that kanidm gets the correct information.
* Bump mozilla-actions/sccache-action from 0.0.7 to 0.0.8 in the all group
* fix: remove manual specification of sccache version from github actions
Bumps the all group with 1 update: [mozilla-actions/sccache-action](https://github.com/mozilla-actions/sccache-action).
Updates `mozilla-actions/sccache-action` from 0.0.7 to 0.0.8
- [Release notes](https://github.com/mozilla-actions/sccache-action/releases)
- [Commits](https://github.com/mozilla-actions/sccache-action/compare/v0.0.7...v0.0.8)
---
updated-dependencies:
- dependency-name: mozilla-actions/sccache-action
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: all
...
Signed-off-by: dependabot[bot] <support@github.com>
Add support for ldap servers in integration tests
This allows the ldap interface to be enabled during tests, which is
a final requirement to complete ldap application passwords.
Some oauth2 clients apparently ignore what we tell them
and request response modes we don't support.
First, we should deserialise these and error correctly.
Second, to maintain temporary compatibility, we remap
form-post to query. This will be removed in future.
* Ignore tests that are no longer used.
Each time a library or binary is added, that requires compilation to create
the *empty* test harness, which then is executed and takes multiple seconds
to start up, do nothing, and return success.
This removes test's for libraries that aren't actually using or running
any tests.
Additionally, each time a new test binary is added, that adds a ton of
compilation time, but also test execution time as the binary for each
test runner must start up, execute, and shutdown. So this merges all
the testkit integration tests to a single running which significantly
speeds up test execution.
* Improve IDL exists behaviour, improve memberof verification
Again to improve test performance. This improves the validation of idx
existance to be a faster SQLite call, caches the results as needed.
Memberof was taking up a large amount of time in verify phases of test
finalisation, and so a better in memory version has been added.
* Disable TLS native roots when not needed
* Cleanup tests that are hitting native certs, or do nothing at all
Migrations and server bootstrap are very interconnected processes
and in this we'll be addressing and improving both.
Server bootstrap was performed by creating base entries in phases,
eventually bringing up enough of the *oldest* supported server
minimum remigration level, to then allow triggering of migrations.
Migrations then applied "patches" effectively ontop of this minimum
level to update entries to what they should be in newer versions of
the server.
This scheme has it's pros and cons, but the major con was that to
remove a migration meant squashing it's content back into the
minimum remigration level, and this was a human process that was
quite error prone and difficult to automate. As well, this scheme
also led to cases where the patch migrations would sometimes *not*
reflect all the needed changes or content, or in one case was actually
undone by a patchlevel fix up that was required to address a bug.
Invariably this led to issues, and cases where a new server may have
different content to a migrated one - not exactly what we want!
This is a new migration scheme that addresses this fragility. However
what it trades is verbosity of the content.
Rather than having a base set of entries and patching/updating small
sections ontop, we have migration data folders that contain the full
set of entries as they should appear at that migration level. This
makes the bootstrap process easier as we can just apply the migration
level as a whole, and targetted to what precise version we want.
This also makes migrations more durable as the content is explicitly
copied and all entries fully applied, so there is no risk that a
migration or data change can be forgotten or applied incorrectly. We
are expressing the full state of what our builtin and provided entries
should be.
Finally this rips out a number of places where migration data was being
used as test case data. Not all of these have been replaced (notably
in authsession with Account), but the majority have and have been replaced
with clearer use of constants rather than building whole entries just to
access the name and throw them away for example.
idm_admin and admin should be exempted from the denied names process,
as these values will already be denied due to attribute uniqueness.
Additionally improved the denied names check to only validate the
name during a change, not during a modifification. This way entries
that become denied can get themself out of the pickle.