Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions docs/Server/Replication.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,8 @@ sequence number, the configured origin, a timestamp, the entry's UUID and DN, an
entry.

A consumer presents a cookie, which is a marker of how far it has already read. The server then sends the changes that
came after that point. The journal is the single source behind sync and change auditing.
came after that point. The journal is the single source behind sync and change auditing. A consumer can also receive a
delete for an entry it never held, such as one added and then removed since its cookie, and should ignore it.

## Linked Attributes

Expand Down Expand Up @@ -107,6 +108,12 @@ A sync consumer receives full entries, including attributes that are hidden from
so grant it only to identities you trust with the whole directory. Attribute read rules like the secure default's
`userPassword` restriction still apply to normal searches by that identity; they do not trim what replication sends.

Entries the sync identity cannot search are not sent. An entry that becomes hidden from it, or leaves the synced
content, is sent as a delete named by the last DN where the identity could see it, and not at all if it never could.
This follows changes to the entry itself. Changing the access rules, or the groups they name, announces nothing. After
a change that hides or reveals entries for a restricted sync identity, remove the replica's checkpoint to have it
reload. A sync identity with full access is not affected.

## Poll vs Listen

The consumer chooses how it reads; the provider supports both:
Expand Down Expand Up @@ -252,7 +259,8 @@ A replica requires PDO storage (SQLite or MySQL) on either runner. Any other sto

Access control is enforced independently on each side, and FreeDSx ACL is server configuration that does not replicate:

* On the provider, the read access granted to the replica's bind identity bounds what data reaches the replica.
* On the provider, the read access granted to the replica's bind identity bounds what data reaches the replica. An
entry moved where that identity cannot search is removed from the replica.
* On the replica, its own access control bounds what clients reading from it may see. A replica with a more permissive
policy than the provider can expose data the provider would not.

Expand Down
123 changes: 123 additions & 0 deletions src/FreeDSx/Ldap/Sync/Provider/ChangeWindow.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
<?php

declare(strict_types=1);

/**
* This file is part of the FreeDSx LDAP package.
*
* (c) Chad Sikorra <Chad.Sikorra@gmail.com>
*
* For the full copyright and license information, please view the LICENSE
* file that was distributed with this source code.
*/

namespace FreeDSx\Ldap\Sync\Provider;

use Closure;
use FreeDSx\Ldap\Entry\Dn;
use FreeDSx\Ldap\Server\Backend\Storage\Journal\Change\ChangeRecord;
use FreeDSx\Ldap\Server\Backend\Storage\Journal\Change\ChangeType;
use FreeDSx\Ldap\Server\Backend\Storage\Journal\Read\ChangeScope;

use function array_key_exists;

/**
* What a window of journal changes says about each entry: its latest change, and the last DN the consumer could see.
*
* @author Chad Sikorra <Chad.Sikorra@gmail.com>
*/
final class ChangeWindow
{
/**
* @var array<string, int>
*/
private array $latestSeqs = [];

/**
* @var array<string, string> the last in-scope DN the sync identity could see each entry at
*/
private array $lastVisible = [];

/**
* @var array<string, true>
*/
private array $changedEarlier = [];

/**
* @param Closure(Dn): bool $isVisible
*/
public function __construct(
private readonly ChangeScope $scope,
private readonly Closure $isVisible,
) {}

/**
* Read in journal order, which makes an entry's first rename the one that tells where the consumer held it.
*/
public function record(ChangeRecord $record): void
{
$change = $record->change;
$uuid = $change->entryUuid;
$isFirst = !array_key_exists(
$uuid,
$this->latestSeqs,
);
$this->latestSeqs[$uuid] = $record->seq;

if (!$isFirst) {
$this->changedEarlier[$uuid] = true;
}

if ($isFirst && $change->changeType === ChangeType::ModRdn && $change->previousDn !== null) {
$this->see(
$uuid,
$change->previousDn,
);
}

$this->see(
$uuid,
$change->dn,
);
}

public function isLatest(ChangeRecord $record): bool
{
return ($this->latestSeqs[$record->change->entryUuid] ?? null) === $record->seq;
}

/**
* Whether the window changed the entry before its latest change.
*/
public function hasEarlierChange(string $entryUuid): bool
{
return isset($this->changedEarlier[$entryUuid]);
}

/**
* The last in-scope DN the sync identity could see the entry at, or null when it never could in this window.
*/
public function lastVisible(string $entryUuid): ?Dn
{
$dn = $this->lastVisible[$entryUuid] ?? null;

return $dn === null
? null
: new Dn($dn);
}

private function see(
string $uuid,
Dn $dn,
): void {
$name = $dn->toString();

if (($this->lastVisible[$uuid] ?? null) === $name) {
return;
}

if ($this->scope->contains($dn) && ($this->isVisible)($dn)) {
$this->lastVisible[$uuid] = $name;
}
}
}
55 changes: 35 additions & 20 deletions src/FreeDSx/Ldap/Sync/Provider/SyncPersistStreamer.php
Original file line number Diff line number Diff line change
Expand Up @@ -70,19 +70,21 @@ public function projectSince(
TokenInterface $token,
): iterable {
$scope = $this->scopeFor($request);
$latestSeqs = $this->latestSeqByUuid(
$window = $this->windowOf(
$sinceSeq,
$throughSeq,
$scope,
$token,
);

foreach ($this->stream->since($sinceSeq, $scope, $throughSeq) as $record) {
if (($latestSeqs[$record->change->entryUuid] ?? null) !== $record->seq) {
if (!$window->isLatest($record)) {
continue;
}

$result = $this->project(
$record->change,
$window,
$scope,
$request,
$token,
Expand Down Expand Up @@ -213,40 +215,48 @@ private function pushChanges(
}

/**
* The seq of each entry's latest change in range.
*
* @return array<string, int>
*
* @throws JournalLapsedException
*/
private function latestSeqByUuid(
private function windowOf(
int $sinceSeq,
int $throughSeq,
ChangeScope $scope,
): array {
$latestSeqs = [];
TokenInterface $token,
): ChangeWindow {
$window = new ChangeWindow(
$scope,
fn(Dn $dn): bool => $this->projector->isVisibleAt(
$token,
$dn,
),
);

foreach ($this->stream->since($sinceSeq, $scope, $throughSeq) as $record) {
$latestSeqs[$record->change->entryUuid] = $record->seq;
$window->record($record);
}

return $latestSeqs;
return $window;
}

private function project(
PendingChange $change,
ChangeWindow $window,
ChangeScope $scope,
SearchRequest $request,
TokenInterface $token,
): ?SyncResult {
$lastVisible = $window->lastVisible($change->entryUuid);

return $change->changeType === ChangeType::Delete
? $this->projector->projectDeleted(
$change,
$lastVisible,
$window->hasEarlierChange($change->entryUuid),
$request,
$token,
)
: $this->fetchAndProject(
$change,
$lastVisible,
$scope,
$request,
$token,
Expand All @@ -258,6 +268,7 @@ private function project(
*/
private function fetchAndProject(
PendingChange $change,
?Dn $lastVisible,
ChangeScope $scope,
SearchRequest $request,
TokenInterface $token,
Expand All @@ -267,23 +278,27 @@ private function fetchAndProject(
$this->projector->fetchProjectionFor($request),
);

// Deleted since, and its own journaled delete announces that.
// Deleted since, and announced now because a later delete judged by its pre-image alone could be withheld.
if ($entry === null) {
return null;
return $this->projector->projectLeft(
$change->entryUuid,
$lastVisible,
);
}

// Left the content, so RFC 4533 §4.1 wants it deleted rather than sent from where it went.
// RFC 4533 §4.1 wants an entry that left the content deleted rather than sent from where it went.
if (!$scope->contains($entry->getDn())) {
return $this->projector->projectMovedOut(
$entry,
$change,
$token,
return $this->projector->projectLeft(
$change->entryUuid,
$lastVisible,
);
}

return $this->projector->projectFetched(
$entry,
$change,
$change->entryUuid,
$change->changeType === ChangeType::Add,
$lastVisible,
$request,
$token,
);
Expand Down
Loading
Loading