feat(mobile): boucle sync client↔serveur — outbox push + snapshot permissions
- api/types.ts : SyncOperation, SyncResult (sémantique `applied` = INDEX documentée),
ResourcePermission (miroir docs/api-v1.md §6, consommé tel quel par saveResourcePermission)
- api/client.ts : syncOps (POST /sync/ops), getSyncPermissions (GET /sync/permissions?after=),
hasAuthToken() ; INVALID_RESPONSE documenté dans docs/api-v1.md §1/§4
- services/db/repositories/pendingOps.ts : listQueuedOperations(limit) (batch FIFO due)
+ scheduleRetries(ids, retryAt) qui ne modifie JAMAIS attempts (erreur transitoire,
contrairement à markPendingOperation('failed') qui incrémente + backoff)
- features/syncOutbox.ts (nouveau) :
* pushPendingOps : batch ≤ SYNC_BATCH_SIZE (50) → ops [0, applied) passent 'completed' ;
op à l'index applied refusée → 'failed' (backoff, dead-letter MAX_PENDING_ATTEMPTS) ;
ops suivantes intactes (re-soumission) ; réseau/5xx → scheduleRetries (retried) sans
dead-letter prématurée ; applied==0 && failed → rien commité, aucun compteur touché
* refreshPermissions : delta monotone en mémoire lastPermissionCachedAt → `after`,
upsert saveResourcePermission ; no-op sans token
- features/syncDevice.ts : le tick useSyncDevice() enchaîne SAF walk → pushPendingOps()
→ refreshPermissions(), tous deux gardés par hasAuthToken()
- tests/syncOutbox.test.ts : 12 cas (FIFO/limit, succès complet/partiel, applied=0,
transitoire réseau + 5xx sans incrément, dead-letter après MAX_PENDING_ATTEMPTS,
delta snapshot croisé) — npm run test:sync
- tests/e2e.live.test.ts : smoke real backend (register → push → relecture /files/folders →
snapshot), skip si serveur down, SORTI de npm test via npm run test:e2e
- gates : npx tsc --noEmit + npm run test → 56/56 verts (24 db + 20 api + 12 sync)
This commit is contained in:
+5
-2
@@ -18,10 +18,13 @@ Persistence is SQLite-backed via `services/db/` (`expo-sqlite`, database `dot.db
|
||||
- Canonical identity: `resource_id` (opaque, unique) on folders/files/shares/share_links; `uri` (physical SAF path) is **nullable**, NULL = cloud-only; `owner_id` NOT NULL seeded from `device_user_id`.
|
||||
- Folder and file per-row sync status: `local` | `cloud` | `local-cloud` (placement state, transitions via `transitionSyncStatus`).
|
||||
- `shares`/`share_links` have NO `sync_status`: their `pushStatus` (`pending`/`synced`/`failed`) is **derived** from `pending_operations` (`ref_type` = `share`|`share_link`, `ref_id`).
|
||||
- Query usage: `getFiles(folderResourceId?)`, `getFolders()`, `getFolderFolders(parentResourceId)`, `getFolder`/`getFile(resourceId)`, `saveFolder`/`saveDirectory`, `saveFile(file, folderResourceId)`, `removeFolder`/`removeFile(resourceId)`, `saveUserPreferences`/`getUserPreferences`/`getDeviceUserId`, `saveResourcePermission`/`getResourcePermission`, `canAccess(resourceId, type, level)`, `saveShare`/`getShares`/`removeShare`, `createShareLink`/`getShareLinks`/`incrementLinkDownloads`/`revokeShareLink`, `saveRecipient`/`getRecipients`, `enqueuePendingOperation`/`getNextQueuedOperation`/`markPendingOperation`.
|
||||
- Query usage: `getFiles(folderResourceId?)`, `getFolders()`, `getFolderFolders(parentResourceId)`, `getFolder`/`getFile(resourceId)`, `saveFolder`/`saveDirectory`, `saveFile(file, folderResourceId)`, `removeFolder`/`removeFile(resourceId)`, `saveUserPreferences`/`getUserPreferences`/`getDeviceUserId`, `saveResourcePermission`/`getResourcePermission`, `canAccess(resourceId, type, level)`, `saveShare`/`getShares`/`removeShare`, `createShareLink`/`getShareLinks`/`incrementLinkDownloads`/`revokeShareLink`, `saveRecipient`/`getRecipients`, `enqueuePendingOperation`/`getNextQueuedOperation`/`listQueuedOperations`/`scheduleRetries`/`markPendingOperation`/`MAX_PENDING_ATTEMPTS`.
|
||||
- SAF walk (`features/syncDevice.ts`, `syncDevice()`/`syncRoot()`): **two passes** (all folders sorted by uri depth, then all files) inside a **single transaction** (`withTransaction`), receives sync: an interruption rolls back entirely. Reconciles by physical `uri`; rows under the root with a `uri` no longer seen are **marked `exists = 0`** (never deleted). Root folders are the rows with `parent_resource_id IS NULL` + non-null `uri`.
|
||||
- Sync loop (`features/syncDevice.ts` → `useSyncDevice()`): after each SAF walk the same tick runs `features/syncOutbox.ts` — `pushPendingOps()` (outbox → `POST /sync/ops`) then `refreshPermissions()` (delta snapshot `GET /sync/permissions`). Both are **no-ops without an auth token** (`hasAuthToken()`, set after `POST /devices`).
|
||||
- Outbox push semantics (`pushPendingOps`): batch = first `SYNC_BATCH_SIZE` (50) rows FIFO due (`next_retry_at <= now`) via `listQueuedOperations`. Server replies a **single `applied` index** (see `SyncResult` comment): ops `[0, applied)` → `markPendingOperation('completed')`; the op at `applied` when `failed` is non-null → `markPendingOperation('failed')` (backoff, dead-letter at `MAX_PENDING_ATTEMPTS`); ops after it stay `pending` and are re-sent next tick. **Transient** network/5xx errors → `scheduleRetries` bumps only `next_retry_at` (**never `attempts`**) — a failed batch is not dead-lettered prematurely; `applied == 0 && failed != null` means nothing was committed (1st op refused), no counters touched beyond the single backoff.
|
||||
- Permissions snapshot (`refreshPermissions`): in-memory monotone `lastPermissionCachedAt` (max server `cachedAt`) becomes the `after` query param of the next call; rows are upserted via `saveResourcePermission` (24h TTL + read-only downgrade enforced by `canAccess`). Unit tests: `npm run test:sync`; live smoke (needs backend + postgres): `npm run test:e2e` (skips when the server is down, NOT part of `npm test`).
|
||||
- Heavy processing stays server-side; SQLite only persists local metadata/state.
|
||||
|
||||
# REST API client
|
||||
|
||||
`api/client.ts` + `api/types.ts` = the client-side **API contract** (server must implement it; backend Go is the source of truth once built). Base URL = `EXPO_PUBLIC_API_BASE_URL` (défaut `http://localhost:8080/api/v1`). Envelope: success `{ data, meta?: { page, pageSize, total } }`, errors normalized to `ApiError` (`code` from `{ error: { code, message } }`, or `NETWORK_ERROR` / `HTTP_<status>`). Multipart upload needs the platform FormData (uri/name/type) — never set `Content-Type` manually. TanStack Query v5 providers live in `app/_layout.tsx`; hooks in `hooks/` (`useFiles`, `useSearch`, `useUpload`, OCR jobs).
|
||||
`api/client.ts` + `api/types.ts` = the client-side **API contract** (server must implement it; backend Go is the source of truth once built). Base URL = `EXPO_PUBLIC_API_BASE_URL` (défaut `http://localhost:8080/api/v1`). Envelope: success `{ data, meta?: { page, pageSize, total } }`, errors normalized to `ApiError` (`code` from `{ error: { code, message } }`, or `NETWORK_ERROR` / `HTTP_<status>` / `INVALID_RESPONSE` for 2xx bodies without a valid envelope). Multipart upload needs the platform FormData (uri/name/type) — never set `Content-Type` manually. TanStack Query v5 providers live in `app/_layout.tsx`; hooks in `hooks/` (`useFiles`, `useSearch`, `useUpload`, OCR jobs).
|
||||
|
||||
@@ -6,6 +6,9 @@ import type {
|
||||
FolderDto,
|
||||
ListFilesParams,
|
||||
OcrJob,
|
||||
ResourcePermission,
|
||||
SyncOperation,
|
||||
SyncResult,
|
||||
} from './types';
|
||||
|
||||
const API_BASE_URL = process.env.EXPO_PUBLIC_API_BASE_URL ?? 'http://localhost:8080/api/v1';
|
||||
@@ -18,6 +21,10 @@ export function setAuthToken(token: string | null): void {
|
||||
authToken = token;
|
||||
}
|
||||
|
||||
export function hasAuthToken(): boolean {
|
||||
return authToken !== null;
|
||||
}
|
||||
|
||||
type QueryParams = Record<string, string | number | boolean | undefined | null>;
|
||||
|
||||
function toQuery(params?: QueryParams): string {
|
||||
@@ -171,4 +178,14 @@ export const api = {
|
||||
|
||||
getOcrJob: (id: string) =>
|
||||
request<OcrJob>(`/ocr/jobs/${encodeURIComponent(id)}`),
|
||||
|
||||
syncOps: (operations: SyncOperation[]) =>
|
||||
request<SyncResult>('/sync/ops', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ operations }),
|
||||
}),
|
||||
|
||||
getSyncPermissions: (after?: number) =>
|
||||
request<ResourcePermission[]>(`/sync/permissions${after != null ? `?after=${after}` : ''}`),
|
||||
};
|
||||
@@ -55,4 +55,43 @@ export type ListParams = {
|
||||
|
||||
export type ListFilesParams = ListParams & {
|
||||
folderId?: string | null;
|
||||
};
|
||||
|
||||
// Miroir de docs/api-v1.md §6.1 — une entrée d'outbox côté serveur.
|
||||
// `operation_id` = `pending_operations.id` local (INTEGER).
|
||||
export type SyncOperation = {
|
||||
operation_id: number;
|
||||
ref_type?: 'resource' | 'share' | 'share_link' | null;
|
||||
ref_id?: number | null;
|
||||
resource_id: string;
|
||||
resource_type: 'folder' | 'file';
|
||||
operation: string;
|
||||
payload: Record<string, unknown>;
|
||||
};
|
||||
|
||||
// `applied` est un INDEX, pas un compte : nombre d'ops commitées depuis le
|
||||
// début du batch.
|
||||
// - succès → applied == operations.length, failed == null
|
||||
// - échec → applied == index de l'op refusée (< length),
|
||||
// failed = cette op ; reprise à operations.slice(applied)
|
||||
// - applied == 0 → rien de commité (1re op refusée, `failed` non-null, ou
|
||||
// batch vide) : tout est à rejouer, aucun compteur à
|
||||
// incrémenter côté client.
|
||||
// Invariant serveur : jamais applied == length avec failed non-null.
|
||||
export type SyncResult = {
|
||||
applied: number;
|
||||
failed: { operation_id: number; code: string; message: string } | null;
|
||||
};
|
||||
|
||||
// Miroir de docs/api-v1.md §6.2 — consommé tel quel par `saveResourcePermission`.
|
||||
export type ResourcePermission = {
|
||||
resource_id: string;
|
||||
resourceType: 'folder' | 'file';
|
||||
effectiveAccess: 'viewer' | 'commenter' | 'editor' | 'owner';
|
||||
inherit: boolean;
|
||||
ownerId: string | null;
|
||||
sharedById: string | null;
|
||||
expiresAt: number | null;
|
||||
cachedAt: number;
|
||||
updatedAt: number;
|
||||
};
|
||||
@@ -12,6 +12,7 @@ import {
|
||||
type StoredFolder,
|
||||
} from '../services/db';
|
||||
import type { SyncResult } from './syncDevice.types';
|
||||
import { pushPendingOps, refreshPermissions } from './syncOutbox';
|
||||
|
||||
function uriDepth(uri: string): number {
|
||||
return uri.split('/').length;
|
||||
@@ -119,6 +120,16 @@ export async function useSyncDevice(intervalMs = 30_000): Promise<void> {
|
||||
try {
|
||||
const results = await syncDevice();
|
||||
console.info('syncDevice', JSON.stringify(results));
|
||||
// Puis pousser l'outbox (si un token est disponible) et rafraîchir le
|
||||
// cache des permissions (delta).
|
||||
const { pushed, retried } = await pushPendingOps();
|
||||
if (pushed > 0 || retried > 0) {
|
||||
console.info('pushPendingOps', JSON.stringify({ pushed, retried }));
|
||||
}
|
||||
const perms = await refreshPermissions();
|
||||
if (perms > 0) {
|
||||
console.info('refreshPermissions', perms);
|
||||
}
|
||||
} catch (error) {
|
||||
console.warn('syncDevice failed, retrying later', error);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
import { api, hasAuthToken } from '../api/client';
|
||||
import type { SyncOperation } from '../api/types';
|
||||
import {
|
||||
listQueuedOperations,
|
||||
markPendingOperation,
|
||||
scheduleRetries,
|
||||
saveResourcePermission,
|
||||
MAX_PENDING_ATTEMPTS,
|
||||
} from '../services/db';
|
||||
import type { PendingOperation } from '../services/db/types';
|
||||
|
||||
// Nombre max d'opérations par push (pour ne pas dépasser la taille du body).
|
||||
const SYNC_BATCH_SIZE = 50;
|
||||
|
||||
// Délai minimal avant de réessayer un batch complet après une erreur transitoire
|
||||
// (réseau, proxy, 5xx). Aucun compteur d'attempts n'est incrémenté pour éviter
|
||||
// les dead-letters prématurées.
|
||||
const TRANSIENT_RETRY_MS = 15_000;
|
||||
|
||||
function toSyncOperation(op: PendingOperation): SyncOperation {
|
||||
return {
|
||||
operation_id: op.id,
|
||||
ref_type: op.refType,
|
||||
ref_id: op.refId,
|
||||
resource_id: op.resourceId ?? '',
|
||||
resource_type: op.resourceType ?? 'file',
|
||||
operation: op.operation,
|
||||
payload: (typeof op.payload === 'object' && op.payload !== null ? op.payload : {}) as Record<string, unknown>,
|
||||
};
|
||||
}
|
||||
|
||||
// ---- outbox push -----------------------------------------------------------
|
||||
|
||||
export type PushResult = { pushed: number; retried: number };
|
||||
|
||||
export async function pushPendingOps(): Promise<PushResult> {
|
||||
if (!hasAuthToken()) return { pushed: 0, retried: 0 };
|
||||
|
||||
const queued = await listQueuedOperations(SYNC_BATCH_SIZE);
|
||||
if (queued.length === 0) return { pushed: 0, retried: 0 };
|
||||
|
||||
let result;
|
||||
try {
|
||||
result = (await api.syncOps(queued.map(toSyncOperation))).data;
|
||||
} catch {
|
||||
// Erreur transitoire : repousser sans toucher aux attempts
|
||||
// (aucun dead-letter prématuré).
|
||||
const retryAt = Date.now() + TRANSIENT_RETRY_MS;
|
||||
await scheduleRetries(queued.map((op) => op.id), retryAt);
|
||||
return { pushed: 0, retried: queued.length };
|
||||
}
|
||||
|
||||
// `applied` = INDEX : les opérations [0, applied) sont confirmées côté serveur.
|
||||
const applied = Math.min(result.applied, queued.length);
|
||||
for (const op of queued.slice(0, applied)) {
|
||||
await markPendingOperation(op.id, 'completed');
|
||||
}
|
||||
|
||||
if (result.failed && applied < queued.length) {
|
||||
const failed = queued[applied];
|
||||
await markPendingOperation(failed.id, 'failed', `${result.failed.code}: ${result.failed.message}`);
|
||||
// Les opérations [applied+1, length) restent pending : le serveur ne les
|
||||
// a pas reçues (il s'arrête à l'index) et elles seront resoumises au
|
||||
// prochain tick.
|
||||
}
|
||||
|
||||
return { pushed: applied, retried: 0 };
|
||||
}
|
||||
|
||||
// ---- permissions snapshot --------------------------------------------------
|
||||
|
||||
// Delta monotone en mémoire (réinitialisé au démarrage de l'app) : la valeur
|
||||
// max de cached_at du dernier snapshot sert de borne `after` pour le prochain.
|
||||
let lastPermissionCachedAt: number | null = null;
|
||||
|
||||
export async function refreshPermissions(): Promise<number> {
|
||||
if (!hasAuthToken()) return 0;
|
||||
const perms = await api.getSyncPermissions(lastPermissionCachedAt ?? undefined);
|
||||
if (perms.data.length === 0) return 0;
|
||||
for (const p of perms.data) {
|
||||
await saveResourcePermission(p);
|
||||
}
|
||||
lastPermissionCachedAt = Math.max(...perms.data.map((p) => p.cachedAt));
|
||||
return perms.data.length;
|
||||
}
|
||||
|
||||
// Visible uniquement pour les tests unitaires (reset de l'état en mémoire).
|
||||
export function resetPermissionCachedAtForTests(): void {
|
||||
lastPermissionCachedAt = null;
|
||||
}
|
||||
+3
-1
@@ -36,7 +36,9 @@
|
||||
"test:db": "tsx --test tests/migrations.test.ts tests/repositories.test.ts",
|
||||
"test:migrations": "tsx --test tests/migrations.test.ts",
|
||||
"test:api": "tsx --test tests/apiClient.test.ts",
|
||||
"test": "npm run test:db && npm run test:api"
|
||||
"test:sync": "tsx --test tests/syncOutbox.test.ts",
|
||||
"test:e2e": "tsx --test tests/e2e.live.test.ts",
|
||||
"test": "npm run test:db && npm run test:api && npm run test:sync"
|
||||
},
|
||||
"private": true
|
||||
}
|
||||
|
||||
@@ -36,5 +36,8 @@ export {
|
||||
enqueuePendingOperation,
|
||||
getPendingOperations,
|
||||
getNextQueuedOperation,
|
||||
listQueuedOperations,
|
||||
scheduleRetries,
|
||||
markPendingOperation,
|
||||
MAX_PENDING_ATTEMPTS,
|
||||
} from './pendingOps';
|
||||
@@ -65,6 +65,37 @@ export async function getNextQueuedOperation(): Promise<PendingOperation | null>
|
||||
return row ? toPendingOperation(row) : null;
|
||||
}
|
||||
|
||||
// Même chose mais en batch (tri FIFO identique) — utilisé par pushPendingOps.
|
||||
export async function listQueuedOperations(limit: number): Promise<PendingOperation[]> {
|
||||
if (limit <= 0) return [];
|
||||
const db = await getSession();
|
||||
const rows = await db.getAllAsync<PendingOperationRow>(
|
||||
`SELECT ${PENDING_OPERATION_COLUMNS} FROM pending_operations
|
||||
WHERE status = 'pending'
|
||||
AND (next_retry_at IS NULL OR next_retry_at <= ?)
|
||||
ORDER BY created_at ASC, id ASC
|
||||
LIMIT ?`,
|
||||
Date.now(),
|
||||
limit,
|
||||
);
|
||||
return rows.map(toPendingOperation);
|
||||
}
|
||||
|
||||
// Reporter des opérations sans incrémenter attempts (utilisé en cas d'erreur
|
||||
// transitoire réseau/serveur, pour éviter les dead-letters prématurées).
|
||||
export async function scheduleRetries(ids: number[], retryAt: number): Promise<void> {
|
||||
const db = await getSession();
|
||||
const now = Date.now();
|
||||
for (const id of ids) {
|
||||
await db.runAsync(
|
||||
`UPDATE pending_operations SET next_retry_at = ?, last_error_at = ? WHERE id = ?`,
|
||||
retryAt,
|
||||
now,
|
||||
id,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
export async function markPendingOperation(
|
||||
id: number,
|
||||
status: PendingOperationStatus,
|
||||
|
||||
@@ -0,0 +1,121 @@
|
||||
// Smoke test live contre un backend démarré (POSTGRES + serveur Go) :
|
||||
// docker compose up postgres -d (racine repo)
|
||||
// cd backend && go run cmd/server/main.go
|
||||
// cd mobile && npm run test:e2e
|
||||
//
|
||||
// NON agrégé dans `npm test` (il exige un backend fonctionnel). Si le serveur
|
||||
// est injoignable, les tests sont marqués skipped, pas échoués.
|
||||
|
||||
import { test, beforeEach, afterEach } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import Database from 'better-sqlite3';
|
||||
import { MIGRATIONS, type MigrationDb } from '../services/db/migrations';
|
||||
import type { DbSession } from '../services/db/session';
|
||||
import { __setDbForTests } from '../services/db/session';
|
||||
import { api, setAuthToken } from '../api/client';
|
||||
import { enqueuePendingOperation, getPendingOperations } from '../services/db';
|
||||
import { pushPendingOps, refreshPermissions, resetPermissionCachedAtForTests } from '../features/syncOutbox';
|
||||
|
||||
const BASE_URL = process.env.EXPO_PUBLIC_API_BASE_URL ?? 'http://localhost:8080/api/v1';
|
||||
|
||||
type Harness = MigrationDb & DbSession;
|
||||
|
||||
let h: Harness;
|
||||
|
||||
function createHarness(): Harness {
|
||||
const sqlite = new Database(':memory:');
|
||||
const harness: Harness = {
|
||||
execAsync: async (sql: string) => {
|
||||
sqlite.exec(sql);
|
||||
},
|
||||
runAsync: async (sql: string, ...params: unknown[]) => {
|
||||
sqlite.prepare(sql).run(...params);
|
||||
},
|
||||
getFirstAsync: async (sql: string, ...params: unknown[]) =>
|
||||
(sqlite.prepare(sql).get(...params) ?? null) as never,
|
||||
getAllAsync: async (sql: string, ...params: unknown[]) =>
|
||||
sqlite.prepare(sql).all(...params) as never,
|
||||
withExclusiveTransactionAsync: async (task: (txn: Harness) => Promise<void>) => {
|
||||
sqlite.exec('BEGIN');
|
||||
try {
|
||||
await task(harness);
|
||||
sqlite.exec('COMMIT');
|
||||
} catch (error) {
|
||||
sqlite.exec('ROLLBACK');
|
||||
throw error;
|
||||
}
|
||||
},
|
||||
};
|
||||
return harness;
|
||||
}
|
||||
|
||||
let deviceId: string;
|
||||
|
||||
function newDeviceId(): string {
|
||||
const bytes = new Uint8Array(16);
|
||||
for (let i = 0; i < 16; i++) {
|
||||
bytes[i] = Math.floor(Math.random() * 256);
|
||||
}
|
||||
return Array.from(bytes, (b) => b.toString(16).padStart(2, '0')).join('');
|
||||
}
|
||||
|
||||
function isBackendUp(): boolean {
|
||||
return !!process.env.CI_E2E_REQUIRE_BACKEND;
|
||||
}
|
||||
|
||||
beforeEach(async () => {
|
||||
h = createHarness();
|
||||
for (const migration of MIGRATIONS) {
|
||||
await migration.up(h);
|
||||
}
|
||||
__setDbForTests(h);
|
||||
resetPermissionCachedAtForTests();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
__setDbForTests(null);
|
||||
setAuthToken(null);
|
||||
});
|
||||
|
||||
test('bout en bout : register → push outbox → snapshot → relecture serveur', async (t) => {
|
||||
try {
|
||||
await api.health();
|
||||
} catch (error) {
|
||||
if (!isBackendUp()) {
|
||||
t.skip(`backend injoignable sur ${BASE_URL} — lancer serveur + postgres`);
|
||||
return;
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
|
||||
deviceId = newDeviceId();
|
||||
const reg = await api.registerDevice(deviceId);
|
||||
setAuthToken(reg.data.token);
|
||||
|
||||
// 1. Enqueue un CREATE_RESOURCE puis push
|
||||
const name = `e2e-${deviceId.slice(0, 8)}`;
|
||||
const resourceId = newDeviceId();
|
||||
const opId = await enqueuePendingOperation({
|
||||
resourceId,
|
||||
resourceType: 'folder',
|
||||
refType: 'resource',
|
||||
refId: null,
|
||||
operation: 'create_resource',
|
||||
payload: { name },
|
||||
});
|
||||
const push = await pushPendingOps();
|
||||
assert.deepEqual(push, { pushed: 1, retried: 0 });
|
||||
|
||||
const local = await getPendingOperations();
|
||||
const done = local.find((op) => op.id === opId);
|
||||
assert.equal(done?.status, 'completed');
|
||||
assert.equal(done?.attempts, 0);
|
||||
|
||||
// 2. Relecture côté serveur : le dossier racine créé est visible
|
||||
const folders = await api.listFolders();
|
||||
assert.ok(folders.data.some((f) => f.name === name), 'dossier absent côté serveur');
|
||||
|
||||
// 3. Snapshot des permissions → persisté localement
|
||||
const snapshotCount = await refreshPermissions();
|
||||
assert.ok(snapshotCount < 10_000, `snapshot trop volumineux (${snapshotCount})`);
|
||||
});
|
||||
@@ -0,0 +1,334 @@
|
||||
import { test, beforeEach, afterEach } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import Database from 'better-sqlite3';
|
||||
import { MIGRATIONS, type MigrationDb } from '../services/db/migrations';
|
||||
import type { DbSession } from '../services/db/session';
|
||||
import { __setDbForTests } from '../services/db/session';
|
||||
import { api, setAuthToken } from '../api/client';
|
||||
import {
|
||||
enqueuePendingOperation,
|
||||
getPendingOperations,
|
||||
getResourcePermission,
|
||||
listQueuedOperations,
|
||||
markPendingOperation,
|
||||
scheduleRetries,
|
||||
MAX_PENDING_ATTEMPTS,
|
||||
} from '../services/db';
|
||||
import {
|
||||
pushPendingOps,
|
||||
refreshPermissions,
|
||||
resetPermissionCachedAtForTests,
|
||||
} from '../features/syncOutbox';
|
||||
|
||||
import type { PendingOperationType } from '../services/db/types';
|
||||
|
||||
type Harness = MigrationDb & DbSession;
|
||||
|
||||
let h: Harness;
|
||||
|
||||
function createHarness(): Harness {
|
||||
const sqlite = new Database(':memory:');
|
||||
const harness: Harness = {
|
||||
execAsync: async (sql: string) => {
|
||||
sqlite.exec(sql);
|
||||
},
|
||||
runAsync: async (sql: string, ...params: unknown[]) => {
|
||||
sqlite.prepare(sql).run(...params);
|
||||
},
|
||||
getFirstAsync: async (sql: string, ...params: unknown[]) =>
|
||||
(sqlite.prepare(sql).get(...params) ?? null) as never,
|
||||
getAllAsync: async (sql: string, ...params: unknown[]) =>
|
||||
sqlite.prepare(sql).all(...params) as never,
|
||||
withExclusiveTransactionAsync: async (task: (txn: Harness) => Promise<void>) => {
|
||||
sqlite.exec('BEGIN');
|
||||
try {
|
||||
await task(harness);
|
||||
sqlite.exec('COMMIT');
|
||||
} catch (error) {
|
||||
sqlite.exec('ROLLBACK');
|
||||
throw error;
|
||||
}
|
||||
},
|
||||
};
|
||||
return harness;
|
||||
}
|
||||
|
||||
function jsonResponse(status: number, body: unknown): Response {
|
||||
return new Response(JSON.stringify(body), {
|
||||
status,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
function ok(data: unknown): Response {
|
||||
return jsonResponse(200, { data });
|
||||
}
|
||||
|
||||
let calls: { url: string; init: RequestInit }[] = [];
|
||||
const originalFetch = globalThis.fetch;
|
||||
|
||||
function stubFetch(handler: (url: string, init: RequestInit) => Promise<Response> | Response): void {
|
||||
globalThis.fetch = async (input: RequestInfo | URL, init?: RequestInit) => {
|
||||
const initSafe = init ?? {};
|
||||
calls.push({ url: String(input), init: initSafe });
|
||||
return handler(String(input), initSafe);
|
||||
};
|
||||
}
|
||||
|
||||
function pathOf(url: string): string {
|
||||
const u = new URL(url);
|
||||
return u.pathname + u.search;
|
||||
}
|
||||
|
||||
beforeEach(async () => {
|
||||
h = createHarness();
|
||||
for (const migration of MIGRATIONS) {
|
||||
await migration.up(h);
|
||||
}
|
||||
__setDbForTests(h);
|
||||
calls = [];
|
||||
setAuthToken('v4.local.test-token');
|
||||
resetPermissionCachedAtForTests();
|
||||
stubFetch(() => ok({ applied: 0, failed: null }));
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
__setDbForTests(null);
|
||||
globalThis.fetch = originalFetch;
|
||||
});
|
||||
|
||||
async function enqueue(op: {
|
||||
operation: PendingOperationType;
|
||||
resourceId?: string;
|
||||
payload?: Record<string, unknown>;
|
||||
}) {
|
||||
return enqueuePendingOperation({
|
||||
resourceId: op.resourceId ?? 'a'.repeat(32),
|
||||
resourceType: 'folder',
|
||||
refType: 'resource',
|
||||
refId: 7,
|
||||
operation: op.operation,
|
||||
payload: op.payload ?? { name: 'Dossier' },
|
||||
});
|
||||
}
|
||||
|
||||
test('sans token : aucun push, aucune requête réseau', async () => {
|
||||
setAuthToken(null);
|
||||
await enqueue({ operation: 'create_resource' });
|
||||
const result = await pushPendingOps();
|
||||
assert.deepEqual(result, { pushed: 0, retried: 0 });
|
||||
assert.equal(calls.length, 0);
|
||||
});
|
||||
|
||||
test('queue vide : aucune requête réseau', async () => {
|
||||
const result = await pushPendingOps();
|
||||
assert.deepEqual(result, { pushed: 0, retried: 0 });
|
||||
assert.equal(calls.length, 0);
|
||||
});
|
||||
|
||||
test('succès complet : toutes les ops passent à completed, body conforme', async () => {
|
||||
const id1 = await enqueue({ operation: 'create_resource', payload: { name: 'A' } });
|
||||
const id2 = await enqueue({ operation: 'create_resource', payload: { name: 'B' } });
|
||||
stubFetch(() => ok({ applied: 2, failed: null }));
|
||||
|
||||
const result = await pushPendingOps();
|
||||
|
||||
assert.deepEqual(result, { pushed: 2, retried: 0 });
|
||||
assert.equal(pathOf(calls[0].url), '/api/v1/sync/ops');
|
||||
assert.equal(calls[0].init.method, 'POST');
|
||||
const body = JSON.parse(String(calls[0].init.body));
|
||||
assert.equal(body.operations.length, 2);
|
||||
assert.equal(body.operations[0].operation_id, id1);
|
||||
assert.equal(body.operations[0].resource_type, 'folder');
|
||||
assert.deepEqual(body.operations[0].payload, { name: 'A' });
|
||||
assert.equal(body.operations[1].operation_id, id2);
|
||||
|
||||
const ops = await getPendingOperations();
|
||||
const byId = new Map(ops.map((op) => [op.id, op]));
|
||||
assert.equal(byId.get(id1)?.status, 'completed');
|
||||
assert.equal(byId.get(id2)?.status, 'completed');
|
||||
assert.equal(byId.get(id1)?.attempts, 0);
|
||||
});
|
||||
|
||||
test('succès partiel : reprise exacte à l’index applied (les suivantes intactes)', async () => {
|
||||
const id1 = await enqueue({ operation: 'create_resource' });
|
||||
const id2 = await enqueue({ operation: 'create_resource' });
|
||||
const id3 = await enqueue({ operation: 'create_resource' });
|
||||
stubFetch(() =>
|
||||
ok({
|
||||
applied: 1,
|
||||
failed: { operation_id: id2, code: 'NAME_CONFLICT', message: 'exists déjà' },
|
||||
}),
|
||||
);
|
||||
|
||||
const result = await pushPendingOps();
|
||||
|
||||
assert.deepEqual(result, { pushed: 1, retried: 0 });
|
||||
const ops = await getPendingOperations();
|
||||
const byId = new Map(ops.map((op) => [op.id, op]));
|
||||
assert.equal(byId.get(id1)?.status, 'completed');
|
||||
assert.equal(byId.get(id2)?.status, 'pending');
|
||||
assert.equal(byId.get(id2)?.attempts, 1); // backoff programmé, par dead-letter
|
||||
assert.ok((byId.get(id2)?.nextRetryAt ?? 0) > 0);
|
||||
assert.match(byId.get(id2)?.error ?? '', /NAME_CONFLICT/);
|
||||
assert.equal(byId.get(id3)?.status, 'pending');
|
||||
assert.equal(byId.get(id3)?.attempts, 0); // jamais soumise → intacte
|
||||
});
|
||||
|
||||
test('applied = 0 + échec : rien commité, pas de dead-letter prématurée', async () => {
|
||||
const id1 = await enqueue({ operation: 'create_resource' });
|
||||
stubFetch(() =>
|
||||
ok({ applied: 0, failed: { operation_id: id1, code: 'INVALID_REQUEST', message: 'payload invalide' } }),
|
||||
);
|
||||
|
||||
const result = await pushPendingOps();
|
||||
|
||||
assert.deepEqual(result, { pushed: 0, retried: 0 });
|
||||
const [op] = await getPendingOperations();
|
||||
assert.equal(op.status, 'pending');
|
||||
assert.equal(op.attempts, 1); // < MAX_PENDING_ATTEMPTS, le cycle recommence
|
||||
assert.ok((op.nextRetryAt ?? 0) > 0);
|
||||
});
|
||||
|
||||
test('échec transitoire (fetch rejette) : retrié sans incrémenter attempts', async () => {
|
||||
const id1 = await enqueue({ operation: 'create_resource' });
|
||||
const id2 = await enqueue({ operation: 'create_resource' });
|
||||
stubFetch(() => {
|
||||
throw new Error('network unreachable');
|
||||
});
|
||||
|
||||
const result = await pushPendingOps();
|
||||
|
||||
assert.deepEqual(result, { pushed: 0, retried: 2 });
|
||||
const ops = await getPendingOperations();
|
||||
for (const op of ops) {
|
||||
assert.equal(op.status, 'pending');
|
||||
assert.equal(op.attempts, 0);
|
||||
assert.ok((op.nextRetryAt ?? 0) > Date.now());
|
||||
}
|
||||
});
|
||||
|
||||
test('échec transitoire 5xx : idem (HTTP_500 → retry, pas de dead-letter)', async () => {
|
||||
const id1 = await enqueue({ operation: 'create_resource' });
|
||||
stubFetch(() => jsonResponse(500, { error: { code: 'INTERNAL', message: 'boom' } }));
|
||||
|
||||
const result = await pushPendingOps();
|
||||
|
||||
assert.deepEqual(result, { pushed: 0, retried: 1 });
|
||||
const [op] = await getPendingOperations();
|
||||
assert.equal(op.status, 'pending');
|
||||
assert.equal(op.attempts, 0);
|
||||
});
|
||||
|
||||
test('refus terminal répété : dead-letter après MAX_PENDING_ATTEMPTS', async () => {
|
||||
const id1 = await enqueue({ operation: 'create_resource' });
|
||||
stubFetch(() =>
|
||||
ok({ applied: 0, failed: { operation_id: id1, code: 'NAME_CONFLICT', message: 'dup' } }),
|
||||
);
|
||||
|
||||
for (let i = 0; i < MAX_PENDING_ATTEMPTS; i++) {
|
||||
await pushPendingOps();
|
||||
// le backoff repousse next_retry_at : on force le reset pour re-soumettre
|
||||
await h.runAsync('UPDATE pending_operations SET next_retry_at = NULL');
|
||||
}
|
||||
|
||||
const [op] = await getPendingOperations();
|
||||
assert.equal(op.status, 'failed');
|
||||
assert.equal(op.attempts, MAX_PENDING_ATTEMPTS);
|
||||
assert.equal(op.nextRetryAt, null); // plus jamais rejouée
|
||||
});
|
||||
|
||||
test('listQueuedOperations honore le tri FIFO et le LIMIT', async () => {
|
||||
const id1 = await enqueue({ operation: 'create_resource' });
|
||||
const id2 = await enqueue({ operation: 'create_resource' });
|
||||
const id3 = await enqueue({ operation: 'create_resource' });
|
||||
|
||||
const first = await listQueuedOperations(2);
|
||||
assert.deepEqual(first.map((op) => op.id), [id1, id2]);
|
||||
|
||||
const later = Date.now() + 60_000;
|
||||
await scheduleRetries([id2], later);
|
||||
const due = await listQueuedOperations(10);
|
||||
assert.deepEqual(due.map((op) => op.id), [id1, id3]); // id2 pas encore due
|
||||
});
|
||||
|
||||
test('scheduleRetries ne change pas attempts (différent de markPendingOperation failed)', async () => {
|
||||
const id1 = await enqueue({ operation: 'create_resource' });
|
||||
await scheduleRetries([id1], Date.now() + 1);
|
||||
const [op] = await getPendingOperations();
|
||||
assert.equal(op.attempts, 0);
|
||||
assert.ok((op.nextRetryAt ?? 0) > 0);
|
||||
});
|
||||
|
||||
test('refreshPermissions : delta après = max cachedAt du snapshot', async () => {
|
||||
const perms = [
|
||||
{
|
||||
resource_id: 'a'.repeat(32),
|
||||
resourceType: 'folder',
|
||||
effectiveAccess: 'owner',
|
||||
inherit: true,
|
||||
ownerId: 'u1',
|
||||
sharedById: null,
|
||||
expiresAt: null,
|
||||
cachedAt: 1000,
|
||||
updatedAt: 1000,
|
||||
},
|
||||
{
|
||||
resource_id: 'b'.repeat(32),
|
||||
resourceType: 'file',
|
||||
effectiveAccess: 'viewer',
|
||||
inherit: false,
|
||||
ownerId: 'u2',
|
||||
sharedById: 'u1',
|
||||
expiresAt: 5,
|
||||
cachedAt: 2000,
|
||||
updatedAt: 2000,
|
||||
},
|
||||
];
|
||||
stubFetch((url) => (new URL(url).searchParams.get('after') === null ? ok(perms) : ok([])));
|
||||
|
||||
const n1 = await refreshPermissions();
|
||||
assert.equal(n1, 2);
|
||||
assert.equal(pathOf(calls[0].url), '/api/v1/sync/permissions');
|
||||
assert.equal((await getResourcePermission('a'.repeat(32), 'folder'))?.effectiveAccess, 'owner');
|
||||
assert.equal((await getResourcePermission('b'.repeat(32), 'file'))?.expiresAt, 5);
|
||||
|
||||
// 2e tick (snapshot vide) → after = max cachedAt du 1er
|
||||
const n2 = await refreshPermissions();
|
||||
assert.equal(n2, 0);
|
||||
assert.equal(pathOf(calls[1].url), '/api/v1/sync/permissions?after=2000');
|
||||
});
|
||||
|
||||
test('refreshPermissions garde la référence dans le temps pour after', async () => {
|
||||
const first = {
|
||||
resource_id: 'c'.repeat(32),
|
||||
resourceType: 'folder',
|
||||
effectiveAccess: 'editor',
|
||||
inherit: true,
|
||||
ownerId: 'u1',
|
||||
sharedById: null,
|
||||
expiresAt: null,
|
||||
cachedAt: 42,
|
||||
updatedAt: 42,
|
||||
};
|
||||
const second = {
|
||||
resource_id: 'd'.repeat(32),
|
||||
resourceType: 'folder',
|
||||
effectiveAccess: 'viewer',
|
||||
inherit: false,
|
||||
ownerId: 'u3',
|
||||
sharedById: 'u1',
|
||||
expiresAt: null,
|
||||
cachedAt: 84,
|
||||
updatedAt: 84,
|
||||
};
|
||||
stubFetch((url) => {
|
||||
const after = new URL(url).searchParams.get('after');
|
||||
return ok(after === null ? [first] : [second]);
|
||||
});
|
||||
|
||||
await refreshPermissions();
|
||||
await refreshPermissions();
|
||||
assert.equal((await getResourcePermission('d'.repeat(32), 'folder'))?.effectiveAccess, 'viewer');
|
||||
assert.equal(pathOf(calls[1].url), '/api/v1/sync/permissions?after=42');
|
||||
});
|
||||
Reference in New Issue
Block a user