feat(mobile): boucle sync client↔serveur — outbox push + snapshot permissions

- api/types.ts : SyncOperation, SyncResult (sémantique `applied` = INDEX documentée),
  ResourcePermission (miroir docs/api-v1.md §6, consommé tel quel par saveResourcePermission)
- api/client.ts : syncOps (POST /sync/ops), getSyncPermissions (GET /sync/permissions?after=),
  hasAuthToken() ; INVALID_RESPONSE documenté dans docs/api-v1.md §1/§4
- services/db/repositories/pendingOps.ts : listQueuedOperations(limit) (batch FIFO due)
  + scheduleRetries(ids, retryAt) qui ne modifie JAMAIS attempts (erreur transitoire,
  contrairement à markPendingOperation('failed') qui incrémente + backoff)
- features/syncOutbox.ts (nouveau) :
  * pushPendingOps : batch ≤ SYNC_BATCH_SIZE (50) → ops [0, applied) passent 'completed' ;
    op à l'index applied refusée → 'failed' (backoff, dead-letter MAX_PENDING_ATTEMPTS) ;
    ops suivantes intactes (re-soumission) ; réseau/5xx → scheduleRetries (retried) sans
    dead-letter prématurée ; applied==0 && failed → rien commité, aucun compteur touché
  * refreshPermissions : delta monotone en mémoire lastPermissionCachedAt → `after`,
    upsert saveResourcePermission ; no-op sans token
- features/syncDevice.ts : le tick useSyncDevice() enchaîne SAF walk → pushPendingOps()
  → refreshPermissions(), tous deux gardés par hasAuthToken()
- tests/syncOutbox.test.ts : 12 cas (FIFO/limit, succès complet/partiel, applied=0,
  transitoire réseau + 5xx sans incrément, dead-letter après MAX_PENDING_ATTEMPTS,
  delta snapshot croisé) — npm run test:sync
- tests/e2e.live.test.ts : smoke real backend (register → push → relecture /files/folders →
  snapshot), skip si serveur down, SORTI de npm test via npm run test:e2e
- gates : npx tsc --noEmit + npm run test → 56/56 verts (24 db + 20 api + 12 sync)
This commit is contained in:
m
2026-09-10 20:23:55 +02:00
parent ea848fa9c7
commit 197c7db8a9
12 changed files with 657 additions and 5 deletions
+1
View File
@@ -52,6 +52,7 @@ cd mobile && npm run test:db
- `db/` — SQLite persistence, see `mobile/AGENTS.md` for the full contract (schema, migrations, repositories, tests) - `db/` — SQLite persistence, see `mobile/AGENTS.md` for the full contract (schema, migrations, repositories, tests)
- `localStorage.ts` — thin re-export of `services/db` (legacy alias) - `localStorage.ts` — thin re-export of `services/db` (legacy alias)
- `features/syncDevice.ts` — device sync orchestration (two-pass SAF walk, single transaction per root, `exists = 0` reconciliation) - `features/syncDevice.ts` — device sync orchestration (two-pass SAF walk, single transaction per root, `exists = 0` reconciliation)
- `features/syncOutbox.ts` — pulls `pending_operations` to `POST /sync/ops` (resume-at-`applied` index, transient retries never bump `attempts`) and `GET /sync/permissions` delta snapshot; both no-ops without an auth token
- `context/AuthContext.tsx` — session context: exposes `deviceUserId` (bootstrapped from `getDeviceUserId()`) and starts the background `syncDevice` loop - `context/AuthContext.tsx` — session context: exposes `deviceUserId` (bootstrapped from `getDeviceUserId()`) and starts the background `syncDevice` loop
- `app/` — expo-router screens: `index.tsx` (dossiers racines + ajout SAF), `folder/[id].tsx` (sous-dossiers + fichiers) - `app/` — expo-router screens: `index.tsx` (dossiers racines + ajout SAF), `folder/[id].tsx` (sous-dossiers + fichiers)
- `api/` — REST client (`client.ts` fetch wrapper + `types.ts` = contrat d'API : enveloppe `{ data, meta }`, erreurs `{ error: { code, message } }`) - `api/` — REST client (`client.ts` fetch wrapper + `types.ts` = contrat d'API : enveloppe `{ data, meta }`, erreurs `{ error: { code, message } }`)
+2 -2
View File
@@ -12,7 +12,7 @@ Références : `V2.md` (modèle cible), `mobile/services/db/` (conventions sync)
- JSON partout, sauf `POST /files/upload` (multipart). - JSON partout, sauf `POST /files/upload` (multipart).
- Enveloppe succès : `{ "data": T, "meta"?: { "page": int, "pageSize": int, "total": int } }` (`meta` présent sur les listes paginées). - Enveloppe succès : `{ "data": T, "meta"?: { "page": int, "pageSize": int, "total": int } }` (`meta` présent sur les listes paginées).
- Erreur : `{ "error": { "code": string, "message": string } }` + statut HTTP adéquat. - Erreur : `{ "error": { "code": string, "message": string } }` + statut HTTP adéquat.
- Côté client, toute réponse non-`2xx` est normalisée en `ApiError` : `code` du body si présent, sinon `HTTP_<status>` ; échec réseau → `NETWORK_ERROR`. - Côté client, toute réponse non-`2xx` est normalisée en `ApiError` : `code` du body si présent, sinon `HTTP_<status>` ; échec réseau → `NETWORK_ERROR`. Une réponse `2xx` mais dont le body n'est pas du JSON d'enveloppe valide (HTML, corps vide, JSON mal formé, absence de la clé `data`) → `INVALID_RESPONSE` (client-only).
## 2. Identité et identifiants (invariants) ## 2. Identité et identifiants (invariants)
@@ -133,4 +133,4 @@ type ResourcePermission = {
## 7. Codes d'erreur courants ## 7. Codes d'erreur courants
`NOT_FOUND`, `NOT_IMPLEMENTED` (501 temporaire sur les routes non construites — état actuel : **toutes les routes V1 sont réelles** : files CRUD/upload/search, folders, devices, health, sync/ops, sync/permissions, ocr/jobs), `FILE_TOO_LARGE` (413), `NAME_CONFLICT` (409 — même nom dans le même parent, cf. `UNIQUE(parent_id, name)`, **ou à la racine**, index partiel `(owner_id, name) WHERE parent_id IS NULL`), `NETWORK_ERROR` (côté client), `HTTP_<status>` (fallback). Statut `SERVICE_UNAVAILABLE` (503) si le backend n'est pas initialisé. `NOT_FOUND`, `NOT_IMPLEMENTED` (501 temporaire sur les routes non construites — état actuel : **toutes les routes V1 sont réelles** : files CRUD/upload/search, folders, devices, health, sync/ops, sync/permissions, ocr/jobs), `FILE_TOO_LARGE` (413), `NAME_CONFLICT` (409 — même nom dans le même parent, cf. `UNIQUE(parent_id, name)`, **ou à la racine**, index partiel `(owner_id, name) WHERE parent_id IS NULL`), `NETWORK_ERROR` (côté client), `INVALID_RESPONSE` (côté client — 2xx mais corps d'enveloppe invalide), `HTTP_<status>` (fallback). Statut `SERVICE_UNAVAILABLE` (503) si le backend n'est pas initialisé.
+5 -2
View File
@@ -18,10 +18,13 @@ Persistence is SQLite-backed via `services/db/` (`expo-sqlite`, database `dot.db
- Canonical identity: `resource_id` (opaque, unique) on folders/files/shares/share_links; `uri` (physical SAF path) is **nullable**, NULL = cloud-only; `owner_id` NOT NULL seeded from `device_user_id`. - Canonical identity: `resource_id` (opaque, unique) on folders/files/shares/share_links; `uri` (physical SAF path) is **nullable**, NULL = cloud-only; `owner_id` NOT NULL seeded from `device_user_id`.
- Folder and file per-row sync status: `local` | `cloud` | `local-cloud` (placement state, transitions via `transitionSyncStatus`). - Folder and file per-row sync status: `local` | `cloud` | `local-cloud` (placement state, transitions via `transitionSyncStatus`).
- `shares`/`share_links` have NO `sync_status`: their `pushStatus` (`pending`/`synced`/`failed`) is **derived** from `pending_operations` (`ref_type` = `share`|`share_link`, `ref_id`). - `shares`/`share_links` have NO `sync_status`: their `pushStatus` (`pending`/`synced`/`failed`) is **derived** from `pending_operations` (`ref_type` = `share`|`share_link`, `ref_id`).
- Query usage: `getFiles(folderResourceId?)`, `getFolders()`, `getFolderFolders(parentResourceId)`, `getFolder`/`getFile(resourceId)`, `saveFolder`/`saveDirectory`, `saveFile(file, folderResourceId)`, `removeFolder`/`removeFile(resourceId)`, `saveUserPreferences`/`getUserPreferences`/`getDeviceUserId`, `saveResourcePermission`/`getResourcePermission`, `canAccess(resourceId, type, level)`, `saveShare`/`getShares`/`removeShare`, `createShareLink`/`getShareLinks`/`incrementLinkDownloads`/`revokeShareLink`, `saveRecipient`/`getRecipients`, `enqueuePendingOperation`/`getNextQueuedOperation`/`markPendingOperation`. - Query usage: `getFiles(folderResourceId?)`, `getFolders()`, `getFolderFolders(parentResourceId)`, `getFolder`/`getFile(resourceId)`, `saveFolder`/`saveDirectory`, `saveFile(file, folderResourceId)`, `removeFolder`/`removeFile(resourceId)`, `saveUserPreferences`/`getUserPreferences`/`getDeviceUserId`, `saveResourcePermission`/`getResourcePermission`, `canAccess(resourceId, type, level)`, `saveShare`/`getShares`/`removeShare`, `createShareLink`/`getShareLinks`/`incrementLinkDownloads`/`revokeShareLink`, `saveRecipient`/`getRecipients`, `enqueuePendingOperation`/`getNextQueuedOperation`/`listQueuedOperations`/`scheduleRetries`/`markPendingOperation`/`MAX_PENDING_ATTEMPTS`.
- SAF walk (`features/syncDevice.ts`, `syncDevice()`/`syncRoot()`): **two passes** (all folders sorted by uri depth, then all files) inside a **single transaction** (`withTransaction`), receives sync: an interruption rolls back entirely. Reconciles by physical `uri`; rows under the root with a `uri` no longer seen are **marked `exists = 0`** (never deleted). Root folders are the rows with `parent_resource_id IS NULL` + non-null `uri`. - SAF walk (`features/syncDevice.ts`, `syncDevice()`/`syncRoot()`): **two passes** (all folders sorted by uri depth, then all files) inside a **single transaction** (`withTransaction`), receives sync: an interruption rolls back entirely. Reconciles by physical `uri`; rows under the root with a `uri` no longer seen are **marked `exists = 0`** (never deleted). Root folders are the rows with `parent_resource_id IS NULL` + non-null `uri`.
- Sync loop (`features/syncDevice.ts``useSyncDevice()`): after each SAF walk the same tick runs `features/syncOutbox.ts``pushPendingOps()` (outbox → `POST /sync/ops`) then `refreshPermissions()` (delta snapshot `GET /sync/permissions`). Both are **no-ops without an auth token** (`hasAuthToken()`, set after `POST /devices`).
- Outbox push semantics (`pushPendingOps`): batch = first `SYNC_BATCH_SIZE` (50) rows FIFO due (`next_retry_at <= now`) via `listQueuedOperations`. Server replies a **single `applied` index** (see `SyncResult` comment): ops `[0, applied)``markPendingOperation('completed')`; the op at `applied` when `failed` is non-null → `markPendingOperation('failed')` (backoff, dead-letter at `MAX_PENDING_ATTEMPTS`); ops after it stay `pending` and are re-sent next tick. **Transient** network/5xx errors → `scheduleRetries` bumps only `next_retry_at` (**never `attempts`**) — a failed batch is not dead-lettered prematurely; `applied == 0 && failed != null` means nothing was committed (1st op refused), no counters touched beyond the single backoff.
- Permissions snapshot (`refreshPermissions`): in-memory monotone `lastPermissionCachedAt` (max server `cachedAt`) becomes the `after` query param of the next call; rows are upserted via `saveResourcePermission` (24h TTL + read-only downgrade enforced by `canAccess`). Unit tests: `npm run test:sync`; live smoke (needs backend + postgres): `npm run test:e2e` (skips when the server is down, NOT part of `npm test`).
- Heavy processing stays server-side; SQLite only persists local metadata/state. - Heavy processing stays server-side; SQLite only persists local metadata/state.
# REST API client # REST API client
`api/client.ts` + `api/types.ts` = the client-side **API contract** (server must implement it; backend Go is the source of truth once built). Base URL = `EXPO_PUBLIC_API_BASE_URL` (défaut `http://localhost:8080/api/v1`). Envelope: success `{ data, meta?: { page, pageSize, total } }`, errors normalized to `ApiError` (`code` from `{ error: { code, message } }`, or `NETWORK_ERROR` / `HTTP_<status>`). Multipart upload needs the platform FormData (uri/name/type) — never set `Content-Type` manually. TanStack Query v5 providers live in `app/_layout.tsx`; hooks in `hooks/` (`useFiles`, `useSearch`, `useUpload`, OCR jobs). `api/client.ts` + `api/types.ts` = the client-side **API contract** (server must implement it; backend Go is the source of truth once built). Base URL = `EXPO_PUBLIC_API_BASE_URL` (défaut `http://localhost:8080/api/v1`). Envelope: success `{ data, meta?: { page, pageSize, total } }`, errors normalized to `ApiError` (`code` from `{ error: { code, message } }`, or `NETWORK_ERROR` / `HTTP_<status>` / `INVALID_RESPONSE` for 2xx bodies without a valid envelope). Multipart upload needs the platform FormData (uri/name/type) — never set `Content-Type` manually. TanStack Query v5 providers live in `app/_layout.tsx`; hooks in `hooks/` (`useFiles`, `useSearch`, `useUpload`, OCR jobs).
+17
View File
@@ -6,6 +6,9 @@ import type {
FolderDto, FolderDto,
ListFilesParams, ListFilesParams,
OcrJob, OcrJob,
ResourcePermission,
SyncOperation,
SyncResult,
} from './types'; } from './types';
const API_BASE_URL = process.env.EXPO_PUBLIC_API_BASE_URL ?? 'http://localhost:8080/api/v1'; const API_BASE_URL = process.env.EXPO_PUBLIC_API_BASE_URL ?? 'http://localhost:8080/api/v1';
@@ -18,6 +21,10 @@ export function setAuthToken(token: string | null): void {
authToken = token; authToken = token;
} }
export function hasAuthToken(): boolean {
return authToken !== null;
}
type QueryParams = Record<string, string | number | boolean | undefined | null>; type QueryParams = Record<string, string | number | boolean | undefined | null>;
function toQuery(params?: QueryParams): string { function toQuery(params?: QueryParams): string {
@@ -171,4 +178,14 @@ export const api = {
getOcrJob: (id: string) => getOcrJob: (id: string) =>
request<OcrJob>(`/ocr/jobs/${encodeURIComponent(id)}`), request<OcrJob>(`/ocr/jobs/${encodeURIComponent(id)}`),
syncOps: (operations: SyncOperation[]) =>
request<SyncResult>('/sync/ops', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ operations }),
}),
getSyncPermissions: (after?: number) =>
request<ResourcePermission[]>(`/sync/permissions${after != null ? `?after=${after}` : ''}`),
}; };
+39
View File
@@ -56,3 +56,42 @@ export type ListParams = {
export type ListFilesParams = ListParams & { export type ListFilesParams = ListParams & {
folderId?: string | null; folderId?: string | null;
}; };
// Miroir de docs/api-v1.md §6.1 — une entrée d'outbox côté serveur.
// `operation_id` = `pending_operations.id` local (INTEGER).
export type SyncOperation = {
operation_id: number;
ref_type?: 'resource' | 'share' | 'share_link' | null;
ref_id?: number | null;
resource_id: string;
resource_type: 'folder' | 'file';
operation: string;
payload: Record<string, unknown>;
};
// `applied` est un INDEX, pas un compte : nombre d'ops commitées depuis le
// début du batch.
// - succès → applied == operations.length, failed == null
// - échec → applied == index de l'op refusée (< length),
// failed = cette op ; reprise à operations.slice(applied)
// - applied == 0 → rien de commité (1re op refusée, `failed` non-null, ou
// batch vide) : tout est à rejouer, aucun compteur à
// incrémenter côté client.
// Invariant serveur : jamais applied == length avec failed non-null.
export type SyncResult = {
applied: number;
failed: { operation_id: number; code: string; message: string } | null;
};
// Miroir de docs/api-v1.md §6.2 — consommé tel quel par `saveResourcePermission`.
export type ResourcePermission = {
resource_id: string;
resourceType: 'folder' | 'file';
effectiveAccess: 'viewer' | 'commenter' | 'editor' | 'owner';
inherit: boolean;
ownerId: string | null;
sharedById: string | null;
expiresAt: number | null;
cachedAt: number;
updatedAt: number;
};
+11
View File
@@ -12,6 +12,7 @@ import {
type StoredFolder, type StoredFolder,
} from '../services/db'; } from '../services/db';
import type { SyncResult } from './syncDevice.types'; import type { SyncResult } from './syncDevice.types';
import { pushPendingOps, refreshPermissions } from './syncOutbox';
function uriDepth(uri: string): number { function uriDepth(uri: string): number {
return uri.split('/').length; return uri.split('/').length;
@@ -119,6 +120,16 @@ export async function useSyncDevice(intervalMs = 30_000): Promise<void> {
try { try {
const results = await syncDevice(); const results = await syncDevice();
console.info('syncDevice', JSON.stringify(results)); console.info('syncDevice', JSON.stringify(results));
// Puis pousser l'outbox (si un token est disponible) et rafraîchir le
// cache des permissions (delta).
const { pushed, retried } = await pushPendingOps();
if (pushed > 0 || retried > 0) {
console.info('pushPendingOps', JSON.stringify({ pushed, retried }));
}
const perms = await refreshPermissions();
if (perms > 0) {
console.info('refreshPermissions', perms);
}
} catch (error) { } catch (error) {
console.warn('syncDevice failed, retrying later', error); console.warn('syncDevice failed, retrying later', error);
} }
+90
View File
@@ -0,0 +1,90 @@
import { api, hasAuthToken } from '../api/client';
import type { SyncOperation } from '../api/types';
import {
listQueuedOperations,
markPendingOperation,
scheduleRetries,
saveResourcePermission,
MAX_PENDING_ATTEMPTS,
} from '../services/db';
import type { PendingOperation } from '../services/db/types';
// Nombre max d'opérations par push (pour ne pas dépasser la taille du body).
const SYNC_BATCH_SIZE = 50;
// Délai minimal avant de réessayer un batch complet après une erreur transitoire
// (réseau, proxy, 5xx). Aucun compteur d'attempts n'est incrémenté pour éviter
// les dead-letters prématurées.
const TRANSIENT_RETRY_MS = 15_000;
function toSyncOperation(op: PendingOperation): SyncOperation {
return {
operation_id: op.id,
ref_type: op.refType,
ref_id: op.refId,
resource_id: op.resourceId ?? '',
resource_type: op.resourceType ?? 'file',
operation: op.operation,
payload: (typeof op.payload === 'object' && op.payload !== null ? op.payload : {}) as Record<string, unknown>,
};
}
// ---- outbox push -----------------------------------------------------------
export type PushResult = { pushed: number; retried: number };
export async function pushPendingOps(): Promise<PushResult> {
if (!hasAuthToken()) return { pushed: 0, retried: 0 };
const queued = await listQueuedOperations(SYNC_BATCH_SIZE);
if (queued.length === 0) return { pushed: 0, retried: 0 };
let result;
try {
result = (await api.syncOps(queued.map(toSyncOperation))).data;
} catch {
// Erreur transitoire : repousser sans toucher aux attempts
// (aucun dead-letter prématuré).
const retryAt = Date.now() + TRANSIENT_RETRY_MS;
await scheduleRetries(queued.map((op) => op.id), retryAt);
return { pushed: 0, retried: queued.length };
}
// `applied` = INDEX : les opérations [0, applied) sont confirmées côté serveur.
const applied = Math.min(result.applied, queued.length);
for (const op of queued.slice(0, applied)) {
await markPendingOperation(op.id, 'completed');
}
if (result.failed && applied < queued.length) {
const failed = queued[applied];
await markPendingOperation(failed.id, 'failed', `${result.failed.code}: ${result.failed.message}`);
// Les opérations [applied+1, length) restent pending : le serveur ne les
// a pas reçues (il s'arrête à l'index) et elles seront resoumises au
// prochain tick.
}
return { pushed: applied, retried: 0 };
}
// ---- permissions snapshot --------------------------------------------------
// Delta monotone en mémoire (réinitialisé au démarrage de l'app) : la valeur
// max de cached_at du dernier snapshot sert de borne `after` pour le prochain.
let lastPermissionCachedAt: number | null = null;
export async function refreshPermissions(): Promise<number> {
if (!hasAuthToken()) return 0;
const perms = await api.getSyncPermissions(lastPermissionCachedAt ?? undefined);
if (perms.data.length === 0) return 0;
for (const p of perms.data) {
await saveResourcePermission(p);
}
lastPermissionCachedAt = Math.max(...perms.data.map((p) => p.cachedAt));
return perms.data.length;
}
// Visible uniquement pour les tests unitaires (reset de l'état en mémoire).
export function resetPermissionCachedAtForTests(): void {
lastPermissionCachedAt = null;
}
+3 -1
View File
@@ -36,7 +36,9 @@
"test:db": "tsx --test tests/migrations.test.ts tests/repositories.test.ts", "test:db": "tsx --test tests/migrations.test.ts tests/repositories.test.ts",
"test:migrations": "tsx --test tests/migrations.test.ts", "test:migrations": "tsx --test tests/migrations.test.ts",
"test:api": "tsx --test tests/apiClient.test.ts", "test:api": "tsx --test tests/apiClient.test.ts",
"test": "npm run test:db && npm run test:api" "test:sync": "tsx --test tests/syncOutbox.test.ts",
"test:e2e": "tsx --test tests/e2e.live.test.ts",
"test": "npm run test:db && npm run test:api && npm run test:sync"
}, },
"private": true "private": true
} }
+3
View File
@@ -36,5 +36,8 @@ export {
enqueuePendingOperation, enqueuePendingOperation,
getPendingOperations, getPendingOperations,
getNextQueuedOperation, getNextQueuedOperation,
listQueuedOperations,
scheduleRetries,
markPendingOperation, markPendingOperation,
MAX_PENDING_ATTEMPTS,
} from './pendingOps'; } from './pendingOps';
@@ -65,6 +65,37 @@ export async function getNextQueuedOperation(): Promise<PendingOperation | null>
return row ? toPendingOperation(row) : null; return row ? toPendingOperation(row) : null;
} }
// Même chose mais en batch (tri FIFO identique) — utilisé par pushPendingOps.
export async function listQueuedOperations(limit: number): Promise<PendingOperation[]> {
if (limit <= 0) return [];
const db = await getSession();
const rows = await db.getAllAsync<PendingOperationRow>(
`SELECT ${PENDING_OPERATION_COLUMNS} FROM pending_operations
WHERE status = 'pending'
AND (next_retry_at IS NULL OR next_retry_at <= ?)
ORDER BY created_at ASC, id ASC
LIMIT ?`,
Date.now(),
limit,
);
return rows.map(toPendingOperation);
}
// Reporter des opérations sans incrémenter attempts (utilisé en cas d'erreur
// transitoire réseau/serveur, pour éviter les dead-letters prématurées).
export async function scheduleRetries(ids: number[], retryAt: number): Promise<void> {
const db = await getSession();
const now = Date.now();
for (const id of ids) {
await db.runAsync(
`UPDATE pending_operations SET next_retry_at = ?, last_error_at = ? WHERE id = ?`,
retryAt,
now,
id,
);
}
}
export async function markPendingOperation( export async function markPendingOperation(
id: number, id: number,
status: PendingOperationStatus, status: PendingOperationStatus,
+121
View File
@@ -0,0 +1,121 @@
// Smoke test live contre un backend démarré (POSTGRES + serveur Go) :
// docker compose up postgres -d (racine repo)
// cd backend && go run cmd/server/main.go
// cd mobile && npm run test:e2e
//
// NON agrégé dans `npm test` (il exige un backend fonctionnel). Si le serveur
// est injoignable, les tests sont marqués skipped, pas échoués.
import { test, beforeEach, afterEach } from 'node:test';
import assert from 'node:assert/strict';
import Database from 'better-sqlite3';
import { MIGRATIONS, type MigrationDb } from '../services/db/migrations';
import type { DbSession } from '../services/db/session';
import { __setDbForTests } from '../services/db/session';
import { api, setAuthToken } from '../api/client';
import { enqueuePendingOperation, getPendingOperations } from '../services/db';
import { pushPendingOps, refreshPermissions, resetPermissionCachedAtForTests } from '../features/syncOutbox';
const BASE_URL = process.env.EXPO_PUBLIC_API_BASE_URL ?? 'http://localhost:8080/api/v1';
type Harness = MigrationDb & DbSession;
let h: Harness;
function createHarness(): Harness {
const sqlite = new Database(':memory:');
const harness: Harness = {
execAsync: async (sql: string) => {
sqlite.exec(sql);
},
runAsync: async (sql: string, ...params: unknown[]) => {
sqlite.prepare(sql).run(...params);
},
getFirstAsync: async (sql: string, ...params: unknown[]) =>
(sqlite.prepare(sql).get(...params) ?? null) as never,
getAllAsync: async (sql: string, ...params: unknown[]) =>
sqlite.prepare(sql).all(...params) as never,
withExclusiveTransactionAsync: async (task: (txn: Harness) => Promise<void>) => {
sqlite.exec('BEGIN');
try {
await task(harness);
sqlite.exec('COMMIT');
} catch (error) {
sqlite.exec('ROLLBACK');
throw error;
}
},
};
return harness;
}
let deviceId: string;
function newDeviceId(): string {
const bytes = new Uint8Array(16);
for (let i = 0; i < 16; i++) {
bytes[i] = Math.floor(Math.random() * 256);
}
return Array.from(bytes, (b) => b.toString(16).padStart(2, '0')).join('');
}
function isBackendUp(): boolean {
return !!process.env.CI_E2E_REQUIRE_BACKEND;
}
beforeEach(async () => {
h = createHarness();
for (const migration of MIGRATIONS) {
await migration.up(h);
}
__setDbForTests(h);
resetPermissionCachedAtForTests();
});
afterEach(() => {
__setDbForTests(null);
setAuthToken(null);
});
test('bout en bout : register → push outbox → snapshot → relecture serveur', async (t) => {
try {
await api.health();
} catch (error) {
if (!isBackendUp()) {
t.skip(`backend injoignable sur ${BASE_URL} — lancer serveur + postgres`);
return;
}
throw error;
}
deviceId = newDeviceId();
const reg = await api.registerDevice(deviceId);
setAuthToken(reg.data.token);
// 1. Enqueue un CREATE_RESOURCE puis push
const name = `e2e-${deviceId.slice(0, 8)}`;
const resourceId = newDeviceId();
const opId = await enqueuePendingOperation({
resourceId,
resourceType: 'folder',
refType: 'resource',
refId: null,
operation: 'create_resource',
payload: { name },
});
const push = await pushPendingOps();
assert.deepEqual(push, { pushed: 1, retried: 0 });
const local = await getPendingOperations();
const done = local.find((op) => op.id === opId);
assert.equal(done?.status, 'completed');
assert.equal(done?.attempts, 0);
// 2. Relecture côté serveur : le dossier racine créé est visible
const folders = await api.listFolders();
assert.ok(folders.data.some((f) => f.name === name), 'dossier absent côté serveur');
// 3. Snapshot des permissions → persisté localement
const snapshotCount = await refreshPermissions();
assert.ok(snapshotCount < 10_000, `snapshot trop volumineux (${snapshotCount})`);
});
+334
View File
@@ -0,0 +1,334 @@
import { test, beforeEach, afterEach } from 'node:test';
import assert from 'node:assert/strict';
import Database from 'better-sqlite3';
import { MIGRATIONS, type MigrationDb } from '../services/db/migrations';
import type { DbSession } from '../services/db/session';
import { __setDbForTests } from '../services/db/session';
import { api, setAuthToken } from '../api/client';
import {
enqueuePendingOperation,
getPendingOperations,
getResourcePermission,
listQueuedOperations,
markPendingOperation,
scheduleRetries,
MAX_PENDING_ATTEMPTS,
} from '../services/db';
import {
pushPendingOps,
refreshPermissions,
resetPermissionCachedAtForTests,
} from '../features/syncOutbox';
import type { PendingOperationType } from '../services/db/types';
type Harness = MigrationDb & DbSession;
let h: Harness;
function createHarness(): Harness {
const sqlite = new Database(':memory:');
const harness: Harness = {
execAsync: async (sql: string) => {
sqlite.exec(sql);
},
runAsync: async (sql: string, ...params: unknown[]) => {
sqlite.prepare(sql).run(...params);
},
getFirstAsync: async (sql: string, ...params: unknown[]) =>
(sqlite.prepare(sql).get(...params) ?? null) as never,
getAllAsync: async (sql: string, ...params: unknown[]) =>
sqlite.prepare(sql).all(...params) as never,
withExclusiveTransactionAsync: async (task: (txn: Harness) => Promise<void>) => {
sqlite.exec('BEGIN');
try {
await task(harness);
sqlite.exec('COMMIT');
} catch (error) {
sqlite.exec('ROLLBACK');
throw error;
}
},
};
return harness;
}
function jsonResponse(status: number, body: unknown): Response {
return new Response(JSON.stringify(body), {
status,
headers: { 'Content-Type': 'application/json' },
});
}
function ok(data: unknown): Response {
return jsonResponse(200, { data });
}
let calls: { url: string; init: RequestInit }[] = [];
const originalFetch = globalThis.fetch;
function stubFetch(handler: (url: string, init: RequestInit) => Promise<Response> | Response): void {
globalThis.fetch = async (input: RequestInfo | URL, init?: RequestInit) => {
const initSafe = init ?? {};
calls.push({ url: String(input), init: initSafe });
return handler(String(input), initSafe);
};
}
function pathOf(url: string): string {
const u = new URL(url);
return u.pathname + u.search;
}
beforeEach(async () => {
h = createHarness();
for (const migration of MIGRATIONS) {
await migration.up(h);
}
__setDbForTests(h);
calls = [];
setAuthToken('v4.local.test-token');
resetPermissionCachedAtForTests();
stubFetch(() => ok({ applied: 0, failed: null }));
});
afterEach(() => {
__setDbForTests(null);
globalThis.fetch = originalFetch;
});
async function enqueue(op: {
operation: PendingOperationType;
resourceId?: string;
payload?: Record<string, unknown>;
}) {
return enqueuePendingOperation({
resourceId: op.resourceId ?? 'a'.repeat(32),
resourceType: 'folder',
refType: 'resource',
refId: 7,
operation: op.operation,
payload: op.payload ?? { name: 'Dossier' },
});
}
test('sans token : aucun push, aucune requête réseau', async () => {
setAuthToken(null);
await enqueue({ operation: 'create_resource' });
const result = await pushPendingOps();
assert.deepEqual(result, { pushed: 0, retried: 0 });
assert.equal(calls.length, 0);
});
test('queue vide : aucune requête réseau', async () => {
const result = await pushPendingOps();
assert.deepEqual(result, { pushed: 0, retried: 0 });
assert.equal(calls.length, 0);
});
test('succès complet : toutes les ops passent à completed, body conforme', async () => {
const id1 = await enqueue({ operation: 'create_resource', payload: { name: 'A' } });
const id2 = await enqueue({ operation: 'create_resource', payload: { name: 'B' } });
stubFetch(() => ok({ applied: 2, failed: null }));
const result = await pushPendingOps();
assert.deepEqual(result, { pushed: 2, retried: 0 });
assert.equal(pathOf(calls[0].url), '/api/v1/sync/ops');
assert.equal(calls[0].init.method, 'POST');
const body = JSON.parse(String(calls[0].init.body));
assert.equal(body.operations.length, 2);
assert.equal(body.operations[0].operation_id, id1);
assert.equal(body.operations[0].resource_type, 'folder');
assert.deepEqual(body.operations[0].payload, { name: 'A' });
assert.equal(body.operations[1].operation_id, id2);
const ops = await getPendingOperations();
const byId = new Map(ops.map((op) => [op.id, op]));
assert.equal(byId.get(id1)?.status, 'completed');
assert.equal(byId.get(id2)?.status, 'completed');
assert.equal(byId.get(id1)?.attempts, 0);
});
test('succès partiel : reprise exacte à lindex applied (les suivantes intactes)', async () => {
const id1 = await enqueue({ operation: 'create_resource' });
const id2 = await enqueue({ operation: 'create_resource' });
const id3 = await enqueue({ operation: 'create_resource' });
stubFetch(() =>
ok({
applied: 1,
failed: { operation_id: id2, code: 'NAME_CONFLICT', message: 'exists déjà' },
}),
);
const result = await pushPendingOps();
assert.deepEqual(result, { pushed: 1, retried: 0 });
const ops = await getPendingOperations();
const byId = new Map(ops.map((op) => [op.id, op]));
assert.equal(byId.get(id1)?.status, 'completed');
assert.equal(byId.get(id2)?.status, 'pending');
assert.equal(byId.get(id2)?.attempts, 1); // backoff programmé, par dead-letter
assert.ok((byId.get(id2)?.nextRetryAt ?? 0) > 0);
assert.match(byId.get(id2)?.error ?? '', /NAME_CONFLICT/);
assert.equal(byId.get(id3)?.status, 'pending');
assert.equal(byId.get(id3)?.attempts, 0); // jamais soumise → intacte
});
test('applied = 0 + échec : rien commité, pas de dead-letter prématurée', async () => {
const id1 = await enqueue({ operation: 'create_resource' });
stubFetch(() =>
ok({ applied: 0, failed: { operation_id: id1, code: 'INVALID_REQUEST', message: 'payload invalide' } }),
);
const result = await pushPendingOps();
assert.deepEqual(result, { pushed: 0, retried: 0 });
const [op] = await getPendingOperations();
assert.equal(op.status, 'pending');
assert.equal(op.attempts, 1); // < MAX_PENDING_ATTEMPTS, le cycle recommence
assert.ok((op.nextRetryAt ?? 0) > 0);
});
test('échec transitoire (fetch rejette) : retrié sans incrémenter attempts', async () => {
const id1 = await enqueue({ operation: 'create_resource' });
const id2 = await enqueue({ operation: 'create_resource' });
stubFetch(() => {
throw new Error('network unreachable');
});
const result = await pushPendingOps();
assert.deepEqual(result, { pushed: 0, retried: 2 });
const ops = await getPendingOperations();
for (const op of ops) {
assert.equal(op.status, 'pending');
assert.equal(op.attempts, 0);
assert.ok((op.nextRetryAt ?? 0) > Date.now());
}
});
test('échec transitoire 5xx : idem (HTTP_500 → retry, pas de dead-letter)', async () => {
const id1 = await enqueue({ operation: 'create_resource' });
stubFetch(() => jsonResponse(500, { error: { code: 'INTERNAL', message: 'boom' } }));
const result = await pushPendingOps();
assert.deepEqual(result, { pushed: 0, retried: 1 });
const [op] = await getPendingOperations();
assert.equal(op.status, 'pending');
assert.equal(op.attempts, 0);
});
test('refus terminal répété : dead-letter après MAX_PENDING_ATTEMPTS', async () => {
const id1 = await enqueue({ operation: 'create_resource' });
stubFetch(() =>
ok({ applied: 0, failed: { operation_id: id1, code: 'NAME_CONFLICT', message: 'dup' } }),
);
for (let i = 0; i < MAX_PENDING_ATTEMPTS; i++) {
await pushPendingOps();
// le backoff repousse next_retry_at : on force le reset pour re-soumettre
await h.runAsync('UPDATE pending_operations SET next_retry_at = NULL');
}
const [op] = await getPendingOperations();
assert.equal(op.status, 'failed');
assert.equal(op.attempts, MAX_PENDING_ATTEMPTS);
assert.equal(op.nextRetryAt, null); // plus jamais rejouée
});
test('listQueuedOperations honore le tri FIFO et le LIMIT', async () => {
const id1 = await enqueue({ operation: 'create_resource' });
const id2 = await enqueue({ operation: 'create_resource' });
const id3 = await enqueue({ operation: 'create_resource' });
const first = await listQueuedOperations(2);
assert.deepEqual(first.map((op) => op.id), [id1, id2]);
const later = Date.now() + 60_000;
await scheduleRetries([id2], later);
const due = await listQueuedOperations(10);
assert.deepEqual(due.map((op) => op.id), [id1, id3]); // id2 pas encore due
});
test('scheduleRetries ne change pas attempts (différent de markPendingOperation failed)', async () => {
const id1 = await enqueue({ operation: 'create_resource' });
await scheduleRetries([id1], Date.now() + 1);
const [op] = await getPendingOperations();
assert.equal(op.attempts, 0);
assert.ok((op.nextRetryAt ?? 0) > 0);
});
test('refreshPermissions : delta après = max cachedAt du snapshot', async () => {
const perms = [
{
resource_id: 'a'.repeat(32),
resourceType: 'folder',
effectiveAccess: 'owner',
inherit: true,
ownerId: 'u1',
sharedById: null,
expiresAt: null,
cachedAt: 1000,
updatedAt: 1000,
},
{
resource_id: 'b'.repeat(32),
resourceType: 'file',
effectiveAccess: 'viewer',
inherit: false,
ownerId: 'u2',
sharedById: 'u1',
expiresAt: 5,
cachedAt: 2000,
updatedAt: 2000,
},
];
stubFetch((url) => (new URL(url).searchParams.get('after') === null ? ok(perms) : ok([])));
const n1 = await refreshPermissions();
assert.equal(n1, 2);
assert.equal(pathOf(calls[0].url), '/api/v1/sync/permissions');
assert.equal((await getResourcePermission('a'.repeat(32), 'folder'))?.effectiveAccess, 'owner');
assert.equal((await getResourcePermission('b'.repeat(32), 'file'))?.expiresAt, 5);
// 2e tick (snapshot vide) → after = max cachedAt du 1er
const n2 = await refreshPermissions();
assert.equal(n2, 0);
assert.equal(pathOf(calls[1].url), '/api/v1/sync/permissions?after=2000');
});
test('refreshPermissions garde la référence dans le temps pour after', async () => {
const first = {
resource_id: 'c'.repeat(32),
resourceType: 'folder',
effectiveAccess: 'editor',
inherit: true,
ownerId: 'u1',
sharedById: null,
expiresAt: null,
cachedAt: 42,
updatedAt: 42,
};
const second = {
resource_id: 'd'.repeat(32),
resourceType: 'folder',
effectiveAccess: 'viewer',
inherit: false,
ownerId: 'u3',
sharedById: 'u1',
expiresAt: null,
cachedAt: 84,
updatedAt: 84,
};
stubFetch((url) => {
const after = new URL(url).searchParams.get('after');
return ok(after === null ? [first] : [second]);
});
await refreshPermissions();
await refreshPermissions();
assert.equal((await getResourcePermission('d'.repeat(32), 'folder'))?.effectiveAccess, 'viewer');
assert.equal(pathOf(calls[1].url), '/api/v1/sync/permissions?after=42');
});