V3 S4 : recherche FTS + facettes (serveur + mobile)

Backend : search_tsv french + GIN (000020), SearchDocumentsVisible
(nom + TS + faits, rang nom > TS > fait sûr > fait douteux, jamais
d'exclusion), facettes issuer/type/montant/période sur
GET /files/search (400 si malformée). Tests repo + handlers + migrations.

Mobile (Room v21) : FTS4 files_fts + triggers partagés
(fresh-install/migration, entité déclarée pour KSP, SQL aligné 21.json),
searchDocuments offline (même rang, accents préservés) + UI filtres
(émetteur, types chips, montants, période ISO, compteur, effacer).

Contrat docs/api-v1.md, V3.md (S4 livré, migrations renumérotées
links->000021, token->000022).
This commit is contained in:
venture
2026-09-26 10:53:56 +02:00
parent ac8bd9acb4
commit 2d984fd665
25 changed files with 2362 additions and 25 deletions
+1
View File
@@ -12,6 +12,7 @@ Le format suit [Keep a Changelog](https://keepachangelog.com/fr/1.1.0/) et le ve
- Migrations `000017` (index corbeille) + `000018` (versions), contrat `docs/api-v1.md` §3/§6.1 étendu, test de contrat `router_test.go`.
- Extract générique V3 (S2 backend) : package `extract` (règles FR stdlib — dates/échéances/périodes/montants/émetteurs/références masquées/`doc_type` suggéré, confiances, corpus testé), table `resource_facts` (migration `000019`, `origin` auto/user), hook worker OCR (`OnText` + backfill boot borné, best-effort, anti-panic), `GET /files/:id/facts` (viewer+, `[]` si vide, 404 si trashé/étranger).
- Corrections V3 (S3) : `PATCH /files/:id/facts` + op `correct_facts` (full-replace du jeu `user`, editor+, validation Go → 400 jamais 500), Room v20 (`resource_facts` + `FactsDao`, migration 19→20), fiche « Ce qu'on a compris » sur le viewer (chips, confiance grisée + suggéré, édition par type, reset auto), `FactsRepository` (sync remplace-en-bloc, correction optimiste + outbox atomique), `GET /files/:id/facts` consommé côté client.
- Recherche V3 (S4) : colonne `search_tsv` (migration `000020`, `french`, index GIN) + `SearchDocumentsVisible` (nom LIKE + TS + faits, rang nom > TS > fait sûr > fait douteux, facettes émetteur/type/montant/période), `GET /files/search` étendu (400 si facette malformée), Room v21 (FTS4 `files_fts` + triggers partagés fresh-install/migration, entité déclarée pour KSP), `searchDocuments` offline (même rang, accents préservés) + UI filtres (émetteur, types chips, montants, période ISO, compteur, effacer).
- Corbeille V3 (S1 mobile, Room v19) : colonne `trashed_at` (files + folders, migration 18→19), tombstone `exists = 0` (requêtes `exists = 1` inchangées), `FileDeleter`/`FolderDeleter`/`SafFileDeleter` → ops `trash_resource` (plus de `delete_resource` émis), garde walk anti-résurrection + reconcile, `TrashRepository` (restore/purge/convergence `GET /trash`/expiration locale 31j), écran Corbeille (Réglages → restore/purge avec confirmation), expiration tombstones dans `RetentionWorker`, dialogue de suppression rewordé (corbeille 30j, plus « irréversible »).
## [1.1.0] - 2026-09-25
+9 -7
View File
@@ -51,10 +51,12 @@ Sans ces 4 items, les piliers grincent. Ils sont petits et intégrés aux slices
- `000017_trash` : rien à créer — ajouter index partiel `idx_resources_deleted_at ON resources(deleted_at) WHERE deleted_at IS NOT NULL` + colonne `trashed_at TIMESTAMPTZ` (NULL = vivant ; purge = `trashed_at < NOW()-30j`). Le `DELETE` actuel devient trash (soft), pas de breaking change API.
- `000018_versions` : table `resource_versions (id 32-hex PK, resource_id FK CASCADE, version_no INT, size_bytes BIGINT, content_hash TEXT, stored_path TEXT, created_at)` + `UNIQUE(resource_id, version_no)`. À chaque overwrite d'octets (upload ciblé), la copie précédente est poussée en version (garder N=3, purger la plus vieille).
- `000019_facts` : table `resource_facts (id 32-hex PK, resource_id FK CASCADE, fact_type TEXT CHECK IN ('date','due_date','period','amount','issuer','reference','doc_type'), value_text TEXT, value_num NUMERIC NULL, value_date DATE NULL, confidence REAL CHECK 0..1, origin TEXT CHECK IN ('auto','user'), created_at, updated_at)` + index `(resource_id)`, `(fact_type, value_text)`. `doc_type` n'est qu'un fait comme un autre (suggestion, pas statut).
- `000020_links` : table `resource_links (id 32-hex PK, from_id FK CASCADE, to_id FK CASCADE, link_type TEXT CHECK IN ('same_scan','duplicate_of','version_of','same_issuer','same_reference'), confidence REAL, created_by TEXT CHECK IN ('auto','user'), CHECK(from_id != to_id), UNIQUE(from_id,to_id,link_type))`.
- `000021_token_version` : `ALTER TABLE users ADD COLUMN token_version INT NOT NULL DEFAULT 0` — `PATCH /users/me/password` bump ; claim paseto `tv`, middleware `RequireAuth` rejette `tv != users.token_version` (lève la limite V1 « tokens non révoqués »).
- `000022_search_fts` : colonne `search_tsv TSVECTOR` sur `resources` (générée : `name + ocr_text` ; les facts sont joints à la requête, pas dupliqués) + index GIN + trigger `UPDATE` sur `name/ocr_text`. Fait après P2 pour ne pas indexer du vide.
- `000023_quota` (A2) : `ALTER TABLE users ADD COLUMN storage_quota_bytes BIGINT NOT NULL DEFAULT 10737418240` (10 Go, V2.md §3.1) + vue/fonction `user_used_bytes(user_id)` (somme `size_bytes`hors trash). Check à l'upload et à l'overwrite versionnée → `413 QUOTA_EXCEEDED` (nouveau code §7).
- `000020_search_fts` (livré S4) : colonne `search_tsv TSVECTOR` générée (`name + ocr_text`, `french`) + index GIN ; faits joints à la requête (rang, jamais d'exclusion).
- `000021_links` (S5) : table `resource_links (id 32-hex PK, from_id FK CASCADE, to_id FK CASCADE, link_type TEXT CHECK IN ('same_scan','duplicate_of','version_of','same_issuer','same_reference'), confidence REAL, created_by TEXT CHECK IN ('auto','user'), CHECK(from_id != to_id), UNIQUE(from_id,to_id,link_type))`.
- `000022_token_version` (S6) : `ALTER TABLE users ADD COLUMN token_version INT NOT NULL DEFAULT 0` — `PATCH /users/me/password` bump ; claim paseto `tv`, middleware `RequireAuth` rejette `tv != users.token_version` (lève la limite V1 « tokens non révoqués »).
- `000023_quota` (A2, S6) : `ALTER TABLE users ADD COLUMN storage_quota_bytes BIGINT NOT NULL DEFAULT 10737418240` (10 Go, V2.md §3.1) + vue/fonction `user_used_bytes(user_id)` (somme `size_bytes`hors trash). Check à l'upload et à l'overwrite versionnée → `413 QUOTA_EXCEEDED` (nouveau code §7).
> Ordre imposé par golang-migrate (montée stricte, pas de trous) : FTS a pris `000020` en S4, décalant links→`000021` et token→`000022`.
### 3.2 Extract — nouveau worker (miroir `service/ocr.go`, `cmd/server/main.go`)
@@ -108,9 +110,9 @@ Sans ces 4 items, les piliers grincent. Ils sont petits et intégrés aux slices
1. **S1 — Filet + A3** (livré : BE `000017/000018` + trash/restore/purge + cron purge 30j + mobile v19 + `FileDeleter` → trash + gardes walk + écran Corbeille) — done quand : supprimer → restaure depuis un 2e device, le walk ne ressuscite pas.
2. **S2 — Extract générique** (livré : `extract/rules.go` + `service/extract.go` + `000019` + `UpsertAuto` + `GET /files/:id/facts`) — done quand : un scan EDF propose émetteur/montant/échéance.
3. **S3 — Fiche + correction** (livré : `PATCH /files/:id/facts` + op `correct_facts` editor+ + Room v20 facts + fiche viewer + `FactsRepository` + B4 a11y fiche) — done quand : corriger en un tap converge serveur ↔ 2e device.
4. **S4 — Recherche** (`000022` FTS + facettes serveur + Room FTS v21 + barre à chips + B4 a11y facettes) — done quand : « facture edf mars » trouve `scan_042.pdf` offline ET online.
5. **S5 — Relations + B2** (`000020` + `duplicate/same_issuer` auto + `same_scan` via `scanSessionId` + fiche « Voir aussi » + collections locales + écran Doublons « garder un seul ») — done quand : fiche CNI → verso + doublon listés, invalidables ; doublons chiffrés en Mo récupérables.
6. **S6 — Partage sobre + A1/A2/A4 + B1 (+B5 si temps)** (expiration défaut 7j + `000021` token_version + `000023` quota + `413 QUOTA_EXCEEDED` + `GET /devices` + écran Appareils + `update_metadata{baseUpdatedAt}` + flag `conflict` + import « classeur » unifié B1 + onboarding cercle B5) — done quand : lien RIB expiré → `NOT_FOUND`, password changé → autres devices re-logués, device perdu révoqué, quota plein → 413 propre, rename concurrent → badge « renommé ailleurs ».
4. **S4 — Recherche** (livré : `000020` FTS + facettes serveur + Room FTS v21 + UI filtres + B4 a11y) — done quand : « facture edf mars » trouve `scan_042.pdf` offline ET online.
5. **S5 — Relations + B2** (`000021` links + `duplicate/same_issuer` auto + `same_scan` via `scanSessionId` + fiche « Voir aussi » + collections locales + écran Doublons « garder un seul ») — done quand : fiche CNI → verso + doublon listés, invalidables ; doublons chiffrés en Mo récupérables.
6. **S6 — Partage sobre + A1/A2/A4 + B1 (+B5 si temps)** (expiration défaut 7j + `000022` token_version + `000023` quota + `413 QUOTA_EXCEEDED` + `GET /devices` + écran Appareils + `update_metadata{baseUpdatedAt}` + flag `conflict` + import « classeur » unifié B1 + onboarding cercle B5) — done quand : lien RIB expiré → `NOT_FOUND`, password changé → autres devices re-logués, device perdu révoqué, quota plein → 413 propre, rename concurrent → badge « renommé ailleurs ».
7. **S7 — Preview 1ère page** (`pdf_preview` V2.md §3.3, une seule taille) + **C3 export chiffré** (`GET /export` zip + manifest) + B3 impression si temps + polish + release 1.2.0. Arbitrage stretch : **export > embeddings** (vrai 3-2-1 perçu, zéro GPU).
## 6. Tests (mêmes harnais, pas de nouveau framework)
@@ -0,0 +1,2 @@
DROP INDEX IF EXISTS idx_resources_search_tsv;
ALTER TABLE resources DROP COLUMN IF EXISTS search_tsv;
@@ -0,0 +1,13 @@
-- V3 recherche (S4) : plein-texte français sur (nom + OCR).
-- Colonne générée stockée (`to_tsvector(regconfig, text)` est IMMUTABLE,
-- donc autorisée en GENERATED) : pas de trigger, backfill automatique des
-- lignes existantes. Index GIN. Les faits (`resource_facts`) sont joints à
-- la requête (jamais dupliqués) : ils pondèrent le rang sans jamais exclure
-- (humble par design — une confiance < 0.5 déclasse, cf. SearchDocuments).
ALTER TABLE resources
ADD COLUMN search_tsv TSVECTOR
GENERATED ALWAYS AS (
to_tsvector('french', COALESCE(name, '') || ' ' || COALESCE(ocr_text, ''))
) STORED;
CREATE INDEX IF NOT EXISTS idx_resources_search_tsv ON resources USING GIN (search_tsv);
+25
View File
@@ -422,6 +422,31 @@ func TestMigrationsUpDown(t *testing.T) {
t.Error("CHECK fact_type manquant sur resource_facts après 000019")
}
// --- 000020 : plein-texte ----------------------------------------------
var tsvCol int
err = conn.QueryRow(`
SELECT COUNT(*) FROM information_schema.columns
WHERE table_schema = 'public' AND table_name = 'resources' AND column_name = 'search_tsv'`).Scan(&tsvCol)
if err != nil {
t.Fatalf("search_tsv column: %v", err)
}
if tsvCol == 0 {
t.Error("colonne resources.search_tsv manquante après 000020")
}
var tsvIdx int
err = conn.QueryRow(`
SELECT COUNT(*) FROM pg_index i
JOIN pg_class t ON t.oid = i.indrelid
WHERE t.relname = 'resources' AND i.indisvalid
AND pg_get_indexdef(i.indexrelid) LIKE '%idx_resources_search_tsv%'`).Scan(&tsvIdx)
if err != nil {
t.Fatalf("000020 index: %v", err)
}
if tsvIdx == 0 {
t.Error("index idx_resources_search_tsv manquant après 000020")
}
if err := MigrateDownDatabase(url); err != nil {
t.Fatalf("migrate down: %v", err)
}
+46 -1
View File
@@ -5,9 +5,11 @@ import (
"net/http"
"strconv"
"strings"
"time"
"github.com/gin-gonic/gin"
"github.com/kazier/backend/pkg/api"
"github.com/kazier/backend/repository"
"github.com/kazier/backend/service"
)
@@ -408,10 +410,53 @@ func FilesSearch(c *gin.Context) {
if !ok {
return
}
files, total, err := Store.SearchFiles(userID, q, page, pageSize)
facets, ok := parseSearchFacets(c)
if !ok {
return
}
files, total, err := Store.SearchFiles(userID, q, page, pageSize, facets)
if err != nil {
writeError(c, err)
return
}
api.OKList(c, files, page, pageSize, total)
}
// parseSearchFacets lit les facettes V3 (`issuer`, `from`/`to` ISO,
// `min_amount`/`max_amount` numériques, `doc_type`) — 400 si malformées.
func parseSearchFacets(c *gin.Context) (repository.SearchFacets, bool) {
var facets repository.SearchFacets
facets.Issuer = strings.TrimSpace(c.Query("issuer"))
facets.DocType = strings.TrimSpace(c.Query("doc_type"))
if v := strings.TrimSpace(c.Query("from")); v != "" {
if _, err := time.Parse("2006-01-02", v); err != nil {
api.Error(c, http.StatusBadRequest, "INVALID_REQUEST", "query param `from` must be ISO YYYY-MM-DD")
return facets, false
}
facets.From = v
}
if v := strings.TrimSpace(c.Query("to")); v != "" {
if _, err := time.Parse("2006-01-02", v); err != nil {
api.Error(c, http.StatusBadRequest, "INVALID_REQUEST", "query param `to` must be ISO YYYY-MM-DD")
return facets, false
}
facets.To = v
}
if v := strings.TrimSpace(c.Query("min_amount")); v != "" {
f, err := strconv.ParseFloat(strings.ReplaceAll(v, ",", "."), 64)
if err != nil {
api.Error(c, http.StatusBadRequest, "INVALID_REQUEST", "query param `min_amount` must be numeric")
return facets, false
}
facets.MinAmount = &f
}
if v := strings.TrimSpace(c.Query("max_amount")); v != "" {
f, err := strconv.ParseFloat(strings.ReplaceAll(v, ",", "."), 64)
if err != nil {
api.Error(c, http.StatusBadRequest, "INVALID_REQUEST", "query param `max_amount` must be numeric")
return facets, false
}
facets.MaxAmount = &f
}
return facets, true
}
+77
View File
@@ -360,6 +360,83 @@ func TestSearchFiles(t *testing.T) {
}
}
// TestSearchFacets : filtres V3 sur faits + 400 si malformés.
func TestSearchFacets(t *testing.T) {
r, _, repo := setup(t)
device := repository.NewID()
token, user := registerAndLogin(t, r, repo, testUserUsername(device, "sf"), "search-test-password", device)
edfID := repository.NewID()
if err := repo.Resources.InsertFile(user, edfID, "scan_1.pdf", "", 100, nil, nil); err != nil {
t.Fatalf("insert: %v", err)
}
if err := repo.Resources.UpdateOcrText(user, edfID, "Facture EDF de mars"); err != nil {
t.Fatalf("ocr: %v", err)
}
num := 84.20
if err := repo.Resources.UpsertAutoFacts(edfID, []repository.FactParam{
{Type: "issuer", ValueText: "EDF", Confidence: 0.9},
{Type: "amount", ValueText: "84.20 €", ValueNum: &num, Confidence: 0.9},
{Type: "doc_type", ValueText: "facture", Confidence: 0.8},
}); err != nil {
t.Fatalf("facts: %v", err)
}
otherID := repository.NewID()
if err := repo.Resources.InsertFile(user, otherID, "scan_2.pdf", "", 100, nil, nil); err != nil {
t.Fatalf("insert: %v", err)
}
get := func(query string) []fileDTO {
t.Helper()
rec, _ := doRequest(t, r, http.MethodGet, "/api/v1/files/search"+query, token, nil, "")
env := expectOK(t, rec, "search-facets:"+query)
var files []fileDTO
if err := json.Unmarshal(env.Data, &files); err != nil {
t.Fatalf("unmarshal %s: %v", query, err)
}
return files
}
// FTS via OCR (nom opaque).
if files := get("?q=EDF"); len(files) == 0 || files[0].ID != edfID {
t.Errorf("q=EDF via OCR: %+v", files)
}
// Facette émetteur : filtre.
if files := get("?q=facture&issuer=edf"); len(files) != 1 || files[0].ID != edfID {
t.Errorf("issuer=edf: %+v", files)
}
if files := get("?q=facture&issuer=engie"); len(files) != 0 {
t.Errorf("issuer=engie doit filtrer: %+v", files)
}
// Facette montant.
if files := get("?q=facture&min_amount=100"); len(files) != 0 {
t.Errorf("min_amount=100 doit exclure 84.20: %+v", files)
}
if files := get("?q=facture&max_amount=100"); len(files) != 1 {
t.Errorf("max_amount=100: %+v", files)
}
// Facette doc_type.
if files := get("?q=scan&doc_type=facture"); len(files) != 1 || files[0].ID != edfID {
t.Errorf("doc_type=facture: %+v", files)
}
// Malformées → 400.
for _, query := range []string{
"?q=facture&from=12/03/2026",
"?q=facture&to=demain",
"?q=facture&min_amount=beaucoup",
"?q=facture&max_amount=",
} {
// max_amount vide = absent (pas d'erreur) — seul le cas non-vide
// non-numérique est rejeté ; on l'exclut ici.
if query == "?q=facture&max_amount=" {
continue
}
rec, _ := doRequest(t, r, http.MethodGet, "/api/v1/files/search"+query, token, nil, "")
expectError(t, rec, http.StatusBadRequest, "INVALID_REQUEST", "search-facets-400:"+query)
}
}
func TestSearchFilesEscapesPercent(t *testing.T) {
r, _, repo := setup(t)
device := repository.NewID()
+169
View File
@@ -0,0 +1,169 @@
package repository
import (
"fmt"
"strings"
)
// SearchFacets filters document search on extracted facts (V3 S4).
// Empty = no filter. From/To are ISO dates (YYYY-MM-DD) applied to
// date/due_date facts and overlapped with period facts (YYYY-MM).
type SearchFacets struct {
Issuer string
From, To string
MinAmount, MaxAmount *float64
DocType string
}
// searchFileColumns mirrors fileColumns qualified for `resources r`.
const searchFileColumns = `r.resource_id, r.name, r.size_bytes, COALESCE(r.mime_type, ''), COALESCE(r.parent_id, ''), r.created_at, r.updated_at, COALESCE(r.ocr_text, ''), COALESCE(r.has_content, FALSE)`
// SearchDocumentsVisible returns the user's accessible files (owned or
// shared, viewer+) matching q across name (ILIKE, compat), full-text
// (french TS, name+OCR) and extracted facts (ILIKE on value_text),
// optionally narrowed by facets. Ranked: TS rank + fact bonus (high-
// confidence fact match +1.0, low-confidence +0.5 — humble by design :
// low-confidence demotes, never excludes), then name.
func (r *Resources) SearchDocumentsVisible(userID, q string, facets SearchFacets, limit, offset int) ([]FileRow, int, error) {
where, order, args := r.searchDocumentsClauses(userID, q, facets)
base := `WITH RECURSIVE ` + grantedCTE + ` SELECT %s FROM resources r WHERE ` + where
var total int
if err := r.DB.QueryRow(fmt.Sprintf(base, "COUNT(*)"), args...).Scan(&total); err != nil {
return nil, 0, err
}
rows, err := r.DB.Query(
fmt.Sprintf(base+` ORDER BY %s, r.name ASC LIMIT $%d OFFSET $%d`, searchFileColumns, order, len(args)+1, len(args)+2),
append(args, limit, offset)...,
)
if err != nil {
return nil, 0, err
}
defer rows.Close()
files := make([]FileRow, 0)
for rows.Next() {
row, err := r.scanFile(rows.Scan)
if err != nil {
return nil, 0, err
}
files = append(files, row)
}
return files, total, rows.Err()
}
// searchDocumentsClauses builds the WHERE (match) and ORDER BY (rank)
// fragments plus bound args. $1 is always userID (grantedCTE).
func (r *Resources) searchDocumentsClauses(userID, q string, facets SearchFacets) (where, order string, args []any) {
args = []any{userID}
next := 2
ph := func(v any) string {
args = append(args, v)
p := fmt.Sprintf("$%d", next)
next++
return p
}
// plainto_tsquery placeholder partagé match/rang (le COUNT ne voit que
// le WHERE : aucun placeholder ne doit n'exister que dans le ORDER BY).
// Même règle pour le pattern LIKE du nom (bonus de rang).
tsPh := ph(q)
namePh := ph(`%` + escapeLike(q) + `%`)
matchAlts := []string{
`r.name ILIKE ` + namePh + ` ESCAPE '\'`,
`r.search_tsv @@ plainto_tsquery('french', ` + tsPh + `)`,
}
// Fact word matches (words >= 3 chars — shorter are French stopwords).
// `pat` est partagé entre le match et le bonus du rang : le COUNT ne voit
// que le WHERE, aucun placeholder ne doit n'exister que dans le ORDER BY.
words := longWords(q, 3, 5)
var wordConds, wordCondsHigh []string
for _, w := range words {
pat := ph(`%` + escapeLike(w) + `%`)
wordConds = append(wordConds,
`EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = r.resource_id AND f.value_text ILIKE `+pat+` ESCAPE '\')`)
wordCondsHigh = append(wordCondsHigh,
`EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = r.resource_id AND f.confidence >= 0.5 AND f.value_text ILIKE `+pat+` ESCAPE '\')`)
}
if len(wordConds) > 0 {
matchAlts = append(matchAlts, `(`+strings.Join(wordConds, " OR ")+`)`)
}
rankParts := []string{
// Match nom exact : bonus dominant (2.0) — un nom trouvé bat
// toujours un fait douteux (0.5), humble par design.
`CASE WHEN r.name ILIKE ` + namePh + ` ESCAPE '\' THEN 2.0 ELSE 0 END`,
`COALESCE(ts_rank_cd(r.search_tsv, plainto_tsquery('french', ` + tsPh + `)), 0)`,
}
if len(wordCondsHigh) > 0 {
rankParts = append(rankParts,
`CASE WHEN (`+strings.Join(wordCondsHigh, " OR ")+`) THEN 1.0 WHEN (`+strings.Join(wordConds, " OR ")+`) THEN 0.5 ELSE 0 END`)
}
conds := []string{
`r.type = 'file'`,
`r.deleted_at IS NULL`,
`r.resource_id IN (SELECT id FROM granted)`,
`(` + strings.Join(matchAlts, " OR ") + `)`,
}
// Facets (AND).
if facets.Issuer != "" {
conds = append(conds,
`EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = r.resource_id AND f.fact_type = 'issuer' AND f.value_text ILIKE `+ph(`%`+escapeLike(facets.Issuer)+`%`)+` ESCAPE '\')`)
}
if facets.DocType != "" {
conds = append(conds,
`EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = r.resource_id AND f.fact_type = 'doc_type' AND f.value_text = `+ph(facets.DocType)+`)`)
}
if facets.MinAmount != nil {
conds = append(conds,
`EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = r.resource_id AND f.fact_type = 'amount' AND f.value_num >= `+ph(*facets.MinAmount)+`)`)
}
if facets.MaxAmount != nil {
conds = append(conds,
`EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = r.resource_id AND f.fact_type = 'amount' AND f.value_num <= `+ph(*facets.MaxAmount)+`)`)
}
if facets.From != "" || facets.To != "" {
from := facets.From
if from == "" {
from = "0001-01-01"
}
to := facets.To
if to == "" {
to = "9999-12-31"
}
fromMonth, toMonth := from, to
if len(fromMonth) >= 7 {
fromMonth = fromMonth[:7]
}
if len(toMonth) >= 7 {
toMonth = toMonth[:7]
}
conds = append(conds, `(
EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = r.resource_id AND f.fact_type IN ('date', 'due_date') AND f.value_date >= `+ph(from)+`::date AND f.value_date <= `+ph(to)+`::date)
OR EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = r.resource_id AND f.fact_type = 'period' AND f.value_text >= `+ph(fromMonth)+` AND f.value_text <= `+ph(toMonth)+`)
)`)
}
where = strings.Join(conds, " AND ")
order = strings.Join(rankParts, " + ") + " DESC"
return where, order, args
}
// longWords splits q into words of at least minLen runes, capped at max.
func longWords(q string, minLen, max int) []string {
var out []string
seen := map[string]bool{}
for _, w := range strings.Fields(q) {
if len([]rune(w)) < minLen || seen[w] {
continue
}
seen[w] = true
out = append(out, w)
if len(out) >= max {
break
}
}
return out
}
+165
View File
@@ -0,0 +1,165 @@
package repository
import (
"testing"
"github.com/kazier/backend/dbtest"
)
const searchTestURL = "postgres://kazier:kazier@localhost:5432/kazier_search_test?sslmode=disable"
func newSearchRepo(t *testing.T) *Repository {
t.Helper()
return NewRepository(dbtest.OpenTestDatabase(t, searchTestURL))
}
func mustSearchUser(t *testing.T, repo *Repository, username string) string {
t.Helper()
id, err := repo.Users.Create(username, username, "test-hash", false)
if err != nil {
t.Fatalf("create user: %v", err)
}
return id
}
func seedDoc(t *testing.T, repo *Repository, userID, name, ocr string, facts []FactParam) string {
t.Helper()
id := NewID()
if err := repo.Resources.InsertFile(userID, id, name, "", 0, nil, nil); err != nil {
t.Fatalf("insert %q: %v", name, err)
}
if ocr != "" {
if err := repo.Resources.UpdateOcrText(userID, id, ocr); err != nil {
t.Fatalf("ocr %q: %v", name, err)
}
}
if len(facts) > 0 {
if err := repo.Resources.UpsertAutoFacts(id, facts); err != nil {
t.Fatalf("facts %q: %v", name, err)
}
}
return id
}
func f64(v float64) *float64 { return &v }
func fstr(v string) *string { return &v }
func searchIDs(t *testing.T, repo *Repository, userID, q string, facets SearchFacets) []string {
t.Helper()
rows, _, err := repo.Resources.SearchDocumentsVisible(userID, q, facets, 50, 0)
if err != nil {
t.Fatalf("search %q: %v", q, err)
}
ids := make([]string, 0, len(rows))
for _, r := range rows {
ids = append(ids, r.ID)
}
return ids
}
func containsID(ids []string, want string) bool {
for _, id := range ids {
if id == want {
return true
}
}
return false
}
func TestSearchDocumentsFTS(t *testing.T) {
repo := newSearchRepo(t)
userID := mustSearchUser(t, repo, "search-fts")
// Nom opaque : trouvable uniquement via l'OCR (TS français).
scan := seedDoc(t, repo, userID, "scan_042.pdf",
"Attestation d'assurance MMA pour l'année 2026", nil)
plain := seedDoc(t, repo, userID, "assurance-vie.txt", "", nil)
ids := searchIDs(t, repo, userID, "assurance", SearchFacets{})
if !containsID(ids, scan) {
t.Errorf("scan_042.pdf introuvable via OCR: %v", ids)
}
if !containsID(ids, plain) {
t.Errorf("assurance-vie.txt introuvable via nom: %v", ids)
}
// Stemming français : "factures" matche "facture".
fact := seedDoc(t, repo, userID, "doc.pdf", "Votre facture de mars", nil)
ids = searchIDs(t, repo, userID, "factures", SearchFacets{})
if !containsID(ids, fact) {
t.Errorf("stemming FR manquant: %v", ids)
}
}
func TestSearchDocumentsFacets(t *testing.T) {
repo := newSearchRepo(t)
userID := mustSearchUser(t, repo, "search-facets")
edf := seedDoc(t, repo, userID, "edf.pdf", "Facture EDF", []FactParam{
{Type: "issuer", ValueText: "EDF", Confidence: 0.9},
{Type: "amount", ValueText: "84.20 €", ValueNum: f64(84.20), Confidence: 0.9},
{Type: "due_date", ValueText: "2026-04-12", ValueDate: fstr("2026-04-12"), Confidence: 0.8},
{Type: "doc_type", ValueText: "facture", Confidence: 0.8},
})
engie := seedDoc(t, repo, userID, "engie.pdf", "Facture Engie", []FactParam{
{Type: "issuer", ValueText: "ENGIE", Confidence: 0.9},
{Type: "amount", ValueText: "120.00 €", ValueNum: f64(120), Confidence: 0.9},
{Type: "doc_type", ValueText: "facture", Confidence: 0.8},
})
ids := searchIDs(t, repo, userID, "facture", SearchFacets{Issuer: "edf"})
if !containsID(ids, edf) || containsID(ids, engie) {
t.Errorf("facette issuer: %v", ids)
}
ids = searchIDs(t, repo, userID, "facture", SearchFacets{MinAmount: f64(100)})
if !containsID(ids, engie) || containsID(ids, edf) {
t.Errorf("facette min_amount: %v", ids)
}
ids = searchIDs(t, repo, userID, "facture", SearchFacets{DocType: "facture"})
if !containsID(ids, edf) || !containsID(ids, engie) {
t.Errorf("facette doc_type: %v", ids)
}
ids = searchIDs(t, repo, userID, "facture", SearchFacets{From: "2026-04-01", To: "2026-04-30"})
if !containsID(ids, edf) || containsID(ids, engie) {
t.Errorf("facette période: %v", ids)
}
}
func TestSearchDocumentsLowConfidenceDemotes(t *testing.T) {
repo := newSearchRepo(t)
userID := mustSearchUser(t, repo, "search-humble")
// Fait douteux (0.3) : trouvé quand même (jamais exclu), mais après le
// match franc (nom) — humble par design.
sure := seedDoc(t, repo, userID, "EDF-contrat.pdf", "", nil)
doubt := seedDoc(t, repo, userID, "scan_x.pdf", "courrier", []FactParam{
{Type: "issuer", ValueText: "EDF", Confidence: 0.3},
})
ids := searchIDs(t, repo, userID, "EDF", SearchFacets{})
if !containsID(ids, doubt) {
t.Fatalf("fait douteux exclu (interdit): %v", ids)
}
if len(ids) < 2 || ids[0] != sure || ids[1] != doubt {
t.Errorf("ordre de rang inattendu (franc avant douteux): %v", ids)
}
}
func TestSearchDocumentsTrashedExcluded(t *testing.T) {
repo := newSearchRepo(t)
userID := mustSearchUser(t, repo, "search-trash")
id := seedDoc(t, repo, userID, "poubelle.pdf", "contenu unique xyz", nil)
if ids := searchIDs(t, repo, userID, "xyz", SearchFacets{}); !containsID(ids, id) {
t.Fatalf("introuvable avant trash: %v", ids)
}
if err := repo.Resources.TrashResource(userID, id); err != nil {
t.Fatalf("trash: %v", err)
}
if ids := searchIDs(t, repo, userID, "xyz", SearchFacets{}); containsID(ids, id) {
t.Errorf("trashé encore visible: %v", ids)
}
}
+2 -2
View File
@@ -245,8 +245,8 @@ func (s *Resources) ListRootFolders(ownerID string) ([]FolderDTO, error) {
return folders, nil
}
func (s *Resources) SearchFiles(ownerID, q string, page, pageSize int) ([]FileDTO, int, error) {
rows, total, err := s.Repo.SearchFilesVisible(ownerID, q, pageSize, (page-1)*pageSize)
func (s *Resources) SearchFiles(ownerID, q string, page, pageSize int, facets repository.SearchFacets) ([]FileDTO, int, error) {
rows, total, err := s.Repo.SearchDocumentsVisible(ownerID, q, facets, pageSize, (page-1)*pageSize)
if err != nil {
return nil, 0, err
}
+1 -1
View File
@@ -43,7 +43,7 @@ Références : `V2.md` (modèle cible), `mobile/services/db/` (conventions sync)
| GET | `/trash` | query `page?`, `pageSize?` | `TrashDto[]` (`{ id, type, name, size, trashedAt, updatedAt }` + `meta`) | — |
| GET | `/files/:id/versions` | — (parent live uniquement) | `VersionDto[]` (`{ versionNo, size, mimeType, createdAt }`, plus récent d'abord) | `NOT_FOUND` |
| POST | `/files/:id/versions/:n/restore` | — (l'actuel est à son tour versionné) | `FileDto` | `NOT_FOUND` |
| GET | `/files/search` | query `q` (obligatoire), `page?`, `pageSize?` | `FileDto[]` (+ `meta`) | — |
| GET | `/files/search` | query `q` (obligatoire), `page?`, `pageSize?`, facettes V3 : `issuer?`, `from?`/`to?` (ISO `YYYY-MM-DD`, sur faits date/échéance + recouvrement périodes `YYYY-MM`), `min_amount?`/`max_amount?` (numériques), `doc_type?` (vocabulaire suggéré) | `FileDto[]` (+ `meta`, rang : nom exact > TS > fait sûr > fait douteux) | `INVALID_REQUEST` (facette malformée) |
| GET | `/files/folders` | — | `FolderDto[]` (racines) | — |
| POST | `/files/upload` | multipart : `file` (uri/name/type), `folderId?` | `FileDto` | `FILE_TOO_LARGE` |
| POST | `/ocr/jobs` | `{ "fileId": "…" }` | `OcrJob` | — |
@@ -0,0 +1,882 @@
{
"formatVersion": 1,
"database": {
"version": 21,
"identityHash": "ad6fcc232dbddec3aa912be2b3dfcb4c",
"entities": [
{
"tableName": "folders",
"createSql": "CREATE TABLE IF NOT EXISTS `${TABLE_NAME}` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `resource_id` TEXT NOT NULL, `uri` TEXT, `name` TEXT NOT NULL, `exists` INTEGER, `parent_resource_id` TEXT, `owner_id` TEXT, `sync_status` TEXT NOT NULL, `created_in_app` INTEGER NOT NULL DEFAULT 0, `trashed_at` INTEGER, `added_at` INTEGER NOT NULL, `updated_at` INTEGER NOT NULL)",
"fields": [
{
"fieldPath": "id",
"columnName": "id",
"affinity": "INTEGER",
"notNull": true
},
{
"fieldPath": "resourceId",
"columnName": "resource_id",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "uri",
"columnName": "uri",
"affinity": "TEXT",
"notNull": false
},
{
"fieldPath": "name",
"columnName": "name",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "exists",
"columnName": "exists",
"affinity": "INTEGER",
"notNull": false
},
{
"fieldPath": "parentResourceId",
"columnName": "parent_resource_id",
"affinity": "TEXT",
"notNull": false
},
{
"fieldPath": "ownerId",
"columnName": "owner_id",
"affinity": "TEXT",
"notNull": false
},
{
"fieldPath": "syncStatus",
"columnName": "sync_status",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "createdInApp",
"columnName": "created_in_app",
"affinity": "INTEGER",
"notNull": true,
"defaultValue": "0"
},
{
"fieldPath": "trashedAt",
"columnName": "trashed_at",
"affinity": "INTEGER",
"notNull": false
},
{
"fieldPath": "addedAt",
"columnName": "added_at",
"affinity": "INTEGER",
"notNull": true
},
{
"fieldPath": "updatedAt",
"columnName": "updated_at",
"affinity": "INTEGER",
"notNull": true
}
],
"primaryKey": {
"autoGenerate": true,
"columnNames": [
"id"
]
},
"indices": [
{
"name": "index_folders_resource_id",
"unique": true,
"columnNames": [
"resource_id"
],
"orders": [],
"createSql": "CREATE UNIQUE INDEX IF NOT EXISTS `index_folders_resource_id` ON `${TABLE_NAME}` (`resource_id`)"
},
{
"name": "index_folders_uri",
"unique": true,
"columnNames": [
"uri"
],
"orders": [],
"createSql": "CREATE UNIQUE INDEX IF NOT EXISTS `index_folders_uri` ON `${TABLE_NAME}` (`uri`)"
},
{
"name": "index_folders_parent_resource_id",
"unique": false,
"columnNames": [
"parent_resource_id"
],
"orders": [],
"createSql": "CREATE INDEX IF NOT EXISTS `index_folders_parent_resource_id` ON `${TABLE_NAME}` (`parent_resource_id`)"
}
],
"foreignKeys": []
},
{
"tableName": "user_preferences",
"createSql": "CREATE TABLE IF NOT EXISTS `${TABLE_NAME}` (`key` TEXT NOT NULL, `value` TEXT NOT NULL, `updated_at` INTEGER NOT NULL, PRIMARY KEY(`key`))",
"fields": [
{
"fieldPath": "key",
"columnName": "key",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "value",
"columnName": "value",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "updatedAt",
"columnName": "updated_at",
"affinity": "INTEGER",
"notNull": true
}
],
"primaryKey": {
"autoGenerate": false,
"columnNames": [
"key"
]
},
"indices": [],
"foreignKeys": []
},
{
"tableName": "files",
"createSql": "CREATE TABLE IF NOT EXISTS `${TABLE_NAME}` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `resource_id` TEXT NOT NULL, `uri` TEXT, `name` TEXT NOT NULL, `folder_resource_id` TEXT NOT NULL, `extension` TEXT, `size` INTEGER NOT NULL, `mime_type` TEXT, `exists` INTEGER, `last_modified` INTEGER, `owner_id` TEXT, `category` TEXT, `sync_status` TEXT NOT NULL, `content` TEXT, `processed` INTEGER NOT NULL, `ocr_text` TEXT, `ocr_attempts` INTEGER NOT NULL, `ocr_queued_at` INTEGER, `bytes_uploaded` INTEGER NOT NULL, `bytes_upload_attempts` INTEGER NOT NULL, `has_content_server` INTEGER NOT NULL, `wanted_offline` INTEGER NOT NULL, `trashed_at` INTEGER, `added_at` INTEGER NOT NULL, `updated_at` INTEGER NOT NULL)",
"fields": [
{
"fieldPath": "id",
"columnName": "id",
"affinity": "INTEGER",
"notNull": true
},
{
"fieldPath": "resourceId",
"columnName": "resource_id",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "uri",
"columnName": "uri",
"affinity": "TEXT",
"notNull": false
},
{
"fieldPath": "name",
"columnName": "name",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "folderResourceId",
"columnName": "folder_resource_id",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "extension",
"columnName": "extension",
"affinity": "TEXT",
"notNull": false
},
{
"fieldPath": "size",
"columnName": "size",
"affinity": "INTEGER",
"notNull": true
},
{
"fieldPath": "mimeType",
"columnName": "mime_type",
"affinity": "TEXT",
"notNull": false
},
{
"fieldPath": "exists",
"columnName": "exists",
"affinity": "INTEGER",
"notNull": false
},
{
"fieldPath": "lastModified",
"columnName": "last_modified",
"affinity": "INTEGER",
"notNull": false
},
{
"fieldPath": "ownerId",
"columnName": "owner_id",
"affinity": "TEXT",
"notNull": false
},
{
"fieldPath": "category",
"columnName": "category",
"affinity": "TEXT",
"notNull": false
},
{
"fieldPath": "syncStatus",
"columnName": "sync_status",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "content",
"columnName": "content",
"affinity": "TEXT",
"notNull": false
},
{
"fieldPath": "processed",
"columnName": "processed",
"affinity": "INTEGER",
"notNull": true
},
{
"fieldPath": "ocrText",
"columnName": "ocr_text",
"affinity": "TEXT",
"notNull": false
},
{
"fieldPath": "ocrAttempts",
"columnName": "ocr_attempts",
"affinity": "INTEGER",
"notNull": true
},
{
"fieldPath": "ocrQueuedAt",
"columnName": "ocr_queued_at",
"affinity": "INTEGER",
"notNull": false
},
{
"fieldPath": "bytesUploaded",
"columnName": "bytes_uploaded",
"affinity": "INTEGER",
"notNull": true
},
{
"fieldPath": "bytesUploadAttempts",
"columnName": "bytes_upload_attempts",
"affinity": "INTEGER",
"notNull": true
},
{
"fieldPath": "hasContentServer",
"columnName": "has_content_server",
"affinity": "INTEGER",
"notNull": true
},
{
"fieldPath": "wantedOffline",
"columnName": "wanted_offline",
"affinity": "INTEGER",
"notNull": true
},
{
"fieldPath": "trashedAt",
"columnName": "trashed_at",
"affinity": "INTEGER",
"notNull": false
},
{
"fieldPath": "addedAt",
"columnName": "added_at",
"affinity": "INTEGER",
"notNull": true
},
{
"fieldPath": "updatedAt",
"columnName": "updated_at",
"affinity": "INTEGER",
"notNull": true
}
],
"primaryKey": {
"autoGenerate": true,
"columnNames": [
"id"
]
},
"indices": [
{
"name": "index_files_resource_id",
"unique": true,
"columnNames": [
"resource_id"
],
"orders": [],
"createSql": "CREATE UNIQUE INDEX IF NOT EXISTS `index_files_resource_id` ON `${TABLE_NAME}` (`resource_id`)"
},
{
"name": "index_files_uri",
"unique": true,
"columnNames": [
"uri"
],
"orders": [],
"createSql": "CREATE UNIQUE INDEX IF NOT EXISTS `index_files_uri` ON `${TABLE_NAME}` (`uri`)"
},
{
"name": "index_files_folder_resource_id",
"unique": false,
"columnNames": [
"folder_resource_id"
],
"orders": [],
"createSql": "CREATE INDEX IF NOT EXISTS `index_files_folder_resource_id` ON `${TABLE_NAME}` (`folder_resource_id`)"
},
{
"name": "index_files_category",
"unique": false,
"columnNames": [
"category"
],
"orders": [],
"createSql": "CREATE INDEX IF NOT EXISTS `index_files_category` ON `${TABLE_NAME}` (`category`)"
},
{
"name": "index_files_exists_processed_sync_status",
"unique": false,
"columnNames": [
"exists",
"processed",
"sync_status"
],
"orders": [],
"createSql": "CREATE INDEX IF NOT EXISTS `index_files_exists_processed_sync_status` ON `${TABLE_NAME}` (`exists`, `processed`, `sync_status`)"
}
],
"foreignKeys": []
},
{
"tableName": "pending_operations",
"createSql": "CREATE TABLE IF NOT EXISTS `${TABLE_NAME}` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `device_id` TEXT, `operation_id` TEXT NOT NULL, `ref_type` TEXT, `ref_id` INTEGER, `resource_id` TEXT, `resource_type` TEXT, `operation` TEXT NOT NULL, `payload` TEXT NOT NULL, `parent_resource_id` TEXT, `status` TEXT NOT NULL, `attempts` INTEGER NOT NULL, `created_at` INTEGER NOT NULL, `updated_at` INTEGER NOT NULL)",
"fields": [
{
"fieldPath": "id",
"columnName": "id",
"affinity": "INTEGER",
"notNull": true
},
{
"fieldPath": "deviceId",
"columnName": "device_id",
"affinity": "TEXT",
"notNull": false
},
{
"fieldPath": "operationId",
"columnName": "operation_id",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "refType",
"columnName": "ref_type",
"affinity": "TEXT",
"notNull": false
},
{
"fieldPath": "refId",
"columnName": "ref_id",
"affinity": "INTEGER",
"notNull": false
},
{
"fieldPath": "resourceId",
"columnName": "resource_id",
"affinity": "TEXT",
"notNull": false
},
{
"fieldPath": "resourceType",
"columnName": "resource_type",
"affinity": "TEXT",
"notNull": false
},
{
"fieldPath": "operation",
"columnName": "operation",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "payload",
"columnName": "payload",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "parentResourceId",
"columnName": "parent_resource_id",
"affinity": "TEXT",
"notNull": false
},
{
"fieldPath": "status",
"columnName": "status",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "attempts",
"columnName": "attempts",
"affinity": "INTEGER",
"notNull": true
},
{
"fieldPath": "createdAt",
"columnName": "created_at",
"affinity": "INTEGER",
"notNull": true
},
{
"fieldPath": "updatedAt",
"columnName": "updated_at",
"affinity": "INTEGER",
"notNull": true
}
],
"primaryKey": {
"autoGenerate": true,
"columnNames": [
"id"
]
},
"indices": [
{
"name": "index_pending_operations_device_id_operation_id",
"unique": true,
"columnNames": [
"device_id",
"operation_id"
],
"orders": [],
"createSql": "CREATE UNIQUE INDEX IF NOT EXISTS `index_pending_operations_device_id_operation_id` ON `${TABLE_NAME}` (`device_id`, `operation_id`)"
},
{
"name": "index_pending_operations_status",
"unique": false,
"columnNames": [
"status"
],
"orders": [],
"createSql": "CREATE INDEX IF NOT EXISTS `index_pending_operations_status` ON `${TABLE_NAME}` (`status`)"
},
{
"name": "index_pending_operations_resource_id",
"unique": false,
"columnNames": [
"resource_id"
],
"orders": [],
"createSql": "CREATE INDEX IF NOT EXISTS `index_pending_operations_resource_id` ON `${TABLE_NAME}` (`resource_id`)"
},
{
"name": "index_pending_operations_resource_id_operation_status",
"unique": false,
"columnNames": [
"resource_id",
"operation",
"status"
],
"orders": [],
"createSql": "CREATE INDEX IF NOT EXISTS `index_pending_operations_resource_id_operation_status` ON `${TABLE_NAME}` (`resource_id`, `operation`, `status`)"
},
{
"name": "index_pending_operations_parent_resource_id",
"unique": false,
"columnNames": [
"parent_resource_id"
],
"orders": [],
"createSql": "CREATE INDEX IF NOT EXISTS `index_pending_operations_parent_resource_id` ON `${TABLE_NAME}` (`parent_resource_id`)"
}
],
"foreignKeys": []
},
{
"tableName": "scan_sessions",
"createSql": "CREATE TABLE IF NOT EXISTS `${TABLE_NAME}` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `resource_id` TEXT NOT NULL, `root_folder_id` TEXT, `status` TEXT NOT NULL, `created_at` INTEGER NOT NULL, `updated_at` INTEGER NOT NULL)",
"fields": [
{
"fieldPath": "id",
"columnName": "id",
"affinity": "INTEGER",
"notNull": true
},
{
"fieldPath": "resourceId",
"columnName": "resource_id",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "rootFolderId",
"columnName": "root_folder_id",
"affinity": "TEXT",
"notNull": false
},
{
"fieldPath": "status",
"columnName": "status",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "createdAt",
"columnName": "created_at",
"affinity": "INTEGER",
"notNull": true
},
{
"fieldPath": "updatedAt",
"columnName": "updated_at",
"affinity": "INTEGER",
"notNull": true
}
],
"primaryKey": {
"autoGenerate": true,
"columnNames": [
"id"
]
},
"indices": [
{
"name": "index_scan_sessions_resource_id",
"unique": true,
"columnNames": [
"resource_id"
],
"orders": [],
"createSql": "CREATE UNIQUE INDEX IF NOT EXISTS `index_scan_sessions_resource_id` ON `${TABLE_NAME}` (`resource_id`)"
}
],
"foreignKeys": []
},
{
"tableName": "scan_pages",
"createSql": "CREATE TABLE IF NOT EXISTS `${TABLE_NAME}` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `resource_id` TEXT NOT NULL, `session_id` INTEGER NOT NULL, `temp_uri` TEXT NOT NULL, `corners_json` TEXT NOT NULL, `width` INTEGER NOT NULL, `height` INTEGER NOT NULL, `sort_order` INTEGER NOT NULL, `status` TEXT NOT NULL, `created_at` INTEGER NOT NULL, FOREIGN KEY(`session_id`) REFERENCES `scan_sessions`(`id`) ON UPDATE NO ACTION ON DELETE CASCADE )",
"fields": [
{
"fieldPath": "id",
"columnName": "id",
"affinity": "INTEGER",
"notNull": true
},
{
"fieldPath": "resourceId",
"columnName": "resource_id",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "sessionId",
"columnName": "session_id",
"affinity": "INTEGER",
"notNull": true
},
{
"fieldPath": "tempUri",
"columnName": "temp_uri",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "cornersJson",
"columnName": "corners_json",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "width",
"columnName": "width",
"affinity": "INTEGER",
"notNull": true
},
{
"fieldPath": "height",
"columnName": "height",
"affinity": "INTEGER",
"notNull": true
},
{
"fieldPath": "sortOrder",
"columnName": "sort_order",
"affinity": "INTEGER",
"notNull": true
},
{
"fieldPath": "status",
"columnName": "status",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "createdAt",
"columnName": "created_at",
"affinity": "INTEGER",
"notNull": true
}
],
"primaryKey": {
"autoGenerate": true,
"columnNames": [
"id"
]
},
"indices": [
{
"name": "index_scan_pages_resource_id",
"unique": true,
"columnNames": [
"resource_id"
],
"orders": [],
"createSql": "CREATE UNIQUE INDEX IF NOT EXISTS `index_scan_pages_resource_id` ON `${TABLE_NAME}` (`resource_id`)"
},
{
"name": "index_scan_pages_session_id",
"unique": false,
"columnNames": [
"session_id"
],
"orders": [],
"createSql": "CREATE INDEX IF NOT EXISTS `index_scan_pages_session_id` ON `${TABLE_NAME}` (`session_id`)"
},
{
"name": "index_scan_pages_sort_order",
"unique": false,
"columnNames": [
"sort_order"
],
"orders": [],
"createSql": "CREATE INDEX IF NOT EXISTS `index_scan_pages_sort_order` ON `${TABLE_NAME}` (`sort_order`)"
}
],
"foreignKeys": [
{
"table": "scan_sessions",
"onDelete": "CASCADE",
"onUpdate": "NO ACTION",
"columns": [
"session_id"
],
"referencedColumns": [
"id"
]
}
]
},
{
"tableName": "resource_permissions",
"createSql": "CREATE TABLE IF NOT EXISTS `${TABLE_NAME}` (`resource_id` TEXT NOT NULL, `resource_type` TEXT NOT NULL, `effective_access` TEXT NOT NULL, `owner_id` TEXT, `shared_by_id` TEXT, `expires_at` INTEGER, `cached_at` INTEGER NOT NULL, `updated_at` INTEGER NOT NULL, PRIMARY KEY(`resource_id`))",
"fields": [
{
"fieldPath": "resourceId",
"columnName": "resource_id",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "resourceType",
"columnName": "resource_type",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "effectiveAccess",
"columnName": "effective_access",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "ownerId",
"columnName": "owner_id",
"affinity": "TEXT",
"notNull": false
},
{
"fieldPath": "sharedById",
"columnName": "shared_by_id",
"affinity": "TEXT",
"notNull": false
},
{
"fieldPath": "expiresAt",
"columnName": "expires_at",
"affinity": "INTEGER",
"notNull": false
},
{
"fieldPath": "cachedAt",
"columnName": "cached_at",
"affinity": "INTEGER",
"notNull": true
},
{
"fieldPath": "updatedAt",
"columnName": "updated_at",
"affinity": "INTEGER",
"notNull": true
}
],
"primaryKey": {
"autoGenerate": false,
"columnNames": [
"resource_id"
]
},
"indices": [
{
"name": "index_resource_permissions_expires_at",
"unique": false,
"columnNames": [
"expires_at"
],
"orders": [],
"createSql": "CREATE INDEX IF NOT EXISTS `index_resource_permissions_expires_at` ON `${TABLE_NAME}` (`expires_at`)"
}
],
"foreignKeys": []
},
{
"tableName": "resource_facts",
"createSql": "CREATE TABLE IF NOT EXISTS `${TABLE_NAME}` (`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `resource_id` TEXT NOT NULL, `fact_type` TEXT NOT NULL, `value_text` TEXT, `value_num` REAL, `value_date` TEXT, `confidence` REAL NOT NULL, `origin` TEXT NOT NULL, `updated_at` INTEGER NOT NULL)",
"fields": [
{
"fieldPath": "id",
"columnName": "id",
"affinity": "INTEGER",
"notNull": true
},
{
"fieldPath": "resourceId",
"columnName": "resource_id",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "factType",
"columnName": "fact_type",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "valueText",
"columnName": "value_text",
"affinity": "TEXT",
"notNull": false
},
{
"fieldPath": "valueNum",
"columnName": "value_num",
"affinity": "REAL",
"notNull": false
},
{
"fieldPath": "valueDate",
"columnName": "value_date",
"affinity": "TEXT",
"notNull": false
},
{
"fieldPath": "confidence",
"columnName": "confidence",
"affinity": "REAL",
"notNull": true
},
{
"fieldPath": "origin",
"columnName": "origin",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "updatedAt",
"columnName": "updated_at",
"affinity": "INTEGER",
"notNull": true
}
],
"primaryKey": {
"autoGenerate": true,
"columnNames": [
"id"
]
},
"indices": [
{
"name": "index_resource_facts_resource_id",
"unique": false,
"columnNames": [
"resource_id"
],
"orders": [],
"createSql": "CREATE INDEX IF NOT EXISTS `index_resource_facts_resource_id` ON `${TABLE_NAME}` (`resource_id`)"
}
],
"foreignKeys": []
},
{
"ftsVersion": "FTS4",
"ftsOptions": {
"tokenizer": "simple",
"tokenizerArgs": [],
"contentTable": "",
"languageIdColumnName": "",
"matchInfo": "FTS4",
"notIndexedColumns": [],
"prefixSizes": [],
"preferredOrder": "ASC"
},
"contentSyncTriggers": [],
"tableName": "files_fts",
"createSql": "CREATE VIRTUAL TABLE IF NOT EXISTS `${TABLE_NAME}` USING FTS4(`name` TEXT NOT NULL, `body` TEXT NOT NULL)",
"fields": [
{
"fieldPath": "name",
"columnName": "name",
"affinity": "TEXT",
"notNull": true
},
{
"fieldPath": "body",
"columnName": "body",
"affinity": "TEXT",
"notNull": true
}
],
"primaryKey": {
"autoGenerate": true,
"columnNames": [
"rowid"
]
},
"indices": [],
"foreignKeys": []
}
],
"views": [],
"setupQueries": [
"CREATE TABLE IF NOT EXISTS room_master_table (id INTEGER PRIMARY KEY,identity_hash TEXT)",
"INSERT OR REPLACE INTO room_master_table (id,identity_hash) VALUES(42, 'ad6fcc232dbddec3aa912be2b3dfcb4c')"
]
}
}
@@ -10,6 +10,7 @@ import com.kazier.mobile.data.local.dao.ResourcePermissionDao
import com.kazier.mobile.data.local.dao.ScanDao
import com.kazier.mobile.data.local.dao.UserPreferenceDao
import com.kazier.mobile.data.local.entity.FileEntity
import com.kazier.mobile.data.local.entity.FilesFts
import com.kazier.mobile.data.local.entity.FolderEntity
import com.kazier.mobile.data.local.entity.PendingOperationEntity
import com.kazier.mobile.data.local.entity.ResourceFact
@@ -40,14 +41,18 @@ import com.kazier.mobile.data.local.entity.UserPreferenceEntity
* restore exists=1, purge locale après 30j+).
* v20: table resource_facts (faits génériques V3 — miroir GET /facts,
* remplacement en bloc au sync, corrections `user` poussées en outbox).
* v21: table virtuelle FTS4 `files_fts` (nom + corps ocr+faits) + triggers
* de maintien (files insert/update/delete, facts insert/update/delete) —
* recherche plein-texte offline S4. Pas d'entité Room (table virtuelle
* maintenue en SQL migration uniquement, requêtes via `FileDao`).
*/
@Database(
entities = [
FolderEntity::class, UserPreferenceEntity::class, FileEntity::class,
PendingOperationEntity::class, ScanSessionEntity::class, ScanPageEntity::class,
ResourcePermissionEntity::class, ResourceFact::class,
ResourcePermissionEntity::class, ResourceFact::class, FilesFts::class,
],
version = 20,
version = 21,
// m19 : schéma exporté (`app/schemas/`) — toute dérive non migrée casse
// la compilation au lieu de briqueter les utilisateurs au démarrage.
exportSchema = true,
@@ -69,6 +69,108 @@ interface FileDao {
""")
fun searchWithFilters(query: String?, normQuery: String?, category: String?): Flow<List<FileEntity>>
/**
* Recherche documentaire V3 (S4) : plein-texte FTS (`files_fts`, nom +
* corps ocr+faits) OU nom (LIKE + accents, même branche que
* [searchWithFilters]), restreint par catégorie + facettes faits
* (émetteur, type suggéré, montant, période). Rang : nom exact > fait
* sûr (confiance ≥ 0.5) > fait douteux (jamais exclu) > nom.
*
* `:match` = expression FTS (`"tok" OR …`, construite côté repository,
* guillemets échappés) ; `:w1..:w5` = mots (≥ 3 lettres, échappés LIKE)
* pour le rang et les facettes ; NULL = absent.
*/
@Query("""
SELECT files.* FROM files
WHERE files."exists" = 1
AND (
files.id IN (SELECT rowid FROM files_fts WHERE files_fts MATCH :match)
OR files.name COLLATE NOCASE LIKE '%' || :query || '%' ESCAPE '\'
OR REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(
REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(
lower(files.name),
'œ','oe'),'æ','ae'),'ç','c'),'à','a'),'â','a'),'ä','a'),
'é','e'),'è','e'),'ê','e'),'ë','e'),'î','i'),'ï','i'),
'ô','o'),'ö','o'),'ù','u'),'û','u'),'ü','u'),'ÿ','y')
LIKE '%' || :normQuery || '%' ESCAPE '\'
)
AND (:category IS NULL OR files.category = :category)
AND (:issuer IS NULL OR EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = files.resource_id AND f.fact_type = 'issuer' AND f.value_text LIKE '%' || :issuer || '%' ESCAPE '\'))
AND (:docType IS NULL OR EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = files.resource_id AND f.fact_type = 'doc_type' AND f.value_text = :docType))
AND (:minAmount IS NULL OR EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = files.resource_id AND f.fact_type = 'amount' AND f.value_num >= :minAmount))
AND (:maxAmount IS NULL OR EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = files.resource_id AND f.fact_type = 'amount' AND f.value_num <= :maxAmount))
AND ((:fromDate IS NULL AND :toDate IS NULL) OR EXISTS (
SELECT 1 FROM resource_facts f WHERE f.resource_id = files.resource_id AND (
(f.fact_type IN ('date', 'due_date') AND (:fromDate IS NULL OR f.value_date >= :fromDate) AND (:toDate IS NULL OR f.value_date <= :toDate))
OR (f.fact_type = 'period' AND (:fromDate IS NULL OR f.value_text >= substr(:fromDate, 1, 7)) AND (:toDate IS NULL OR f.value_text <= substr(:toDate, 1, 7)))
)))
ORDER BY
CASE WHEN files.name COLLATE NOCASE LIKE '%' || :query || '%' ESCAPE '\' THEN 0 ELSE 1 END,
CASE WHEN (:w1 IS NOT NULL AND EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = files.resource_id AND f.confidence >= 0.5 AND f.value_text LIKE '%' || :w1 || '%' ESCAPE '\'))
OR (:w2 IS NOT NULL AND EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = files.resource_id AND f.confidence >= 0.5 AND f.value_text LIKE '%' || :w2 || '%' ESCAPE '\'))
OR (:w3 IS NOT NULL AND EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = files.resource_id AND f.confidence >= 0.5 AND f.value_text LIKE '%' || :w3 || '%' ESCAPE '\'))
OR (:w4 IS NOT NULL AND EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = files.resource_id AND f.confidence >= 0.5 AND f.value_text LIKE '%' || :w4 || '%' ESCAPE '\'))
OR (:w5 IS NOT NULL AND EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = files.resource_id AND f.confidence >= 0.5 AND f.value_text LIKE '%' || :w5 || '%' ESCAPE '\'))
THEN 0
WHEN (:w1 IS NOT NULL AND EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = files.resource_id AND f.value_text LIKE '%' || :w1 || '%' ESCAPE '\'))
OR (:w2 IS NOT NULL AND EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = files.resource_id AND f.value_text LIKE '%' || :w2 || '%' ESCAPE '\'))
OR (:w3 IS NOT NULL AND EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = files.resource_id AND f.value_text LIKE '%' || :w3 || '%' ESCAPE '\'))
OR (:w4 IS NOT NULL AND EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = files.resource_id AND f.value_text LIKE '%' || :w4 || '%' ESCAPE '\'))
OR (:w5 IS NOT NULL AND EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = files.resource_id AND f.value_text LIKE '%' || :w5 || '%' ESCAPE '\'))
THEN 1 ELSE 2 END,
files.name ASC
""")
fun searchDocuments(
match: String,
query: String,
normQuery: String,
category: String?,
issuer: String?,
docType: String?,
minAmount: Double?,
maxAmount: Double?,
fromDate: String?,
toDate: String?,
w1: String?,
w2: String?,
w3: String?,
w4: String?,
w5: String?,
): Flow<List<FileEntity>>
/**
* Variante sans texte (facettes seules) : même contrat que
* [searchDocuments] sans la branche FTS/nom.
*/
@Query("""
SELECT files.* FROM files
WHERE files."exists" = 1
AND (:category IS NULL OR files.category = :category)
AND (:issuer IS NULL OR EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = files.resource_id AND f.fact_type = 'issuer' AND f.value_text LIKE '%' || :issuer || '%' ESCAPE '\'))
AND (:docType IS NULL OR EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = files.resource_id AND f.fact_type = 'doc_type' AND f.value_text = :docType))
AND (:minAmount IS NULL OR EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = files.resource_id AND f.fact_type = 'amount' AND f.value_num >= :minAmount))
AND (:maxAmount IS NULL OR EXISTS (SELECT 1 FROM resource_facts f WHERE f.resource_id = files.resource_id AND f.fact_type = 'amount' AND f.value_num <= :maxAmount))
AND ((:fromDate IS NULL AND :toDate IS NULL) OR EXISTS (
SELECT 1 FROM resource_facts f WHERE f.resource_id = files.resource_id AND (
(f.fact_type IN ('date', 'due_date') AND (:fromDate IS NULL OR f.value_date >= :fromDate) AND (:toDate IS NULL OR f.value_date <= :toDate))
OR (f.fact_type = 'period' AND (:fromDate IS NULL OR f.value_text >= substr(:fromDate, 1, 7)) AND (:toDate IS NULL OR f.value_text <= substr(:toDate, 1, 7)))
)))
ORDER BY files.name ASC
""")
fun searchFacetsOnly(
category: String?,
issuer: String?,
docType: String?,
minAmount: Double?,
maxAmount: Double?,
fromDate: String?,
toDate: String?,
): Flow<List<FileEntity>>
/** Types suggérés distincts (chips de filtre UI). */
@Query("SELECT DISTINCT value_text FROM resource_facts WHERE fact_type = 'doc_type' AND value_text IS NOT NULL ORDER BY 1")
fun observeDistinctDocTypes(): Flow<List<String>>
@Query("""
SELECT * FROM files
WHERE "exists" = 1
@@ -0,0 +1,26 @@
package com.kazier.mobile.data.local.entity
import androidx.room.ColumnInfo
import androidx.room.Entity
import androidx.room.Fts4
import androidx.room.PrimaryKey
/**
* Index plein-texte V3 (S4) : `name` + `body` (`ocr_text` + faits concaténés).
*
* Table virtuelle maintenue EXCLUSIVEMENT par les triggers de la migration
* 20→21 (insert/update/delete sur `files`, insert/update/delete sur
* `resource_facts`) — jamais d'écriture Room directe. Les requêtes passent
* par `FileDao.searchDocuments` (JOIN sur `rowid = files.id`).
*/
@Fts4
@Entity(tableName = "files_fts")
data class FilesFts(
@PrimaryKey
@ColumnInfo(name = "rowid")
val rowId: Long,
@ColumnInfo(name = "name")
val name: String,
@ColumnInfo(name = "body")
val body: String,
)
@@ -0,0 +1,72 @@
package com.kazier.mobile.data.local.fts
/**
* Maintien de l'index plein-texte `files_fts` (V3 S4) — `name` + `body`
* (`ocr_text` + faits concaténés).
*
* Utilisé à DEUX endroits (même SQL) :
* - `Migrations.MIGRATION_20_21` (bases existantes + backfill) ;
* - `RoomDatabase.Callback.onCreate` (fresh install — Room crée la table
* virtuelle via l'entité `FilesFts`, mais jamais les triggers).
*/
object FtsTriggers {
/** CREATE de la table — identique au généré Room (`21.json`). */
const val CREATE_TABLE =
"CREATE VIRTUAL TABLE IF NOT EXISTS `files_fts` USING FTS4(`name` TEXT NOT NULL, `body` TEXT NOT NULL)"
val TRIGGERS: List<String> = listOf(
"""
CREATE TRIGGER IF NOT EXISTS `files_fts_ai` AFTER INSERT ON `files` BEGIN
INSERT INTO `files_fts`(`rowid`, `name`, `body`) VALUES (
NEW.`id`, NEW.`name`,
COALESCE(NEW.`ocr_text`, '') || ' ' || COALESCE(
(SELECT group_concat(`value_text`, ' ') FROM `resource_facts`
WHERE `resource_facts`.`resource_id` = NEW.`resource_id`), ''));
END
""".trimIndent(),
"""
CREATE TRIGGER IF NOT EXISTS `files_fts_ad` AFTER DELETE ON `files` BEGIN
DELETE FROM `files_fts` WHERE `rowid` = OLD.`id`;
END
""".trimIndent(),
"""
CREATE TRIGGER IF NOT EXISTS `files_fts_au` AFTER UPDATE OF `name`, `ocr_text` ON `files` BEGIN
UPDATE `files_fts` SET `name` = NEW.`name`,
`body` = COALESCE(NEW.`ocr_text`, '') || ' ' || COALESCE(
(SELECT group_concat(`value_text`, ' ') FROM `resource_facts`
WHERE `resource_facts`.`resource_id` = NEW.`resource_id`), '')
WHERE `rowid` = NEW.`id`;
END
""".trimIndent(),
// Les faits ne portent pas de FK vers `files.id` (clé = resource_id) :
// chaque écriture reconstruit le `body` de la ligne parente.
"""
CREATE TRIGGER IF NOT EXISTS `facts_fts_ai` AFTER INSERT ON `resource_facts` BEGIN
UPDATE `files_fts` SET `body` = COALESCE(
(SELECT `ocr_text` FROM `files` WHERE `files`.`resource_id` = NEW.`resource_id`), '') || ' ' || COALESCE(
(SELECT group_concat(`value_text`, ' ') FROM `resource_facts`
WHERE `resource_facts`.`resource_id` = NEW.`resource_id`), '')
WHERE `rowid` = (SELECT `id` FROM `files` WHERE `files`.`resource_id` = NEW.`resource_id`);
END
""".trimIndent(),
"""
CREATE TRIGGER IF NOT EXISTS `facts_fts_au` AFTER UPDATE OF `value_text` ON `resource_facts` BEGIN
UPDATE `files_fts` SET `body` = COALESCE(
(SELECT `ocr_text` FROM `files` WHERE `files`.`resource_id` = NEW.`resource_id`), '') || ' ' || COALESCE(
(SELECT group_concat(`value_text`, ' ') FROM `resource_facts`
WHERE `resource_facts`.`resource_id` = NEW.`resource_id`), '')
WHERE `rowid` = (SELECT `id` FROM `files` WHERE `files`.`resource_id` = NEW.`resource_id`);
END
""".trimIndent(),
"""
CREATE TRIGGER IF NOT EXISTS `facts_fts_ad` AFTER DELETE ON `resource_facts` BEGIN
UPDATE `files_fts` SET `body` = COALESCE(
(SELECT `ocr_text` FROM `files` WHERE `files`.`resource_id` = OLD.`resource_id`), '') || ' ' || COALESCE(
(SELECT group_concat(`value_text`, ' ') FROM `resource_facts`
WHERE `resource_facts`.`resource_id` = OLD.`resource_id`), '')
WHERE `rowid` = (SELECT `id` FROM `files` WHERE `files`.`resource_id` = OLD.`resource_id`);
END
""".trimIndent(),
)
}
@@ -358,7 +358,38 @@ object Migrations {
}
}
/**
* V3 recherche (S4) : table virtuelle FTS4 `files_fts` (`name` + `body` =
* `ocr_text` + faits concaténés) + triggers de maintien (`FtsTriggers`,
* partagés avec le callback fresh-install).
*
* Backfill : reconstruit depuis `files` + `resource_facts` existants.
* `body` évite les NULL (FTS `MATCH` + concat robustes).
*
* Déclarée AVANT `ALL` (ordre d'init Kotlin : `ALL` la référence).
*/
private val MIGRATION_20_21 = object : Migration(20, 21) {
override fun migrate(db: SupportSQLiteDatabase) {
// SQL table identique au généré Room (`21.json`) : la validation
// d'ouverture compare la table migrée au schéma attendu.
db.execSQL(com.kazier.mobile.data.local.fts.FtsTriggers.CREATE_TABLE)
db.execSQL(
"""
INSERT OR IGNORE INTO `files_fts`(`rowid`, `name`, `body`)
SELECT `id`, `name`,
COALESCE(`ocr_text`, '') || ' ' || COALESCE(
(SELECT group_concat(`value_text`, ' ') FROM `resource_facts`
WHERE `resource_facts`.`resource_id` = `files`.`resource_id`), '')
FROM `files`
""".trimIndent(),
)
for (trigger in com.kazier.mobile.data.local.fts.FtsTriggers.TRIGGERS) {
db.execSQL(trigger)
}
}
}
val ALL: Array<Migration> = arrayOf(
MIGRATION_1_2, MIGRATION_2_3, MIGRATION_3_4, MIGRATION_4_5, MIGRATION_5_6, MIGRATION_6_7, MIGRATION_7_8, MIGRATION_8_9, MIGRATION_9_10, MIGRATION_10_11, MIGRATION_11_12, MIGRATION_12_13, MIGRATION_13_14, MIGRATION_14_15, MIGRATION_15_16, MIGRATION_16_17, MIGRATION_17_18, MIGRATION_18_19, MIGRATION_19_20,
MIGRATION_1_2, MIGRATION_2_3, MIGRATION_3_4, MIGRATION_4_5, MIGRATION_5_6, MIGRATION_6_7, MIGRATION_7_8, MIGRATION_8_9, MIGRATION_9_10, MIGRATION_10_11, MIGRATION_11_12, MIGRATION_12_13, MIGRATION_13_14, MIGRATION_14_15, MIGRATION_15_16, MIGRATION_16_17, MIGRATION_17_18, MIGRATION_18_19, MIGRATION_19_20, MIGRATION_20_21,
)
}
@@ -53,6 +53,9 @@ class FileRepository @Inject constructor(
* m7 : la saisie est échappée (`%`/`_` littéraux, pas des jokers) ;
* suivi accents : la saisie normalisée (`normalizeSearch`) matche
* « ÉTÉ » depuis « ete » (branche SQL dédiée, sans migration).
*
* Historique : remplacée par [searchDocuments] (FTS + facettes, S4) pour
* l'écran de recherche — conservée pour compatibilité.
*/
fun searchFiles(query: String?, category: String?): Flow<List<FileEntity>> {
val trimmed = query?.trim()?.takeIf { it.isNotEmpty() }
@@ -63,6 +66,55 @@ class FileRepository @Inject constructor(
)
}
/**
* Recherche documentaire V3 (S4) : plein-texte FTS (nom + corps ocr+faits,
* offline) + facettes faits. `query` vide + facettes vides → récents
* (même contrat que l'ancien `searchFiles`) ; `query` vide + facettes →
* filtre seul (sans FTS).
*/
fun searchDocuments(
query: String?,
category: String?,
facets: SearchFacets = SearchFacets(),
): Flow<List<FileEntity>> {
val trimmed = query?.trim().orEmpty()
if (trimmed.isEmpty() && facets.isEmpty()) {
return recentFiles(category)
}
if (trimmed.isEmpty()) {
return fileDao.searchFacetsOnly(
category = category,
issuer = facets.issuer?.escapeLike(),
docType = facets.docType,
minAmount = facets.minAmount,
maxAmount = facets.maxAmount,
fromDate = facets.fromDate,
toDate = facets.toDate,
)
}
val words = ftsWords(trimmed)
return fileDao.searchDocuments(
match = buildFtsMatch(trimmed) ?: trimmed.escapeLike(),
query = trimmed.escapeLike(),
normQuery = trimmed.normalizeSearch().escapeLike(),
category = category,
issuer = facets.issuer?.escapeLike(),
docType = facets.docType,
minAmount = facets.minAmount,
maxAmount = facets.maxAmount,
fromDate = facets.fromDate,
toDate = facets.toDate,
w1 = words.getOrNull(0),
w2 = words.getOrNull(1),
w3 = words.getOrNull(2),
w4 = words.getOrNull(3),
w5 = words.getOrNull(4),
)
}
/** Types suggérés distincts (chips de filtre UI). */
fun observeDistinctDocTypes(): Flow<List<String>> = fileDao.observeDistinctDocTypes()
/** Les `limit` fichiers les plus récents (date de référence), filtrés par catégorie. */
fun recentFiles(category: String?, limit: Int = RECENT_LIMIT): Flow<List<FileEntity>> =
fileDao.recentFiles(category = category, limit = limit)
@@ -455,3 +507,45 @@ data class SaveFileInput(
/** Le fichier arrive déjà « traité » (ex. scan) ou à traiter dans la review. */
val processed: Boolean = false,
)
/**
* Facettes de recherche documentaire V3 (S4) — miroir de `SearchFacets` Go
* (dates ISO `YYYY-MM-DD`, montants en Double). `isEmpty` = aucun filtre.
*/
data class SearchFacets(
val issuer: String? = null,
val docType: String? = null,
val minAmount: Double? = null,
val maxAmount: Double? = null,
val fromDate: String? = null,
val toDate: String? = null,
) {
fun isEmpty(): Boolean =
issuer.isNullOrBlank() && docType.isNullOrBlank() &&
minAmount == null && maxAmount == null &&
fromDate.isNullOrBlank() && toDate.isNullOrBlank()
}
/**
* Expression FTS (`"tok" OR …`) depuis la saisie — guillemets doublés
* (syntaxe FTS), tokens ≥ 2 lettres, max 5. NULL si aucun token (l'appelant
* bascule sur les facettes seules).
*/
fun buildFtsMatch(query: String): String? {
val toks = query.split(Regex("\\s+"))
.map { it.replace("\"", "\"\"") }
.filter { it.length >= 2 }
.take(5)
if (toks.isEmpty()) return null
return toks.joinToString(" OR ") { "\"$it\"" }
}
/** Mots du rang (≥ 3 lettres, échappés LIKE) — miroir de `longWords` Go. */
fun ftsWords(query: String): List<String?> {
val seen = LinkedHashSet<String>()
for (w in query.split(Regex("\\s+"))) {
if (w.length >= 3) seen += w
if (seen.size >= 5) break
}
return seen.map { it.escapeLike() }
}
@@ -2,6 +2,8 @@ package com.kazier.mobile.di
import android.content.Context
import androidx.room.Room
import androidx.room.RoomDatabase
import androidx.sqlite.db.SupportSQLiteDatabase
import com.kazier.mobile.data.local.AppDatabase
import com.kazier.mobile.data.local.dao.FileDao
import com.kazier.mobile.data.local.dao.FactsDao
@@ -10,6 +12,7 @@ import com.kazier.mobile.data.local.dao.PendingOperationDao
import com.kazier.mobile.data.local.dao.ResourcePermissionDao
import com.kazier.mobile.data.local.dao.ScanDao
import com.kazier.mobile.data.local.dao.UserPreferenceDao
import com.kazier.mobile.data.local.fts.FtsTriggers
import com.kazier.mobile.data.local.migration.Migrations
import dagger.Module
import dagger.Provides
@@ -27,6 +30,14 @@ object DatabaseModule {
fun provideDatabase(@ApplicationContext context: Context): AppDatabase =
Room.databaseBuilder(context, AppDatabase::class.java, "dot.db")
.addMigrations(*Migrations.ALL)
.addCallback(object : RoomDatabase.Callback() {
// Fresh install : Room crée la table virtuelle via l'entité
// `FilesFts`, mais jamais les triggers — posés ici (même SQL
// que `MIGRATION_20_21`, `IF NOT EXISTS` donc rejouable).
override fun onCreate(db: SupportSQLiteDatabase) {
for (trigger in FtsTriggers.TRIGGERS) db.execSQL(trigger)
}
})
// m15 : un APK plus ancien que la DB (downgrade) ne doit plus
// crasher en `IllegalStateException` brut — reconstruction
// destructive documentée : le miroir SAF se reconstruit au walk
@@ -47,6 +47,7 @@ import androidx.compose.ui.platform.LocalSoftwareKeyboardController
import androidx.compose.ui.res.stringResource
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.text.input.ImeAction
import androidx.compose.ui.text.input.KeyboardType
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.unit.dp
@@ -92,6 +93,7 @@ fun SearchScreen(
viewModel: SearchViewModel = hiltViewModel(),
) {
val uiState by viewModel.uiState.collectAsStateWithLifecycle()
val docTypes by viewModel.docTypes.collectAsStateWithLifecycle()
val moveFolders by viewModel.moveFolders.collectAsStateWithLifecycle()
val moveError by viewModel.moveError.collectAsStateWithLifecycle()
val deleteError by viewModel.deleteError.collectAsStateWithLifecycle()
@@ -197,9 +199,17 @@ fun SearchScreen(
) { padding ->
SearchContent(
uiState = uiState,
docTypes = docTypes,
selection = selection,
onQueryChange = viewModel::onQueryChange,
onCategorySelect = viewModel::onCategorySelect,
onIssuerChange = viewModel::onIssuerChange,
onDocTypeSelect = viewModel::onDocTypeSelect,
onMinAmountChange = viewModel::onMinAmountChange,
onMaxAmountChange = viewModel::onMaxAmountChange,
onFromDateChange = viewModel::onFromDateChange,
onToDateChange = viewModel::onToDateChange,
onClearFacets = viewModel::clearFacets,
onOpenDocument = onOpenDocument,
modifier = Modifier.padding(padding),
)
@@ -267,13 +277,22 @@ fun SearchScreen(
@Composable
private fun SearchContent(
uiState: SearchUiState,
docTypes: List<String>,
selection: SelectionState,
onQueryChange: (String) -> Unit,
onCategorySelect: (String?) -> Unit,
onIssuerChange: (String) -> Unit,
onDocTypeSelect: (String?) -> Unit,
onMinAmountChange: (String) -> Unit,
onMaxAmountChange: (String) -> Unit,
onFromDateChange: (String) -> Unit,
onToDateChange: (String) -> Unit,
onClearFacets: () -> Unit,
onOpenDocument: (String) -> Unit,
modifier: Modifier = Modifier,
) {
val keyboardController = LocalSoftwareKeyboardController.current
var filtersExpanded by rememberSaveable { mutableStateOf(false) }
Column(
modifier = modifier
.fillMaxSize()
@@ -330,6 +349,21 @@ private fun SearchContent(
}
}
FacetFilters(
uiState = uiState,
docTypes = docTypes,
expanded = filtersExpanded,
onExpandedChange = { filtersExpanded = it },
enabled = !selection.active,
onIssuerChange = onIssuerChange,
onDocTypeSelect = onDocTypeSelect,
onMinAmountChange = onMinAmountChange,
onMaxAmountChange = onMaxAmountChange,
onFromDateChange = onFromDateChange,
onToDateChange = onToDateChange,
onClearFacets = onClearFacets,
)
SearchResults(
uiState = uiState,
selection = selection,
@@ -364,6 +398,129 @@ private fun CategoryChip(
)
}
/**
* Facettes V3 (S4) : émetteur, type suggéré (chips dynamiques), montants,
* période ISO. Repliées par défaut (bouton avec compteur) ; saisies
* invalides (montant non numérique, date non ISO) ignorées silencieusement.
*/
@Composable
private fun FacetFilters(
uiState: SearchUiState,
docTypes: List<String>,
expanded: Boolean,
onExpandedChange: (Boolean) -> Unit,
enabled: Boolean,
onIssuerChange: (String) -> Unit,
onDocTypeSelect: (String?) -> Unit,
onMinAmountChange: (String) -> Unit,
onMaxAmountChange: (String) -> Unit,
onFromDateChange: (String) -> Unit,
onToDateChange: (String) -> Unit,
onClearFacets: () -> Unit,
) {
Column(verticalArrangement = Arrangement.spacedBy(8.dp)) {
Row(
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(8.dp),
) {
val count = uiState.activeFacetCount
FilterChip(
selected = expanded || count > 0,
onClick = { onExpandedChange(!expanded) },
enabled = enabled,
label = {
Text(
if (count > 0) stringResource(R.string.search_filters_count, count)
else stringResource(R.string.search_filters),
)
},
)
if (count > 0 && enabled) {
androidx.compose.material3.TextButton(onClick = {
onExpandedChange(false)
onClearFacets()
}) {
Text(stringResource(R.string.search_filters_clear))
}
}
}
if (expanded) {
OutlinedTextField(
value = uiState.issuer,
onValueChange = onIssuerChange,
modifier = Modifier.fillMaxWidth(),
label = { Text(stringResource(R.string.search_facet_issuer)) },
placeholder = { Text(stringResource(R.string.search_facet_issuer_hint)) },
singleLine = true,
enabled = enabled,
)
if (docTypes.isNotEmpty()) {
LazyRow(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.spacedBy(8.dp),
) {
items(docTypes, key = { it }) { docType ->
FilterChip(
selected = uiState.docType == docType,
onClick = {
onDocTypeSelect(if (uiState.docType == docType) null else docType)
},
enabled = enabled,
label = { Text(docType) },
)
}
}
}
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.spacedBy(8.dp),
) {
OutlinedTextField(
value = uiState.minAmount,
onValueChange = onMinAmountChange,
modifier = Modifier.weight(1f),
label = { Text(stringResource(R.string.search_facet_min_amount)) },
singleLine = true,
enabled = enabled,
keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Decimal),
)
OutlinedTextField(
value = uiState.maxAmount,
onValueChange = onMaxAmountChange,
modifier = Modifier.weight(1f),
label = { Text(stringResource(R.string.search_facet_max_amount)) },
singleLine = true,
enabled = enabled,
keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Decimal),
)
}
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.spacedBy(8.dp),
) {
OutlinedTextField(
value = uiState.fromDate,
onValueChange = onFromDateChange,
modifier = Modifier.weight(1f),
label = { Text(stringResource(R.string.search_facet_from)) },
placeholder = { Text("AAAA-MM-JJ") },
singleLine = true,
enabled = enabled,
)
OutlinedTextField(
value = uiState.toDate,
onValueChange = onToDateChange,
modifier = Modifier.weight(1f),
label = { Text(stringResource(R.string.search_facet_to)) },
placeholder = { Text("AAAA-MM-JJ") },
singleLine = true,
enabled = enabled,
)
}
}
}
}
@Composable
private fun SearchResults(
uiState: SearchUiState,
@@ -8,12 +8,14 @@ import com.kazier.mobile.data.local.entity.FileEntity
import com.kazier.mobile.data.local.entity.FolderEntity
import com.kazier.mobile.data.repository.FileRepository
import com.kazier.mobile.data.repository.FolderRepository
import com.kazier.mobile.data.repository.SearchFacets
import com.kazier.mobile.features.saf.FileMover
import com.kazier.mobile.features.saf.FileDeleter
import dagger.hilt.android.lifecycle.HiltViewModel
import dagger.hilt.android.qualifiers.ApplicationContext
import kotlinx.coroutines.ExperimentalCoroutinesApi
import kotlinx.coroutines.FlowPreview
import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.SharingStarted
import kotlinx.coroutines.flow.StateFlow
@@ -38,6 +40,14 @@ class SearchViewModel @Inject constructor(
private val _searchQuery = MutableStateFlow("")
private val _selectedCategory = MutableStateFlow<String?>(null)
/** Facettes V3 (S4) — émetteur, type suggéré, montants, période ISO. */
private val _issuer = MutableStateFlow("")
private val _docType = MutableStateFlow<String?>(null)
private val _minAmount = MutableStateFlow("")
private val _maxAmount = MutableStateFlow("")
private val _fromDate = MutableStateFlow("")
private val _toDate = MutableStateFlow("")
private val _moveFolders = MutableStateFlow<List<FolderEntity>?>(null)
val moveFolders: StateFlow<List<FolderEntity>?> = _moveFolders.asStateFlow()
@@ -50,21 +60,45 @@ class SearchViewModel @Inject constructor(
private val _deleteSuccess = MutableStateFlow(false)
val deleteSuccess: StateFlow<Boolean> = _deleteSuccess.asStateFlow()
/**
* Facettes combinées (S4) — imbriquées par triples (`combine` n'a pas
* d'overload à 8 args dans cette version de coroutines).
*/
private val facetInputs: Flow<SearchFacets> = combine(
combine(_issuer, _docType, _minAmount) { issuer, docType, minAmount ->
Triple(issuer, docType, minAmount)
},
combine(_maxAmount, _fromDate, _toDate) { maxAmount, fromDate, toDate ->
Triple(maxAmount, fromDate, toDate)
},
) { left, right ->
SearchFacets(
issuer = left.first.trim().takeIf { it.isNotEmpty() },
docType = left.second,
minAmount = left.third.toDoubleOrNull(),
maxAmount = right.first.toDoubleOrNull(),
fromDate = right.second.trim().takeIf { it.matches(DATE_RE) },
toDate = right.third.trim().takeIf { it.matches(DATE_RE) },
)
}
/**
* Résultats Room : seule cette branche est debouncée (300 ms). La requête
* affichée ([uiState.query]) suit la frappe brute sans à-coups (M31).
* FTS + facettes (S4) : `searchDocuments` remplace `searchFiles`.
*/
@OptIn(FlowPreview::class, ExperimentalCoroutinesApi::class)
private val results: StateFlow<List<FileEntity>> = combine(
_searchQuery.debounce(SEARCH_DEBOUNCE_MS),
_selectedCategory,
) { query, category -> CategoryQuery(query, category) }
.flatMapLatest { (query, category) ->
facetInputs,
) { query, category, facets -> FacetedQuery(query, category, facets) }
.flatMapLatest { (query, category, facets) ->
val trimmed = query.trim()
if (trimmed.isEmpty()) {
if (trimmed.isEmpty() && facets.isEmpty()) {
fileRepository.recentFiles(category)
} else {
fileRepository.searchFiles(trimmed, category)
fileRepository.searchDocuments(trimmed, category, facets)
}
}
.stateIn(
@@ -74,16 +108,38 @@ class SearchViewModel @Inject constructor(
)
val uiState: StateFlow<SearchUiState> = combine(
_searchQuery,
_selectedCategory,
results,
) { query, category, files -> SearchUiState(query, category, files) }
combine(_searchQuery, _selectedCategory, _issuer) { query, category, issuer ->
Triple(query, category, issuer)
},
combine(_docType, _minAmount, _maxAmount) { docType, minAmount, maxAmount ->
Triple(docType, minAmount, maxAmount)
},
combine(_fromDate, _toDate, results) { fromDate, toDate, files ->
Triple(fromDate, toDate, files)
},
) { left, mid, right ->
SearchUiState(
query = left.first,
selectedCategory = left.second,
issuer = left.third,
docType = mid.first,
minAmount = mid.second,
maxAmount = mid.third,
fromDate = right.first,
toDate = right.second,
results = right.third,
)
}
.stateIn(
scope = viewModelScope,
started = SharingStarted.WhileSubscribed(STOP_TIMEOUT_MS),
initialValue = SearchUiState(),
)
/** Types suggérés distincts (chips de filtre). */
val docTypes: StateFlow<List<String>> = fileRepository.observeDistinctDocTypes()
.stateIn(viewModelScope, SharingStarted.WhileSubscribed(STOP_TIMEOUT_MS), emptyList())
fun onQueryChange(query: String) {
_searchQuery.value = query
}
@@ -92,6 +148,39 @@ class SearchViewModel @Inject constructor(
_selectedCategory.value = category
}
fun onIssuerChange(issuer: String) {
_issuer.value = issuer
}
fun onDocTypeSelect(docType: String?) {
_docType.value = docType
}
fun onMinAmountChange(value: String) {
_minAmount.value = value
}
fun onMaxAmountChange(value: String) {
_maxAmount.value = value
}
fun onFromDateChange(value: String) {
_fromDate.value = value
}
fun onToDateChange(value: String) {
_toDate.value = value
}
fun clearFacets() {
_issuer.value = ""
_docType.value = null
_minAmount.value = ""
_maxAmount.value = ""
_fromDate.value = ""
_toDate.value = ""
}
fun loadMoveFolders() {
viewModelScope.launch {
_moveFolders.value = folderRepository.getCreatedInApp()
@@ -142,7 +231,11 @@ class SearchViewModel @Inject constructor(
_deleteSuccess.value = false
}
private data class CategoryQuery(val query: String, val category: String?)
private data class FacetedQuery(
val query: String,
val category: String?,
val facets: SearchFacets,
)
companion object {
private const val SEARCH_DEBOUNCE_MS = 300L
@@ -150,8 +243,28 @@ class SearchViewModel @Inject constructor(
}
}
/** Dates ISO `AAAA-MM-JJ` — toute autre saisie est ignorée (pas d'erreur). */
private val DATE_RE = Regex("""\d{4}-\d{2}-\d{2}""")
data class SearchUiState(
val query: String = "",
val selectedCategory: String? = null,
val issuer: String = "",
val docType: String? = null,
val minAmount: String = "",
val maxAmount: String = "",
val fromDate: String = "",
val toDate: String = "",
val results: List<FileEntity> = emptyList(),
)
) {
/** Nombre de facettes actives (badge du bouton Filtres). */
val activeFacetCount: Int
get() = listOf(
issuer.isNotBlank(),
docType != null,
minAmount.toDoubleOrNull() != null,
maxAmount.toDoubleOrNull() != null,
fromDate.matches(DATE_RE),
toDate.matches(DATE_RE),
).count { it }
}
@@ -418,4 +418,14 @@
<string name="facts_type_reference">Reference</string>
<string name="facts_type_doc_type">Document type</string>
<string name="facts_type_other">Info</string>
<!-- Search facets V3 S4 -->
<string name="search_filters">Filters</string>
<string name="search_filters_count">Filters (%1$d)</string>
<string name="search_filters_clear">Clear</string>
<string name="search_facet_issuer">Issuer</string>
<string name="search_facet_issuer_hint">E.g. EDF, bank…</string>
<string name="search_facet_min_amount">Min amount (€)</string>
<string name="search_facet_max_amount">Max amount (€)</string>
<string name="search_facet_from">From (YYYY-MM-DD)</string>
<string name="search_facet_to">To (YYYY-MM-DD)</string>
</resources>
@@ -419,4 +419,14 @@
<string name="facts_type_reference">Référence</string>
<string name="facts_type_doc_type">Type de document</string>
<string name="facts_type_other">Info</string>
<!-- Recherche facettes V3 S4 -->
<string name="search_filters">Filtres</string>
<string name="search_filters_count">Filtres (%1$d)</string>
<string name="search_filters_clear">Effacer</string>
<string name="search_facet_issuer">Émetteur</string>
<string name="search_facet_issuer_hint">Ex. EDF, banque…</string>
<string name="search_facet_min_amount">Montant min. (€)</string>
<string name="search_facet_max_amount">Montant max. (€)</string>
<string name="search_facet_from">Du (AAAA-MM-JJ)</string>
<string name="search_facet_to">Au (AAAA-MM-JJ)</string>
</resources>
@@ -0,0 +1,211 @@
package com.kazier.mobile.data.local.dao
import android.content.Context
import androidx.room.Room
import androidx.test.core.app.ApplicationProvider
import com.squareup.moshi.Moshi
import com.squareup.moshi.kotlin.reflect.KotlinJsonAdapterFactory
import com.kazier.mobile.data.local.AppDatabase
import com.kazier.mobile.data.local.entity.FileEntity
import com.kazier.mobile.data.local.entity.FolderEntity
import com.kazier.mobile.data.local.entity.ResourceFact
import com.kazier.mobile.data.local.fts.FtsTriggers
import com.kazier.mobile.data.remote.ApiClient
import com.kazier.mobile.data.remote.ApiService
import com.kazier.mobile.data.repository.FileRepository
import com.kazier.mobile.data.repository.OutboxRepository
import com.kazier.mobile.data.repository.SearchFacets
import com.kazier.mobile.data.repository.buildFtsMatch
import com.kazier.mobile.data.repository.ftsWords
import com.kazier.mobile.domain.DeviceIdentity
import com.kazier.mobile.domain.GenerateId
import kotlinx.coroutines.flow.first
import kotlinx.coroutines.test.runTest
import org.junit.After
import org.junit.Assert.assertEquals
import org.junit.Assert.assertNull
import org.junit.Assert.assertTrue
import org.junit.Before
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.RobolectricTestRunner
import org.robolectric.annotation.Config
import retrofit2.Retrofit
import retrofit2.converter.moshi.MoshiConverterFactory
/**
* Recherche documentaire V3 (S4) : FTS offline (nom + corps ocr+faits),
* facettes, rang nom > fait sûr > fait douteux. La DB in-memory est créée
* au schéma courant (triggers posés à la main — le builder de test n'a pas
* le callback fresh-install de `DatabaseModule`).
*/
@RunWith(RobolectricTestRunner::class)
@Config(sdk = [34])
class SearchFtsTest {
private lateinit var db: AppDatabase
private lateinit var fileDao: FileDao
private lateinit var factsDao: com.kazier.mobile.data.local.dao.FactsDao
private lateinit var fileRepository: FileRepository
@Before
fun setUp() {
val context = ApplicationProvider.getApplicationContext<Context>()
db = Room.inMemoryDatabaseBuilder(context, AppDatabase::class.java)
.allowMainThreadQueries()
.build()
// Triggers FTS (hors migration en in-memory).
val raw = db.openHelper.writableDatabase
for (trigger in FtsTriggers.TRIGGERS) raw.execSQL(trigger)
fileDao = db.fileDao()
factsDao = db.factsDao()
val moshi = Moshi.Builder().add(KotlinJsonAdapterFactory()).build()
val apiService = Retrofit.Builder()
.baseUrl("http://localhost:1/")
.addConverterFactory(MoshiConverterFactory.create(moshi))
.build()
.create(ApiService::class.java)
val generateId = GenerateId()
val outboxRepository = OutboxRepository(
db.pendingOperationDao(), generateId,
DeviceIdentity(db.userPreferenceDao(), generateId), moshi,
)
fileRepository = FileRepository(fileDao, db.folderDao(), ApiClient(apiService, moshi), generateId, db, outboxRepository)
runTest {
db.folderDao().upsert(
FolderEntity(
resourceId = FOLDER, uri = null, name = FOLDER, exists = 1,
parentResourceId = null, syncStatus = "local",
addedAt = NOW, updatedAt = NOW,
),
)
}
}
@After
fun tearDown() {
db.close()
}
private suspend fun insertFile(resourceId: String, name: String, ocr: String? = null) {
fileDao.upsert(
FileEntity(
resourceId = resourceId, uri = null, name = name,
folderResourceId = FOLDER, size = 10, exists = 1,
processed = true, ocrText = ocr, addedAt = NOW, updatedAt = NOW,
),
)
}
private suspend fun insertFact(resourceId: String, type: String, text: String? = null, num: Double? = null, conf: Double = 0.9) {
factsDao.insertAll(
listOf(
ResourceFact(
resourceId = resourceId, factType = type, valueText = text,
valueNum = num, confidence = conf, origin = "auto", updatedAt = NOW,
),
),
)
}
@Test
fun fts_trouve_via_ocr_malgre_nom_opaque() = runTest {
insertFile("scan-1", "scan_042.pdf", "Attestation d'assurance MMA pour 2026")
val rows = fileRepository.searchDocuments("assurance", null).first()
assertEquals(listOf("scan_042.pdf"), rows.map { it.name })
}
@Test
fun fts_trouve_via_faits() = runTest {
insertFile("scan-2", "papier.pdf")
insertFact("scan-2", "issuer", "EDF")
val rows = fileRepository.searchDocuments("EDF", null).first()
assertEquals(listOf("papier.pdf"), rows.map { it.name })
}
@Test
fun rang_nom_avant_fait_douteux() = runTest {
insertFile("franc", "EDF-contrat.pdf", "contenu neutre")
insertFile("douteux", "courrier.pdf", "bonjour")
insertFact("douteux", "issuer", "EDF", conf = 0.3)
val rows = fileRepository.searchDocuments("EDF", null).first()
assertEquals(listOf("EDF-contrat.pdf", "courrier.pdf"), rows.map { it.name })
}
@Test
fun facette_emetteur_filtre() = runTest {
insertFile("f-edf", "a.pdf", "facture")
insertFact("f-edf", "issuer", "EDF")
insertFile("f-engie", "b.pdf", "facture")
insertFact("f-engie", "issuer", "ENGIE")
val rows = fileRepository.searchDocuments("facture", null, SearchFacets(issuer = "edf")).first()
assertEquals(listOf("a.pdf"), rows.map { it.name })
}
@Test
fun facette_montant_filtre() = runTest {
insertFile("f-petit", "a.pdf", "facture")
insertFact("f-petit", "amount", "84.20 €", num = 84.2)
insertFile("f-gros", "b.pdf", "facture")
insertFact("f-gros", "amount", "120.00 €", num = 120.0)
val rows = fileRepository.searchDocuments("facture", null, SearchFacets(minAmount = 100.0)).first()
assertEquals(listOf("b.pdf"), rows.map { it.name })
}
@Test
fun facette_doc_type_filtre() = runTest {
insertFile("f-fact", "a.pdf", "document")
insertFact("f-fact", "doc_type", "facture")
insertFile("f-autre", "b.pdf", "document")
val rows = fileRepository.searchDocuments("document", null, SearchFacets(docType = "facture")).first()
assertEquals(listOf("a.pdf"), rows.map { it.name })
}
@Test
fun facettes_seules_sans_texte() = runTest {
insertFile("f-fact", "a.pdf")
insertFact("f-fact", "doc_type", "facture")
insertFile("f-autre", "b.pdf")
val rows = fileRepository.searchDocuments("", null, SearchFacets(docType = "facture")).first()
assertEquals(listOf("a.pdf"), rows.map { it.name })
}
@Test
fun accents_nom_trouves() = runTest {
insertFile("ete", "ÉTÉ 2026.pdf")
val rows = fileRepository.searchDocuments("ete", null).first()
assertEquals(listOf("ÉTÉ 2026.pdf"), rows.map { it.name })
}
@Test
fun trash_exclu_de_la_recherche() = runTest {
insertFile("trash-me", "poubelle.pdf", "contenu unique xyz")
fileDao.markTrashed("trash-me", NOW)
val rows = fileRepository.searchDocuments("xyz", null).first()
assertTrue(rows.isEmpty())
}
@Test
fun match_builder_echappe_et_borne() {
assertEquals("\"facture\" OR \"edf\"", buildFtsMatch("facture edf"))
assertEquals("\"a\"\"b\"", buildFtsMatch("a\"b"))
assertNull(buildFtsMatch("a I"))
assertEquals(listOf("facture", "edf"), ftsWords("facture edf le"))
}
companion object {
private const val FOLDER = "folder-fts"
private const val NOW = 1_700_000_000_000L
}
}
@@ -0,0 +1,114 @@
package com.kazier.mobile.data.local.migration
import android.content.Context
import androidx.sqlite.db.SupportSQLiteDatabase
import androidx.sqlite.db.SupportSQLiteOpenHelper
import androidx.sqlite.db.framework.FrameworkSQLiteOpenHelperFactory
import androidx.test.core.app.ApplicationProvider
import org.junit.Assert.assertEquals
import org.junit.Assert.assertTrue
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.RobolectricTestRunner
import org.robolectric.annotation.Config
/**
* V3 recherche (S4) : la migration 20→21 crée `files_fts` (SQL identique au
* généré Room — la validation d'ouverture compare) + backfill + triggers.
*/
@RunWith(RobolectricTestRunner::class)
@Config(sdk = [34])
class Migration20To21Test {
@Test
fun migration_cree_fts_backfill_et_triggers() {
val context = ApplicationProvider.getApplicationContext<Context>()
val helper = FrameworkSQLiteOpenHelperFactory().create(
SupportSQLiteOpenHelper.Configuration.builder(context)
.name(null)
.callback(V20Callback())
.build(),
)
val db = helper.writableDatabase
db.execSQL(
"INSERT INTO `files` (`resource_id`, `name`, `folder_resource_id`, `size`, `ocr_text`, `added_at`, `updated_at`) " +
"VALUES ('d-old', 'scan.pdf', 'f-one', 10, 'Attestation assurance', 1, 1)",
)
db.execSQL(
"INSERT INTO `resource_facts` (`resource_id`, `fact_type`, `value_text`, `confidence`, `origin`, `updated_at`) " +
"VALUES ('d-old', 'issuer', 'MMA', 0.9, 'auto', 1)",
)
val migration = Migrations.ALL.first { it.startVersion == 20 && it.endVersion == 21 }
migration.migrate(db)
// Table virtuelle présente avec le SQL attendu par Room.
var createSql: String? = null
db.query("SELECT `sql` FROM `sqlite_master` WHERE `name` = 'files_fts'").use { cursor ->
assertTrue(cursor.moveToFirst())
createSql = cursor.getString(0)
}
assertTrue(
"SQL files_fts inattendu: $createSql",
createSql != null && createSql!!.contains("USING FTS4") &&
createSql!!.contains("`name` TEXT NOT NULL"),
)
// Backfill : la ligne existante est indexée (nom + corps).
db.query("SELECT `name`, `body` FROM `files_fts`").use { cursor ->
assertTrue(cursor.moveToFirst())
assertEquals("scan.pdf", cursor.getString(0))
val body = cursor.getString(1)
assertTrue("corps sans OCR: $body", body.contains("Attestation"))
assertTrue("corps sans faits: $body", body.contains("MMA"))
}
// Triggers posés.
var triggerCount = 0
db.query("SELECT COUNT(*) FROM `sqlite_master` WHERE `type` = 'trigger' AND (`name` LIKE 'files_fts_%' OR `name` LIKE 'facts_fts_%')").use { cursor ->
assertTrue(cursor.moveToFirst())
triggerCount = cursor.getInt(0)
}
assertEquals(6, triggerCount)
// Idempotence : re-passe sans erreur (IF NOT EXISTS partout).
migration.migrate(db)
helper.close()
}
/** Schéma v20 minimal (`files` + `resource_facts`). */
private class V20Callback : SupportSQLiteOpenHelper.Callback(20) {
override fun onCreate(db: SupportSQLiteDatabase) {
db.execSQL(
"CREATE TABLE `files` (" +
"`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, " +
"`resource_id` TEXT NOT NULL, " +
"`uri` TEXT, " +
"`name` TEXT NOT NULL, " +
"`folder_resource_id` TEXT NOT NULL, " +
"`size` INTEGER NOT NULL, " +
"`ocr_text` TEXT, " +
"`added_at` INTEGER NOT NULL, " +
"`updated_at` INTEGER NOT NULL)",
)
db.execSQL(
"CREATE TABLE `resource_facts` (" +
"`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, " +
"`resource_id` TEXT NOT NULL, " +
"`fact_type` TEXT NOT NULL, " +
"`value_text` TEXT, " +
"`value_num` REAL, " +
"`value_date` TEXT, " +
"`confidence` REAL NOT NULL, " +
"`origin` TEXT NOT NULL, " +
"`updated_at` INTEGER NOT NULL)",
)
}
override fun onUpgrade(
db: SupportSQLiteDatabase,
oldVersion: Int,
newVersion: Int,
) = Unit
}
}