refactor: introduce database provider abstraction, isolate MSSQL layer

- Add provider contract (src/providers/provider.js): query() -> {rows,
  rowsAffected, insertId}, callback transaction(fn), exec() for stored
  procs, initialize/purge lifecycle
- Add MssqlProvider (src/providers/mssql/index.js) wrapping an mssql
  ConnectionPool with normalized results, tx commit/rollback and
  constraint-error classifiers (unique 2627/2601, FK 547)
- LibDashStore constructor now takes a provider instead of a pool
  (breaking); adds LibDashStore.create() async factory; keeps pool as
  transitional bridge until store modules are converted (see TODO.md)
- Move MSSQL-specific code into the provider: src/sql/* ->
  src/providers/mssql/queries/*, src/schema.js ->
  src/providers/mssql/schema.js, purge SQL -> purgeSql.js
- Update scripts, generate-schema-graph.js and package.json exports
  (./providers/mssql, ./providers/provider)
- Add TODO.md with the remaining provider/SQLite migration plan

No SQL or behavioral changes.
This commit is contained in:
seb
2026-09-20 13:51:01 +02:00
parent 67b3f8b6bf
commit 53a20c4353
184 changed files with 508 additions and 192 deletions

125
TODO.md Normal file
View File

@@ -0,0 +1,125 @@
# TODO — Database provider reorganization
Milestones 1–3 are done (provider contract, MSSQL provider, repo reorg).
Everything below is outstanding.
## 1. Convert store modules to the provider API (per area, keep behavior)
Replace raw `pool` usage in `src/store/**` with `provider.query()` /
`provider.transaction()` / `provider.exec()` and the normalized result shape
(`{ rows, rowsAffected, insertId }` instead of `recordset` / `rowsAffected[]`).
For every module:
- [ ] `pool.request().input(...).query(SQL)` → `provider.query(SQL, params)`
- [ ] `pool.transaction(); begin; commit; catch rollback` → `provider.transaction(async (tx) => { ... })`
- [ ] `result.recordset` → `result.rows`; `result.rowsAffected[0]` → `result.rowsAffected`
- [ ] `request.js` helper (`(tx ?? pool).request()`) becomes obsolete — delete when unused
- [ ] `recordset[0].id` after `OUTPUT INSERTED.id` → keep, but prefer `insertId` where trivial
Conversion order (bottom-up by dependency):
- [ ] `lang`, `tax` (leaf helpers, used by company create)
- [ ] `google_auth`
- [ ] `companyuser` (resolveCompanyAccess / resolveSelectedCompanyAccess first)
- [ ] `company`, `companysetting`
- [ ] `customergroup`, `entity`, `entityaddress`, `entitycontact`, `entityhistory`
- [ ] `unit` (incl. `ensureDefault`), `brand`, `supplier`
- [ ] `numbers` (see §2 — stored procs!)
- [ ] `category` (recursive CTEs!)
- [ ] `article` + articletext/articlepic/articleean/price/articlehistory
- [ ] `articlesupplier`
- [ ] `quote`, `order`, `delivery` (shared manifest helpers: `manifestLines`, `manifestDoc`, `manifestattachment`)
- [ ] `posterminal`, `pic`
- [ ] `initialize.js`, `purge.js` final wiring (drop `provider.pool` fallback from `index.js`)
- [ ] Delete `src/store/request.js` once no module uses it
## 2. Replace MSSQL stored procedures (blocker for SQLite)
- [ ] `numbers.nextNumber` uses proc `getnextnumberwithprefixandpadding` (output param)
- [ ] `numbers.peekNumber` uses proc `peeknextnumberwithprefixandpadding` (output param)
- [ ] Implement both in plain JS inside the store layer: `SELECT next/prefix/padding`,
compute `prefix + padded next`, `UPDATE numbers SET next = next + 1 WHERE ...`.
Must stay atomic: use `provider.transaction()` with row locking semantics
(MSSQL: `WITH (UPDLOCK, ROWLOCK)` in the SELECT; SQLite: within its write tx).
- [ ] Drop the proc definitions from schema/init scripts (`scripts/init-schema.js`).
## 3. provider contract additions / hygiene
- [ ] Add `isUniqueConstraintError` / `isForeignKeyConstraintError` to the
contract and use them in stores instead of raw mssql error numbers
(currently error messages like `Customer group in use`, `Brand in use`,
`Unit in use` rely on FK/unique detection — grep for these).
- [ ] Make `initialize()`/`purge()` real provider methods (schema migrations move
into the provider; store delegates). Decide per §5.
- [ ] Document provider contract (JSDoc typedef in `src/providers/provider.js`).
## 4. better-sqlite3 provider
- [ ] Add `src/providers/better-sqlite3/index.js` implementing the same contract:
- `query()` → prepare + `.all()`/`.get()`/`.run()`; normalize
`lastInsertRowid` → `insertId`, `changes` → `rowsAffected`
- `transaction(fn)` → `db.transaction()` wrapper (note: better-sqlite3 is sync;
wrap in async signature so store code stays unchanged)
- Named params: mssql uses `@name`; better-sqlite3 uses `:name` / `@name` /
`$name` — define one (e.g. `@name`) in SQLite SQL so params pass through
- Pragmas on open: `foreign_keys = ON`, `journal_mode = WAL`, `busy_timeout = 5000`
- `BLOB` handling: return `Buffer` for pic/image/attachment binaries
- Error classification: unique → `SQLITE_CONSTRAINT_UNIQUE`, FK → `SQLITE_CONSTRAINT_FOREIGNKEY`
- Do NOT close the caller-owned db in the provider; expose `close()`
- [ ] `better-sqlite3` as peerDependency (optional) in `package.json`
- [ ] package.json exports: `"./providers/better-sqlite3"`
## 5. SQLite schema (native, not mechanically translated)
- [ ] `src/providers/better-sqlite3/schema.js` + migrations:
- `INT IDENTITY` → `INTEGER PRIMARY KEY` (rowid)
- `NVARCHAR(n)` → `TEXT`; `VARBINARY(MAX)` → `BLOB`
- `DATETIME2 DEFAULT SYSUTCDATETIME()` → `TEXT` ISO-8601 UTC
(`strftime('%Y-%m-%dT%H:%M:%fZ','now')`)
- `IF NOT EXISTS ... sys.tables` → `CREATE TABLE IF NOT EXISTS`
- `OUTPUT INSERTED...` in queries → `RETURNING ...`
- `MERGE companysetting` → `INSERT ... ON CONFLICT ... DO UPDATE`
- `[identifier]` brackets → rename or quote (`"number"`)
- [ ] **Entity number uniqueness** (MSSQL uses indexed view `v_entity_company_number`):
- Add `company_id` column to `entity` (denormalized from customergroup, kept
in sync on insert + customergroup moves) and enforce
`CREATE UNIQUE INDEX uq_entity_company_number ON entity(company_id, number) WHERE number IS NOT NULL`
- Add the equivalent MSSQL migration for parity; remove the indexed view
- [ ] Versioned migrations via `meta` table (already exists) instead of pure
`IF NOT EXISTS` scripts for both providers
- [ ] Implement `initialize()` (migrations) and `purge()` (drop tables, FK-order)
in the SQLite provider; port `purgeSql.js` logic (currently topological
drop order — replicate)
## 6. SQL translation per area (SQLite queries)
- [ ] `TOP 1` → `LIMIT 1` (mind: TOP without ORDER BY — keep identical ordering)
- [ ] `OUTER APPLY (SELECT TOP 1 ...)` (article/category first-pic joins) →
correlated subquery or `JOIN` on min(sort,id)
- [ ] `FOR JSON PATH` (getBySku, listSearchIndex, listExport, getTreeForCompany) →
build JSON in JS from plain rows (preferred: keeps providers dumb)
- [ ] Recursive CTEs (`SEARCH_ARTICLES`, `RESOLVE_ARTICLE_ACCESS_BY_SKU`,
`GET_CATEGORY_TREE_FLAT`) → SQLite `WITH RECURSIVE` (mostly compatible)
- [ ] `ISNULL` / `COALESCE` differences; `FLOAT` vs `REAL`
- [ ] Case-insensitive comparisons relying on MSSQL default collation
(e.g. name-uniqueness checks) → `COLLATE NOCASE` in SQLite schema or
explicit `LOWER()` in both dialects
## 7. Contract / parity tests
- [ ] Test harness that runs the identical suite against both providers
(SQLite in-memory for speed; MSSQL in CI)
- [ ] Cover: return shapes, auth errors (`No company selected`, `Not authorized`),
domain errors, tx rollback, FK/unique behavior (`Brand in use` etc.),
binary round-trips (pic, manifestattachment, google_auth picturebinary),
date handling, recursive category tree, importRows two-pass V/K linking,
numbers.next/peek atomicity
- [ ] `scripts/test-api.js`: parametrize over provider
- [ ] `scripts/init-schema.js`, `reset-db.js`, `purge-db.js`: accept provider
selection (e.g. `--provider=better-sqlite3` / env `DASH_DB_PROVIDER`)
## 8. Docs & tooling
- [ ] README: provider-based usage, both examples, migration notes for dashback
- [ ] `API.md` / `API.*.md` generators (`generate-api-doc.js`): update paths if
they scan `src/sql` / `src/schema`
- [ ] `generate-schema-graph.js` / `schema-graph.dot`: point at both schemas
- [ ] Deprecation note: `new LibDashStore(pool)` removed — constructor now takes
a provider (breaking change; coordinate with dashback)
## 9. Cleanup
- [ ] Remove transitional `this.pool` / `provider.pool` usage from `index.js`
- [ ] Remove `mssql` from peerDependencies only if no runtime coupling remains
(queries/schema stay mssql-specific, so likely keep it)
- [ ] Decide export for `SCHEMA_SCRIPTS` (currently re-exported from `index.js`
from the moved schema file)