136 lines
8.2 KiB
Markdown
136 lines
8.2 KiB
Markdown
# TODO — Database provider reorganization
|
||
|
||
Milestones 1–3 are done (provider contract, MSSQL provider, repo reorg).
|
||
SQLite provider, native schema, and query translations now exist; store wiring,
|
||
migrations, lifecycle methods, and parity tests remain outstanding.
|
||
|
||
## 1. Convert store modules to the provider API (per area, keep behavior)
|
||
Replace raw `pool` usage in `src/store/**` with `provider.query()` /
|
||
`provider.transaction()` / `provider.exec()` and the normalized result shape
|
||
(`{ rows, rowsAffected, insertId }` instead of `recordset` / `rowsAffected[]`).
|
||
For every module:
|
||
- [ ] `pool.request().input(...).query(SQL)` → `provider.query(SQL, params)`
|
||
- [ ] `pool.transaction(); begin; commit; catch rollback` → `provider.transaction(async (tx) => { ... })`
|
||
- [ ] `result.recordset` → `result.rows`; `result.rowsAffected[0]` → `result.rowsAffected`
|
||
- [ ] `request.js` helper (`(tx ?? pool).request()`) becomes obsolete — delete when unused
|
||
- [x] `src/store/runner.js` bridges mixed call sites: converted modules accept a
|
||
provider runner *or* a legacy mssql Transaction/`pool` (wrapped on the fly)
|
||
so partially converted callers keep working
|
||
- [ ] `recordset[0].id` after `OUTPUT INSERTED.id` → keep, but prefer `insertId` where trivial
|
||
|
||
Conversion order (bottom-up by dependency):
|
||
- [x] `lang`, `tax` (leaf helpers, used by company create)
|
||
- [x] `google_auth`
|
||
- [ ] `companyuser` (resolveCompanyAccess / resolveSelectedCompanyAccess first)
|
||
- [ ] `company`, `companysetting`
|
||
- [ ] `customergroup`, `entity`, `entityaddress`, `entitycontact`, `entityhistory`
|
||
- [ ] `unit` (incl. `ensureDefault`), `brand`, `supplier`
|
||
- [ ] `numbers` (see §2 — stored procs!)
|
||
- [ ] `category` (recursive CTEs!)
|
||
- [ ] `article` + articletext/articlepic/articleean/price/articlehistory
|
||
- [ ] `articlesupplier`
|
||
- [ ] `quote`, `order`, `delivery` (shared manifest helpers: `manifestLines`, `manifestDoc`, `manifestattachment`)
|
||
- [ ] `posterminal`, `pic`
|
||
- [ ] `initialize.js`, `purge.js` final wiring (drop `provider.pool` fallback from `index.js`)
|
||
- [ ] Delete `src/store/request.js` once no module uses it
|
||
|
||
## 2. Replace MSSQL stored procedures (blocker for SQLite)
|
||
- [ ] `numbers.nextNumber` uses proc `getnextnumberwithprefixandpadding` (output param)
|
||
- [ ] `numbers.peekNumber` uses proc `peeknextnumberwithprefixandpadding` (output param)
|
||
- [ ] Implement both in plain JS inside the store layer: `SELECT next/prefix/padding`,
|
||
compute `prefix + padded next`, `UPDATE numbers SET next = next + 1 WHERE ...`.
|
||
Must stay atomic: use `provider.transaction()` with row locking semantics
|
||
(MSSQL: `WITH (UPDLOCK, ROWLOCK)` in the SELECT; SQLite: within its write tx).
|
||
- [ ] Drop the proc definitions from schema/init scripts (`scripts/init-schema.js`).
|
||
|
||
## 3. provider contract additions / hygiene
|
||
- [ ] Add `isUniqueConstraintError` / `isForeignKeyConstraintError` to the
|
||
contract and use them in stores instead of raw mssql error numbers
|
||
(currently error messages like `Customer group in use`, `Brand in use`,
|
||
`Unit in use` rely on FK/unique detection — grep for these).
|
||
- [ ] Make `initialize()`/`purge()` real provider methods (schema migrations move
|
||
into the provider; store delegates). Decide per §5.
|
||
- [ ] Document provider contract (JSDoc typedef in `src/providers/provider.js`).
|
||
|
||
## 4. better-sqlite3 provider
|
||
- [x] Add `src/providers/better-sqlite3/index.js` implementing the same contract:
|
||
- `query()` → prepare + `.all()`/`.get()`/`.run()`; normalize
|
||
`lastInsertRowid` → `insertId`, `changes` → `rowsAffected`
|
||
- `transaction(fn)` → `db.transaction()` wrapper (note: better-sqlite3 is sync;
|
||
wrap in async signature so store code stays unchanged)
|
||
- Named params: mssql uses `@name`; better-sqlite3 uses `:name` / `@name` /
|
||
`$name` — define one (e.g. `@name`) in SQLite SQL so params pass through
|
||
- Pragmas on open: `foreign_keys = ON`, `journal_mode = WAL`, `busy_timeout = 5000`
|
||
- `BLOB` handling: return `Buffer` for pic/image/attachment binaries
|
||
- Error classification: unique → `SQLITE_CONSTRAINT_UNIQUE`, FK → `SQLITE_CONSTRAINT_FOREIGNKEY`
|
||
- Do NOT close the caller-owned db in the provider; expose `close()`
|
||
- [x] `better-sqlite3` as peerDependency (optional) in `package.json`
|
||
- [x] package.json exports: `"./providers/better-sqlite3"`
|
||
|
||
## 5. SQLite schema (native, not mechanically translated)
|
||
- [x] Native schema in `src/providers/sqlite3/schema.js`:
|
||
- `INT IDENTITY` → `INTEGER PRIMARY KEY` (rowid)
|
||
- `NVARCHAR(n)` → `TEXT`; `VARBINARY(MAX)` → `BLOB`
|
||
- `DATETIME2 DEFAULT SYSUTCDATETIME()` → `TEXT` ISO-8601 UTC
|
||
(`strftime('%Y-%m-%dT%H:%M:%fZ','now')`)
|
||
- `IF NOT EXISTS ... sys.tables` → `CREATE TABLE IF NOT EXISTS`
|
||
- `OUTPUT INSERTED...` in queries → `RETURNING ...`
|
||
- `MERGE companysetting` → `INSERT ... ON CONFLICT ... DO UPDATE`
|
||
- `[identifier]` brackets → rename or quote (`"number"`)
|
||
- [ ] Add versioned SQLite migrations around the native schema
|
||
- [ ] **Entity number uniqueness parity** (MSSQL still uses indexed view
|
||
`v_entity_company_number`):
|
||
- [x] SQLite: add `company_id` to `entity` (denormalized from customergroup, kept
|
||
in sync on insert + customergroup moves) and enforce
|
||
`CREATE UNIQUE INDEX uq_entity_company_number ON entity(company_id, number) WHERE number IS NOT NULL`
|
||
- [ ] Add the equivalent MSSQL migration for parity; remove the indexed view
|
||
- [ ] Versioned migrations via `meta` table (already exists) instead of pure
|
||
`IF NOT EXISTS` scripts for both providers
|
||
- [ ] Implement `initialize()` (migrations) and `purge()` (drop tables, FK-order)
|
||
in the SQLite provider; port `purgeSql.js` logic (currently topological
|
||
drop order — replicate)
|
||
|
||
## 6. SQL translation per area (SQLite queries)
|
||
- [x] Add all 32 modules under `src/providers/sqlite3/queries/`, preserving the
|
||
MSSQL query modules' named-export API (plus SQLite number helpers)
|
||
- [x] `TOP 1` → `LIMIT 1` (mind: TOP without ORDER BY — keep identical ordering)
|
||
- [x] `OUTER APPLY (SELECT TOP 1 ...)` (article/category first-pic joins) →
|
||
correlated subquery or `JOIN` on min(sort,id)
|
||
- [x] `FOR JSON PATH` queries → SQLite JSON1 aggregation with shapes compatible
|
||
with the current store parsers (building JSON in JS remains a future cleanup)
|
||
- [x] Recursive CTEs (`SEARCH_ARTICLES`, `RESOLVE_ARTICLE_ACCESS_BY_SKU`,
|
||
`GET_CATEGORY_TREE_FLAT`) → SQLite `WITH RECURSIVE` (mostly compatible)
|
||
- [x] `ISNULL` / `COALESCE` differences; `FLOAT` vs `REAL`
|
||
- [x] Case-insensitive comparisons relying on MSSQL default collation
|
||
(e.g. name-uniqueness checks) → `COLLATE NOCASE` in SQLite schema or
|
||
explicit `LOWER()` in both dialects
|
||
- [x] Compile all 212 complete SQLite query exports against an in-memory native
|
||
schema (zero syntax/schema errors)
|
||
|
||
## 7. Contract / parity tests
|
||
- [ ] Test harness that runs the identical suite against both providers
|
||
(SQLite in-memory for speed; MSSQL in CI)
|
||
- [ ] Cover: return shapes, auth errors (`No company selected`, `Not authorized`),
|
||
domain errors, tx rollback, FK/unique behavior (`Brand in use` etc.),
|
||
binary round-trips (pic, manifestattachment, google_auth picturebinary),
|
||
date handling, recursive category tree, importRows two-pass V/K linking,
|
||
numbers.next/peek atomicity
|
||
- [ ] `scripts/test-api.js`: parametrize over provider
|
||
- [ ] `scripts/init-schema.js`, `reset-db.js`, `purge-db.js`: accept provider
|
||
selection (e.g. `--provider=better-sqlite3` / env `DASH_DB_PROVIDER`)
|
||
|
||
## 8. Docs & tooling
|
||
- [ ] README: provider-based usage, both examples, migration notes for dashback
|
||
- [ ] `API.md` / `API.*.md` generators (`generate-api-doc.js`): update paths if
|
||
they scan `src/sql` / `src/schema`
|
||
- [ ] `generate-schema-graph.js` / `schema-graph.dot`: point at both schemas
|
||
- [ ] Deprecation note: `new LibDashStore(pool)` removed — constructor now takes
|
||
a provider (breaking change; coordinate with dashback)
|
||
|
||
## 9. Cleanup
|
||
- [ ] Remove transitional `this.pool` / `provider.pool` usage from `index.js`
|
||
- [ ] Remove `mssql` from peerDependencies only if no runtime coupling remains
|
||
(queries/schema stay mssql-specific, so likely keep it)
|
||
- [ ] Decide export for `SCHEMA_SCRIPTS` (currently re-exported from `index.js`
|
||
from the moved schema file)
|