dashlibstorage/TODO.md

TODO — Database provider reorganization

Milestones 1–3 are done (provider contract, MSSQL provider, repo reorg). Everything below is outstanding.

1. Convert store modules to the provider API (per area, keep behavior)

Replace raw pool usage in src/store/** with provider.query() / provider.transaction() / provider.exec() and the normalized result shape ({ rows, rowsAffected, insertId } instead of recordset / rowsAffected[]). For every module:

• pool.request().input(...).query(SQL) → provider.query(SQL, params)
• pool.transaction(); begin; commit; catch rollback → provider.transaction(async (tx) => { ... })
• result.recordset → result.rows; result.rowsAffected[0] → result.rowsAffected
• request.js helper ((tx ?? pool).request()) becomes obsolete — delete when unused
• recordset[0].id after OUTPUT INSERTED.id → keep, but prefer insertId where trivial

Conversion order (bottom-up by dependency):

• lang, tax (leaf helpers, used by company create)
• google_auth
• companyuser (resolveCompanyAccess / resolveSelectedCompanyAccess first)
• company, companysetting
• customergroup, entity, entityaddress, entitycontact, entityhistory
• unit (incl. ensureDefault), brand, supplier
• numbers (see §2 — stored procs!)
• category (recursive CTEs!)
• article + articletext/articlepic/articleean/price/articlehistory
• articlesupplier
• quote, order, delivery (shared manifest helpers: manifestLines, manifestDoc, manifestattachment)
• posterminal, pic
• initialize.js, purge.js final wiring (drop provider.pool fallback from index.js)
• Delete src/store/request.js once no module uses it

2. Replace MSSQL stored procedures (blocker for SQLite)

• numbers.nextNumber uses proc getnextnumberwithprefixandpadding (output param)
• numbers.peekNumber uses proc peeknextnumberwithprefixandpadding (output param)
• Implement both in plain JS inside the store layer: SELECT next/prefix/padding, compute prefix + padded next, UPDATE numbers SET next = next + 1 WHERE .... Must stay atomic: use provider.transaction() with row locking semantics (MSSQL: WITH (UPDLOCK, ROWLOCK) in the SELECT; SQLite: within its write tx).
• Drop the proc definitions from schema/init scripts (scripts/init-schema.js).

3. provider contract additions / hygiene

• Add isUniqueConstraintError / isForeignKeyConstraintError to the contract and use them in stores instead of raw mssql error numbers (currently error messages like Customer group in use, Brand in use, Unit in use rely on FK/unique detection — grep for these).
• Make initialize()/purge() real provider methods (schema migrations move into the provider; store delegates). Decide per §5.
• Document provider contract (JSDoc typedef in src/providers/provider.js).

4. better-sqlite3 provider

• Add src/providers/better-sqlite3/index.js implementing the same contract:
  • query() → prepare + .all()/.get()/.run(); normalize lastInsertRowid → insertId, changes → rowsAffected
  • transaction(fn) → db.transaction() wrapper (note: better-sqlite3 is sync; wrap in async signature so store code stays unchanged)
  • Named params: mssql uses @name; better-sqlite3 uses :name / @name / $name — define one (e.g. @name) in SQLite SQL so params pass through
  • Pragmas on open: foreign_keys = ON, journal_mode = WAL, busy_timeout = 5000
  • BLOB handling: return Buffer for pic/image/attachment binaries
  • Error classification: unique → SQLITE_CONSTRAINT_UNIQUE, FK → SQLITE_CONSTRAINT_FOREIGNKEY
  • Do NOT close the caller-owned db in the provider; expose close()
• better-sqlite3 as peerDependency (optional) in package.json
• package.json exports: "./providers/better-sqlite3"

5. SQLite schema (native, not mechanically translated)

• src/providers/better-sqlite3/schema.js + migrations:
  • INT IDENTITY → INTEGER PRIMARY KEY (rowid)
  • NVARCHAR(n) → TEXT; VARBINARY(MAX) → BLOB
  • DATETIME2 DEFAULT SYSUTCDATETIME() → TEXT ISO-8601 UTC (strftime('%Y-%m-%dT%H:%M:%fZ','now'))
  • IF NOT EXISTS ... sys.tables → CREATE TABLE IF NOT EXISTS
  • OUTPUT INSERTED... in queries → RETURNING ...
  • MERGE companysetting → INSERT ... ON CONFLICT ... DO UPDATE
  • [identifier] brackets → rename or quote ("number")
• Entity number uniqueness (MSSQL uses indexed view v_entity_company_number):
  • Add company_id column to entity (denormalized from customergroup, kept in sync on insert + customergroup moves) and enforce CREATE UNIQUE INDEX uq_entity_company_number ON entity(company_id, number) WHERE number IS NOT NULL
  • Add the equivalent MSSQL migration for parity; remove the indexed view
• Versioned migrations via meta table (already exists) instead of pure IF NOT EXISTS scripts for both providers
• Implement initialize() (migrations) and purge() (drop tables, FK-order) in the SQLite provider; port purgeSql.js logic (currently topological drop order — replicate)

6. SQL translation per area (SQLite queries)

• TOP 1 → LIMIT 1 (mind: TOP without ORDER BY — keep identical ordering)
• OUTER APPLY (SELECT TOP 1 ...) (article/category first-pic joins) → correlated subquery or JOIN on min(sort,id)
• FOR JSON PATH (getBySku, listSearchIndex, listExport, getTreeForCompany) → build JSON in JS from plain rows (preferred: keeps providers dumb)
• Recursive CTEs (SEARCH_ARTICLES, RESOLVE_ARTICLE_ACCESS_BY_SKU, GET_CATEGORY_TREE_FLAT) → SQLite WITH RECURSIVE (mostly compatible)
• ISNULL / COALESCE differences; FLOAT vs REAL
• Case-insensitive comparisons relying on MSSQL default collation (e.g. name-uniqueness checks) → COLLATE NOCASE in SQLite schema or explicit LOWER() in both dialects

7. Contract / parity tests

• Test harness that runs the identical suite against both providers (SQLite in-memory for speed; MSSQL in CI)
• Cover: return shapes, auth errors (No company selected, Not authorized), domain errors, tx rollback, FK/unique behavior (Brand in use etc.), binary round-trips (pic, manifestattachment, google_auth picturebinary), date handling, recursive category tree, importRows two-pass V/K linking, numbers.next/peek atomicity
• scripts/test-api.js: parametrize over provider
• scripts/init-schema.js, reset-db.js, purge-db.js: accept provider selection (e.g. --provider=better-sqlite3 / env DASH_DB_PROVIDER)

8. Docs & tooling

• README: provider-based usage, both examples, migration notes for dashback
• API.md / API.*.md generators (generate-api-doc.js): update paths if they scan src/sql / src/schema
• generate-schema-graph.js / schema-graph.dot: point at both schemas
• Deprecation note: new LibDashStore(pool) removed — constructor now takes a provider (breaking change; coordinate with dashback)

9. Cleanup

• Remove transitional this.pool / provider.pool usage from index.js
• Remove mssql from peerDependencies only if no runtime coupling remains (queries/schema stay mssql-specific, so likely keep it)
• Decide export for SCHEMA_SCRIPTS (currently re-exported from index.js from the moved schema file)