u
This commit is contained in:
seb
101
c++_port_of_jtlsrv_f5ce4ce8.plan.md
Normal file
101
c++_port_of_jtlsrv_f5ce4ce8.plan.md
Normal file
@@ -0,0 +1,101 @@
|
||||
---
|
||||
name: C++ port of jtlsrv
|
||||
overview: Port the Node.js JTL-POS sync server (~1,400 LOC) to C++17 on Linux, using libuv for the event loop, OpenSSL for HTTPS, unixODBC + msodbcsql18 for MSSQL, and libvips for image resizing — with all blocking work (ODBC, vips) dispatched to uv_queue_work threads.
|
||||
todos:
|
||||
- id: skeleton
|
||||
content: CMake project, .env config loader, logger, libuv TCP skeleton
|
||||
status: pending
|
||||
- id: tls-http
|
||||
content: TLS layer (OpenSSL memory-BIO over uv_tcp) + llhttp request parsing
|
||||
status: pending
|
||||
- id: router
|
||||
content: Router, pairing store, client/init endpoints (no DB)
|
||||
status: pending
|
||||
- id: odbc
|
||||
content: ODBC connection pool + uv_queue_work plumbing + simple queries
|
||||
status: pending
|
||||
- id: queries
|
||||
content: Port remaining list queries and JSON shaping (product, category, composite, deleted, attributes)
|
||||
status: pending
|
||||
- id: images
|
||||
content: "Image endpoints: blob fetch + libvips thumbnail in worker threads"
|
||||
status: pending
|
||||
- id: polish
|
||||
content: Request/order logging, init suppression, graceful shutdown, side-by-side verification vs Node server
|
||||
status: pending
|
||||
isProject: false
|
||||
---
|
||||
|
||||
# Port jtlsrv to C++ (libuv + ODBC + libvips)
|
||||
|
||||
## Answers to your questions
|
||||
|
||||
- **libuv?** Yes. Single event-loop thread owns all sockets; libuv's built-in thread pool (`uv_queue_work`) handles blocking work. Set `UV_THREADPOOL_SIZE=8` or so.
|
||||
- **MSSQL via ODBC?** Yes — unixODBC + Microsoft's `msodbcsql18` driver. It supports named parameters' equivalent (`?` placeholders via `SQLBindParameter`), reading `varbinary(max)` image blobs with `SQLGetData` in chunks, and TLS with `TrustServerCertificate=yes` matching the current `.env` options.
|
||||
- **ODBC in `uv_queue_work()`?** Yes, mandatory — ODBC calls are fully blocking. Maintain a small connection pool (e.g. 4 `SQLHDBC` handles guarded by a mutex/semaphore); each work item checks out a connection, runs the query, marshals rows into plain structs, and the after-work callback (back on the loop thread) builds JSON and writes the response.
|
||||
- **Image processing in `uv_queue_work()`?** Yes — same work item as the DB fetch: fetch blob via ODBC, then `vips_thumbnail_buffer()` (200px, fit-inside, no enlargement — exact equivalent of the sharp call in [src/image-resize.js](src/image-resize.js)), return the encoded buffer to the loop thread for sending.
|
||||
|
||||
## Caveat worth knowing upfront
|
||||
|
||||
libuv has **no TLS support**. The one genuinely new piece of work in this port is an HTTPS layer: OpenSSL memory-BIOs pumped over `uv_tcp_t` (~300 lines, well-trodden pattern), plus **llhttp** (Node's own HTTP parser, plain C) for request parsing. Everything else is a mechanical translation. If you'd rather skip that, `libhv` gives you a libuv-style loop with HTTPS built in — but the plan below assumes plain libuv as requested.
|
||||
|
||||
## Dependencies (all in Ubuntu/Debian repos except the MS driver)
|
||||
|
||||
- `libuv1-dev`, `libssl-dev`, `libvips-dev`, `unixodbc-dev`, `msodbcsql18` (MS apt repo)
|
||||
- Vendored single-header/small: `llhttp` (HTTP parser), `nlohmann/json` (or `yyjson` if you want max speed; nlohmann is fine at this scale)
|
||||
- Build: CMake ≥ 3.16
|
||||
|
||||
## Project layout
|
||||
|
||||
```
|
||||
jtlsrv-cpp/
|
||||
CMakeLists.txt
|
||||
src/
|
||||
main.cpp <- server.js: config, signals, startup
|
||||
config.hpp <- .env loader (tiny hand-rolled parser, ~40 lines)
|
||||
log.{hpp,cpp} <- logger.js + request-log.js + order-log.js
|
||||
tls_server.{hpp,cpp}<- uv_tcp + OpenSSL BIO pump (the new part)
|
||||
http.{hpp,cpp} <- llhttp glue; Request/Response structs; send_json/send_binary
|
||||
router.{hpp,cpp} <- jtl-server.js: method+path map, init-log suppression
|
||||
pairing.hpp <- pairing.js (trivial in-memory map)
|
||||
db/
|
||||
pool.{hpp,cpp} <- ODBC env + connection pool, work-item helpers
|
||||
row.hpp <- variant-ish cell type (int64/double/string/blob/null)
|
||||
endpoints/ <- one .cpp per file in src/endpoints/
|
||||
queries/ <- one .cpp per file in src/queries/ (SQL strings copied verbatim,
|
||||
@name params -> ? placeholders)
|
||||
image.{hpp,cpp} <- image-handler.js + image-resize.js via vips_thumbnail_buffer
|
||||
```
|
||||
|
||||
## Concurrency model
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Client -->|TLS| LoopThread["libuv loop thread: uv_tcp + OpenSSL BIO + llhttp + routing"]
|
||||
LoopThread -->|uv_queue_work| Workers["uv threadpool workers"]
|
||||
Workers -->|checkout SQLHDBC| OdbcPool["ODBC connection pool (4 conns)"]
|
||||
Workers -->|"vips_thumbnail_buffer (image endpoints)"| Vips[libvips]
|
||||
Workers -->|after_work: rows/buffer| LoopThread
|
||||
LoopThread -->|JSON / binary response| Client
|
||||
```
|
||||
|
||||
Rules: no libuv handle touched off-loop; work items own their input/output structs (heap-allocated, freed in after-work); JSON serialization happens on the loop thread (cheap at these payload sizes); errors in workers carried back as a status + message field, mapped to the same 500/404 JSON bodies as [src/jtl-server.js](src/jtl-server.js).
|
||||
|
||||
## Translation notes per area
|
||||
|
||||
- **HTTPS/TLS**: load `certs/key.pem`/`cert.pem` as now; `SSL_CTX` with TLS ≥ 1.2. Keep-alive supported via llhttp message-complete callbacks.
|
||||
- **Router**: same normalization as [src/http.js](src/http.js) (strip leading `/api` before `/v1/`), `"METHOD path"` key into an `unordered_map`.
|
||||
- **Queries**: the 12 files in [src/queries](src/queries) port 1:1. `STRING_AGG`, CTEs etc. stay server-side, untouched. The dynamic `IN (...)` in `product-list.js` price overrides stays string-built from integer IDs (safe, they come from the DB). Post-processing (price/gross calc, attribute maps) becomes plain C++ over row structs.
|
||||
- **Images**: `getImageByHash` reads `bBild`/`bVorschauBild` blobs (chunked `SQLGetData` into `std::vector<uint8_t>`), picks preview vs full exactly as [src/queries/image.js](src/queries/image.js), then vips thumbnail keeping the source format's encoder (jpg/png/gif/webp — libvips handles all).
|
||||
- **Logging**: `requests.log` / `orders.log` appended via `uv_fs_write` or plain buffered `FILE*` on the loop thread (writes are tiny); replicate the init-suppression window logic from [server.js](server.js) verbatim.
|
||||
- **Shutdown**: SIGINT/SIGTERM via `uv_signal_t` → stop accepting, drain pool, `uv_stop`.
|
||||
|
||||
## Milestones (each independently testable)
|
||||
|
||||
1. Skeleton: CMake, config/env, logger, plain-TCP libuv echo — builds and runs.
|
||||
2. TLS + llhttp layer serving a hardcoded 404 JSON — verify with `curl -k`.
|
||||
3. Router + pairing + the no-DB endpoints (`client`, `init`) — the existing [test-client.js](test-client.js) should pass against it.
|
||||
4. ODBC pool + `uv_queue_work` plumbing + `shop`/`customer-groups`/counts queries.
|
||||
5. Remaining list queries (product, category, composite, deleted-entity, attributes).
|
||||
6. Image endpoints with libvips; byte-compare output against the Node server using [check-image-rv.mjs](check-image-rv.mjs)-style spot checks.
|
||||
7. Request/order logs, init-log suppression, graceful shutdown; side-by-side diff of responses vs the Node server on a real POS sync.
|
||||
Reference in New Issue
Block a user