seb/fibdash
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Remove data.csv file and update README to reflect new features including CSV import and banking account management. Enhance TransactionsTable and KreditorTable components with banking account handling, including UI updates and validation logic. Update SQL schema to support banking accounts and adjust API routes for improved data handling. Implement new document rendering logic for banking transactions and enhance recipient rendering with banking account status. Add new views and indexes for better transaction management.

Browse Source
This commit is contained in:
sebseb7
2025-08-01 13:26:26 +02:00
parent 6cde543938
commit fbfd918d81
13 changed files with 1774 additions and 1409 deletions
Download Patch File Download Diff File Expand all files Collapse all files

478
README.md
View File

@@ -1,18 +1,20 @@
# FibDash
A modern React Material-UI dashboard application with Google SSO authentication and MSSQL database integration.
A modern React Material-UI dashboard for financial reconciliation with Google SSO authentication, CSV import/analysis, DATEV export, and optional MSSQL integration with JTL tables.
## Features
- 🚀 **React 18** with class components
- 🎨 **Material-UI (MUI)** for beautiful, modern UI
- 🔐 **Google SSO Authentication**
- 🗄️ **MSSQL Database** integration
- ⚡ **Webpack Dev Server** with Hot Module Reload
- 🔄 **Express API** with hot reload via nodemon
- 🛡️ **JWT Authentication** for API security
- 📱 **Responsive Design**
- 🎯 **Production Ready** build configuration
- 🚀 React 18 (class components)
- 🎨 Material-UI (MUI) UI with responsive layout
- 🔐 Google SSO (Google Identity Services) + JWT API auth
- 🗄️ MSSQL integration (optional; app runs without DB)
- 📥 CSV import of bank transactions (German MT940-like CSV)
- 🔍 Reconciliation view: CSV vs JTL (if DB available)
- 📤 DATEV export for selected month/quarter/year
- 🧩 Admin data management (Kreditor, Konto, BU)
- ⚡ Webpack dev server (HMR) + nodemon hot-reload
- 🛡️ Email allowlist authorization
- 🧰 Production single-process build (Express serves React)
## Architecture
@@ -27,337 +29,249 @@ fibdash/
│ └── public/
│ └── index.html # HTML template
├── src/ # Backend Express API
│ ├── config/ # Database configuration
│ ├── middleware/ # Authentication middleware
│ ├── routes/ # API routes
│ ├── database/ # Database schema
│ ├── config/ # Database configuration (MSSQL)
│ ├── middleware/ # Auth + email allowlist middleware
│ ├── routes/ # API routes (auth, data, admin, dashboard)
│ ├── database/ # SQL schema and CSV import schema
│ └── index.js # Express server entry point
├── webpack.config.js # Webpack configuration
├── nginx.*.conf # Nginx reverse-proxy configs (dev/prod/simple)
├── webpack*.config.js # Webpack configs
├── docker-compose.dev.yml # Optional dev docker-compose for proxying
├── data.csv # Sample CSV for local analysis
└── package.json # Dependencies and scripts
```
## Functional Overview
Authentication and Authorization
- Login: Frontend uses Google Identity Services. The backend validates the ID token and issues a JWT.
- Email allowlist: Only emails in AUTHORIZED_EMAILS or matching DB rule are allowed.
- ENV allowlist: fast path check.
- DB check: optional, queries JTL tables to verify access by attributes 219/220.
- JWT middleware guards all /api routes.
CSV Analysis and Reconciliation
- Upload or place CSV at project root (data.csv) for quick testing.
- CSV parsing: German semicolon-separated format with headers like Buchungstag, Betrag, Verwendungszweck, IBAN, etc.
- Reconciliation:
- If MSSQL available: fetch JTL transactions (tZahlungsabgleichUmsatz), related PDFs (tUmsatzBeleg, tPdfObjekt), and links; match by date+amount.
- Kreditor lookup: optional mapping via fibdash.Kreditor using IBAN; also supports banking accounts via is_banking.
- Summary totals: income, expenses, net, match counts.
DATEV Export
- Endpoint returns CSV in DATEV format for the chosen period (month, quarter, year).
- Headers and column mapping created server-side; amounts normalized; encoding served as text/csv.
Admin Management
- Kreditor: CRUD for name, kreditorId, IBAN (optional if is_banking=true).
- Konto: CRUD account numbers/names used for accounting.
- BU (Buchungsschlüssel): CRUD including optional VSt.
- System info endpoint.
Health
- /api/health returns OK and timestamp.
## API Surface (key endpoints)
Auth
- POST /api/auth/google — Google token exchange → JWT
- GET /api/auth/verify — Validate JWT, returns user
- POST /api/auth/logout — Stateless logout success
Dashboard
- GET /api/dashboard — Mock dashboard stats
- GET /api/dashboard/user — Returns current JWT user
Data and Reconciliation
- GET /api/data/months — Available months inferred from data.csv
- GET /api/data/transactions/:timeRange — Combined CSV + JTL view with summary
- timeRange supports YYYY-MM, YYYY, YYYY-Q1..Q4
- GET /api/data/datev/:timeRange — Download DATEV CSV
- GET /api/data/pdf/umsatzbeleg/:kUmsatzBeleg — Stream PDF from tUmsatzBeleg
- GET /api/data/pdf/pdfobject/:kPdfObjekt — Stream PDF from tPdfObjekt
Kreditor, Konto, BU
- GET /api/data/kreditors
- GET /api/data/kreditors/:id
- POST /api/data/kreditors
- PUT /api/data/kreditors/:id
- DELETE /api/data/kreditors/:id
- GET /api/data/assignable-kreditors — only non-banking
- BankingAccountTransactions assignments: POST/PUT/DELETE, and GET /api/data/banking-transactions/:transactionId
- Admin counterparts exist under /api/admin for Kreditor/Konto/BU CRUD.
CSV Import (to DB)
- POST /api/data/import-csv-transactions — Validates rows and inserts into fibdash.CSVTransactions
- GET /api/data/csv-transactions — Paginated list with kreditor joins and assignment info
- GET /api/data/csv-import-batches — Import batch summaries
## Frontend UX Summary
App shell
- Top AppBar with title, tabs (Dashboard, Stammdaten), current user, DATEV export button when applicable, and logout.
Login
- Button triggers Google prompt; robust error messaging for SSO, service unavailability, authorization.
Dashboard view
- Month selector, summary header, and a transactions table with reconciliation indicators (CSV-only, JTL-only, matched), PDF links when available.
Stammdaten view
- Management UI for Kreditors, Konten, and Buchungsschlüssel (class-based components under client/src/components/admin).
CSV Import
- Modal dialog with drag-and-drop or file picker, header detection, basic validation, progress, and results summary; uses /api/data/import-csv-transactions.
## Prerequisites
- Node.js (v16 or higher)
- MSSQL Server instance
- Google Cloud Platform account for OAuth setup
- Node.js (v16+)
- Optionally: MSSQL Server for JTL and fibdash schema
- Google Cloud project and OAuth 2.0 Client ID
## Setup Instructions
### 1. Clone and Install Dependencies
## Setup
1) Clone and install
```bash
git clone <your-repo-url>
cd fibdash
npm install
```
### 2. Environment Configuration
Copy the example environment file and configure your settings:
2) Environment
```bash
cp .env.example .env
# then edit .env
```
Edit `.env` with your actual configuration:
Required variables
- GOOGLE_CLIENT_ID, GOOGLE_CLIENT_SECRET
- REACT_APP_GOOGLE_CLIENT_ID (must match GOOGLE_CLIENT_ID)
- JWT_SECRET
- Optional authorization: AUTHORIZED_EMAILS=admin@company.com,user@company.com
- MSSQL: DB_SERVER, DB_DATABASE, DB_USERNAME, DB_PASSWORD, DB_PORT=1433
- Server: PORT=5000, NODE_ENV=development
```env
# Google OAuth Configuration
GOOGLE_CLIENT_ID=your_google_client_id_here
GOOGLE_CLIENT_SECRET=your_google_client_secret_here
3) Google OAuth
- Create OAuth 2.0 Web Client in Google Cloud Console.
- Authorized origins: http://localhost:5001 (dev), your domain(s).
- Authorized redirects: matching roots (e.g., http://localhost:5001/).
- See detailed guide: [docs/GOOGLE_OAUTH_SETUP.md](docs/GOOGLE_OAUTH_SETUP.md)
# Frontend Environment Variables (REACT_APP_ prefix required)
REACT_APP_GOOGLE_CLIENT_ID=your_google_client_id_here
# JWT Secret (generate a secure random string)
JWT_SECRET=your_jwt_secret_here
# MSSQL Database Configuration
DB_SERVER=your_mssql_server_here
DB_DATABASE=your_database_name_here
DB_USERNAME=your_db_username_here
DB_PASSWORD=your_db_password_here
DB_PORT=1433
# Server Configuration
PORT=5000
NODE_ENV=development
```
### 3. Google OAuth Setup
1. Go to the [Google Cloud Console](https://console.cloud.google.com/)
2. Create a new project or select an existing one
3. Enable the Google+ API
4. Create OAuth 2.0 credentials:
- Application type: Web application
- Authorized JavaScript origins: `http://localhost:3000`
- Authorized redirect URIs: `http://localhost:3000`
5. Copy the Client ID and Client Secret to your `.env` file
### 4. Database Setup
1. Create a new MSSQL database
2. Run the schema creation script:
4) Database (optional but required for JTL features)
- Create DB and run schemas:
- Core schema: see src/database/schema.sql
- CSV import table schemas if needed
- The app will run without DB; JTL features and admin CRUD will error if DB not configured.
5) Development
```bash
# Connect to your MSSQL server and run:
sqlcmd -S your_server -d your_database -i src/database/schema.sql
```
Or manually execute the SQL commands in `src/database/schema.sql`
### 5. Start Development Servers
Run both frontend and backend development servers:
```bash
npm run dev
```
This will start:
- Frontend dev server on `http://localhost:5001` (with hot reload)
- Backend API server on `http://localhost:5000` (with nodemon auto-restart)
Or run them separately:
```bash
# Frontend only
npm run dev # runs frontend at 5001 and backend at 5000
# or separately:
npm run dev:frontend
# Backend only
npm run dev:backend
```
### 6. Optional: Nginx Setup for Development
For a more production-like development environment, you can set up nginx as a reverse proxy:
#### Automatic Setup (Linux/macOS)
6) Optional Nginx for dev
- Automatic:
```bash
npm run setup:nginx
```
#### Manual Setup
1. Install nginx on your system
2. Copy the nginx configuration:
- Manual:
```bash
sudo cp nginx.simple.conf /etc/nginx/sites-available/fibdash-dev
sudo ln -s /etc/nginx/sites-available/fibdash-dev /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx
```
- Hosts entry (optional): 127.0.0.1 fibdash.local
3. Add to your hosts file (optional):
With nginx:
- App: http://localhost/ or http://fibdash.local/
- API: http://localhost/api/
- Direct FE: http://localhost:5001/
- Direct BE: http://localhost:5000/
## Production
Single-process model
- Express serves built React app and handles APIs and auth.
Build and run
```bash
echo "127.0.0.1 fibdash.local" | sudo tee -a /etc/hosts
npm start # build frontend and start backend
# or:
npm run build
npm run start:prod
```
With nginx setup, you can access:
- **Main app**: `http://localhost/` or `http://fibdash.local/`
- **API**: `http://localhost/api/` or `http://fibdash.local/api/`
- **Direct Frontend**: `http://localhost:5001/`
- **Direct Backend**: `http://localhost:5000/`
### Production Nginx Setup
For production deployment, use the production nginx configuration:
Nginx production reverse proxy
```bash
# Copy production nginx config
sudo cp nginx.prod.conf /etc/nginx/sites-available/fibdash-prod
sudo ln -s /etc/nginx/sites-available/fibdash-prod /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx
```
The production config includes:
- **Security headers** (CSP, XSS protection, etc.)
- **Static asset caching** (1 year for JS/CSS/images)
- **Gzip compression** for better performance
- **SSL/HTTPS support** (commented out, ready to enable)
Prod features:
- Static asset caching, gzip, security headers, optional TLS.
## Email Authorization
## Environment variables
FibDash includes built-in email authorization to restrict access to specific users.
- GOOGLE_CLIENT_ID, GOOGLE_CLIENT_SECRET
- REACT_APP_GOOGLE_CLIENT_ID
- JWT_SECRET
- AUTHORIZED_EMAILS (comma-separated; if unset/empty, no users can access)
- DB_SERVER, DB_DATABASE, DB_USERNAME, DB_PASSWORD, DB_PORT
- PORT, NODE_ENV
### Setup Authorization
## Data model (MSSQL)
1. **Add authorized emails to your `.env` file:**
```env
AUTHORIZED_EMAILS=admin@yourcompany.com,user1@yourcompany.com,user2@yourcompany.com
```
Core tables referenced
- eazybusiness.dbo.tZahlungsabgleichUmsatz (+ tUmsatzKontierung)
- tUmsatzBeleg (PDF storage), tPdfObjekt (PDF objects), tZahlungsabgleichUmsatzLink (links)
- fibdash.Kreditor (id, iban, name, kreditorId, is_banking)
- fibdash.Konto (id, konto, name)
- fibdash.BU (id, bu, name, vst)
- fibdash.BankingAccountTransactions (assignment records)
- fibdash.CSVTransactions (imported CSV rows)
2. **First email is admin**: The first email in the list automatically gets admin privileges
See SQL in src/database/schema.sql and src/database/csv_transactions_schema.sql for exact DDL.
3. **No authorization**: If `AUTHORIZED_EMAILS` is not set or empty, **NO USERS** can access the app
## Developer notes
### Admin Features
- Backend auto-detects DB availability. If DB env is missing, it logs a warning and continues. Endpoints that require DB will respond with 500/404 as applicable.
- data.csv at repo root is used by /api/data/months and /api/data/transactions/* for local CSV-based analysis.
- Matching logic between CSV and JTL uses exact amount and same-day date to mark matches.
- DATEV export uses well-formed header, consistent number formatting, and limited text field lengths.
Admins (first email in the authorized list) can:
- View all authorized emails: `GET /api/admin/authorized-emails`
- Add new authorized email: `POST /api/admin/authorized-emails`
- Remove authorized email: `DELETE /api/admin/authorized-emails/:email`
- View system info: `GET /api/admin/system-info`
## Scripts
**Note**: Admin changes via API are temporary. For permanent changes, update the `.env` file.
### Authorization Flow
1. User signs in with Google
2. Backend checks if email is in `AUTHORIZED_EMAILS` list
3. If authorized → login succeeds
4. If not authorized → "Access denied" error
5. All API endpoints check authorization via middleware
## Development
### Frontend Development
- Frontend code is in the `client/` directory
- Uses Webpack dev server with hot module reload
- All components are class components (no function components)
- Material-UI for styling and components
- Authentication handled via Google SSO
### Backend Development
- Backend code is in the `src/` directory
- Express.js API server with CORS enabled
- JWT authentication middleware
- MSSQL database integration
- Auto-restart on file changes via nodemon
### API Endpoints
- `POST /api/auth/google` - Google OAuth login
- `GET /api/auth/verify` - Verify JWT token
- `POST /api/auth/logout` - Logout user
- `GET /api/dashboard` - Get dashboard data
- `GET /api/dashboard/user` - Get user-specific data
- `GET /api/health` - Health check
### Available Scripts
| Script | Description |
|--------|-------------|
| `npm run dev` | Start both frontend and backend in development mode |
| `npm run dev:frontend` | Start only the frontend webpack dev server |
| `npm run dev:backend` | Start only the backend API server with nodemon |
| `npm run build` | Create production build of frontend |
| `npm run build:prod` | Build and start production server |
| `npm start` | Build frontend and start production server |
| `npm run start:prod` | Start production server (assumes build exists) |
| `npm run setup:nginx` | Automatically setup nginx for development |
| `npm run nginx:test` | Test nginx configuration |
| `npm run nginx:reload` | Reload nginx configuration |
| `npm run nginx:start` | Start nginx service |
| `npm run nginx:stop` | Stop nginx service |
| `npm run nginx:status` | Check nginx service status |
## Production Deployment
### Single-Process Production Setup
In production, the Express backend builds the React frontend and serves it as static files. No separate frontend server is needed.
#### Build and Start Production Server
```bash
# Build frontend and start server in one command
npm start
# Or build and start separately
npm run build
npm run start:prod
```
#### Production Architecture
```
Production Setup:
┌─────────────────────────────────────┐
│ Single Express Server (Port 5000) │
├─────────────────────────────────────┤
│ • Serves static React files │
│ • Handles API routes (/api/*) │
│ • Manages authentication │
│ • Connects to MSSQL database │
└─────────────────────────────────────┘
```
#### Production Features
- **Single Process**: One Node.js server handles everything
- **Static File Serving**: Built React app served with caching headers
- **Optimized Build**: Minified JS/CSS with content hashing
- **Code Splitting**: Vendor libraries separated for better caching
- **Production Security**: CSP headers and security optimizations
## Project Structure Details
### Frontend (`client/`)
- **Components**: All React components using class-based architecture
- **Services**: API communication classes
- **Material-UI**: Modern UI components with custom theming
- **Hot Reload**: Webpack dev server with HMR enabled
### Backend (`src/`)
- **Express Server**: RESTful API with middleware
- **Authentication**: Google OAuth + JWT tokens
- **Database**: MSSQL with connection pooling
- **Hot Reload**: Nodemon for automatic server restart
## Environment Variables
| Variable | Description | Required |
|----------|-------------|----------|
| `GOOGLE_CLIENT_ID` | Google OAuth Client ID (backend) | Yes |
| `GOOGLE_CLIENT_SECRET` | Google OAuth Client Secret (backend) | Yes |
| `REACT_APP_GOOGLE_CLIENT_ID` | Google OAuth Client ID (frontend) | Yes |
| `JWT_SECRET` | Secret for JWT token signing | Yes |
| `AUTHORIZED_EMAILS` | Comma-separated list of authorized email addresses | No* |
| `DB_SERVER` | MSSQL server address | Yes |
| `DB_DATABASE` | Database name | Yes |
| `DB_USERNAME` | Database username | Yes |
| `DB_PASSWORD` | Database password | Yes |
| `DB_PORT` | Database port (default: 1433) | No |
| `PORT` | API server port (default: 5000) | No |
| `NODE_ENV` | Environment mode | No |
*If `AUTHORIZED_EMAILS` is not set or empty, **NO USERS** can access the application. Only email addresses listed in `AUTHORIZED_EMAILS` can log in.
- npm run dev — run FE+BE
- npm run dev:frontend — FE only (HMR)
- npm run dev:backend — BE only (nodemon)
- npm run build — FE production build
- npm run build:prod — build and start production server
- npm start — build FE and start BE
- npm run start:prod — start BE with existing build
- npm run setup:nginx — dev nginx setup
- npm run nginx:test|reload|start|stop|status — helpers
## Troubleshooting
### Database Connection Issues
Database
- Ensure MSSQL reachable; set DB_* env; firewall allows 1433; check logs.
- Without DB, JTL and admin features won’t function; CSV-only features still work.
1. Verify your MSSQL server is running and accessible
2. Check firewall settings allow connections on port 1433
3. Ensure your database credentials are correct
4. Check the server logs for detailed error messages
Google OAuth
- Ensure both GOOGLE_CLIENT_ID and REACT_APP_GOOGLE_CLIENT_ID set and equal.
- Add dev/prod origins and redirects in Google Cloud Console.
- Use HTTPS in production and ensure CSP allows Google domains.
- See docs/GOOGLE_OAUTH_SETUP.md.
### Google OAuth Issues
CORS/Headers
- Dev CORS is open. Tighten in prod behind nginx.
1. Verify your Google Client ID is correctly set
2. Check that your domain is authorized in Google Cloud Console
3. Ensure the OAuth consent screen is configured
4. Make sure you're testing on the correct domain/port
**For detailed Google OAuth troubleshooting, see: [docs/GOOGLE_OAUTH_SETUP.md](docs/GOOGLE_OAUTH_SETUP.md)**
Common fixes for CORS/GSI errors:
- Add your domain to Google Cloud Console authorized origins
- Ensure both `GOOGLE_CLIENT_ID` and `REACT_APP_GOOGLE_CLIENT_ID` are set
- Use the "Alternative Google Sign-In" button as fallback
- Consider enabling HTTPS for better OAuth compatibility
### Hot Reload Not Working
1. Check that both dev servers are running
2. Verify webpack proxy configuration
3. Clear browser cache and restart dev servers
Hot reload
- Make sure both dev servers run; clear cache; check proxy configs.
## License
ISC
ISC