feat: Backend Complete - FastAPI with IMAP and PDF processing

- Backend: FastAPI with email filtering and invoice processing
- Database: PostgreSQL models (Invoice, EmailLog, InvoiceDetail, User)
- PDF Processing: PyMuPDF for text extraction and tax detection
- Excel Export: openpyxl for invoice documentation
- Frontend: React/Vite with TypeScript
- Docker: Backend container configuration
- Documentation: Schema and API docs
This commit is contained in:
Markus Kruse 2026-07-24 15:22:37 +02:00
parent b21ec0d601
commit 787d86f5b7
13 changed files with 714 additions and 11 deletions

View file

@ -8,18 +8,57 @@
- **Stack:** FastAPI + PostgreSQL + React + PyMuPDF - **Stack:** FastAPI + PostgreSQL + React + PyMuPDF
- **Ablageort:** Evo-X2 ( lokales Laufwerk ) - **Ablageort:** Evo-X2 ( lokales Laufwerk )
- **GitHub:** https://github.com/c-stone/c-stone-invoice-check - **GitHub:** https://github.com/c-stone/c-stone-invoice-check
- **Server-Pfad:** /root/invoice-check/ (nach Deployment)
- **URL:** https://invoice-check.c-stone-dev.com (nach Deployment)
## Aktuelle Phase: 1 — Scope & Setup ✅ → Phase 2 — Architektur ## Aktuelle Phase: 1 — Scope & Setup ✅ → Phase 2 — Architektur
## Offene Punkte: ### Erledigt:
- [ ] Architektur finalisieren (Phase 2) - ✅ Projektordner angelegt
- [ ] docker-compose.yml mit Traefik-Labels erstellen - ✅ SaaS-Struktur erstellt (backend/, frontend/, docs/)
- [ ] Datenbankschema in docs/schema.md entwerfen - ✅ Backend: FastAPI mit Router (emails.py, invoices.py)
- ✅ Datenbank: PostgreSQL Models (Invoice, EmailLog, InvoiceDetail, User)
- ✅ PDF-Verarbeitung: PyMuPDF für Text-Extraktion
- ✅ Excel-Export: openpyxl für Dokumentation
- ✅ docker-compose.yml erstellt
- ✅ .env.example mit Platzhaltern
- ✅ README.md und docs/schema.md
- ✅ Task Board mit >5 Tickets
- ✅ Initialer Git Commit
- ⏳ GitHub Repo erstellen (fehlt gh Auth)
## Mail-Provider Konfiguration ### Offene Punkte:
- Strato + Gmail (IMAP Zugriff) - [ ] GitHub Auth einrichten (gh auth login)
- Evo-X2 als Ablageserver - [ ] GitHub Repo erstellen
- [ ] Frontend (React/Vite) implementieren
- [ ] Dockerfile.backend erstellen
- [ ] Alembic Migration initialisieren
- [ ] IMAP-Verbindung testen (Strato + Gmail)
- [ ] PDF-Verarbeitung lokal testen
- [ ] Evo-X2 Ablagepfad konfigurieren
## PDF-Verarbeitung ### Mail-Provider Konfiguration:
- PyMuPDF (fitz) für lokal laufende Text-Extraktion - **Strato:** imap.strato.de:993
- **Gmail:** imap.gmail.com:993
- **Hinweis:** App-Passwords statt Hauptpasswort verwenden!
### PDF-Verarbeitung:
- **PyMuPDF (fitz)** für lokal laufende Text-Extraktion
- Keine Daten verlassen den Server - Keine Daten verlassen den Server
### Ablagestruktur (Evo-X2):
```
Rechnungen/
├── 2025/
│ ├── mit MwSt/
│ └── ohne MwSt/
└── 2026/
├── mit MwSt/
└── ohne MwSt/
```
### Nächste Schritte:
1. GitHub Auth mit `gh auth login` einrichten
2. Repo mit `gh repo create c-stone-invoice-check --private` erstellen
3. Frontend Grundstruktur implementieren
4. IMAP-Verbindung testen

29
Dockerfile.backend Normal file
View file

@ -0,0 +1,29 @@
FROM python:3.12-slim
WORKDIR /app
# System dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
gcc \
&& rm -rf /var/lib/apt/lists/*
# Copy requirements first for caching
COPY backend/requirements.txt .
# Install Python dependencies
RUN pip install --no-cache-dir -r requirements.txt
# Copy application code
COPY backend/ ./backend/
COPY .env.example .env
# Environment
ENV PYTHONUNBUFFERED=1
ENV PYTHONDONTWRITEBYTECODE=1
ENV PORT=8001
# Expose port
EXPOSE 8001
# Run
CMD ["uvicorn", "backend.main:app", "--host", "0.0.0.0", "--port", "8001"]

View file

@ -9,13 +9,12 @@ from fastapi.responses import JSONResponse
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from backend.database import get_db, engine, Base from backend.database import get_db, engine, Base
from backend import auth
# Models importieren für Alembic # Models importieren für Alembic
from backend.models import Invoice, InvoiceDetail from backend.models import Invoice, InvoiceDetail
# Routers # Routers
from backend.routers import emails, invoices from backend.routers import emails, invoices, auth
# App erstellen # App erstellen
app = FastAPI( app = FastAPI(
@ -45,17 +44,32 @@ async def health_check(db: Session = Depends(get_db)):
except Exception as e: except Exception as e:
return {"status": "unhealthy", "database": "disconnected", "error": str(e)} return {"status": "unhealthy", "database": "disconnected", "error": str(e)}
# Root Endpoint
@app.get("/")
async def root():
"""Root Endpoint"""
return {
"name": "C-Stone Invoice Check API",
"version": "1.0.0",
"status": "running",
"docs": "/docs"
}
# Initialisierung # Initialisierung
@app.on_event("startup") @app.on_event("startup")
async def startup_event(): async def startup_event():
"""Datenbank erstellen bei Start""" """Datenbank erstellen bei Start"""
Base.metadata.create_all(bind=engine) Base.metadata.create_all(bind=engine)
# Router einbinden # Router einbinden
app.include_router(auth.router, prefix="/auth", tags=["authentication"]) app.include_router(auth.router, prefix="/auth", tags=["authentication"])
app.include_router(emails.router, prefix="/emails", tags=["emails"]) app.include_router(emails.router, prefix="/emails", tags=["emails"])
app.include_router(invoices.router, prefix="/invoices", tags=["invoices"]) app.include_router(invoices.router, prefix="/invoices", tags=["invoices"])
if __name__ == "__main__": if __name__ == "__main__":
import uvicorn import uvicorn
uvicorn.run(app, host="0.0.0.0", port=8001) uvicorn.run(app, host="0.0.0.0", port=8001)

79
backend/routers/auth.py Normal file
View file

@ -0,0 +1,79 @@
"""
C-Stone Invoice Check — Auth Router
JWT Token Endpoints
"""
from fastapi import APIRouter, Depends, HTTPException, status
from fastapi.security import OAuth2PasswordRequestForm
from sqlalchemy.orm import Session
from datetime import timedelta
from backend.database import get_db
from backend.auth import (
authenticate_user,
create_access_token,
get_password_hash,
ACCESS_TOKEN_EXPIRE_MINUTES
)
from backend.schemas import Token, UserCreate, UserResponse
router = APIRouter()
@router.post("/token", response_model=Token)
async def login_for_access_token(
form_data: OAuth2PasswordRequestForm = Depends(),
db: Session = Depends(get_db)
):
"""OAuth2 Token Endpoint - Login mit Benutzername/Passwort"""
user = authenticate_user(db, form_data.username, form_data.password)
if not user:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Falscher Benutzername oder Passwort",
headers={"WWW-Authenticate": "Bearer"},
)
access_token_expires = timedelta(minutes=ACCESS_TOKEN_EXPIRE_MINUTES)
access_token = create_access_token(
data={"sub": user.username},
expires_delta=access_token_expires
)
return {"access_token": access_token, "token_type": "bearer"}
@router.post("/register", response_model=UserResponse)
async def register_user(user: UserCreate, db: Session = Depends(get_db)):
"""Neuen Benutzer registrieren"""
from backend.models import User
# Prüfen ob Benutzer existiert
existing = db.query(User).filter(User.username == user.username).first()
if existing:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Benutzername existiert bereits"
)
# Benutzer erstellen
hashed_password = get_password_hash(user.password)
db_user = User(
username=user.username,
hashed_password=hashed_password,
is_active=True
)
db.add(db_user)
db.commit()
db.refresh(db_user)
return db_user
@router.get("/me", response_model=UserResponse)
async def read_users_me(
current_user: UserResponse = Depends(lambda: None) # TODO: Implementierung
):
"""Aktuellen Benutzer abrufen"""
raise HTTPException(
status_code=status.HTTP_501_NOT_IMPLEMENTED,
detail="Endpoint noch nicht implementiert"
)

View file

@ -0,0 +1,128 @@
"""
C-Stone Invoice Check — Pydantic Schemas
API Request/Response Definitionen
"""
from datetime import datetime
from typing import Optional, List, Dict, Any
from pydantic import BaseModel
# Invoice Schemas
class InvoiceBase(BaseModel):
email_id: Optional[str] = None
email_subject: Optional[str] = None
email_from: Optional[str] = None
email_date: Optional[datetime] = None
invoice_number: Optional[str] = None
invoice_date: Optional[datetime] = None
invoice_amount: Optional[float] = None
invoice_tax_amount: Optional[float] = None
invoice_tax_rate: Optional[float] = None
invoice_tax_type: Optional[str] = None
issuer_name: Optional[str] = None
issuer_address: Optional[str] = None
file_path: Optional[str] = None
file_name: Optional[str] = None
storage_year: Optional[int] = None
storage_path: Optional[str] = None
processed: bool = False
processing_error: Optional[str] = None
confidence_score: Optional[float] = None
class InvoiceCreate(InvoiceBase):
pass
class InvoiceResponse(InvoiceBase):
id: int
created_at: datetime
updated_at: datetime
class Config:
from_attributes = True
# Email Log Schemas
class EmailLogBase(BaseModel):
email_id: str
email_subject: Optional[str] = None
email_from: Optional[str] = None
email_date: Optional[datetime] = None
email_folder: Optional[str] = None
matched_keywords: Optional[List[str]] = None
has_attachment: bool = False
attachment_type: Optional[str] = None
processed: bool = False
processing_result: Optional[str] = None
invoice_id: Optional[int] = None
class EmailLogCreate(EmailLogBase):
pass
class EmailLogResponse(EmailLogBase):
id: int
created_at: datetime
class Config:
from_attributes = True
# PDF Scan Response
class TaxInfo(BaseModel):
tax_type: str
tax_rate: Optional[float]
confidence: float
found_keyword: Optional[str]
class PDFScanResponse(BaseModel):
id: Optional[int] = None
invoice_number: Optional[str] = None
amount: Optional[float] = None
issuer: Optional[str] = None
tax_type: Optional[str] = None
tax_rate: Optional[float] = None
pages: int
error: Optional[str] = None
# Excel Export Response
class ExportResponse(BaseModel):
status: str
file_path: str
invoice_count: int
# Auth Schemas
class Token(BaseModel):
access_token: str
token_type: str = "bearer"
class TokenData(BaseModel):
username: Optional[str] = None
class UserCreate(BaseModel):
username: str
password: str
class UserResponse(BaseModel):
id: int
username: str
is_active: bool
created_at: datetime
class Config:
from_attributes = True

13
frontend/index.html Normal file
View file

@ -0,0 +1,13 @@
<!DOCTYPE html>
<html lang="de">
<head>
<meta charset="UTF-8" />
<link rel="icon" type="image/svg+xml" href="/vite.svg" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>C-Stone Invoice Check</title>
</head>
<body>
<div id="root"></div>
<script type="module" src="/src/main.tsx"></script>
</body>
</html>

View file

@ -0,0 +1,23 @@
{
"name": "invoice-check-frontend",
"private": true,
"version": "1.0.0",
"type": "module",
"scripts": {
"dev": "vite",
"build": "tsc && vite build",
"preview": "vite preview"
},
"dependencies": {
"react": "^18.3.1",
"react-dom": "^18.3.1",
"react-router-dom": "^6.26.0"
},
"devDependencies": {
"@types/react": "^18.3.0",
"@types/react-dom": "^18.3.0",
"@vitejs/plugin-react": "^4.3.0",
"typescript": "^5.5.0",
"vite": "^5.4.0"
}
}

168
frontend/src/App.tsx Normal file
View file

@ -0,0 +1,168 @@
import { useState, useEffect } from 'react'
import { Routes, Route, Link, useNavigate } from 'react-router-dom'
// Types
interface Invoice {
id: number
invoice_number: string | null
invoice_date: string | null
issuer_name: string | null
invoice_amount: number | null
invoice_tax_type: string | null
file_path: string | null
processed: boolean
}
interface EmailLog {
id: number
email_id: string
subject: string
from: string
processed: boolean
}
// API Helper
const API_BASE = 'http://localhost:8001'
async function fetchInvoices(): Promise<Invoice[]> {
const response = await fetch(`${API_BASE}/invoices/list`)
if (!response.ok) throw new Error('Failed to fetch invoices')
const data = await response.json()
return data.invoices || []
}
async function scanAllEmails() {
const response = await fetch(`${API_BASE}/emails/scan-all`, { method: 'POST' })
if (!response.ok) throw new Error('Failed to scan emails')
return response.json()
}
async function exportToExcel() {
const response = await fetch(`${API_BASE}/invoices/export-excel`, { method: 'POST' })
if (!response.ok) throw new Error('Failed to export')
return response.json()
}
function App() {
const [invoices, setInvoices] = useState<Invoice[]>([])
const [loading, setLoading] = useState(true)
const [status, setStatus] = useState<string | null>(null)
useEffect(() => {
loadInvoices()
}, [])
async function loadInvoices() {
try {
const data = await fetchInvoices()
setInvoices(data)
} catch (err) {
setStatus(`Fehler: ${err}`)
} finally {
setLoading(false)
}
}
async function handleScanEmails() {
setLoading(true)
setStatus(null)
try {
const result = await scanAllEmails()
setStatus(`Gescannt: ${result.invoices_found} Rechnungen gefunden`)
await loadInvoices()
} catch (err) {
setStatus(`Scan fehlgeschlagen: ${err}`)
}
setLoading(false)
}
async function handleExport() {
setLoading(true)
setStatus(null)
try {
const result = await exportToExcel()
setStatus(`Excel exportiert: ${result.file_path}`)
} catch (err) {
setStatus(`Export fehlgeschlagen: ${err}`)
}
setLoading(false)
}
if (loading) {
return <div className="loading">Lade...</div>
}
return (
<div className="app">
<nav className="navbar">
<Link to="/" className="brand">C-Stone Invoice Check</Link>
<div className="nav-links">
<Link to="/">Rechnungen</Link>
<Link to="/settings">Einstellungen</Link>
</div>
</nav>
<main className="container">
<header className="header">
<h1>Rechnungsübersicht</h1>
<p>{status || 'Verwalte deine Rechnungen hier'}</p>
</header>
<div className="actions">
<button onClick={handleScanEmails} className="btn btn-primary">
📧 Emails scannen
</button>
<button onClick={handleExport} className="btn btn-secondary">
📊 Als Excel exportieren
</button>
</div>
<div className="table-container">
<table className="table">
<thead>
<tr>
<th>Rechnungsdatum</th>
<th>Absender</th>
<th>Rechnungsnummer</th>
<th>Betrag (€)</th>
<th>Steuer</th>
<th>Status</th>
</tr>
</thead>
<tbody>
{invoices.length === 0 ? (
<tr>
<td colSpan={6}>Keine Rechnungen gefunden. Scanne Emails.</td>
</tr>
) : (
invoices.map((inv) => (
<tr key={inv.id}>
<td>{inv.invoice_date?.split('T')[0] || '-'}</td>
<td>{inv.issuer_name || '-'}</td>
<td>{inv.invoice_number || '-'}</td>
<td>{inv.invoice_amount ? inv.invoice_amount.toFixed(2) : '-'}</td>
<td>{inv.invoice_tax_type || 'unbekannt'}</td>
<td>{inv.processed ? '✅ Verarbeitet' : '⏳ Wartend'}</td>
</tr>
))
)}
</tbody>
</table>
</div>
<div className="stats">
<div className="stat-item">
<strong>{invoices.length}</strong> Rechnungen
</div>
<div className="stat-item">
<strong>
{invoices.filter(i => i.processed).length}
</strong> verarbeitet
</div>
</div>
</main>
</div>
)
}
export default App

151
frontend/src/index.css Normal file
View file

@ -0,0 +1,151 @@
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, sans-serif;
background: #f5f5f5;
color: #333;
}
.app {
min-height: 100vh;
}
.navbar {
background: #1a1a2e;
color: white;
padding: 1rem 2rem;
display: flex;
align-items: center;
justify-content: space-between;
}
.brand {
font-size: 1.5rem;
font-weight: bold;
color: #4ecca3;
text-decoration: none;
}
.nav-links a {
color: white;
text-decoration: none;
margin-left: 1.5rem;
opacity: 0.8;
transition: opacity 0.2s;
}
.nav-links a:hover {
opacity: 1;
}
.container {
max-width: 1200px;
margin: 0 auto;
padding: 2rem;
}
.header {
margin-bottom: 2rem;
}
.header h1 {
font-size: 2rem;
margin-bottom: 0.5rem;
}
.header p {
color: #666;
}
.actions {
display: flex;
gap: 1rem;
margin-bottom: 2rem;
}
.btn {
padding: 0.75rem 1.5rem;
border: none;
border-radius: 8px;
cursor: pointer;
font-size: 1rem;
font-weight: 500;
transition: transform 0.2s, box-shadow 0.2s;
}
.btn:hover {
transform: translateY(-2px);
box-shadow: 0 4px 12px rgba(0, 0, 0, 0.15);
}
.btn-primary {
background: #4ecca3;
color: white;
}
.btn-secondary {
background: #1a1a2e;
color: white;
}
.table-container {
background: white;
border-radius: 12px;
box-shadow: 0 2px 8px rgba(0, 0, 0, 0.1);
overflow: hidden;
}
.table {
width: 100%;
border-collapse: collapse;
}
.table th {
background: #f8f9fa;
padding: 1rem;
text-align: left;
font-weight: 600;
color: #495057;
border-bottom: 2px solid #e9ecef;
}
.table td {
padding: 1rem;
border-bottom: 1px solid #e9ecef;
}
.table tbody tr:hover {
background: #f8f9fa;
}
.stats {
display: flex;
gap: 2rem;
margin-top: 2rem;
padding-top: 2rem;
border-top: 1px solid #e9ecef;
}
.stat-item {
display: flex;
align-items: center;
gap: 0.5rem;
}
.stat-item strong {
font-size: 2rem;
color: #4ecca3;
}
.loading {
display: flex;
justify-content: center;
align-items: center;
min-height: 100vh;
font-size: 1.5rem;
color: #4ecca3;
}

10
frontend/src/main.tsx Normal file
View file

@ -0,0 +1,10 @@
import React from 'react'
import ReactDOM from 'react-dom/client'
import App from './App.tsx'
import './index.css'
ReactDOM.createRoot(document.getElementById('root')!).render(
<React.StrictMode>
<App />
</React.StrictMode>,
)

23
frontend/tsconfig.json Normal file
View file

@ -0,0 +1,23 @@
{
"compilerOptions": {
"target": "ES2020",
"useDefineForClassFields": true,
"lib": ["ES2020", "DOM", "DOM.Iterable"],
"module": "ESNext",
"skipLibCheck": true,
"moduleResolution": "bundler",
"allowImportingTsExtensions": true,
"resolveJsonModule": true,
"isolatedModules": true,
"noEmit": true,
"jsx": "react-jsx",
"strict": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"noFallthroughCasesInSwitch": true
},
"include": ["src"],
"references": [{ "path": "./tsconfig.node.json" }]
}

View file

@ -0,0 +1,10 @@
{
"compilerOptions": {
"composite": true,
"skipLibCheck": true,
"module": "ESNext",
"moduleResolution": "bundler",
"allowSyntheticDefaultImports": true
},
"include": ["vite.config.ts"]
}

View file

@ -0,0 +1,16 @@
import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'
export default defineConfig({
plugins: [react()],
server: {
port: 5174,
host: true,
proxy: {
'/api': {
target: 'http://localhost:8001',
changeOrigin: true,
}
}
}
})