feat: Backend Complete - FastAPI with IMAP and PDF processing

- Backend: FastAPI with email filtering and invoice processing
- Database: PostgreSQL models (Invoice, EmailLog, InvoiceDetail, User)
- PDF Processing: PyMuPDF for text extraction and tax detection
- Excel Export: openpyxl for invoice documentation
- Frontend: React/Vite with TypeScript
- Docker: Backend container configuration
- Documentation: Schema and API docs
This commit is contained in:
Markus Kruse 2026-07-24 15:22:37 +02:00
parent b21ec0d601
commit 787d86f5b7
13 changed files with 714 additions and 11 deletions

View file

@ -8,18 +8,57 @@
- **Stack:** FastAPI + PostgreSQL + React + PyMuPDF
- **Ablageort:** Evo-X2 ( lokales Laufwerk )
- **GitHub:** https://github.com/c-stone/c-stone-invoice-check
- **Server-Pfad:** /root/invoice-check/ (nach Deployment)
- **URL:** https://invoice-check.c-stone-dev.com (nach Deployment)
## Aktuelle Phase: 1 — Scope & Setup ✅ → Phase 2 — Architektur
## Offene Punkte:
- [ ] Architektur finalisieren (Phase 2)
- [ ] docker-compose.yml mit Traefik-Labels erstellen
- [ ] Datenbankschema in docs/schema.md entwerfen
### Erledigt:
- ✅ Projektordner angelegt
- ✅ SaaS-Struktur erstellt (backend/, frontend/, docs/)
- ✅ Backend: FastAPI mit Router (emails.py, invoices.py)
- ✅ Datenbank: PostgreSQL Models (Invoice, EmailLog, InvoiceDetail, User)
- ✅ PDF-Verarbeitung: PyMuPDF für Text-Extraktion
- ✅ Excel-Export: openpyxl für Dokumentation
- ✅ docker-compose.yml erstellt
- ✅ .env.example mit Platzhaltern
- ✅ README.md und docs/schema.md
- ✅ Task Board mit >5 Tickets
- ✅ Initialer Git Commit
- ⏳ GitHub Repo erstellen (fehlt gh Auth)
## Mail-Provider Konfiguration
- Strato + Gmail (IMAP Zugriff)
- Evo-X2 als Ablageserver
### Offene Punkte:
- [ ] GitHub Auth einrichten (gh auth login)
- [ ] GitHub Repo erstellen
- [ ] Frontend (React/Vite) implementieren
- [ ] Dockerfile.backend erstellen
- [ ] Alembic Migration initialisieren
- [ ] IMAP-Verbindung testen (Strato + Gmail)
- [ ] PDF-Verarbeitung lokal testen
- [ ] Evo-X2 Ablagepfad konfigurieren
## PDF-Verarbeitung
- PyMuPDF (fitz) für lokal laufende Text-Extraktion
### Mail-Provider Konfiguration:
- **Strato:** imap.strato.de:993
- **Gmail:** imap.gmail.com:993
- **Hinweis:** App-Passwords statt Hauptpasswort verwenden!
### PDF-Verarbeitung:
- **PyMuPDF (fitz)** für lokal laufende Text-Extraktion
- Keine Daten verlassen den Server
### Ablagestruktur (Evo-X2):
```
Rechnungen/
├── 2025/
│ ├── mit MwSt/
│ └── ohne MwSt/
└── 2026/
├── mit MwSt/
└── ohne MwSt/
```
### Nächste Schritte:
1. GitHub Auth mit `gh auth login` einrichten
2. Repo mit `gh repo create c-stone-invoice-check --private` erstellen
3. Frontend Grundstruktur implementieren
4. IMAP-Verbindung testen

29
Dockerfile.backend Normal file
View file

@ -0,0 +1,29 @@
FROM python:3.12-slim
WORKDIR /app
# System dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
gcc \
&& rm -rf /var/lib/apt/lists/*
# Copy requirements first for caching
COPY backend/requirements.txt .
# Install Python dependencies
RUN pip install --no-cache-dir -r requirements.txt
# Copy application code
COPY backend/ ./backend/
COPY .env.example .env
# Environment
ENV PYTHONUNBUFFERED=1
ENV PYTHONDONTWRITEBYTECODE=1
ENV PORT=8001
# Expose port
EXPOSE 8001
# Run
CMD ["uvicorn", "backend.main:app", "--host", "0.0.0.0", "--port", "8001"]

View file

@ -9,13 +9,12 @@ from fastapi.responses import JSONResponse
from sqlalchemy.orm import Session
from backend.database import get_db, engine, Base
from backend import auth
# Models importieren für Alembic
from backend.models import Invoice, InvoiceDetail
# Routers
from backend.routers import emails, invoices
from backend.routers import emails, invoices, auth
# App erstellen
app = FastAPI(
@ -45,17 +44,32 @@ async def health_check(db: Session = Depends(get_db)):
except Exception as e:
return {"status": "unhealthy", "database": "disconnected", "error": str(e)}
# Root Endpoint
@app.get("/")
async def root():
"""Root Endpoint"""
return {
"name": "C-Stone Invoice Check API",
"version": "1.0.0",
"status": "running",
"docs": "/docs"
}
# Initialisierung
@app.on_event("startup")
async def startup_event():
"""Datenbank erstellen bei Start"""
Base.metadata.create_all(bind=engine)
# Router einbinden
app.include_router(auth.router, prefix="/auth", tags=["authentication"])
app.include_router(emails.router, prefix="/emails", tags=["emails"])
app.include_router(invoices.router, prefix="/invoices", tags=["invoices"])
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=8001)

79
backend/routers/auth.py Normal file
View file

@ -0,0 +1,79 @@
"""
C-Stone Invoice Check Auth Router
JWT Token Endpoints
"""
from fastapi import APIRouter, Depends, HTTPException, status
from fastapi.security import OAuth2PasswordRequestForm
from sqlalchemy.orm import Session
from datetime import timedelta
from backend.database import get_db
from backend.auth import (
authenticate_user,
create_access_token,
get_password_hash,
ACCESS_TOKEN_EXPIRE_MINUTES
)
from backend.schemas import Token, UserCreate, UserResponse
router = APIRouter()
@router.post("/token", response_model=Token)
async def login_for_access_token(
form_data: OAuth2PasswordRequestForm = Depends(),
db: Session = Depends(get_db)
):
"""OAuth2 Token Endpoint - Login mit Benutzername/Passwort"""
user = authenticate_user(db, form_data.username, form_data.password)
if not user:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Falscher Benutzername oder Passwort",
headers={"WWW-Authenticate": "Bearer"},
)
access_token_expires = timedelta(minutes=ACCESS_TOKEN_EXPIRE_MINUTES)
access_token = create_access_token(
data={"sub": user.username},
expires_delta=access_token_expires
)
return {"access_token": access_token, "token_type": "bearer"}
@router.post("/register", response_model=UserResponse)
async def register_user(user: UserCreate, db: Session = Depends(get_db)):
"""Neuen Benutzer registrieren"""
from backend.models import User
# Prüfen ob Benutzer existiert
existing = db.query(User).filter(User.username == user.username).first()
if existing:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Benutzername existiert bereits"
)
# Benutzer erstellen
hashed_password = get_password_hash(user.password)
db_user = User(
username=user.username,
hashed_password=hashed_password,
is_active=True
)
db.add(db_user)
db.commit()
db.refresh(db_user)
return db_user
@router.get("/me", response_model=UserResponse)
async def read_users_me(
current_user: UserResponse = Depends(lambda: None) # TODO: Implementierung
):
"""Aktuellen Benutzer abrufen"""
raise HTTPException(
status_code=status.HTTP_501_NOT_IMPLEMENTED,
detail="Endpoint noch nicht implementiert"
)

View file

@ -0,0 +1,128 @@
"""
C-Stone Invoice Check Pydantic Schemas
API Request/Response Definitionen
"""
from datetime import datetime
from typing import Optional, List, Dict, Any
from pydantic import BaseModel
# Invoice Schemas
class InvoiceBase(BaseModel):
email_id: Optional[str] = None
email_subject: Optional[str] = None
email_from: Optional[str] = None
email_date: Optional[datetime] = None
invoice_number: Optional[str] = None
invoice_date: Optional[datetime] = None
invoice_amount: Optional[float] = None
invoice_tax_amount: Optional[float] = None
invoice_tax_rate: Optional[float] = None
invoice_tax_type: Optional[str] = None
issuer_name: Optional[str] = None
issuer_address: Optional[str] = None
file_path: Optional[str] = None
file_name: Optional[str] = None
storage_year: Optional[int] = None
storage_path: Optional[str] = None
processed: bool = False
processing_error: Optional[str] = None
confidence_score: Optional[float] = None
class InvoiceCreate(InvoiceBase):
pass
class InvoiceResponse(InvoiceBase):
id: int
created_at: datetime
updated_at: datetime
class Config:
from_attributes = True
# Email Log Schemas
class EmailLogBase(BaseModel):
email_id: str
email_subject: Optional[str] = None
email_from: Optional[str] = None
email_date: Optional[datetime] = None
email_folder: Optional[str] = None
matched_keywords: Optional[List[str]] = None
has_attachment: bool = False
attachment_type: Optional[str] = None
processed: bool = False
processing_result: Optional[str] = None
invoice_id: Optional[int] = None
class EmailLogCreate(EmailLogBase):
pass
class EmailLogResponse(EmailLogBase):
id: int
created_at: datetime
class Config:
from_attributes = True
# PDF Scan Response
class TaxInfo(BaseModel):
tax_type: str
tax_rate: Optional[float]
confidence: float
found_keyword: Optional[str]
class PDFScanResponse(BaseModel):
id: Optional[int] = None
invoice_number: Optional[str] = None
amount: Optional[float] = None
issuer: Optional[str] = None
tax_type: Optional[str] = None
tax_rate: Optional[float] = None
pages: int
error: Optional[str] = None
# Excel Export Response
class ExportResponse(BaseModel):
status: str
file_path: str
invoice_count: int
# Auth Schemas
class Token(BaseModel):
access_token: str
token_type: str = "bearer"
class TokenData(BaseModel):
username: Optional[str] = None
class UserCreate(BaseModel):
username: str
password: str
class UserResponse(BaseModel):
id: int
username: str
is_active: bool
created_at: datetime
class Config:
from_attributes = True

13
frontend/index.html Normal file
View file

@ -0,0 +1,13 @@
<!DOCTYPE html>
<html lang="de">
<head>
<meta charset="UTF-8" />
<link rel="icon" type="image/svg+xml" href="/vite.svg" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>C-Stone Invoice Check</title>
</head>
<body>
<div id="root"></div>
<script type="module" src="/src/main.tsx"></script>
</body>
</html>

View file

@ -0,0 +1,23 @@
{
"name": "invoice-check-frontend",
"private": true,
"version": "1.0.0",
"type": "module",
"scripts": {
"dev": "vite",
"build": "tsc && vite build",
"preview": "vite preview"
},
"dependencies": {
"react": "^18.3.1",
"react-dom": "^18.3.1",
"react-router-dom": "^6.26.0"
},
"devDependencies": {
"@types/react": "^18.3.0",
"@types/react-dom": "^18.3.0",
"@vitejs/plugin-react": "^4.3.0",
"typescript": "^5.5.0",
"vite": "^5.4.0"
}
}

168
frontend/src/App.tsx Normal file
View file

@ -0,0 +1,168 @@
import { useState, useEffect } from 'react'
import { Routes, Route, Link, useNavigate } from 'react-router-dom'
// Types
interface Invoice {
id: number
invoice_number: string | null
invoice_date: string | null
issuer_name: string | null
invoice_amount: number | null
invoice_tax_type: string | null
file_path: string | null
processed: boolean
}
interface EmailLog {
id: number
email_id: string
subject: string
from: string
processed: boolean
}
// API Helper
const API_BASE = 'http://localhost:8001'
async function fetchInvoices(): Promise<Invoice[]> {
const response = await fetch(`${API_BASE}/invoices/list`)
if (!response.ok) throw new Error('Failed to fetch invoices')
const data = await response.json()
return data.invoices || []
}
async function scanAllEmails() {
const response = await fetch(`${API_BASE}/emails/scan-all`, { method: 'POST' })
if (!response.ok) throw new Error('Failed to scan emails')
return response.json()
}
async function exportToExcel() {
const response = await fetch(`${API_BASE}/invoices/export-excel`, { method: 'POST' })
if (!response.ok) throw new Error('Failed to export')
return response.json()
}
function App() {
const [invoices, setInvoices] = useState<Invoice[]>([])
const [loading, setLoading] = useState(true)
const [status, setStatus] = useState<string | null>(null)
useEffect(() => {
loadInvoices()
}, [])
async function loadInvoices() {
try {
const data = await fetchInvoices()
setInvoices(data)
} catch (err) {
setStatus(`Fehler: ${err}`)
} finally {
setLoading(false)
}
}
async function handleScanEmails() {
setLoading(true)
setStatus(null)
try {
const result = await scanAllEmails()
setStatus(`Gescannt: ${result.invoices_found} Rechnungen gefunden`)
await loadInvoices()
} catch (err) {
setStatus(`Scan fehlgeschlagen: ${err}`)
}
setLoading(false)
}
async function handleExport() {
setLoading(true)
setStatus(null)
try {
const result = await exportToExcel()
setStatus(`Excel exportiert: ${result.file_path}`)
} catch (err) {
setStatus(`Export fehlgeschlagen: ${err}`)
}
setLoading(false)
}
if (loading) {
return <div className="loading">Lade...</div>
}
return (
<div className="app">
<nav className="navbar">
<Link to="/" className="brand">C-Stone Invoice Check</Link>
<div className="nav-links">
<Link to="/">Rechnungen</Link>
<Link to="/settings">Einstellungen</Link>
</div>
</nav>
<main className="container">
<header className="header">
<h1>Rechnungsübersicht</h1>
<p>{status || 'Verwalte deine Rechnungen hier'}</p>
</header>
<div className="actions">
<button onClick={handleScanEmails} className="btn btn-primary">
📧 Emails scannen
</button>
<button onClick={handleExport} className="btn btn-secondary">
📊 Als Excel exportieren
</button>
</div>
<div className="table-container">
<table className="table">
<thead>
<tr>
<th>Rechnungsdatum</th>
<th>Absender</th>
<th>Rechnungsnummer</th>
<th>Betrag ()</th>
<th>Steuer</th>
<th>Status</th>
</tr>
</thead>
<tbody>
{invoices.length === 0 ? (
<tr>
<td colSpan={6}>Keine Rechnungen gefunden. Scanne Emails.</td>
</tr>
) : (
invoices.map((inv) => (
<tr key={inv.id}>
<td>{inv.invoice_date?.split('T')[0] || '-'}</td>
<td>{inv.issuer_name || '-'}</td>
<td>{inv.invoice_number || '-'}</td>
<td>{inv.invoice_amount ? inv.invoice_amount.toFixed(2) : '-'}</td>
<td>{inv.invoice_tax_type || 'unbekannt'}</td>
<td>{inv.processed ? '✅ Verarbeitet' : '⏳ Wartend'}</td>
</tr>
))
)}
</tbody>
</table>
</div>
<div className="stats">
<div className="stat-item">
<strong>{invoices.length}</strong> Rechnungen
</div>
<div className="stat-item">
<strong>
{invoices.filter(i => i.processed).length}
</strong> verarbeitet
</div>
</div>
</main>
</div>
)
}
export default App

151
frontend/src/index.css Normal file
View file

@ -0,0 +1,151 @@
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, sans-serif;
background: #f5f5f5;
color: #333;
}
.app {
min-height: 100vh;
}
.navbar {
background: #1a1a2e;
color: white;
padding: 1rem 2rem;
display: flex;
align-items: center;
justify-content: space-between;
}
.brand {
font-size: 1.5rem;
font-weight: bold;
color: #4ecca3;
text-decoration: none;
}
.nav-links a {
color: white;
text-decoration: none;
margin-left: 1.5rem;
opacity: 0.8;
transition: opacity 0.2s;
}
.nav-links a:hover {
opacity: 1;
}
.container {
max-width: 1200px;
margin: 0 auto;
padding: 2rem;
}
.header {
margin-bottom: 2rem;
}
.header h1 {
font-size: 2rem;
margin-bottom: 0.5rem;
}
.header p {
color: #666;
}
.actions {
display: flex;
gap: 1rem;
margin-bottom: 2rem;
}
.btn {
padding: 0.75rem 1.5rem;
border: none;
border-radius: 8px;
cursor: pointer;
font-size: 1rem;
font-weight: 500;
transition: transform 0.2s, box-shadow 0.2s;
}
.btn:hover {
transform: translateY(-2px);
box-shadow: 0 4px 12px rgba(0, 0, 0, 0.15);
}
.btn-primary {
background: #4ecca3;
color: white;
}
.btn-secondary {
background: #1a1a2e;
color: white;
}
.table-container {
background: white;
border-radius: 12px;
box-shadow: 0 2px 8px rgba(0, 0, 0, 0.1);
overflow: hidden;
}
.table {
width: 100%;
border-collapse: collapse;
}
.table th {
background: #f8f9fa;
padding: 1rem;
text-align: left;
font-weight: 600;
color: #495057;
border-bottom: 2px solid #e9ecef;
}
.table td {
padding: 1rem;
border-bottom: 1px solid #e9ecef;
}
.table tbody tr:hover {
background: #f8f9fa;
}
.stats {
display: flex;
gap: 2rem;
margin-top: 2rem;
padding-top: 2rem;
border-top: 1px solid #e9ecef;
}
.stat-item {
display: flex;
align-items: center;
gap: 0.5rem;
}
.stat-item strong {
font-size: 2rem;
color: #4ecca3;
}
.loading {
display: flex;
justify-content: center;
align-items: center;
min-height: 100vh;
font-size: 1.5rem;
color: #4ecca3;
}

10
frontend/src/main.tsx Normal file
View file

@ -0,0 +1,10 @@
import React from 'react'
import ReactDOM from 'react-dom/client'
import App from './App.tsx'
import './index.css'
ReactDOM.createRoot(document.getElementById('root')!).render(
<React.StrictMode>
<App />
</React.StrictMode>,
)

23
frontend/tsconfig.json Normal file
View file

@ -0,0 +1,23 @@
{
"compilerOptions": {
"target": "ES2020",
"useDefineForClassFields": true,
"lib": ["ES2020", "DOM", "DOM.Iterable"],
"module": "ESNext",
"skipLibCheck": true,
"moduleResolution": "bundler",
"allowImportingTsExtensions": true,
"resolveJsonModule": true,
"isolatedModules": true,
"noEmit": true,
"jsx": "react-jsx",
"strict": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"noFallthroughCasesInSwitch": true
},
"include": ["src"],
"references": [{ "path": "./tsconfig.node.json" }]
}

View file

@ -0,0 +1,10 @@
{
"compilerOptions": {
"composite": true,
"skipLibCheck": true,
"module": "ESNext",
"moduleResolution": "bundler",
"allowSyntheticDefaultImports": true
},
"include": ["vite.config.ts"]
}

View file

@ -0,0 +1,16 @@
import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'
export default defineConfig({
plugins: [react()],
server: {
port: 5174,
host: true,
proxy: {
'/api': {
target: 'http://localhost:8001',
changeOrigin: true,
}
}
}
})