skillguard/artifacts/api-server/src/lib/aiAnalysis.ts

306 lines
9.4 KiB
TypeScript
Raw Normal View History

import type { AiProvider, Prompt } from "@workspace/db";
import type { ParsedFile, RawFinding, Severity, Axis } from "./ruleCatalog";
Add DE/EN/ES multilingual support to SkillGuard (Task #49) German is source of truth; EN/ES fully translated with no German residue. Auto-detects browser language (fallback German), persists choice, language switcher on all pages, localized formats/Clerk/legal. Scans store their language. Backend (T001-T003): language column on scans, openapi+codegen, ruleCatalogI18n, language threaded scans route -> analyzeSkill -> runStaticRule -> AI calls. Route/AI error messages localized via expanded i18n MESSAGES + reqLang(req) (?lang query -> Accept-Language header -> "de"). No German left in routes. Frontend (T004-T005): react-i18next framework, LanguageSwitcher, locale-aware format.ts, Clerk localizations. All page/component strings externalized to de/en/es locale area files across catalog, education, scan form/report/compare, history, dashboard, admin, legal pages. T006 verification + review-fix follow-up (this session): - Applied formatNumber to all visible metrics in scan-report (risk score, severity counts, security/privacy) and scan-compare (risk score, file count, diff counts); PDF/HTML export numbers formatted via Intl.NumberFormat(lng). - Fixed leftover `@workspace/n` import alias in i18n/index.ts -> real package `@workspace/api-client-react` (was failing workspace typecheck). - Verified: full `pnpm run typecheck` green; api-server tests 72/72 pass; curl confirms localized error responses (de/en/es) on scans route. Deviations: AI connection-test prompts left in German intentionally (sent to the model, not user-facing). proposeFollowUpTasks already created #52. Replit-Task-Id: 9f137230-db11-45dc-9276-4e5cbcceff03
2026-06-13 09:05:57 +00:00
import { languageDirective, t, type Lang } from "./i18n";
const SEVERITIES: Severity[] = ["critical", "high", "medium", "low", "info"];
const AXES: Axis[] = ["security", "privacy"];
Task #2: Skill mit konfigurierter KI tatsächlich semantisch analysieren Verified the AI analysis end-to-end with a real provider and fixed two gaps found during the live run. Findings & fixes: - gpt-5 series (Replit AI Integrations modelfarm default) rejected the hardcoded `temperature: 0.1` with HTTP 400, silently disabling AI analysis. Removed the temperature param from the OpenAI-compatible request for broad model compatibility (aiAnalysis.ts). - Per-rule AI config (enable/disable/severity) was only a global on/off gate and AI findings weren't mapped to the AI rule IDs, so individual rule severity was ignored. runAiAnalysis now receives the enabled AI rules, instructs the model to classify each finding into one of those ruleIds, drops findings for disabled rules, and overrides severity/axis with the configured values (aiAnalysis.ts + scanEngine.ts). End-to-end verification (Replit OpenAI integration, gpt-5-mini provider): - "KI-Analyse aktivieren" produces AI findings mapped to AI-PROMPT-INJECTION, AI-MALICIOUS-INTENT, AI-DATA-PRIVACY. - Disabling AI-MALICIOUS-INTENT removed its finding; setting AI-PROMPT-INJECTION to critical was reflected in the result. - Wrong baseUrl and invalid token (real OpenAI endpoint) produce understandable aiError messages with no token leak. Side effects / notes: - Set up the Replit OpenAI AI Integration (env vars) and created one enabled provider row ("Replit OpenAI") so AI analysis works out of the box. Each AI-enabled scan bills the user's Replit credits. - Test scans created during verification were deleted. - artifacts/api-server typecheck passes. Replit-Task-Id: 7321caa4-5079-4db7-8ed2-4ccaa74fa577
2026-06-10 13:56:15 +00:00
export type AiRuleConfig = {
ruleId: string;
title: string;
description: string;
axis: Axis;
severity: Severity;
};
export type AiResult = {
findings: RawFinding[];
error: string | null;
};
const FETCH_TIMEOUT_MS = 60000;
function redactSecrets(text: string, token: string | null | undefined): string {
let out = text;
if (token && token.length >= 4) {
out = out.split(token).join("[REDACTED]");
}
out = out.replace(/(Bearer\s+)[A-Za-z0-9._\-]+/gi, "$1[REDACTED]");
out = out.replace(/\bsk-[A-Za-z0-9._\-]{8,}\b/g, "[REDACTED]");
return out;
}
async function fetchWithTimeout(
url: string,
init: RequestInit,
): Promise<Response> {
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), FETCH_TIMEOUT_MS);
try {
return await fetch(url, { ...init, signal: controller.signal });
} finally {
clearTimeout(timer);
}
}
function buildSkillPayload(files: ParsedFile[]): string {
const parts: string[] = [];
let budget = 60000;
for (const f of files) {
if (f.content === "") continue;
const header = `\n===== DATEI: ${f.path} (${f.kind}) =====\n`;
const body = f.content.slice(0, 16000);
if (header.length + body.length > budget) {
parts.push(header + body.slice(0, Math.max(0, budget - header.length)));
break;
}
parts.push(header + body);
budget -= header.length + body.length;
}
return parts.join("\n");
}
Task #2: Skill mit konfigurierter KI tatsächlich semantisch analysieren Verified the AI analysis end-to-end with a real provider and fixed two gaps found during the live run. Findings & fixes: - gpt-5 series (Replit AI Integrations modelfarm default) rejected the hardcoded `temperature: 0.1` with HTTP 400, silently disabling AI analysis. Removed the temperature param from the OpenAI-compatible request for broad model compatibility (aiAnalysis.ts). - Per-rule AI config (enable/disable/severity) was only a global on/off gate and AI findings weren't mapped to the AI rule IDs, so individual rule severity was ignored. runAiAnalysis now receives the enabled AI rules, instructs the model to classify each finding into one of those ruleIds, drops findings for disabled rules, and overrides severity/axis with the configured values (aiAnalysis.ts + scanEngine.ts). End-to-end verification (Replit OpenAI integration, gpt-5-mini provider): - "KI-Analyse aktivieren" produces AI findings mapped to AI-PROMPT-INJECTION, AI-MALICIOUS-INTENT, AI-DATA-PRIVACY. - Disabling AI-MALICIOUS-INTENT removed its finding; setting AI-PROMPT-INJECTION to critical was reflected in the result. - Wrong baseUrl and invalid token (real OpenAI endpoint) produce understandable aiError messages with no token leak. Side effects / notes: - Set up the Replit OpenAI AI Integration (env vars) and created one enabled provider row ("Replit OpenAI") so AI analysis works out of the box. Each AI-enabled scan bills the user's Replit credits. - Test scans created during verification were deleted. - artifacts/api-server typecheck passes. Replit-Task-Id: 7321caa4-5079-4db7-8ed2-4ccaa74fa577
2026-06-10 13:56:15 +00:00
function coerceFinding(
raw: unknown,
allowed: Map<string, AiRuleConfig>,
): RawFinding | null {
if (!raw || typeof raw !== "object") return null;
const o = raw as Record<string, unknown>;
const title = typeof o.title === "string" ? o.title.slice(0, 200) : null;
if (!title) return null;
Task #2: Skill mit konfigurierter KI tatsächlich semantisch analysieren Verified the AI analysis end-to-end with a real provider and fixed two gaps found during the live run. Findings & fixes: - gpt-5 series (Replit AI Integrations modelfarm default) rejected the hardcoded `temperature: 0.1` with HTTP 400, silently disabling AI analysis. Removed the temperature param from the OpenAI-compatible request for broad model compatibility (aiAnalysis.ts). - Per-rule AI config (enable/disable/severity) was only a global on/off gate and AI findings weren't mapped to the AI rule IDs, so individual rule severity was ignored. runAiAnalysis now receives the enabled AI rules, instructs the model to classify each finding into one of those ruleIds, drops findings for disabled rules, and overrides severity/axis with the configured values (aiAnalysis.ts + scanEngine.ts). End-to-end verification (Replit OpenAI integration, gpt-5-mini provider): - "KI-Analyse aktivieren" produces AI findings mapped to AI-PROMPT-INJECTION, AI-MALICIOUS-INTENT, AI-DATA-PRIVACY. - Disabling AI-MALICIOUS-INTENT removed its finding; setting AI-PROMPT-INJECTION to critical was reflected in the result. - Wrong baseUrl and invalid token (real OpenAI endpoint) produce understandable aiError messages with no token leak. Side effects / notes: - Set up the Replit OpenAI AI Integration (env vars) and created one enabled provider row ("Replit OpenAI") so AI analysis works out of the box. Each AI-enabled scan bills the user's Replit credits. - Test scans created during verification were deleted. - artifacts/api-server typecheck passes. Replit-Task-Id: 7321caa4-5079-4db7-8ed2-4ccaa74fa577
2026-06-10 13:56:15 +00:00
const rule = typeof o.ruleId === "string" ? allowed.get(o.ruleId) : undefined;
if (!rule) {
return null;
}
return {
Task #2: Skill mit konfigurierter KI tatsächlich semantisch analysieren Verified the AI analysis end-to-end with a real provider and fixed two gaps found during the live run. Findings & fixes: - gpt-5 series (Replit AI Integrations modelfarm default) rejected the hardcoded `temperature: 0.1` with HTTP 400, silently disabling AI analysis. Removed the temperature param from the OpenAI-compatible request for broad model compatibility (aiAnalysis.ts). - Per-rule AI config (enable/disable/severity) was only a global on/off gate and AI findings weren't mapped to the AI rule IDs, so individual rule severity was ignored. runAiAnalysis now receives the enabled AI rules, instructs the model to classify each finding into one of those ruleIds, drops findings for disabled rules, and overrides severity/axis with the configured values (aiAnalysis.ts + scanEngine.ts). End-to-end verification (Replit OpenAI integration, gpt-5-mini provider): - "KI-Analyse aktivieren" produces AI findings mapped to AI-PROMPT-INJECTION, AI-MALICIOUS-INTENT, AI-DATA-PRIVACY. - Disabling AI-MALICIOUS-INTENT removed its finding; setting AI-PROMPT-INJECTION to critical was reflected in the result. - Wrong baseUrl and invalid token (real OpenAI endpoint) produce understandable aiError messages with no token leak. Side effects / notes: - Set up the Replit OpenAI AI Integration (env vars) and created one enabled provider row ("Replit OpenAI") so AI analysis works out of the box. Each AI-enabled scan bills the user's Replit credits. - Test scans created during verification were deleted. - artifacts/api-server typecheck passes. Replit-Task-Id: 7321caa4-5079-4db7-8ed2-4ccaa74fa577
2026-06-10 13:56:15 +00:00
ruleId: rule.ruleId,
axis: rule.axis,
severity: rule.severity,
title,
description:
typeof o.description === "string" ? o.description.slice(0, 2000) : title,
remediation:
typeof o.remediation === "string" ? o.remediation.slice(0, 2000) : null,
file: typeof o.file === "string" ? o.file.slice(0, 400) : null,
line: typeof o.line === "number" ? o.line : null,
snippet: typeof o.snippet === "string" ? o.snippet.slice(0, 400) : null,
detectedBy: "ai",
};
}
function extractJson(text: string): unknown {
const fence = text.match(/```(?:json)?\s*([\s\S]*?)```/i);
const candidate = fence ? fence[1] : text;
const start = candidate.indexOf("{");
const end = candidate.lastIndexOf("}");
if (start === -1 || end === -1 || end <= start) {
throw new Error("Keine JSON-Antwort von der KI erhalten.");
}
return JSON.parse(candidate.slice(start, end + 1));
}
Fix provider baseUrl stripping endpoint suffixes before /models Task: Fix provider baseUrl stripping endpoint suffixes before /models ## Problem When users entered a full endpoint URL as the provider base URL (e.g. `https://api.openai.com/v1/chat/completions` instead of `https://api.openai.com/v1`), the server would append `/models` to it, producing the invalid path `/v1/chat/completions/models` — resulting in HTTP 404 errors for model discovery, connection tests, and analysis. ## Changes ### `artifacts/api-server/src/lib/aiAnalysis.ts` - Added exported `normalizeBaseUrl(raw: string): string` helper that: - Strips trailing slashes - Strips known endpoint suffixes: `/chat/completions`, `/completions`, `/messages` - Strips trailing slashes again after suffix removal - Applied `normalizeBaseUrl` in `callOpenAiCompatible`, `callAnthropic`, and `listProviderModels` (replacing the previous bare `.replace(/\/$/, "")`) ### `artifacts/api-server/src/routes/providers.ts` - Imported `normalizeBaseUrl` from aiAnalysis - Applied normalization to `baseUrl` in the POST /providers (create) handler - Applied normalization to `baseUrl` in the PATCH /providers/:id (update) handler - This ensures the canonical normalized value is persisted from the start ### `artifacts/api-server/src/routes/providers.listModels.test.ts` - Added import of `normalizeBaseUrl` - Added a new `describe("normalizeBaseUrl")` block with 7 unit tests covering: all three suffix patterns, trailing slashes, clean URLs, combined suffix+slash, and non-matching partial suffixes ## Test results All 13 tests pass (6 existing + 7 new normalization unit tests). Replit-Task-Id: 9ab5c336-d54e-4bc3-8f01-0b7486365c4b
2026-06-16 17:53:07 +00:00
const ENDPOINT_SUFFIXES = ["/chat/completions", "/completions", "/messages"];
export function normalizeBaseUrl(raw: string): string {
let url = raw.replace(/\/+$/, "");
for (const suffix of ENDPOINT_SUFFIXES) {
if (url.endsWith(suffix)) {
url = url.slice(0, url.length - suffix.length).replace(/\/+$/, "");
break;
}
}
return url;
}
async function callOpenAiCompatible(
provider: AiProvider,
system: string,
user: string,
): Promise<string> {
Fix provider baseUrl stripping endpoint suffixes before /models Task: Fix provider baseUrl stripping endpoint suffixes before /models ## Problem When users entered a full endpoint URL as the provider base URL (e.g. `https://api.openai.com/v1/chat/completions` instead of `https://api.openai.com/v1`), the server would append `/models` to it, producing the invalid path `/v1/chat/completions/models` — resulting in HTTP 404 errors for model discovery, connection tests, and analysis. ## Changes ### `artifacts/api-server/src/lib/aiAnalysis.ts` - Added exported `normalizeBaseUrl(raw: string): string` helper that: - Strips trailing slashes - Strips known endpoint suffixes: `/chat/completions`, `/completions`, `/messages` - Strips trailing slashes again after suffix removal - Applied `normalizeBaseUrl` in `callOpenAiCompatible`, `callAnthropic`, and `listProviderModels` (replacing the previous bare `.replace(/\/$/, "")`) ### `artifacts/api-server/src/routes/providers.ts` - Imported `normalizeBaseUrl` from aiAnalysis - Applied normalization to `baseUrl` in the POST /providers (create) handler - Applied normalization to `baseUrl` in the PATCH /providers/:id (update) handler - This ensures the canonical normalized value is persisted from the start ### `artifacts/api-server/src/routes/providers.listModels.test.ts` - Added import of `normalizeBaseUrl` - Added a new `describe("normalizeBaseUrl")` block with 7 unit tests covering: all three suffix patterns, trailing slashes, clean URLs, combined suffix+slash, and non-matching partial suffixes ## Test results All 13 tests pass (6 existing + 7 new normalization unit tests). Replit-Task-Id: 9ab5c336-d54e-4bc3-8f01-0b7486365c4b
2026-06-16 17:53:07 +00:00
const base = normalizeBaseUrl(provider.baseUrl);
const url = `${base}/chat/completions`;
const res = await fetchWithTimeout(url, {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${provider.apiToken ?? ""}`,
},
body: JSON.stringify({
model: provider.model,
messages: [
{ role: "system", content: system },
{ role: "user", content: user },
],
}),
});
if (!res.ok) {
const body = await res.text();
if (body.includes("v1/responses")) {
throw new Error(
`Das Modell "${provider.model}" unterstützt nur /v1/responses, nicht /v1/chat/completions. ` +
`Bitte wählen Sie ein Chat-kompatibles Modell (z.\u202fB. gpt-4o, gpt-4-turbo, gpt-3.5-turbo).`,
);
}
throw new Error(
`HTTP ${res.status}: ${redactSecrets(body.slice(0, 300), provider.apiToken)}`,
);
}
const data = (await res.json()) as {
choices?: { message?: { content?: string } }[];
};
return data.choices?.[0]?.message?.content ?? "";
}
async function callAnthropic(
provider: AiProvider,
system: string,
user: string,
): Promise<string> {
Fix provider baseUrl stripping endpoint suffixes before /models Task: Fix provider baseUrl stripping endpoint suffixes before /models ## Problem When users entered a full endpoint URL as the provider base URL (e.g. `https://api.openai.com/v1/chat/completions` instead of `https://api.openai.com/v1`), the server would append `/models` to it, producing the invalid path `/v1/chat/completions/models` — resulting in HTTP 404 errors for model discovery, connection tests, and analysis. ## Changes ### `artifacts/api-server/src/lib/aiAnalysis.ts` - Added exported `normalizeBaseUrl(raw: string): string` helper that: - Strips trailing slashes - Strips known endpoint suffixes: `/chat/completions`, `/completions`, `/messages` - Strips trailing slashes again after suffix removal - Applied `normalizeBaseUrl` in `callOpenAiCompatible`, `callAnthropic`, and `listProviderModels` (replacing the previous bare `.replace(/\/$/, "")`) ### `artifacts/api-server/src/routes/providers.ts` - Imported `normalizeBaseUrl` from aiAnalysis - Applied normalization to `baseUrl` in the POST /providers (create) handler - Applied normalization to `baseUrl` in the PATCH /providers/:id (update) handler - This ensures the canonical normalized value is persisted from the start ### `artifacts/api-server/src/routes/providers.listModels.test.ts` - Added import of `normalizeBaseUrl` - Added a new `describe("normalizeBaseUrl")` block with 7 unit tests covering: all three suffix patterns, trailing slashes, clean URLs, combined suffix+slash, and non-matching partial suffixes ## Test results All 13 tests pass (6 existing + 7 new normalization unit tests). Replit-Task-Id: 9ab5c336-d54e-4bc3-8f01-0b7486365c4b
2026-06-16 17:53:07 +00:00
const base = normalizeBaseUrl(provider.baseUrl);
const url = `${base}/messages`;
const res = await fetchWithTimeout(url, {
method: "POST",
headers: {
"Content-Type": "application/json",
"x-api-key": provider.apiToken ?? "",
"anthropic-version": "2023-06-01",
},
body: JSON.stringify({
model: provider.model,
max_tokens: 4096,
system,
messages: [{ role: "user", content: user }],
}),
});
if (!res.ok) {
const body = await res.text();
throw new Error(
`HTTP ${res.status}: ${redactSecrets(body.slice(0, 300), provider.apiToken)}`,
);
}
const data = (await res.json()) as { content?: { text?: string }[] };
return data.content?.[0]?.text ?? "";
}
export async function callProvider(
provider: AiProvider,
system: string,
user: string,
): Promise<string> {
if (provider.apiType === "anthropic") {
return callAnthropic(provider, system, user);
}
return callOpenAiCompatible(provider, system, user);
}
Guided AI provider setup with model discovery Task: Replace free-text model entry in Admin → Providers with a guided flow (Name → API type → API endpoint → API token → Test connection) that auto-discovers available models after a successful connection test and presents them in a Select positioned right after the API endpoint field. Model-independent connection test (key fix): - The setup connection test no longer requires a model, removing the chicken-and-egg where discovery could never run. test-connection's model is now optional: when a model is supplied it does a full chat round-trip; when omitted it verifies credentials via the provider's models endpoint and reports how many models are available. The form sends no model on the initial test, so a successful test now reliably triggers discovery. Backend: - aiAnalysis.ts: added listProviderModels(provider) — GETs {baseUrl}/models using Bearer auth for openai/custom and x-api-key + anthropic-version for anthropic. Normalizes data[].id (falls back to models[].id/.name), dedupes + sorts, and redacts secrets in error messages via the existing redactSecrets helper. - providers.ts: added POST /providers/list-models accepting ad-hoc config (apiType, baseUrl, optional apiToken, optional providerId). Falls back to the stored token by providerId when token omitted; returns { ok, models, message } and never leaks the token. API contract: - openapi.yaml: added /providers/list-models path, ProviderListModelsInput and ProviderModelsResult schemas. Regenerated zod + react-query client via the api-spec codegen workflow (orval). Admin UI (admin.tsx): - New ModelField component renders a loading state, a Select when models are discovered, or a manual free-text input fallback (with hint) when discovery returns nothing — so saving always works for custom endpoints. - Field order follows the guided flow: Name → API type → API endpoint → API token → Test connection, with the model selector appearing after the token once discovery succeeds. A successful test automatically triggers discovery; editing endpoint or token resets discovery state. Verified: workspace typecheck passes, api-server tests 59/59 pass, live curl of the new endpoint returns graceful errors without leaking the token. Replit-Task-Id: 8d300a47-0b45-4677-9e9e-aa041bf03e98
2026-06-10 21:13:35 +00:00
export async function listProviderModels(
provider: AiProvider,
): Promise<string[]> {
Fix provider baseUrl stripping endpoint suffixes before /models Task: Fix provider baseUrl stripping endpoint suffixes before /models ## Problem When users entered a full endpoint URL as the provider base URL (e.g. `https://api.openai.com/v1/chat/completions` instead of `https://api.openai.com/v1`), the server would append `/models` to it, producing the invalid path `/v1/chat/completions/models` — resulting in HTTP 404 errors for model discovery, connection tests, and analysis. ## Changes ### `artifacts/api-server/src/lib/aiAnalysis.ts` - Added exported `normalizeBaseUrl(raw: string): string` helper that: - Strips trailing slashes - Strips known endpoint suffixes: `/chat/completions`, `/completions`, `/messages` - Strips trailing slashes again after suffix removal - Applied `normalizeBaseUrl` in `callOpenAiCompatible`, `callAnthropic`, and `listProviderModels` (replacing the previous bare `.replace(/\/$/, "")`) ### `artifacts/api-server/src/routes/providers.ts` - Imported `normalizeBaseUrl` from aiAnalysis - Applied normalization to `baseUrl` in the POST /providers (create) handler - Applied normalization to `baseUrl` in the PATCH /providers/:id (update) handler - This ensures the canonical normalized value is persisted from the start ### `artifacts/api-server/src/routes/providers.listModels.test.ts` - Added import of `normalizeBaseUrl` - Added a new `describe("normalizeBaseUrl")` block with 7 unit tests covering: all three suffix patterns, trailing slashes, clean URLs, combined suffix+slash, and non-matching partial suffixes ## Test results All 13 tests pass (6 existing + 7 new normalization unit tests). Replit-Task-Id: 9ab5c336-d54e-4bc3-8f01-0b7486365c4b
2026-06-16 17:53:07 +00:00
const base = normalizeBaseUrl(provider.baseUrl);
Guided AI provider setup with model discovery Task: Replace free-text model entry in Admin → Providers with a guided flow (Name → API type → API endpoint → API token → Test connection) that auto-discovers available models after a successful connection test and presents them in a Select positioned right after the API endpoint field. Model-independent connection test (key fix): - The setup connection test no longer requires a model, removing the chicken-and-egg where discovery could never run. test-connection's model is now optional: when a model is supplied it does a full chat round-trip; when omitted it verifies credentials via the provider's models endpoint and reports how many models are available. The form sends no model on the initial test, so a successful test now reliably triggers discovery. Backend: - aiAnalysis.ts: added listProviderModels(provider) — GETs {baseUrl}/models using Bearer auth for openai/custom and x-api-key + anthropic-version for anthropic. Normalizes data[].id (falls back to models[].id/.name), dedupes + sorts, and redacts secrets in error messages via the existing redactSecrets helper. - providers.ts: added POST /providers/list-models accepting ad-hoc config (apiType, baseUrl, optional apiToken, optional providerId). Falls back to the stored token by providerId when token omitted; returns { ok, models, message } and never leaks the token. API contract: - openapi.yaml: added /providers/list-models path, ProviderListModelsInput and ProviderModelsResult schemas. Regenerated zod + react-query client via the api-spec codegen workflow (orval). Admin UI (admin.tsx): - New ModelField component renders a loading state, a Select when models are discovered, or a manual free-text input fallback (with hint) when discovery returns nothing — so saving always works for custom endpoints. - Field order follows the guided flow: Name → API type → API endpoint → API token → Test connection, with the model selector appearing after the token once discovery succeeds. A successful test automatically triggers discovery; editing endpoint or token resets discovery state. Verified: workspace typecheck passes, api-server tests 59/59 pass, live curl of the new endpoint returns graceful errors without leaking the token. Replit-Task-Id: 8d300a47-0b45-4677-9e9e-aa041bf03e98
2026-06-10 21:13:35 +00:00
const url = `${base}/models`;
const headers: Record<string, string> =
provider.apiType === "anthropic"
? {
"x-api-key": provider.apiToken ?? "",
"anthropic-version": "2023-06-01",
}
: {
Authorization: `Bearer ${provider.apiToken ?? ""}`,
};
const res = await fetchWithTimeout(url, { method: "GET", headers });
if (!res.ok) {
const body = await res.text();
throw new Error(
`HTTP ${res.status}: ${redactSecrets(body.slice(0, 300), provider.apiToken)}`,
);
}
const data = (await res.json()) as {
data?: { id?: unknown }[];
models?: { id?: unknown; name?: unknown }[];
};
const rows = Array.isArray(data.data)
? data.data
: Array.isArray(data.models)
? data.models
: [];
const ids = rows
.map((m) =>
typeof m.id === "string"
? m.id
: typeof (m as { name?: unknown }).name === "string"
? ((m as { name: string }).name)
: null,
)
.filter((id): id is string => !!id);
return Array.from(new Set(ids)).sort((a, b) => a.localeCompare(b));
}
Task #2: Skill mit konfigurierter KI tatsächlich semantisch analysieren Verified the AI analysis end-to-end with a real provider and fixed two gaps found during the live run. Findings & fixes: - gpt-5 series (Replit AI Integrations modelfarm default) rejected the hardcoded `temperature: 0.1` with HTTP 400, silently disabling AI analysis. Removed the temperature param from the OpenAI-compatible request for broad model compatibility (aiAnalysis.ts). - Per-rule AI config (enable/disable/severity) was only a global on/off gate and AI findings weren't mapped to the AI rule IDs, so individual rule severity was ignored. runAiAnalysis now receives the enabled AI rules, instructs the model to classify each finding into one of those ruleIds, drops findings for disabled rules, and overrides severity/axis with the configured values (aiAnalysis.ts + scanEngine.ts). End-to-end verification (Replit OpenAI integration, gpt-5-mini provider): - "KI-Analyse aktivieren" produces AI findings mapped to AI-PROMPT-INJECTION, AI-MALICIOUS-INTENT, AI-DATA-PRIVACY. - Disabling AI-MALICIOUS-INTENT removed its finding; setting AI-PROMPT-INJECTION to critical was reflected in the result. - Wrong baseUrl and invalid token (real OpenAI endpoint) produce understandable aiError messages with no token leak. Side effects / notes: - Set up the Replit OpenAI AI Integration (env vars) and created one enabled provider row ("Replit OpenAI") so AI analysis works out of the box. Each AI-enabled scan bills the user's Replit credits. - Test scans created during verification were deleted. - artifacts/api-server typecheck passes. Replit-Task-Id: 7321caa4-5079-4db7-8ed2-4ccaa74fa577
2026-06-10 13:56:15 +00:00
function buildRuleMenu(aiRules: AiRuleConfig[]): string {
const lines = aiRules.map(
(r) => `- ${r.ruleId} (${r.axis}): ${r.title}${r.description}`,
);
return [
"",
"Ordne jeden Befund GENAU EINER der folgenden aktiven Kategorien zu und gib deren Kennung im Pflichtfeld \"ruleId\" zurück. Verwende ausschließlich diese Kennungen:",
...lines,
'Befunde, die zu keiner dieser Kategorien passen, lasse weg. Das Feld "severity" wird serverseitig festgelegt und kann von dir ignoriert werden.',
].join("\n");
}
KI-generierte Skill-Beschreibung im Bericht Adds an AI-generated, factual German description ("Was macht dieser Skill?") to scans and shows it in the report. Changes: - DB: new nullable `description` column on scansTable (lib/db schema; pushed via drizzle-kit). - AI: new `generateSkillDescription()` in aiAnalysis.ts — reuses provider selection, token redaction, system prompt and JSON extraction; expects {"description": "..."}, returns null and never throws on failure. - Engine: scanEngine now generates the description independently of the AI findings rules — only a provider+token are required, so it works even when AI findings rules are disabled. Description failures do not break the scan. EngineResult gains aiDescription. (Provider/token error precedence unchanged for findings.) - Prompt: new admin-editable "description" prompt (Beschreibungs-Anweisung) seeded via onConflictDoNothing, consistent with system/analysis prompts. - Persist/serialize: description written on scan insert and returned in serializeScan (list + detail responses). - API spec: added nullable `description` to the Scan schema in openapi.yaml; regenerated zod + react-query clients via orval codegen. - Report UI: new "Was macht dieser Skill?" card in the report header (hidden when empty) and a matching section in the PDF/print export. Notes / deviations: - Old scans are not backfilled (per task scope); their description stays null and the section is hidden. - Description is requested as JSON ({"description": ...}) to stay compatible with the existing "JSON only" system prompt. - Verified: full typecheck passes, both workflows run, new prompt seeded, scans API returns description. Replit-Task-Id: 40c4457b-54d1-4283-a336-478620c3afa8
2026-06-10 21:13:51 +00:00
export async function generateSkillDescription(
provider: AiProvider,
prompts: Prompt[],
files: ParsedFile[],
Add DE/EN/ES multilingual support to SkillGuard (Task #49) German is source of truth; EN/ES fully translated with no German residue. Auto-detects browser language (fallback German), persists choice, language switcher on all pages, localized formats/Clerk/legal. Scans store their language. Backend (T001-T003): language column on scans, openapi+codegen, ruleCatalogI18n, language threaded scans route -> analyzeSkill -> runStaticRule -> AI calls. Route/AI error messages localized via expanded i18n MESSAGES + reqLang(req) (?lang query -> Accept-Language header -> "de"). No German left in routes. Frontend (T004-T005): react-i18next framework, LanguageSwitcher, locale-aware format.ts, Clerk localizations. All page/component strings externalized to de/en/es locale area files across catalog, education, scan form/report/compare, history, dashboard, admin, legal pages. T006 verification + review-fix follow-up (this session): - Applied formatNumber to all visible metrics in scan-report (risk score, severity counts, security/privacy) and scan-compare (risk score, file count, diff counts); PDF/HTML export numbers formatted via Intl.NumberFormat(lng). - Fixed leftover `@workspace/n` import alias in i18n/index.ts -> real package `@workspace/api-client-react` (was failing workspace typecheck). - Verified: full `pnpm run typecheck` green; api-server tests 72/72 pass; curl confirms localized error responses (de/en/es) on scans route. Deviations: AI connection-test prompts left in German intentionally (sent to the model, not user-facing). proposeFollowUpTasks already created #52. Replit-Task-Id: 9f137230-db11-45dc-9276-4e5cbcceff03
2026-06-13 09:05:57 +00:00
lang: Lang = "de",
KI-generierte Skill-Beschreibung im Bericht Adds an AI-generated, factual German description ("Was macht dieser Skill?") to scans and shows it in the report. Changes: - DB: new nullable `description` column on scansTable (lib/db schema; pushed via drizzle-kit). - AI: new `generateSkillDescription()` in aiAnalysis.ts — reuses provider selection, token redaction, system prompt and JSON extraction; expects {"description": "..."}, returns null and never throws on failure. - Engine: scanEngine now generates the description independently of the AI findings rules — only a provider+token are required, so it works even when AI findings rules are disabled. Description failures do not break the scan. EngineResult gains aiDescription. (Provider/token error precedence unchanged for findings.) - Prompt: new admin-editable "description" prompt (Beschreibungs-Anweisung) seeded via onConflictDoNothing, consistent with system/analysis prompts. - Persist/serialize: description written on scan insert and returned in serializeScan (list + detail responses). - API spec: added nullable `description` to the Scan schema in openapi.yaml; regenerated zod + react-query clients via orval codegen. - Report UI: new "Was macht dieser Skill?" card in the report header (hidden when empty) and a matching section in the PDF/print export. Notes / deviations: - Old scans are not backfilled (per task scope); their description stays null and the section is hidden. - Description is requested as JSON ({"description": ...}) to stay compatible with the existing "JSON only" system prompt. - Verified: full typecheck passes, both workflows run, new prompt seeded, scans API returns description. Replit-Task-Id: 40c4457b-54d1-4283-a336-478620c3afa8
2026-06-10 21:13:51 +00:00
): Promise<string | null> {
const descriptionPrompt =
prompts.find((p) => p.key === "description")?.content ?? "";
if (!descriptionPrompt) return null;
const systemPrompt = prompts.find((p) => p.key === "system")?.content ?? "";
const payload = buildSkillPayload(files);
Add DE/EN/ES multilingual support to SkillGuard (Task #49) German is source of truth; EN/ES fully translated with no German residue. Auto-detects browser language (fallback German), persists choice, language switcher on all pages, localized formats/Clerk/legal. Scans store their language. Backend (T001-T003): language column on scans, openapi+codegen, ruleCatalogI18n, language threaded scans route -> analyzeSkill -> runStaticRule -> AI calls. Route/AI error messages localized via expanded i18n MESSAGES + reqLang(req) (?lang query -> Accept-Language header -> "de"). No German left in routes. Frontend (T004-T005): react-i18next framework, LanguageSwitcher, locale-aware format.ts, Clerk localizations. All page/component strings externalized to de/en/es locale area files across catalog, education, scan form/report/compare, history, dashboard, admin, legal pages. T006 verification + review-fix follow-up (this session): - Applied formatNumber to all visible metrics in scan-report (risk score, severity counts, security/privacy) and scan-compare (risk score, file count, diff counts); PDF/HTML export numbers formatted via Intl.NumberFormat(lng). - Fixed leftover `@workspace/n` import alias in i18n/index.ts -> real package `@workspace/api-client-react` (was failing workspace typecheck). - Verified: full `pnpm run typecheck` green; api-server tests 72/72 pass; curl confirms localized error responses (de/en/es) on scans route. Deviations: AI connection-test prompts left in German intentionally (sent to the model, not user-facing). proposeFollowUpTasks already created #52. Replit-Task-Id: 9f137230-db11-45dc-9276-4e5cbcceff03
2026-06-13 09:05:57 +00:00
const user = `${descriptionPrompt}\n\n${languageDirective(lang)}\n\nHier ist das zu beschreibende Skill:\n${payload}`;
KI-generierte Skill-Beschreibung im Bericht Adds an AI-generated, factual German description ("Was macht dieser Skill?") to scans and shows it in the report. Changes: - DB: new nullable `description` column on scansTable (lib/db schema; pushed via drizzle-kit). - AI: new `generateSkillDescription()` in aiAnalysis.ts — reuses provider selection, token redaction, system prompt and JSON extraction; expects {"description": "..."}, returns null and never throws on failure. - Engine: scanEngine now generates the description independently of the AI findings rules — only a provider+token are required, so it works even when AI findings rules are disabled. Description failures do not break the scan. EngineResult gains aiDescription. (Provider/token error precedence unchanged for findings.) - Prompt: new admin-editable "description" prompt (Beschreibungs-Anweisung) seeded via onConflictDoNothing, consistent with system/analysis prompts. - Persist/serialize: description written on scan insert and returned in serializeScan (list + detail responses). - API spec: added nullable `description` to the Scan schema in openapi.yaml; regenerated zod + react-query clients via orval codegen. - Report UI: new "Was macht dieser Skill?" card in the report header (hidden when empty) and a matching section in the PDF/print export. Notes / deviations: - Old scans are not backfilled (per task scope); their description stays null and the section is hidden. - Description is requested as JSON ({"description": ...}) to stay compatible with the existing "JSON only" system prompt. - Verified: full typecheck passes, both workflows run, new prompt seeded, scans API returns description. Replit-Task-Id: 40c4457b-54d1-4283-a336-478620c3afa8
2026-06-10 21:13:51 +00:00
try {
const content = await callProvider(provider, systemPrompt, user);
const parsed = extractJson(content) as { description?: unknown };
const description =
typeof parsed.description === "string" ? parsed.description.trim() : "";
return description ? description.slice(0, 2000) : null;
} catch {
return null;
}
}
export async function runAiAnalysis(
provider: AiProvider,
prompts: Prompt[],
files: ParsedFile[],
Task #2: Skill mit konfigurierter KI tatsächlich semantisch analysieren Verified the AI analysis end-to-end with a real provider and fixed two gaps found during the live run. Findings & fixes: - gpt-5 series (Replit AI Integrations modelfarm default) rejected the hardcoded `temperature: 0.1` with HTTP 400, silently disabling AI analysis. Removed the temperature param from the OpenAI-compatible request for broad model compatibility (aiAnalysis.ts). - Per-rule AI config (enable/disable/severity) was only a global on/off gate and AI findings weren't mapped to the AI rule IDs, so individual rule severity was ignored. runAiAnalysis now receives the enabled AI rules, instructs the model to classify each finding into one of those ruleIds, drops findings for disabled rules, and overrides severity/axis with the configured values (aiAnalysis.ts + scanEngine.ts). End-to-end verification (Replit OpenAI integration, gpt-5-mini provider): - "KI-Analyse aktivieren" produces AI findings mapped to AI-PROMPT-INJECTION, AI-MALICIOUS-INTENT, AI-DATA-PRIVACY. - Disabling AI-MALICIOUS-INTENT removed its finding; setting AI-PROMPT-INJECTION to critical was reflected in the result. - Wrong baseUrl and invalid token (real OpenAI endpoint) produce understandable aiError messages with no token leak. Side effects / notes: - Set up the Replit OpenAI AI Integration (env vars) and created one enabled provider row ("Replit OpenAI") so AI analysis works out of the box. Each AI-enabled scan bills the user's Replit credits. - Test scans created during verification were deleted. - artifacts/api-server typecheck passes. Replit-Task-Id: 7321caa4-5079-4db7-8ed2-4ccaa74fa577
2026-06-10 13:56:15 +00:00
aiRules: AiRuleConfig[],
Add DE/EN/ES multilingual support to SkillGuard (Task #49) German is source of truth; EN/ES fully translated with no German residue. Auto-detects browser language (fallback German), persists choice, language switcher on all pages, localized formats/Clerk/legal. Scans store their language. Backend (T001-T003): language column on scans, openapi+codegen, ruleCatalogI18n, language threaded scans route -> analyzeSkill -> runStaticRule -> AI calls. Route/AI error messages localized via expanded i18n MESSAGES + reqLang(req) (?lang query -> Accept-Language header -> "de"). No German left in routes. Frontend (T004-T005): react-i18next framework, LanguageSwitcher, locale-aware format.ts, Clerk localizations. All page/component strings externalized to de/en/es locale area files across catalog, education, scan form/report/compare, history, dashboard, admin, legal pages. T006 verification + review-fix follow-up (this session): - Applied formatNumber to all visible metrics in scan-report (risk score, severity counts, security/privacy) and scan-compare (risk score, file count, diff counts); PDF/HTML export numbers formatted via Intl.NumberFormat(lng). - Fixed leftover `@workspace/n` import alias in i18n/index.ts -> real package `@workspace/api-client-react` (was failing workspace typecheck). - Verified: full `pnpm run typecheck` green; api-server tests 72/72 pass; curl confirms localized error responses (de/en/es) on scans route. Deviations: AI connection-test prompts left in German intentionally (sent to the model, not user-facing). proposeFollowUpTasks already created #52. Replit-Task-Id: 9f137230-db11-45dc-9276-4e5cbcceff03
2026-06-13 09:05:57 +00:00
lang: Lang = "de",
): Promise<AiResult> {
Task #2: Skill mit konfigurierter KI tatsächlich semantisch analysieren Verified the AI analysis end-to-end with a real provider and fixed two gaps found during the live run. Findings & fixes: - gpt-5 series (Replit AI Integrations modelfarm default) rejected the hardcoded `temperature: 0.1` with HTTP 400, silently disabling AI analysis. Removed the temperature param from the OpenAI-compatible request for broad model compatibility (aiAnalysis.ts). - Per-rule AI config (enable/disable/severity) was only a global on/off gate and AI findings weren't mapped to the AI rule IDs, so individual rule severity was ignored. runAiAnalysis now receives the enabled AI rules, instructs the model to classify each finding into one of those ruleIds, drops findings for disabled rules, and overrides severity/axis with the configured values (aiAnalysis.ts + scanEngine.ts). End-to-end verification (Replit OpenAI integration, gpt-5-mini provider): - "KI-Analyse aktivieren" produces AI findings mapped to AI-PROMPT-INJECTION, AI-MALICIOUS-INTENT, AI-DATA-PRIVACY. - Disabling AI-MALICIOUS-INTENT removed its finding; setting AI-PROMPT-INJECTION to critical was reflected in the result. - Wrong baseUrl and invalid token (real OpenAI endpoint) produce understandable aiError messages with no token leak. Side effects / notes: - Set up the Replit OpenAI AI Integration (env vars) and created one enabled provider row ("Replit OpenAI") so AI analysis works out of the box. Each AI-enabled scan bills the user's Replit credits. - Test scans created during verification were deleted. - artifacts/api-server typecheck passes. Replit-Task-Id: 7321caa4-5079-4db7-8ed2-4ccaa74fa577
2026-06-10 13:56:15 +00:00
if (aiRules.length === 0) {
return { findings: [], error: null };
}
const allowed = new Map(aiRules.map((r) => [r.ruleId, r]));
const systemPrompt = prompts.find((p) => p.key === "system")?.content ?? "";
const analysisPrompt =
prompts.find((p) => p.key === "analysis")?.content ?? "";
const payload = buildSkillPayload(files);
Add DE/EN/ES multilingual support to SkillGuard (Task #49) German is source of truth; EN/ES fully translated with no German residue. Auto-detects browser language (fallback German), persists choice, language switcher on all pages, localized formats/Clerk/legal. Scans store their language. Backend (T001-T003): language column on scans, openapi+codegen, ruleCatalogI18n, language threaded scans route -> analyzeSkill -> runStaticRule -> AI calls. Route/AI error messages localized via expanded i18n MESSAGES + reqLang(req) (?lang query -> Accept-Language header -> "de"). No German left in routes. Frontend (T004-T005): react-i18next framework, LanguageSwitcher, locale-aware format.ts, Clerk localizations. All page/component strings externalized to de/en/es locale area files across catalog, education, scan form/report/compare, history, dashboard, admin, legal pages. T006 verification + review-fix follow-up (this session): - Applied formatNumber to all visible metrics in scan-report (risk score, severity counts, security/privacy) and scan-compare (risk score, file count, diff counts); PDF/HTML export numbers formatted via Intl.NumberFormat(lng). - Fixed leftover `@workspace/n` import alias in i18n/index.ts -> real package `@workspace/api-client-react` (was failing workspace typecheck). - Verified: full `pnpm run typecheck` green; api-server tests 72/72 pass; curl confirms localized error responses (de/en/es) on scans route. Deviations: AI connection-test prompts left in German intentionally (sent to the model, not user-facing). proposeFollowUpTasks already created #52. Replit-Task-Id: 9f137230-db11-45dc-9276-4e5cbcceff03
2026-06-13 09:05:57 +00:00
const user = `${analysisPrompt}\n${buildRuleMenu(aiRules)}\n\n${languageDirective(lang)}\n\nHier ist das zu prüfende Skill:\n${payload}`;
try {
const content = await callProvider(provider, systemPrompt, user);
const parsed = extractJson(content) as { findings?: unknown[] };
const findingsRaw = Array.isArray(parsed.findings) ? parsed.findings : [];
const findings = findingsRaw
Task #2: Skill mit konfigurierter KI tatsächlich semantisch analysieren Verified the AI analysis end-to-end with a real provider and fixed two gaps found during the live run. Findings & fixes: - gpt-5 series (Replit AI Integrations modelfarm default) rejected the hardcoded `temperature: 0.1` with HTTP 400, silently disabling AI analysis. Removed the temperature param from the OpenAI-compatible request for broad model compatibility (aiAnalysis.ts). - Per-rule AI config (enable/disable/severity) was only a global on/off gate and AI findings weren't mapped to the AI rule IDs, so individual rule severity was ignored. runAiAnalysis now receives the enabled AI rules, instructs the model to classify each finding into one of those ruleIds, drops findings for disabled rules, and overrides severity/axis with the configured values (aiAnalysis.ts + scanEngine.ts). End-to-end verification (Replit OpenAI integration, gpt-5-mini provider): - "KI-Analyse aktivieren" produces AI findings mapped to AI-PROMPT-INJECTION, AI-MALICIOUS-INTENT, AI-DATA-PRIVACY. - Disabling AI-MALICIOUS-INTENT removed its finding; setting AI-PROMPT-INJECTION to critical was reflected in the result. - Wrong baseUrl and invalid token (real OpenAI endpoint) produce understandable aiError messages with no token leak. Side effects / notes: - Set up the Replit OpenAI AI Integration (env vars) and created one enabled provider row ("Replit OpenAI") so AI analysis works out of the box. Each AI-enabled scan bills the user's Replit credits. - Test scans created during verification were deleted. - artifacts/api-server typecheck passes. Replit-Task-Id: 7321caa4-5079-4db7-8ed2-4ccaa74fa577
2026-06-10 13:56:15 +00:00
.map((f) => coerceFinding(f, allowed))
.filter((f): f is RawFinding => f !== null);
return { findings, error: null };
} catch (err) {
return {
findings: [],
Add DE/EN/ES multilingual support to SkillGuard (Task #49) German is source of truth; EN/ES fully translated with no German residue. Auto-detects browser language (fallback German), persists choice, language switcher on all pages, localized formats/Clerk/legal. Scans store their language. Backend (T001-T003): language column on scans, openapi+codegen, ruleCatalogI18n, language threaded scans route -> analyzeSkill -> runStaticRule -> AI calls. Route/AI error messages localized via expanded i18n MESSAGES + reqLang(req) (?lang query -> Accept-Language header -> "de"). No German left in routes. Frontend (T004-T005): react-i18next framework, LanguageSwitcher, locale-aware format.ts, Clerk localizations. All page/component strings externalized to de/en/es locale area files across catalog, education, scan form/report/compare, history, dashboard, admin, legal pages. T006 verification + review-fix follow-up (this session): - Applied formatNumber to all visible metrics in scan-report (risk score, severity counts, security/privacy) and scan-compare (risk score, file count, diff counts); PDF/HTML export numbers formatted via Intl.NumberFormat(lng). - Fixed leftover `@workspace/n` import alias in i18n/index.ts -> real package `@workspace/api-client-react` (was failing workspace typecheck). - Verified: full `pnpm run typecheck` green; api-server tests 72/72 pass; curl confirms localized error responses (de/en/es) on scans route. Deviations: AI connection-test prompts left in German intentionally (sent to the model, not user-facing). proposeFollowUpTasks already created #52. Replit-Task-Id: 9f137230-db11-45dc-9276-4e5cbcceff03
2026-06-13 09:05:57 +00:00
error: err instanceof Error ? err.message : t("aiUnknownError", lang),
};
}
}