セキュリティと高リスク導入

翻訳状況

本文は英語のみです

この研究方向ページは現在英語版のみです。共通ナビゲーションと周辺 UI は、選択した言語のままご覧いただけます。

なぜ今重要か

Guidance from CISA, NSA, and partner agencies on deploying AI securely, first released on April 15, 2024, is still being operationalized in real systems, while the international Guidelines for Secure AI System Development keep pressure on teams to treat the full lifecycle as part of the attack surface. NIST's 2025 adversarial machine learning taxonomy further sharpens the threat vocabulary around model evasion, extraction, and poisoning. The practical lesson is straightforward: security cannot be bolted onto a model endpoint after procurement. It has to be designed through architecture, operations, and user-facing defaults from the start.

現在の優先事項

Make threat models explicit across model, data, retrieval, tool, and vendor layers instead of treating security as a generic risk checkbox.
Use least privilege, segmentation, and isolated execution for sensitive actions or external integrations.
Instrument logs and detection for misuse, drift, prompt-based attacks, and evidence loss.
Couple governance artifacts with runtime controls, not policy documents alone.

システム設計の原則

Secure-by-default architectures

Default settings, permissions, and data paths should assume non-expert operators and common attacker behavior.

Lifecycle threat modeling

Reassess risks at design, procurement, deployment, maintenance, and retirement rather than only during launch.

Auditable operations

Keep logs, versioning, and evidence retention in forms incident responders and compliance teams can actually use.

Containment over optimism

Design for partial compromise, rollback, isolation, and fail-safe degradation instead of assuming the model will stay well behaved.

人間要因とレビュー設計

Alerts should reflect operational consequence, not just anomaly score, so teams can triage under pressure.
Approval paths need clear ownership when AI outputs influence regulated, clinical, or security-sensitive actions.
Audit trails must be searchable and comprehensible to mixed audiences: engineers, risk owners, investigators, and reviewers.
Secure defaults matter because busy users rarely improve risky configurations after deployment.

評価アジェンダ

Adversarial resilience

Probe prompt injection, data poisoning, extraction, model evasion, and tool misuse under realistic attack paths.

Privacy and leakage

Test whether models, logs, and retrieval layers expose sensitive data directly or through inference.

Operational response

Measure detection latency, containment speed, rollback quality, and evidence availability during incidents.

Governance fit

Check whether controls, documentation, and review points satisfy the actual regulatory or institutional workflow.

継続している問い

How do we evaluate safe failure behavior when the system has tool use, retrieval, and external integrations?
What should a minimum viable audit trail contain for regulated or security-sensitive AI workflows?
How can teams compare vendor systems when critical security properties are distributed across model, platform, and operations?
Which deployment controls genuinely reduce risk, and which ones only create paper compliance?