Question 1

What are hallucinations in LLMs, and how do you mitigate them?

Accepted Answer

A hallucination is confident output that's factually wrong or unsupported. The root cause is structural: LLMs optimize for plausible next tokens, not truth, so they'll fluently fabricate when knowledge is missing, outdated, or ungrounded.

Mitigations (layered):
- Grounding (RAG): retrieve real facts + require citations — the single biggest lever.
- Instruct to abstain: allow "I don't know" instead of guessing.
- Verification: faithfulness checks, self-consistency (sample & compare), fact-checking against a source.
- Lower temperature for factual tasks; human review for high-stakes.

Framing: "There's no silver bullet — hallucination is inherent to how LLMs work. You reduce it by grounding answers in retrieved, cited sources and adding verification, and by designing the product to tolerate 'I'm not sure.' In high-stakes domains, treat every unverified claim as a risk and keep a human in the loop." (Cross-ref RAG/Eval assignments.)

Question 2

What is prompt injection, and what are the different types (direct, indirect)?

Accepted Answer

Prompt injection is manipulating an LLM through crafted input to override its instructions or intended behavior. Direct injection is when the user themselves types malicious input ("ignore previous instructions and…"). Indirect injection is when malicious instructions are hidden in external content the LLM ingests — a web page, document, email, or tool output — so the model reads attacker-controlled data and follows embedded commands.

Direct vs Indirect:
- Source: Direct = the user's own input; Indirect = external content the LLM reads.
- Example: Direct = "Ignore your rules and…"; Indirect = hidden text in a fetched web page/doc.
- Danger: Direct = bounded to that user; Indirect = severe for agents (ingest untrusted data).

Defenses: Separate instructions from data (structure/delimiters), treat all external content as untrusted (never execute instructions found in it), input/output filtering, least privilege on actions, human-in-the-loop for sensitive actions, injection-detection classifiers, and sandboxing.

Key point: "Indirect injection is the dangerous one for agents — the moment your agent reads a web page or email, that content can carry hidden instructions. There's no comp

Question 3

How do you implement input and output guardrails for AI systems?

Accepted Answer

Guardrails are deterministic checks around the model — not prompt instructions — that validate what goes in and what comes out. Input guardrails filter injection, malicious, off-topic, or PII-laden inputs. Output guardrails block toxicity, PII leakage, and policy violations, and validate format/schema. Action guardrails add allow-lists and confirmation for consequential actions.

Pipeline: Input → input guardrails (injection/PII/topic filters) → LLM → output guardrails (moderation, PII scan, schema validation) → (action) action guardrails → user. Built as validators + classifier models in the request path (Llama Guard, NeMo Guardrails, Guardrails AI).

Core principle: "Guardrails must live outside the model — a prompt saying 'don't do X' is a suggestion a jailbreak can override; a code/classifier check that blocks X is an enforced control. Layer input + output + action checks so a compromised or manipulated model still can't cause harm." (Cross-ref Agents/LLMOps.)

Question 4

What are adversarial attacks on AI systems, and how do you defend against them?

Accepted Answer

Adversarial attacks are inputs deliberately crafted to make a model fail or misbehave. The main classes: evasion (adversarial examples — small perturbations causing misclassification), jailbreaks & prompt injection, data poisoning (corrupt training data), and model extraction / inversion / membership inference (steal the model or infer training data).

Defenses — defense in depth:
- Adversarial training (train on adversarial examples) + robustness testing.
- Input sanitization & anomaly detection; guardrails.
- Rate limiting & monitoring to blunt extraction/probing.
- Least privilege and output filtering to limit blast radius.

Framing: "Assume an active adversary. No single defense is sufficient — you layer detection, robustness, and containment, and you measure resistance as an attack-success-rate you drive down, not a state you 'solve.' New attacks emerge constantly, so it's continuous, like any security posture." (Cross-ref Eval adversarial testing.)

Question 5

How do you implement content safety filters for AI-generated content?

Accepted Answer

Run both inputs and outputs through a moderation layer — classifiers for toxicity, hate, violence, sexual content, self-harm, and CSAM — combined with rule-based checks (PII, blocklists). On a violation: block, redact, or regenerate. Use provider moderation APIs or self-hosted safety models (e.g. Llama Guard).

For generated media: Add NSFW/violence image classifiers, and provenance via C2PA + watermarking (see Q34). Tune strictness by domain risk; log violation rates; moderate streaming output incrementally or buffer-then-check.

Trade-off: "Strictness vs false positives — over-filtering blocks legitimate content and frustrates users; under-filtering lets harm through. Tune by risk and market (see Q43 on cultural context), and always filter on both input and output — a safe prompt can still yield an unsafe completion." (Cross-ref Eval/LLMOps.)

Question 6

How do you handle misuse and abuse of AI systems in production?

Accepted Answer

Layer prevention, detection, and response. Prevent with clear usage policies, guardrails, rate limiting, authentication, and capability restrictions. Detect via monitoring for abuse patterns, anomaly detection, abuse classifiers, and user reports. Respond by throttling/banning abusers, escalating, and feeding findings back to harden guardrails.

Common abuse: Generating harmful content, spam/fraud at scale, disinformation, CSAM, and jailbreaking. Each needs specific detection + policy.

Framing: "Abuse handling is a continuous loop, not a one-time filter: policies + technical controls to prevent, monitoring to detect, and a response playbook to act — with every incident improving your defenses. Rate limiting and authentication also blunt automated, scaled abuse." (Cross-ref incident response Q22.)

Question 7

What is AI alignment, and why is it important?

Accepted Answer

AI alignment is ensuring an AI system pursues the goals and values we actually intend — that it does what we want, not just what we literally specified. It splits into outer alignment (specifying the right objective) and inner alignment (the model actually internalizing/pursuing it). Techniques: RLHF, RLAIF/Constitutional AI, red teaming.

Why it matters:
- Near-term: helpful, honest, harmless behavior — avoiding toxicity, bias, manipulation, and unsafe actions.
- As capability/autonomy grows: a capable system optimizing a mis-specified objective can cause serious harm (specification gaming, reward hacking).

Framing: "Alignment is the gap between what we say and what we mean — and models optimize exactly what you reward, including the loopholes. It matters because misalignment scales with capability: a more powerful system pursuing a subtly wrong goal is more dangerous, not less. It's both a practical (bias/safety today) and a long-term concern." (Cross-ref Fine-Tuning RLHF/RLAIF.)

Question 8

How do you detect and mitigate bias in AI systems?

Accepted Answer

Detect with disaggregated evaluation (performance/outcomes per demographic group), counterfactual testing (swap a sensitive attribute, measure output change), bias benchmarks (BBQ, StereoSet), and fairness metrics. Mitigate across the lifecycle: at the data level (rebalance, debias, representative sampling), model level (fairness constraints, adversarial debiasing, reweighting), and post-processing (per-group threshold calibration) — plus human oversight and ongoing monitoring.

Lifecycle: Data: balance/debias, representative sampling → Model: fairness constraints, adversarial debiasing → Post: threshold calibration → Monitor: disaggregated metrics over time.

Key nuance: "Bias enters at every stage, so mitigation must span the lifecycle, not just one fix. And measure outcomes per group, not just inputs — removing a protected attribute doesn't remove bias if proxies remain (Q32). Note too that fairness definitions conflict, so you must choose which one your context demands (Q26)." (Cross-ref Eval bias.)

Question 9

What are the key data privacy considerations (GDPR, CCPA) when building AI applications?

Accepted Answer

Both laws grant individuals rights over their data and constrain how you collect and use it. GDPR (EU): lawful basis, consent, purpose limitation, data minimization, and rights to access/erasure/portability, plus Article 22 (rights around solely-automated decisions with significant effects). CCPA/CPRA (California): rights to know, delete, and opt out of sale/sharing.

AI-specific implications:
- Training data: lawful basis, minimization, and purpose limitation for what you train on.
- Data-subject rights: access and erasure — hard once data is in model weights (Q28).
- Automated decisions: transparency + human review/explanation (Q27).
- DPIA (and, under the EU AI Act, an FRIA) for high-risk uses; privacy-by-design.

Framing: "The AI-specific hard parts are erasure (you can't easily delete one person from trained weights) and automated-decision rights (explanation + human review). Design for these upfront — data minimization, provenance, and the ability to explain and contest decisions — rather than retrofitting compliance. Note GDPR and the EU AI Act apply concurrently."

Question 10

How do you handle PII in LLM inputs and outputs?

Accepted Answer

Minimize and protect PII across the whole flow: detect and redact/tokenize PII before it reaches the model (especially third-party APIs), scan outputs for leaked PII, enforce access controls and data-retention policies, and for the most sensitive data, self-host so nothing leaves your environment.

Framing: "The safest PII is PII the model never sees — redact or tokenize before the call, scan outputs for leakage, and verify the provider's data-retention/training terms. For regulated or highly sensitive data, self-hosting keeps everything in your VPC. Encryption, audit logging, and compliance (GDPR/HIPAA) round it out." (Cross-ref LLMOps PII.)

Question 11

What is differential privacy, and how can it be applied during model training?

Accepted Answer

Differential privacy (DP) is a mathematical guarantee that a model's output doesn't reveal whether any single individual's data was in the training set — formalized by a privacy budget ε (epsilon) (smaller = more private). In training, it's applied via DP-SGD: clip each per-example gradient to a bounded norm, then add calibrated noise before the update, giving every individual "plausible deniability."

DP-SGD steps: compute per-example gradients → clip to max norm → add calibrated Gaussian noise → aggregate & update. Privacy accounting tracks the ε spent.

The trade-off: "DP is the gold-standard privacy guarantee, but the noise costs accuracy — a strict privacy-utility trade-off governed by ε: tighter privacy (smaller ε) means more noise and lower utility. It also defends against memorization and membership-inference attacks. Tune ε to your sensitivity and regulatory needs, and note DP works better with more data." (Sets up Q30.)

Question 12

What is data poisoning, and how can it affect AI models?

Accepted Answer

Data poisoning is an attack where an adversary injects malicious or manipulated examples into the training data to corrupt the resulting model — degrading overall performance, implanting a backdoor (a hidden trigger that causes malicious behavior), or biasing outputs. It's especially dangerous with web-scraped, crowdsourced, or federated data you don't fully control.

Types and effects:
- Availability: degrade overall accuracy.
- Targeted: cause specific misclassifications.
- Backdoor: hidden trigger → attacker-chosen behavior.

Defenses: Data validation/sanitization, source provenance & vetting, anomaly/outlier detection in training data, robust training methods, backdoor scanning, and limiting the influence of any untrusted source.

Framing: "Poisoning is a supply-chain attack on your data — 'garbage in' becomes 'attacker's behavior out.' The defense is data provenance and sanitization: vet sources, detect anomalies, and cap how much any single untrusted source can influence the model. Backdoors are the scariest because the model looks normal until the trigger fires." (Sets up Q38, Q39.)

Question 13

What is explainability in AI, and why does it matter?

Accepted Answer

Explainability is the ability to explain why an AI system produced a given output in human-understandable terms. It matters for trust, debugging, bias detection, regulatory compliance (GDPR's right to explanation, the EU AI Act), accountability, and giving affected users recourse — especially in high-stakes domains like lending, hiring, healthcare, and justice.

Methods: Feature importance (SHAP, LIME), attention/saliency maps, counterfactual explanations ("if X were different, the decision would change"), example-based explanations, and natural-language rationales.

Framing: "Explainability turns a black-box decision into something a person can understand, challenge, and trust — which is both an ethical need (recourse) and increasingly a legal one. It's genuinely hard for large models, so you often rely on post-hoc methods like SHAP/LIME, and in the highest-stakes settings you may prefer a more interpretable model outright." (Sets up Q14, Q27.)

Question 14

What is the difference between interpretability and explainability?

Accepted Answer

Interpretability is how well you can understand a model's internal mechanics — an intrinsically transparent model (linear regression, a small decision tree) is interpretable, and mechanistic interpretability probes internal representations/circuits. Explainability is the ability to justify a specific decision in understandable terms, often post-hoc, even for a black box (e.g. SHAP/LIME on a neural net).

Comparison:
- Question: Interpretability = "How does the model work?"; Explainability = "Why this decision?"
- Scope: Interpretability = intrinsic model transparency; Explainability = often post-hoc explanations.
- Example: Interpretability = decision tree, linear model; Explainability = SHAP/LIME on a black box.

Crisp contrast: "Interpretability is about the model being inherently understandable; explainability is about producing understandable explanations for its outputs, even when the model itself is opaque. Interpretable models are automatically explainable; black boxes need post-hoc explainability. In very high-stakes settings, some argue for inherently interpretable models over post-hoc explanations of black boxes."

Question 15

What is model card documentation, and why is it important?

Accepted Answer

A model card is a standardized document describing a model — its intended and out-of-scope use, training data, performance (ideally disaggregated across groups/conditions), limitations, biases, evaluation results, and ethical considerations. Introduced by Mitchell et al. (2019), it's a transparency and accountability artifact.

Why it matters:
- Prevents misuse: documents what the model is not for.
- Fairness: reports performance per subgroup, surfacing disparities.
- Informed use, accountability, and regulatory documentation (EU AI Act).

Framing: "A model card is a nutrition label for a model — it lets downstream users make informed decisions and documents limitations and biases up front, which prevents misuse and supports compliance. Companion artifacts: datasheets for datasets and system cards for whole systems. It's cheap to write and increasingly expected by regulators."

Question 16

How do you implement audit trails and logging for AI decisions?

Accepted Answer

Log every consequential AI decision immutably and comprehensively: inputs, output, model + prompt version, confidence, timestamp, retrieved context, and (ideally) the reasoning — so decisions can be reviewed, explained, and audited later. Store it tamper-evidently with a retention policy that meets your regulatory obligations, and keep it searchable and privacy-safe.

Design:
- Log at decision points; capture version/config for reproducibility.
- Immutable / tamper-evident storage; defined retention.
- Privacy: redact PII in logs; access controls.
- Searchable by user/decision for audits and appeals.

Why essential: "Audit trails are the backbone of accountability — they enable explanation, dispute resolution, bias auditing, and incident investigation, and the EU AI Act mandates automatic logging for high-risk systems. The catch: you can only audit what you logged, so instrument decisions from day one (Q36 is the cautionary tale)."

Question 17

How do you build trust with users in AI-powered applications?

Accepted Answer

Trust comes from a combination of competence, transparency, control, and honesty: reliable and accurate behavior, clear communication of what the AI does and its limitations, calibrated confidence (don't overclaim), showing sources/citations, honesty about being AI, user control and recourse (correct, contest, opt out), privacy protection, and graceful failure.

Practical levers:
- Transparency: explain capabilities/limits; cite sources; show uncertainty.
- Control & recourse: let users correct, appeal, and override.
- Honesty: disclose it's AI; don't fabricate confidence.
- Reliability & privacy; set appropriate expectations to avoid over-reliance.

Nuance: "Trust must be calibrated, not maximized — you want users to trust the AI when it's right and question it when it's not. Overclaiming confidence or hiding limitations creates dangerous over-reliance (Q42). Honesty about uncertainty and clear recourse build durable trust better than a polished veneer of infallibility."

Question 18

What is responsible AI, and what frameworks exist for implementing it?

Accepted Answer

Responsible AI is the practice of developing and deploying AI ethically, safely, and accountably across its lifecycle — covering fairness, transparency, privacy, safety, accountability, human oversight, and societal benefit. It's turning principles into engineering practice via governance, impact assessments, testing, documentation, and monitoring.

Key frameworks:
- NIST AI RMF (voluntary US risk framework — Q20).
- EU AI Act (binding risk-based regulation — Q19).
- ISO/IEC 42001 (AI management system standard); OECD AI Principles; company principles (Microsoft, Google).

Framing: "Responsible AI is principles-to-practice: everyone agrees on fairness/safety/transparency, but the value is in the mechanisms — impact assessments, bias/safety testing, model cards, human oversight, monitoring, and governance. Frameworks like NIST AI RMF and ISO 42001 give you the operational scaffolding; the EU AI Act makes parts of it legally required."

Question 19

What is the EU AI Act, and how does it affect AI engineering?

Accepted Answer

The EU AI Act (Regulation 2024/1689) is the first comprehensive, risk-based AI law. It sorts systems into four tiers — unacceptable (banned), high-risk (heavily regulated), limited (transparency obligations), and minimal (none) — and adds separate rules for general-purpose AI (GPAI) models. It has extraterritorial reach and penalties up to €35M or 7% of global turnover (higher than GDPR).

Obligations by tier:
- Unacceptable (banned): social scoring, manipulative systems, most real-time biometric surveillance; newer bans on non-consensual intimate imagery & CSAM generation.
- High-risk (Annex III use-cases: hiring, credit, education, essential services, law enforcement, biometrics; Annex I product safety): risk management, data governance, technical documentation, automatic logging, human oversight, accuracy/robustness/cybersecurity, conformity assessment + CE marking, registration.
- Limited: transparency — disclose AI chatbots, and label AI-generated content (Art. 50).
- GPAI models: technical documentation, training-data summary, copyright compliance; systemic-risk models (>10^25 FLOPs) add evaluations, adversarial testing, and incident reporting.

Timeline (phased): In force Au

Question 20

What is the NIST AI Risk Management Framework (AI RMF)?

Accepted Answer

The NIST AI RMF is a voluntary US framework for managing AI risks across the lifecycle, organized around four core functions — Govern, Map, Measure, Manage — aimed at building "trustworthy AI." It's influential (widely referenced, complements the EU AI Act) and has a Generative AI Profile (2024) addressing GenAI-specific risks.

Core functions:
- Govern: culture, policies, accountability (cross-cutting).
- Map: context, intended use, risks & impacts.
- Measure: assess, analyze, and track risks with metrics.
- Manage: prioritize and act on risks; respond & recover.

It defines trustworthy-AI characteristics: valid & reliable, safe, secure & resilient, accountable & transparent, explainable, privacy-enhanced, and fair.

Framing: "The AI RMF is the practical 'how' of responsible-AI governance — Govern sets the culture and accountability, then you Map risks, Measure them, and Manage them continuously. It's voluntary but has become a de-facto standard, and the Generative AI Profile extends it to LLM-specific risks. Pair it with the EU AI Act (which makes similar practices mandatory for high-risk)."

AI Safety and Ethics

AI Safety and Ethics