Question 1

What are Transformers and why are they important in Generative AI?

Accepted Answer

Transformers are neural network architectures introduced in the 2017 paper "Attention Is All You Need". They use self-attention mechanisms to process input data in parallel, enabling better handling of long-range dependencies and scalability in models like GPT, BERT, and T5.

Question 2

What is the Attention Mechanism in Transformers?

Accepted Answer

Attention mechanisms allow the model to focus on relevant parts of the input. Key types include:
- Self-Attention (used within the same sequence),
- Cross-Attention (encoder-decoder),
- Multi-Head Attention (multiple attention layers in parallel),
- Causal Attention (used in autoregressive generation),
- Global vs Local Attention (based on context span).

Question 3

How do Transformers compare with RNNs?

Accepted Answer

Transformers:
- Handle input in parallel
- Capture long-range dependencies better
- Scale efficiently with data
- Avoid vanishing gradients

Unlike RNNs, they are more suitable for modern NLP tasks.

Question 4

What are some popular Transformer-based models?

Accepted Answer

- BERT (encoder-only)
- GPT series (decoder-only)
- T5 (encoder-decoder)
- RoBERTa, XLNet, BART — variants with architectural improvements

Question 5

What is a Large Language Model (LLM)?

Accepted Answer

LLMs are transformer-based neural networks trained on large text corpora. Examples: GPT-4, Gemini, LLaMA. They can perform multiple NLP tasks like summarization, translation, coding, and Q&A.

Question 6

What are the advantages and limitations of LLMs?

Accepted Answer

Pros:
- Generalization across domains
- In-context learning
- Zero- and few-shot capabilities

Cons:
- Hallucination
- High compute requirements
- Can reflect bias from training data

Question 7

What are hallucinations in LLMs?

Accepted Answer

Hallucinations are confidently generated but factually incorrect responses from LLMs. They result from extrapolating beyond known data, often due to training data noise or limited context.

Question 8

How can hallucinations be useful or mitigated?

Accepted Answer

Useful: Creativity, idea generation, storytelling

Mitigation:
- Retrieval-Augmented Generation (RAG)
- Prompt engineering
- Human-in-the-loop
- Temperature tuning
- Grounding in structured data

Question 9

What is Prompt Engineering?

Accepted Answer

Prompt engineering involves crafting inputs that guide LLMs to produce desired outputs. It includes role assignment, context definition, and formatting strategies like few-shot and chain-of-thought prompting.

Question 10

What are the common prompting techniques?

Accepted Answer

- Zero-shot
- Few-shot
- Chain-of-Thought (CoT)
- Instruction-based prompting
- ReAct-style prompts (for agents)

Gen AI

Gen AI