Foundation Models, LLMs & Prompting

37 interview prep topics with adaptive MCQ tests.

LLM Fundamentals
Large Language Models (LLMs) have revolutionized AI, enabling applications from sophisticated chatbots to advanced code…
Start Test →
Transformer Architecture
The Transformer architecture, introduced in the seminal 2017 paper 'Attention is All You Need', has fundamentally…
Start Test →
Attention Mechanisms
Attention Mechanisms have revolutionized the field of Artificial Intelligence, particularly in Natural Language…
Start Test →
Tokenization
Tokenization is a fundamental process in Natural Language Processing (NLP) and a cornerstone for modern Large Language…
Start Test →
Fine-Tuning
Fine-tuning is a crucial technique in modern AI, especially with the rise of large language models (LLMs) and…
Start Test →
LoRA & PEFT
LoRA (Low-Rank Adaptation) and PEFT (Parameter-Efficient Fine-Tuning) are critical techniques for adapting large…
Start Test →
Attention Sink
Attention Sink refers to the phenomenon where LLMs, particularly those trained with causal attention, concentrate a…
Start Test →
Context Window Management
Context Window Management is the critical engineering discipline of optimizing how large language models process,…
Start Test →
FlashAttention
FlashAttention is an IO-aware, exact attention algorithm designed to speed up Transformer training and inference by…
Start Test →
Grouped Query Attention (GQA)
Grouped Query Attention (GQA) is a critical architectural optimization for Large Language Models (LLMs) that balances…
Start Test →
KV Cache
The KV (Key-Value) Cache is a critical optimization technique in transformer-based LLM inference that stores the…
Start Test →
Latent Space
Latent space represents the compressed, multi-dimensional vector space where foundation models map raw input data into…
Start Test →
Layer Normalization vs RMSNorm
Layer Normalization (LayerNorm) and Root Mean Square Normalization (RMSNorm) are foundational components in modern…
Start Test →
Long-Context Extension Techniques (YaRN)
Long-context extension techniques are critical methodologies in 2026 for scaling Large Language Models (LLMs) beyond…
Start Test →
Mixture of Depths
Mixture of Depths (MoD) is a sophisticated architectural paradigm in transformer-based language models that enables…
Start Test →
Mixture of Experts (MoE)
Mixture of Experts (MoE) is a neural network architecture that achieves massive parameter counts while maintaining…
Start Test →
Positional Encoding (Absolute vs Relative)
Positional encoding is a fundamental mechanism in Transformer-based architectures that injects sequence order…
Start Test →
RoPE (Rotary Position Embedding)
RoPE (Rotary Position Embedding) has become the industry standard for modern Large Language Models (LLMs) like Llama 3,…
Start Test →
Scaling Laws (Chinchilla)
Scaling Laws (Chinchilla) represent the foundational empirical research that dictates how LLM performance scales with…
Start Test →
Self-Supervised Learning
Self-Supervised Learning (SSL) is the paradigm where models learn representations from unlabeled data by generating…
Start Test →
Sliding Window Attention
Sliding Window Attention is a critical optimization technique in modern Large Language Model (LLM) architectures,…
Start Test →
Alignment Tax in LLMs
The 'Alignment Tax' refers to the measurable performance degradation in a Large Language Model's (LLM) reasoning,…
Start Test →
Catastrophic Forgetting
Catastrophic forgetting is a critical phenomenon in machine learning where a neural network, after being fine-tuned on…
Start Test →
Constitutional AI (RLAIF)
Constitutional AI (CAI), often implemented via Reinforcement Learning from AI Feedback (RLAIF), is a pivotal alignment…
Start Test →
Domain Adaptation for LLMs
Domain adaptation for LLMs is the process of tailoring a general-purpose foundation model to excel in a specific…
Start Test →
DPO (Direct Preference Optimization)
Direct Preference Optimization (DPO) has emerged as the industry standard for aligning Large Language Models (LLMs)…
Start Test →
Instruction Tuning
Instruction tuning is the critical process of fine-tuning a pre-trained base language model on a curated dataset of…
Start Test →
Model Merging and Mergekit
Model merging is a transformative technique in 2026 that allows engineers to combine multiple fine-tuned Large Language…
Start Test →
Reward Modeling
Reward modeling is a critical component of modern LLM alignment, serving as the bridge between human preferences and…
Start Test →
RLHF (Reinforcement Learning from Human Feedback)
Reinforcement Learning from Human Feedback (RLHF) is the critical alignment process used to bridge the gap between base…
Start Test →
Convolution vs Self-Attention
The architectural shift from Convolutional Neural Networks (CNNs) to Transformer models represents one of the most…
Start Test →
Prompt Engineering
Prompt Engineering is the art and science of crafting effective inputs (prompts) to guide Large Language Models (LLMs)…
Start Test →
Context Engineering
Context Engineering is the systematic practice of designing, structuring, optimizing, and managing the input context…
Start Test →
Chain of Thought
Chain of Thought (CoT) prompting has revolutionized how we interact with and utilize Large Language Models (LLMs). By…
Start Test →
Structured Outputs
Structured Outputs refer to the practice of forcing Large Language Models (LLMs) to return responses that strictly…
Start Test →
Function Calling
Function calling is a breakthrough capability in modern Large Language Models (LLMs) that allows them to interact…
Start Test →
MCP
The Model Context Protocol (MCP) is an open standard designed to solve one of the most persistent challenges in modern…
Start Test →
← All Interview Prep Topics