Artificial Intelligence (AI) is the field of building systems that perform tasks which normally require human intelligence: understanding language, recognizing images, making decisions, and solving problems. Modern AI is not explicitly programmed with rules; it learns patterns from data.

Types of AI by Capability -

• Narrow AI (ANI) : AI built for a specific task, such as spam filtering, face recognition, route navigation, or chatbots. All AI in use today is narrow AI, even the most capable systems.
• General AI (AGI) : A hypothetical system that matches human intelligence across any task, able to learn and reason about anything a person can. Not achieved yet; it is the stated goal of major AI labs.
• Super AI (ASI) : A hypothetical system that surpasses human intelligence in all fields. Purely theoretical.

Major Subfields -

• Machine Learning (ML) : Systems that learn from data instead of hand-written rules. The engine behind almost all modern AI.
• Natural Language Processing (NLP) : Understanding and generating human language: translation, summarization, chatbots.
• Computer Vision : Understanding images and video: object detection, face recognition, medical imaging.
• Speech : Converting speech to text (recognition) and text to speech (synthesis).
• Robotics : Combining perception, planning, and control so machines can act in the physical world.

Machine Learning is the practice of training algorithms to find patterns in data and make predictions or decisions without being explicitly programmed for the task. Instead of writing rules ("if the email contains X, mark as spam"), you show the algorithm thousands of examples and it learns the rules itself.

Key Terms -

• Dataset : The collection of examples used for learning, usually split into training data (to learn from) and test data (to check performance on unseen examples).
• Features : The input variables the model looks at, such as an email's sender, subject, and word counts.
• Label : The correct answer attached to a training example, such as "spam" or "not spam".
• Training : The process of adjusting the model using the data until its predictions become accurate.
• Inference : Using the trained model to make predictions on new, unseen input.

Types of Machine Learning -

• Supervised Learning : Learns from labeled examples (input plus correct answer). Used for classification (predicting a category: spam or not, disease or not) and regression (predicting a number: house price, temperature). Examples: linear regression, decision trees, random forests.
• Unsupervised Learning : Finds structure in unlabeled data on its own. Used for clustering (grouping similar customers), anomaly detection (flagging unusual transactions), and dimensionality reduction. Example: k-means clustering.
• Semi-Supervised Learning : Trains on a small labeled set plus a large unlabeled set, useful when labeling is expensive.
• Reinforcement Learning (RL) : An agent learns by acting in an environment and receiving rewards or penalties, improving through trial and error. Used in game-playing AI (AlphaGo), robotics, and for aligning LLM behavior with human preferences (RLHF).

Deep Learning is a branch of machine learning based on neural networks with many layers. Each layer learns increasingly abstract features from the previous one, for example edges, then shapes, then faces. Deep learning powers modern image recognition, speech, and language models, and it scales: more data and more compute keep improving results, which is why training runs on GPU clusters.

• Neural Network : A network of simple units (neurons) organized in layers. Each connection has a weight, and training adjusts these weights so the network's output moves closer to the correct answer (using backpropagation and gradient descent).

Common Architectures -

• CNN (Convolutional Neural Network) : Specialized for grid-like data, mainly images. Used in image classification, object detection, and medical imaging.
• RNN / LSTM : Processes sequences step by step while keeping memory of earlier steps. Formerly standard for text and time series, now largely replaced by transformers.
• Transformer : The architecture behind modern AI. Its attention mechanism lets the model weigh how every part of the input relates to every other part, and it processes sequences in parallel, making huge models trainable. Introduced in the 2017 paper "Attention Is All You Need"; the basis of all LLMs.
• Diffusion Models : Generate images by learning to reverse noise, starting from random noise and refining it into a picture. Used by image generators like Stable Diffusion.

A model is the trained artifact that comes out of machine learning: a mathematical function with learned parameters (weights) that maps inputs to outputs. The "knowledge" lives in the weights, which are just large arrays of numbers. Model size is measured in parameters; modern LLMs range from a few billion to trillions.

Model Lifecycle -

• Pre-Training : Training from scratch on a massive general dataset. Extremely expensive; done by AI labs, not end users.
• Fine-Tuning : Taking a pre-trained model and training it further on a smaller, task-specific dataset, such as adapting a general LLM to medical or legal text.
• Evaluation : Measuring performance on data the model has never seen, using benchmarks and metrics like accuracy.
• Deployment : Serving the model for inference, either through a provider's API or self-hosted on your own hardware.

Key Concepts -

• Overfitting : The model memorizes the training data instead of learning general patterns, so it performs great on training data and poorly on new data. The opposite, underfitting, means the model is too simple to capture the pattern at all.
• Open vs Closed Weights : Open-weight models (Llama, Mistral, Gemma) can be downloaded and run on your own hardware. Closed models (GPT, Claude, Gemini) are accessed only through the provider's API.
• Multimodal Models : Models that handle more than text, taking images, audio, or video as input and sometimes producing them as output.
• Distillation / Quantization : Techniques to shrink models: distillation trains a small model to imitate a large one, and quantization stores weights with fewer bits so the model runs on smaller hardware.

Generative AI creates new content (text, images, audio, video, code) instead of only classifying or predicting. A Large Language Model (LLM) is a generative model trained on massive amounts of text to predict the next token, and from that single skill emerges the ability to answer questions, write code, summarize, translate, and reason. Examples: GPT (OpenAI), Claude (Anthropic), Gemini (Google), Llama (Meta, open weights), Mistral.

How LLMs Work -

• Tokens : LLMs process text as tokens, chunks of a few characters (roughly 3-4 characters of English per token). Both input and output are measured and billed in tokens.
• Context Window : The maximum number of tokens a model can consider at once, including the conversation so far. Anything beyond it is forgotten. Modern models range from about 128K to over 1M tokens.
• Next-Token Prediction : The model repeatedly predicts the most likely next token given everything before it, one token at a time, which is why responses stream word by word.
• Temperature : A setting that controls randomness. Low values give focused, consistent output (good for code and facts); high values give more creative and varied output.
• System Prompt : Hidden instructions given to the model before the user's message that set its role, rules, and style.

Limitations -

• Hallucination : The model can generate confident but false information, because it predicts plausible text rather than looking up facts.
• Knowledge Cutoff : The model only knows what was in its training data, which ends at a fixed date. Anything after that requires giving it the information (search, RAG).
• Context Limits : Long conversations or documents that exceed the context window get truncated or summarized, losing detail.

Prompt engineering is the craft of writing inputs that get reliable, high-quality output from an LLM. The same model can perform badly or brilliantly depending on how the task is described.

Core Techniques -

• Zero-Shot : Just ask, with a clear task description and no examples. Works for common tasks.
• Few-Shot : Include 2-5 examples of input and desired output in the prompt; the model imitates the pattern. Best way to control format.
• Chain of Thought : Ask the model to reason step by step before answering. Improves accuracy on math and logic. Modern "reasoning models" do this internally.
• Role & Constraints : Give the model a role ("you are a code reviewer"), explicit rules, and a required output format (such as JSON). Structure long prompts with sections or tags.

RAG (Retrieval-Augmented Generation) :

RAG gives an LLM knowledge it was not trained on by fetching relevant documents at question time and placing them in the prompt. This grounds answers in real sources, reduces hallucination, and lets the model use private or current data without retraining. Application: a support chatbot that answers from your product documentation.

RAG Pipeline -

• Embeddings : Documents are split into chunks and converted into embeddings, numeric vectors where similar meanings land close together.
• Vector Database : Stores the embeddings and finds the chunks most similar to a query. Examples: Pinecone, Chroma, Qdrant, pgvector.
• Retrieve & Generate : The user's question is embedded, the closest chunks are retrieved, and the LLM answers using those chunks as context, often citing them.

An AI agent is an LLM that does not just answer, but acts: it plans, uses tools, observes the results, and keeps going until a goal is reached. A chatbot answers one message; an agent can take a task like "fix this bug" and edit files, run tests, read the errors, and try again.

The Agent Loop -

Goal → Reason (decide next step) → Act (call a tool) → Observe (read the result) → repeat until done

• Tool / Function Calling : The model is given a list of tools it may call (search the web, run code, query a database, send an email). It responds with a structured call, the application executes it, and the result is fed back to the model.
• Memory : Agents keep short-term memory (the conversation and recent results in context) and sometimes long-term memory (notes or vector stores that persist across sessions).
• Planning : Breaking a large goal into steps, tracking progress, and revising the plan when a step fails.

Types of Agent Systems -

• Workflows : Fixed, predefined sequences of LLM calls and tools. Predictable and reliable; best when the steps are known in advance.
• Autonomous Agents : The LLM decides its own steps and tool calls dynamically. Flexible and powerful for open-ended tasks, but less predictable and needs guardrails.
• Multi-Agent Systems : Several specialized agents cooperate, such as a planner delegating to coder and reviewer agents, coordinated by an orchestrator.

• Examples : Coding agents (Claude Code, GitHub Copilot agent mode, Cursor), research agents that browse and compile reports, and support agents that resolve tickets end to end.
• Frameworks : LangChain / LangGraph, Claude Agent SDK, OpenAI Agents SDK, CrewAI.

The Model Context Protocol (MCP) is an open standard, introduced by Anthropic in 2024, that defines how AI applications connect to external tools and data sources. Instead of writing a custom integration for every app-tool pair, a tool exposes one MCP server and any MCP-capable app can use it, like USB-C for AI integrations.

Architecture -

• Host : The AI application the user interacts with, such as Claude Desktop, Claude Code, or an IDE.
• Client : The connector inside the host that maintains a connection to one server.
• Server : A small program that exposes capabilities from some system (GitHub, a database, the filesystem, Slack) over the protocol. Servers run locally or remotely.

What a Server Exposes -

• Tools : Actions the model can call, such as create_issue or query_database.
• Resources : Data the host can read into context, such as files, schemas, or documents.
• Prompts : Reusable prompt templates the user can invoke.

Application: connect a GitHub MCP server and a Postgres MCP server to an AI assistant, and it can read open issues, query the production schema, and open a pull request, all through one standard protocol. SDKs are available in Python, TypeScript, and other languages.