Reinforcement Learning (RL): A Quick Introduction
What is RL?
Reinforcement Learning is learning by trial and error. A learner (the agent) acts in a world (the environment). After each action it gets a number called the reward. Nobody tells it the correct action. It has to find out for itself which actions bring the most reward over time, like a child learning to ride a bicycle.
How is it different from other machine learning?
- Supervised learning: the correct answer is given for every example.
- Unsupervised learning: there are no answers; the goal is to find patterns.
- Reinforcement learning: only a reward score is given, often late, and the agent’s own actions decide what it sees next.
Key terms (example: a robot vacuum cleaner)
- Agent: the learner and decision-maker (the vacuum’s control program).
- Environment: everything outside the agent (room, furniture, dirt, charging dock).
- State (s): the current situation (position, battery level, areas already cleaned).
- Action (a): a choice the agent can make (forward, turn left, turn right, go to dock).
- Reward (r): one number after each action (+1 for dirt collected, −1 for hitting furniture, −10 if the battery runs out).
- Episode: one complete run from start to end (one cleaning session).
- Return (G): the total reward collected from now on.
- Discount factor (γ): a number from 0 to 1 that makes future rewards count less than rewards now.
- Policy (π): the agent’s rule for choosing an action in each state.
- Value v(s): how much total reward the agent can expect starting from state s.
- Action value q(s,a): how good it is to take action a in state s. This is the “Q” in Q-learning.
- Markov property: the state holds all the information needed; the past adds nothing more.
- Exploration vs exploitation: trying new actions vs using the best action found so far. The ε-greedy rule explores with a small probability ε.
The basic loop
The agent sees the state → chooses an action → receives a reward and a new state → repeats.
Three formulas to remember
- Return: G = r₁ + γ·r₂ + γ²·r₃ + …
- Return now = next reward + γ × (return from the next step)
- Goal: find the policy that gives the highest expected return.
Types of RL methods
- Model-based (plans using knowledge of the environment) vs model-free (learns only from experience).
- Value-based, such as Q-learning, vs policy-based, such as policy gradient.
- On-policy, such as SARSA, vs off-policy, such as Q-learning.
In one line: RL teaches an agent to act well by rewarding good outcomes, not by showing it the right answers.
For full explanations, worked numericals and exercises, download the complete tutorial below.
— @SSRoy
RL Part 2: MDPs, Policies and Planning (Quick Introduction)
Markov Decision Process (MDP)
An MDP is the exact mathematical description of an RL problem. It has five parts, written (S, A, P, R, γ):
- S: the set of states (corridor cells A, B, C, D)
- A: the set of actions (Left, Right)
- P(s′|s,a): the probability of reaching state s′ after action a in state s
- R: the reward for each move (−1 per step, +10 for reaching D)
- γ: the discount factor (e.g. 0.9)
Policy
A policy is a rule that says which action to take in each state, e.g. “always go Right”.
Bellman equation
The value of a state = expected reward now + γ × the value of the next state.
- For a given policy: v(s) = Σ π(a|s) Σ P(s′|s,a) [R + γ·v(s′)]
- For the best policy: v*(s) = maxₐ Σ P(s′|s,a) [R + γ·v*(s′)]
- Best action: π*(s) = the action with the highest q*(s,a)
Planning
Planning means computing the best policy when the model (P and R) is already known, with no trial and error.
- Policy evaluation: find the value of each state under a given policy.
- Policy improvement: in each state, switch to the action with the highest value.
- Policy iteration: repeat evaluate → improve until the policy stops changing.
- Value iteration: repeat the Bellman max update until the values stop changing.
In one line: an MDP describes the problem, the Bellman equation links neighbouring states, and planning uses it to find the best policy.
For full explanations, worked numericals and exercises, download the complete tutorial below.
— @SSRoy
Q-Learning
Winnow Algorithm: A Quick Introduction
The Winnow algorithm is an online learning algorithm: it learns from one training example at a time and keeps learning as new data arrives, without needing the full dataset first. Each input is described by features with yes/no values: 1 means the feature is present and 0 means it is absent, such as “email contains the word prize”. A feature with value 1 is called an active feature. Every feature has a weight showing its importance, and all weights start at 1. For each example, Winnow adds the weights of the active features to get a score. If the score is at least the threshold (for example 2), the prediction is YES; otherwise it is NO. Winnow is mistake-driven: weights change only when a prediction is wrong. If the correct answer was YES, it multiplies the weights of the active features by the learning factor (for example 2); this is called promotion. If the correct answer was NO, it divides them by the learning factor; this is called demotion. For example, a spam email containing “prize” first scores 1, so Winnow predicts NO; the weight of “prize” then doubles to 2, and the next such email is correctly flagged as spam. Applications include spam detection, text classification, web filtering and simple recommendation systems.
For the full worked example and an exercise, download the complete material below.
— @SSRoy
Explainable AI (XAI): An Introduction
Modern AI systems such as deep neural networks and large language models make powerful decisions, but they often work as “black boxes”: they give an answer without saying why. Explainable AI (XAI) is the set of methods that make these decisions understandable to people. With XAI, a doctor can see which part of an X-ray a model looked at, and a bank can tell a customer why a loan was refused. This tutorial covers the essential terms (interpretability, transparency, local and global explanations, faithfulness and counterfactuals); the main methods (SHAP, LIME, Grad-CAM, Integrated Gradients and concept-based explanations); and the newest research on understanding large language models (mechanistic interpretability, sparse autoencoders, circuit tracing and chain-of-thought faithfulness). It also covers key applications, popular tools, laws such as the EU AI Act and India’s AI governance guidelines, and the landmark research that shaped the field. XAI is no longer optional: it is central to building AI that is trustworthy, fair and accountable.
AI Ethics
What is AI ethics?
AI ethics is the set of values and rules that make sure AI systems help people and do not harm them. It asks three questions: Is the AI fair? Is it safe? Who is responsible when it goes wrong? As AI now decides loans, jobs, medical treatment and what news we see, these questions affect everyone.
Core principles (the keywords)
- Fairness and non-discrimination: AI must not treat people unfairly by gender, caste, religion, race, region or age. Algorithmic bias often comes from biased training data.
- Transparency and explainability (XAI): people should know when AI is used and why it reached a decision.
- Accountability: a human or organisation must answer for every AI decision; “the algorithm did it” is never an excuse.
- Privacy and data protection: collect only the data needed, with informed consent, and keep it secure.
- Safety, security and robustness: AI must work reliably and resist hacking, jailbreaks and misuse.
- Human oversight (human-in-the-loop): humans must be able to check, override or stop AI, especially in high-risk uses.
- Beneficence and non-maleficence: do good and avoid harm.
- Inclusiveness and sustainability: AI should benefit all, including the Global South and people with disabilities, and limit its energy and water use.
Key ethical issues today
- Bias: hiring tools that prefer one gender; face recognition that is less accurate for darker skin.
- Deepfakes and misinformation: fake videos and voice clones used for fraud, scams and election manipulation.
- Hallucination: chatbots confidently stating false facts, fake legal cases or fake references.
- Copyright and consent: training AI on books, art and news without permission or payment.
- Surveillance: mass facial recognition and tracking that threaten civil liberties.
- Jobs and inequality: automation displacing workers; the digital divide between countries and communities.
- Children and mental health: AI companion chatbots forming unhealthy bonds with minors.
- Agentic AI: AI agents that act on their own (buy, email, code) raise new questions about who is responsible.
- Autonomous weapons: machines deciding to use lethal force.
- Environmental cost: data centres use large amounts of electricity and water.
- AI safety and alignment: making sure powerful AI systems pursue human goals.
Recent happenings (2025–2026)
- India: amended IT Rules (in force 20 Feb 2026) require AI-generated content to be clearly labelled with permanent metadata, and some takedown orders to be acted on within 3 hours. India hosted the AI Impact Summit (New Delhi, 16–20 Feb 2026), where 88 countries plus the EU endorsed the New Delhi Declaration for “collaborative, trusted, resilient and efficient AI”. MeitY released the India AI Governance Guidelines (Nov 2025).
- European Union: the EU AI Act bans unacceptable-risk AI (e.g. social scoring); its transparency duties for chatbots and deepfakes apply from 2 Aug 2026. The 2026 Digital Omnibus moved high-risk rules to Dec 2027 / Aug 2028.
- United Nations: the first Global Dialogue on AI Governance (Geneva, July 2026) received the preliminary report of the new Independent International Scientific Panel on AI.
- USA: no federal AI law yet; states lead. Nineteen states, including California (SB 243), passed laws on AI chatbots and minors. Colorado replaced its AI Act with a lighter transparency law.
- Copyright: a US court gave final approval (July 2026) to a record $1.5 billion settlement between Anthropic and authors over pirated books used for training.
Key frameworks
UNESCO Recommendation on the Ethics of AI (2021) · OECD AI Principles · EU AI Act · NIST AI Risk Management Framework · ISO/IEC 42001 · India’s Digital Personal Data Protection (DPDP) Act 2023 · Council of Europe Framework Convention on AI.
Putting ethics into practice (Responsible AI)
Use diverse, representative data · run bias audits and red-teaming before release · publish model cards · keep humans in the loop · run AI impact assessments · label AI-generated content and use provenance/watermarking · keep audit trails · give people a way to complain and appeal.
In one line: ethical AI is AI that is fair, transparent, safe, private and accountable, built for people, not just about them.
— @SSRoy