What Actually Happens When You Set Model Weights to Zero (and Why Gradients Still Work)
A common fear in deep learning is: “If I set some weights to zero, won’t I break differentiability?”
Read more →Blog
Short updates, context, and essays that complement the long-form notes in the bookshelf.
Looking for a specific topic? Search all posts.
Popular Tags
Hollywood Color Pipeline: From Dailies to DI (and Why Show LUT Is the Film's Visual DNA)
If you want to understand Hollywood color, you do not start with knobs and curves. You start with a pipeline.
Read more →Taylor Series Expansion: A Local Lens for Functions
Taylor Series Expansion: A Local Lens for Functions
Read more →Learning Rate Schedulers: Intuition, Tradeoffs, and When to Use Which
Learning rate schedules are the steering wheel of optimization. The learning rate controls how far each step moves in parameter space; the schedule controls how that step size e...
Read more →Dijkstra’s Algorithm and Where Machine Learning Uses It
Dijkstra’s algorithm is the workhorse for finding shortest paths in weighted graphs with non‑negative edges. It’s taught in algorithms courses, but it also shows up quietly insi...
Read more →Symmetric Groups: Understanding Permutations from First Principles
Symmetric Groups: Understanding Permutations from First Principles
Read more →Bijective Functions: The Perfect Correspondence
Bijective Functions: The Perfect Correspondence
Read more →Signed Volume: The Geometric Soul of Determinants
This post explores one of the most beautiful concepts in linear algebra: signed volume. We’ll see why volumes need signs, what orientation means geometrically, and how this simp...
Read more →Matrix Determinants: From Leibniz Formula to Geometric Intuition
This post explores the determinant—one of the most fundamental concepts in linear algebra. We’ll build intuition from geometric interpretations, derive the Leibniz formula, and ...
Read more →Mask Refinement in Professional Video Editing: Premiere Pro vs DaVinci Resolve Magic Mask 2.0
This post explores how modern video editing software has integrated advanced matting and mask refinement technology into production workflows. We’ll compare Adobe Premiere Pro’s...
Read more →Video Matting: Temporal Consistency and Real-Time Foreground Extraction
This post explores how to extract foreground objects from video sequences with temporally consistent alpha mattes. We build on image matting concepts to handle the additional ch...
Read more →Image Matting: Estimating Accurate Mask Edges for Professional Compositing
This post explores how to extract foreground objects with accurate transparency information at edges, enabling professional-quality compositing. We’ll cover the mathematical fou...
Read more →Depth Maps in Computer Vision: From Stereo Geometry to Neural Networks
This post explores how computers perceive depth in images, bridging classical geometric methods with modern neural approaches. Familiarity with basic linear algebra, projective ...
Read more →Interactive Depth vs Disparity Visualization
This interactive tool lets you explore the fundamental relationship between depth and disparity in stereo vision systems. Experiment with camera parameters to build intuition ab...
Read more →How Variational Autoencoders Avoid Computing the Partition Function
Reading Time: 40-50 minutes This comprehensive article includes rigorous mathematical derivations requiring careful thought. Plan for extra time to work through the ELBO derivat...
Read more →Expected Value & Expectation: Mathematical Foundations
This post provides the mathematical foundation for understanding expectation, a concept central to probability theory and machine learning. This explanation supports the ELBO de...
Read more →Building Contrast Prediction Models with Unsupervised Learning: When Ground Truth Doesn't Exist
Imagine you have thousands of images and need to build a system that can predict their contrast characteristics. The catch? You have no labeled ground truth. No “correct” contra...
Read more →From Brownian Motion to Modern Generative Models: The Stochastic Foundation of Diffusion and Flow Models
This post builds on concepts from Stochastic Processes and the Art of Sampling Uncertainty. For a broader introduction to differential equations (ODEs, PDEs, SDEs), see The Land...
Read more →About Atul Singh: Technical Portfolio and Expertise
About Atul Singh
Read more →Understanding Color Balance in Images and Video
Color balance is the practice of aligning the relative intensities of primaries so that neutral surfaces appear neutral and hues stay believable under different illuminants. Whe...
Read more →Measuring Contrast Between SDR and HDR Images: Bridging Dynamic Range Domains
Note: This post builds on Understanding Contrast in Images: From Perception to Computation, Understanding Contrast in Color Images: Beyond Luminance, Measuring Contrast Betwe...
Read more →Mathematical Properties of Brownian Motion: A Visual Guide
This post provides the mathematical foundation for understanding Brownian Motion and Modern Generative Models. It can be read independently or as a prerequisite for that post.
Read more →The Landscape of Differential Equations: From ODEs to PDEs to SDEs
Table of Contents
Read more →Comparing Contrast Across Different Images: Content-Independent Metrics
When comparing contrast between two images, there’s a fundamental distinction that’s often overlooked: Are we comparing different versions of the same scene, or are we comparing...
Read more →Measuring Contrast Between Two Color Images: Comparison Metrics and Methods
When working with image processing pipelines, quality assessment, or change detection systems, we often need to quantify how much the contrast differs between two images. This g...
Read more →Itô Calculus: Why We Need New Rules for Stochastic Differential Equations
This post assumes familiarity with basic calculus and Brownian motion. For a broader context on differential equations, see The Landscape of Differential Equations.
Read more →Understanding Contrast in Images: From Perception to Computation
Contrast is one of those terms that feels intuitive until you try to define it precisely. “High contrast” evokes vivid images with deep blacks and brilliant whites, while “low c...
Read more →Understanding Contrast in Color Images: Beyond Luminance
In grayscale images, contrast is relatively straightforward—it’s all about differences in brightness. But when color enters the picture, the story becomes fascinatingly complex....
Read more →Infinite Total Variation of Brownian Motion: Why the Path Length Diverges
This post provides a detailed mathematical analysis of total variation for Brownian motion. It builds on concepts from Mathematical Properties of Brownian Motion and connects to...
Read more →Machine Learning Paradigms: Learning Distributions vs Approximating Functions
This post establishes foundational concepts that underpin many advanced topics. For historical context on why this distinction matters, see Why Discriminative Learning Dominated...
Read more →Why Discriminative Learning Dominated First: The Pragmatic Path to Modern AI
This post provides historical context for the paradigm shift discussed in Machine Learning Paradigms: Learning Distributions vs Approximating Functions.
Read more →The Normalization Constant Problem: Why Computing Z Is So Hard
This post explores one of the fundamental obstacles in generative modeling, discussed in Why Discriminative Learning Dominated First. For the broader context, see Machine Learni...
Read more →The Curse of Dimensionality: Why High Dimensions Break Intuition
This post explores another fundamental obstacle in machine learning, discussed in Why Discriminative Learning Dominated First. For the broader context, see Machine Learning Para...
Read more →From Funnels to Loops: Distribution Shifts in the AI Era
TL;DR
Read more →Adaptive Dual-Teacher Distillation for Lightweight Video Models
TL;DR
Read more →Understanding Reference Types in Modern C++
Modern C++ gives you several kinds of references, each solving different problems around binding, lifetime, and ownership. Before C++11, you had one tool: the lvalue reference. ...
Read more →How Pix2Seq-D Generates Panoptic Masks Without Heavy Inductive Biases
Question: The model learns to generate a panoptic mask, which is an array of discrete tokens, conditioned on an input image. This avoids the inductive biases of traditional m...
Read more →Why Direct Sampling from PDFs or PMFs Is So Hard
Directly sampling from a probability density function (PDF) or probability mass function (PMF) sounds deceptively simple: evaluate the function, interpret the number, and draw a...
Read more →Gripe, Complaint, or Grievance? Understanding Usage in Indian Newspapers (September 2025)
Introduction
Read more →From Elementary Mathematics to Vision Algorithms: The Hidden Life of Normalized Power Sums
From Elementary Mathematics to Vision Algorithms: The Hidden Life of Normalized Power Sums
Read more →How Cameras and Eyes See Light: From Spectra to Illuminants
Light is never just “white.” It has a shape, a fingerprint, a dance across wavelengths. Cameras record it, eyes interpret it, and both struggle in their own ways. In this post, ...
Read more →From Elementary Mathematics to Vision Algorithms: The Hidden Life of Normalized Power Sums
From Elementary Mathematics to Vision Algorithms: The Hidden Life of Normalized Power Sums
Read more →From Linear Equations to Vector Spaces
Linear algebra did not start as an abstract theory. It started as a practical question: when does a system of equations have a solution, and how many are there? From that concre...
Read more →Frequency Domain vs Spatial Domain: How Images Reveal Different Stories
Image filters feel natural in the spatial domain because we see pixels and neighborhoods. Yet many operations become simpler—or even possible only—in the frequency domain. This ...
Read more →Understanding KL Divergence: Measuring One Distribution Against Another
Kullback–Leibler (KL) divergence quantifies how one probability distribution departs from another. It shows up everywhere—from maximum-likelihood training to variational inferen...
Read more →Bitmap Lookups for Fast YUV → RGB Gamut Decisions
Color pipelines that ingest YUV video frames but render or analyze in RGB often have to answer two expensive questions before any correction work begins:
Read more →The Evolution of Optimization: From Solving Equations to Gradient Descent and Poisson Solvers
Table of Contents
Read more →PyTorch Tensor Indexing: From 1D Slices to N-Dimensional Views
Tensor indexing feels natural once you see how dimensions line up. This walkthrough starts with 1D arrays and climbs to N-dimensional tensors, highlighting how PyTorch treats sl...
Read more →Publishing Cadence Summary
Looking for a single place to trace how the notebook has evolved? This reference page aggregates every post that landed on the site, sliced by month, quarter, half-year, and ful...
Read more →From ISA to GPU Kernels: Bridging SIMD Mindsets
If you have written SIMD intrinsics for SSE or AVX, many of the mental models translate directly to CUDA or HIP kernels. Both ecosystems ask you to think in vectors, manage memo...
Read more →Random vs Stochastic: Clarifying Variables, Processes, Sampling, and Optimization
Why the Vocabulary Feels Slippery
Read more →GPU Kernel Programming: Grids, Blocks, and Warps Explained
Modern GPUs deliver petaflops by letting thousands of lightweight threads execute in lockstep. To harness that power, you need to understand how your kernels are scheduled and w...
Read more →Template Programming Frontiers
Why Templates Still Matter
Read more →SIMD Intrinsics: From SSE to AVX2 in Practice
Modern CPUs have wide vector units capable of executing the same operation across multiple data elements at once. Compiler auto-vectorization handles the simple cases, but when ...
Read more →Logarithmic Color Spaces, PCA, and the lαβ Intuition
The lαβ color model from Reinhard, Ashikhmin, Gooch, and Shirley (2001) fuses two simple ideas: compress channel intensities with a logarithm so that multiplicative illumination...
Read more →Beyond Basics: Importance, Gibbs, and Stratified Sampling
When Theory Meets Practice in Sampling
Read more →Stochastic Processes and the Art of Sampling Uncertainty
Why Stochastic Thinking Matters
Read more →Mastering std::async in Modern C++
std::async is the standard library’s shortest path to running work asynchronously and receiving the result as a std::future. Used well, it hides thread management, propagates ex...
Read more →Composing Futures in Modern C++
In the previous post we focused on the contract between a single promise and future. Real systems rarely stop there. Data pipelines, UI flows, and service backends routinely lau...
Read more →Understanding Futures and Promises in Modern C++
Modern C++ puts powerful concurrency tools in your hands, but raw threads still make it easy to race on shared state, forget to join, or swallow exceptions. Futures and promises...
Read more →Culpable: Word Family, Nuance, and Indian Newsroom Examples
Introduction
Read more →Understanding the Implicit Function Theorem
Why the Implicit Function Theorem Matters
Read more →From Gradients to Hessians: How Optimization Shapes Vision & ML
Interactive Figures Spin, zoom, and inspect these plots to see how gradient and Hessian information shapes the local geometry and degeneracies we meet in vision pipelines.
Read more →From Gradients to Hessians: How Optimization Shapes Vision & ML
Interactive Figures
Read more →Why Intersection Fails in Lagrange Multipliers: The Geometry of Optimization
Why Intersection Fails in Lagrange Multipliers: The Geometry of Optimization
Read more →The Hidden Symmetry of Inverse Sine and Cosine
Mathematics often rewards patient observers with patterns that seemed invisible at first glance. One of my favorite examples links two functions that appear to live on separate ...
Read more →Understanding ACES, Scene-Referred Workflows, and Color Space Conversions
Color pipelines can feel abstract until you tie each transform to what your eye sees. In this post we follow a Rec.709 image as it moves into ACES, gets graded in a scene-referr...
Read more →From Tesseract to Transformers: The Evolution of OCR
Optical character recognition rarely gets the spotlight, yet it’s the bridge between physical documents and searchable data. In 2005, HP open-sourced its internal Tesseract engi...
Read more →Understanding the Resent Family of Emotions
Negative feelings often get lumped together, but the words we choose point to very different shades of experience. I like to think of resent as the seed in a small emotional eco...
Read more →MathJax Test Page
MathJax Test Page
Read more →Advanced Topics and Future Directions in Kalman Filtering
This is Part 8 (Final) of an 8-part series on Kalman Filtering. Part 7 covered nonlinear extensions.
Read more →Nonlinear Extensions: EKF, UKF, and Particle Filters
This is Part 7 of an 8-part series on Kalman Filtering. Part 6 explored real-world applications.
Read more →Real-World Applications of Kalman Filtering
This is Part 6 of an 8-part series on Kalman Filtering. Part 5 covered Python implementation details.
Read more →Implementing the Kalman Filter in Python
This is Part 5 of an 8-part series on Kalman Filtering. Part 4 provided the complete mathematical derivation.
Read more →Complete Mathematical Derivation of the Kalman Filter
This is Part 4 of an 8-part series on Kalman Filtering. Part 3 established the Bayesian foundations.
Read more →Bayesian Foundations of Kalman Filtering
This is Part 3 of an 8-part series on Kalman Filtering. Part 2 explored recursive filtering fundamentals.
Read more →Fundamentals of Recursive Filtering
This is Part 2 of an 8-part series on Kalman Filtering. Part 1 introduced state estimation concepts.
Read more →Introduction to Kalman Filtering and State Estimation
This is Part 1 of an 8-part series on Kalman Filtering. This series will take you from basic concepts to advanced applications and implementations.
Read more →How the Learning Collection Gets Built Each Week
One of the benefits of keeping research inside Obsidian is that the vault can double as a content pipeline. Folders map to learning projects, daily notes track reading sessions,...
Read more →Introducing the Atul Singh Notes Blog
The books and long-form research that live in this repository tend to move slowly. They collect notes, citations, and iterative drafts over weeks or months before they finally b...
Read more →Canny Edge Detector in Practice (circa 2013)
Canny’s blend of Gaussian smoothing, gradient estimation, non-maximum suppression, and hysteresis linking made it the go-to detector for general-purpose pipelines in 2013. Below...
Read more →Edge Detection Operators: A 2013 Snapshot
Edge detection has been a foundational pre-processing step in computer vision since the earliest digital imaging pipelines. Around 2013, practitioners still relied on a handful ...
Read more →