linear-algebra-practice
Applied linear algebra — decompositions, least squares, conditioning, and numerical habits for real matrices.
- License 许可
- MIT
- 最近更新
- 2026-11-07
- Stars
- ★ 0
- Forks
- 1
01 前言 Front Matter
- name
- linear-algebra-practice
- category
- scientific
- description
- Applied linear algebra — decompositions, least squares, conditioning, and numerical habits for real matrices.
03 SKILL.md 正文
Overview
Linear algebra is the computational substrate of science: every fit, simulation, and data analysis reduces to matrix operations. This skill covers choosing the right decomposition (LU, QR, SVD, eigendecomposition), solving least-squares problems properly, understanding conditioning and stability, and the numerical habits (never invert explicitly, exploit structure) that separate working code from slow, wrong code.
When to use
- Solving linear systems or least-squares problems
- Choosing between SVD, QR, eigendecomposition, Cholesky for a task
- Diagnosing numerical instability: huge condition numbers, rank deficiency
- Dimensionality reduction (PCA), low-rank approximation, pseudoinverses
- Reviewing numerical code for stability and efficiency
Core concepts
-
The decompositions and their jobs: LU (general square systems), Cholesky (symmetric positive-definite — 2× faster than LU), QR (least squares, orthogonalization), SVD (rank, pseudoinverse, PCA — the most revealing, most expensive), eigendecomposition (symmetric/Hermitian only for guaranteed orthogonal eigenvectors).
-
Condition number κ: κ = σ_max/σ_min — measures error amplification; κ ~ 10^k means losing ~k digits. A large κ is a property of the problem, not the algorithm — regularize or reformulate.
-
Least squares done right: solve via QR or SVD, never via the normal equations (AᵀA squares the condition number: κ²).
lstsqwith SVD is the safe default. -
Rank and nullspace: SVD reveals numerical rank (singular values near zero), nullspace (right singular vectors), and range — the diagnostic toolkit for degenerate problems.
-
Sparsity and structure: sparse (scipy.sparse), banded, Toeplitz, Kronecker — structured solvers are orders of magnitude faster; forming a dense matrix from a sparse problem is the classic performance bug.
-
Floating point: double precision has ~16 digits; catastrophic cancellation (subtracting nearly equal numbers) and overflow in naive formulas (use hypot, log-sum-exp) are the perennial traps.
-
Krylov subspace methods: CG, GMRES, Lanczos build solutions in expanding Krylov spaces — matrix-free (only matvecs needed), the only option for million-degree-of-freedom systems; preconditioning makes or breaks them.
-
Randomized algorithms: randomized SVD and sketching give near-optimal low-rank approximations in a fraction of the time — the practical tool for huge dense matrices in data science.
-
Tensor decompositions: CP and Tucker generalize SVD to multiway arrays — compression and latent-structure discovery for multidimensional scientific data.
Practical workflow
1. Solve systems correctly
import numpy as np
from scipy import linalg
# Least squares via SVD (safe default); never form inv(A) or A.T @ A
x, residuals, rank, sv = linalg.lstsq(A, b)
# Check conditioning before trusting the answer
kappa = sv[0] / sv[-1]
- Never compute the matrix inverse explicitly — solve the system (
solve,lstsq) instead. - Check κ (or the singular-value spectrum) routinely; if κ > 1/√ε (~10⁸ in double), treat results as suspect.
- For SPD systems use Cholesky; for least squares use QR/SVD — match the decomposition to the structure.
2. Diagnose trouble
- Huge κ: rescale columns (column equilibration), remove collinear features, or regularize (ridge/Tikhonov: minimize ‖Ax−b‖² + λ‖x‖², choosing λ by L-curve or cross-validation).
- Rank deficiency: inspect small singular values and their vectors — they tell you which combinations of variables are unidentifiable.
- Residual analysis: large residuals + small κ = model wrong; small residuals + huge κ = solution unstable — different diseases, different cures.
3. Exploit structure
- Sparse systems: iterative solvers (CG for SPD, GMRES/BiCGSTAB general) with preconditioners (incomplete Cholesky/LU) — direct sparse factorization for moderate sizes.
- Use matrix-free formulations (linear operators) when A is too big to form — common in inverse problems and PDE-constrained optimization.
- Randomized SVD for huge matrices when only the top-k singular vectors are needed.
4. PCA and low-rank work
- Center (and consider scaling) before PCA — the first component of uncentered data is usually the mean.
- Choose rank by the singular-value spectrum (elbow, or a variance threshold) and validate that discarded components are noise (compare against a shuffled null).
- Remember PCA is variance-maximizing, not signal-maximizing — high-variance components can be artifacts (batch effects, trends).
5. Solve a large sparse system iteratively
- Choose the solver for the structure: CG for symmetric positive-definite, GMRES/BiCGSTAB otherwise — and always precondition (incomplete Cholesky/LU, or algebraic multigrid for PDE-derived systems).
- Monitor the residual norm and the true error (they differ) — set tolerances on the quantity you actually care about.
- Verify against a direct solve on a small version of the problem — iterative solvers with bad preconditioners converge to wrong answers slowly and confidently.
Common pitfalls
- Explicit inverses:
inv(A) @ bis slower and less accurate thansolve(A, b)— always. - Normal equations: forming AᵀA squares the condition number — the textbook method that fails in practice.
- Ignoring κ: publishing solutions from κ ~ 10¹² systems as if they were precise.
- Uncentered PCA: first component captures the mean, not the interesting variation.
- Dense treatment of sparse problems: forming a million×million dense matrix instead of using sparse/iterative methods.
- Single precision by accident: float32 halves your digits — fine for ML, dangerous for ill-conditioned solves; check dtypes.
- Iterative solver without preconditioning: unpreconditioned Krylov methods on ill-conditioned systems stall for thousands of iterations — preconditioning is not optional.
- Confusing residual with error: a small residual with an ill-conditioned matrix still means a large error — bound the error by κ times the residual.
04 同仓库更多 Skills
web-scrape
Turn web pages into LLM-ready markdown via the Firecrawl API (renders JS, anti-bot handling). Use when browser_open/fetch fails on JS-heavy pages, or when you need clean markdown/structured extraction from a site. Requires FIRECRAWL_API_KEY.
design-taste-frontend
tasteskill v2, the anti-slop frontend skill. Art direction for websites, distilled from a study of 247 curated gallery sites (One Page Love, Awwwards Sites of the Day). Deals researched directions (idea, composition, type, palette, photographs, section blueprints) from a library, enforces a strict no-slop house style with a real-browser inspector, and runs a look-and-fix loop on screenshots. Use this whenever you build, design, redesign, restyle, polish or review a landing page, marketing site, portfolio, product page, one-pager, hero or any public-facing web page, in any stack (HTML/CSS, React, Next.js, Vue, Svelte, Astro, Tailwind) — also when the user says a page looks generic, templated, "AI-made", sloppy, or asks to make it look designed, premium, award-level or less like every other site. Not for dashboards or app screens.
test-driven-development
Use when implementing any feature or bugfix, before writing implementation code
systematic-debugging
Use when encountering any bug, test failure, or unexpected behavior, before proposing fixes
requesting-code-review
Use when completing tasks, implementing major features, or before merging to verify work meets requirements
brainstorming
You MUST use this before any creative work - creating features, building components, adding functionality, modifying behavior, or planning anything new, in software or out of it (a talk, a business, a renovation).