Real Analysis  

Warning: notations are only consistent within one paragraph!

• What is the insight of differential for function y = f(x)?  The differential dy is called principle linear part of the increment \delta y, due to \delta x.   The differential dy differs from \delta y by an infinitesimal of a higher order as compared to \delta x.
• Inequalities: 
  • Holder Inequality: 1/p+1/q=1, E(ab)<= [E(a^p)]^{1/p} * [E(b^q)]^{1/q}, can be used to separate correlated r.v.
  • Cauchy-Schwarz inequality: E(ab)<= [E(a^2)]^{1/2} * [E(b^2)]^{1/2}, can be used to separate correlated r.v.
• Definitions of real numbers (six?)
  • Dedekind cut:  any real number corresponds to a Dedekind cut of the set of rational numbers.

    A Dedekind cut in an ordered field is a partition of it, (A, B), such that A is closed downwards (meaning that whenever a is in A and x ≤ a, then x is in A as well), B is closed upwards and A has no maximum.

    The Dedekind cut is named after Richard Dedekind, who invented this construction in order to represent the real numbers as Dedekind cuts of the rational numbers. A typical Dedekind cut of the rational numbers is given by A = { a in Q : a² < 2 }, B = { b in Q : b² ≥ 2 }. This cut represents the real number √ 2 in Dedekind's construction.

    The existence of representation by Dedekind-cuts of a dense subset is equivalent to separability of a metric space.

  • (Completeness axiom of the real numbers) Least upper bound principle:  
    • Every nonempty set (of real numbers) that is bounded above has a least upper bound.
  • Greatest lower bound principle: 
    • Every nonempty set (of real numbers) that is bounded below has a greatest lower bound.
  • Completeness of the set of all real numbers R:
    • Every Cauchy sequence in R converges to exactly one number in R.
  • Nested interval principle
• Important theorems (six)
  • (Cantor) Nested interval principle (axiom): 
    • An increasing sequence x_n and a decreasing sequence y_n with x_n< y_n, must meet at only one point on the real line (not more than one); that is, lim x_n = lim y_n = c, where c \in union_{n=1}^{\infty} [x_n, y_n].  There is no hole on the real line.  Completeness of R.
  • Bolzano-Weirestrass theorem:
    • Every bounded infinite set has an accumulation point; that is, every bounded sequence must have a convergent subsequence.
    • (interpretation) Every bounded group must have a subgroup with infinite number of points approaching a common objective.
    • An accumulation point is like a kernel with infinite points packed in a very small neighborhood around it.
    • This theorem could serve as another definition of a compact set: a set is compact if every bounded infinite subset has an accumulation point.  (Compare with "every open cover admits a finite subcovering".)
  • Cauchy theorem: 
    • A sequence converges if and only if it is a Cauchy sequence.
  • A sequence that is monotonic and bounded, must converge.
  • Every nonempty set (of real numbers) that is bounded above has a least upper bound; every nonempty set (of real numbers) that is bounded below has a greatest lower bound.
  • Heine-Borel Theorem:
    • A subset of R is compact if and only if it is closed and bounded.
• (Intermediate Value Theorem): 
  • Intermediate Value Property:  A function f is said to have the intermediate value property on the interval [a,b] if given any two points x_1, x_2 \in [a,b] and any number N satisfying f(x_1) < N < f(x_2), then there is at least one value c between x_1 and x_2 for which f(c)=N.    
    • Note that Intermediate Value Property does not necessarily lead to continuity of a function.
    • If f is piecewise monotonic and satisfies the intermediate value property on the interval [a,b], then f is continuous at every point c in (a,b).
  • Intermediate Value Theorem: If a function f is continuous on the interval [a,b], then it has the intermediate value property on this interval.
• On infinity
  • For a function f(x), f(x_0)= infinity means that f(x_0) is unbounded and it does not take any finite value. Infinity is equivalent to unboundedness.
  • A sequence {a_n} converges to a limit A.   It implies that any (infinite) subsequence of {a_n} converges to A.   
  • For a divergent sequence {a_n}, there may not exist an (infinite) subsequence of {a_n} that converges.   But the upper limit (limsup) of {a_n} always exists (it could be +infinity but cannot be negative infinity).
• Sufficient conditions (Dirichlet conditions) for a function f(x) to be representable by a Fourier series over an interval [a,b]:
  • f(x) is integrable over [a,b];   (that is, the coefficients of Fourier series can be computed by the integral formulae proposed by Fourier.)
  • at any discontinuous point, say x_0, the value f(x_0) is equal to the half of the sum of the left limit and the right limit of f(x) at x_0;
  • f(x) is piecewise continuous, bounded, and piecewise monotonic.
• Uniqueness of the Fourier-series representation  (uniqueness of the trigonometric representation):
  • Uniform convergence in general:  
    • A trigonometric series converges uniformly in general if there are a finite number of break points in the interval of the basic period so that on any closed interval that does not include a break point, the series converges uniformly.
  • (Sufficient condition) If the Fourier series of a function f(x) converges uniformly in general, then there is no other trigonometric series representing the same function f(x).
  • A function may be represented by several trigonometric series (i.e., these series converge to the same function) if they do not converge uniformly in general.   That is, multiple different trigonometric series can be equal at every point in the region of convergence.
• Continuity of f(x) at x_0
  • Intuition: if x is very close to x_0, f(x) should also stay very close to f(x_0).
  • Definition using \epsilon - \delta argument:  For any given \epsion>0, there exists a \delta>0 such that for |x-x_0|<\delta, |f(x)-f(x_0)|<\epsilon.
• Infinite series of numbers
  • Condition for groupings (associative law): convergence of the series.
  • Condition for rearrangements (commutative law): 
    • Given an absolutely convergent series, any rearrangement of this series yields another convergent series, converging to the same value.
    • If the series converges conditionally, then for any real number r, we can find a rearrangement of this series that converges to r.

Infinite series of functions

• Infinite series of functions
  • Sufficient condition for continuity: If S= f_1 + f_2 + f_3 + ... converges uniformly over the interval (a,b), and if each of the summands is continuous at every point in (a,b), then the series S is continuous at every point in (a,b).
    • Note that if S converges but not converges uniformly, the continuity of each summand does not guarantee the continuity of S on (a,b).
    • Uniform convergence of the series is not necessary condition of the continuity of S.
  • Sufficient condition for term-by-term differentiation (interchanging differentiations and summations)
    • Let F(x)=f_1(x) +f_2(x) + f_3(x) + ... be an infinite series convergent at x=a and such that each f_k(x) is differentiable at every point in an open interval I containing a.   If \sum_{k=1}^\infty  f_k^{\prime} (x) converges uniformly over I, then F(x) is differentiable at a and F^{\prime}(a) = \sum_{k=1}^\infty  f_k^{\prime} (x).
  • Sufficient condition for term-by-term integration (interchanging integrations and summations)
    • uniform convergence of f_1(x) +f_2(x) + f_3(x) + ... 
• Upper limit of a series of numbers x_n
  • Definition: The upper limit or lim sup of a series x_n is the greatest lower bound (glb) of a set S, where S is defined as below:  a point x is in S if there exists a number N, for all n>N, we have x_n<x.   That is, lim sup x_n = glb S.
    • It does not necessarily mean lim sup > x_n for all n.  The lim sup only cares about infinite terms or asymptotic behavior.  Finite terms can be regarded as experimental errors. 
  • General definition of limsup (limit superior) of a sequence of sets A_n
    • lim sup A_n = intersection_{n=1}^{\infty} union_{m=n}^{\infty} A_m
    • In comparison with limsup for a numerical series,   S= union_{m=n}^{\infty} A_m; and  intersection_{n=1}^{\infty} S = glb S.
    • lim sup A_n = lim_{n  approaches \infty} union_{m=n}^{\infty} A_m
  • lim sup of a series always exists although a series may not converge.
• Lower limit of a series x_n
  • Definition: 
    • The lower limit or lim inf of a series x_n is the least upper bound (lub) of a set S, where S is defined as below:  a point x is in S if there exists a number N, for all n>N, we have x_n>x.   That is, lim inf x_n = lub S.
    • lim inf x_n = lim sup - x_n.
  • General definition of lim inf (limit inferior) of a sequence of sets A_n
    • lim inf A_n = union_{n=1}^{\infty} intersection_{m=n}^{\infty} A_m
    • lim inf A_n = lim_{n  approaches \infty} intersection_{m=n}^{\infty} A_m
• For a continuous function f(x) defined on [a,b], the maximum (maximum value of f(x)) and minimum (mimimum value of f(x)) always exists.   But this is not true for a bounded function defined on [a,b].  Supremum (least upper bound) and infimum (greatest lower bound) always exists for any function defined on [a,b].   They could be positive and negative infinity, respectively.  Supremum and infimum may not be achievable for a bounded function f defined on [a,b]; that is, there may not exist a point x_0 in [a,b] such that f(x_0)=sup f(x) or f(x_0)=inf f(x).
• Uniform continuity of a function f(x) over [a,b].  What is the relation between uniform convergence of a series of functions f_n(x) and the uniform continuity of f(x), to which f_n(x) converges? 
• Distinct functions have distinct Taylor expansions (power series)?   Not true!
• Nice property of power series:   
  • If a power series converges, it converges uniformly.   That is, the power series converges uniformly over its region of convergence (ROC).
  • So on the ROC of the power series, we can interchange the order of integration/differentiation and infinite summation.
  • The power series and the series of derivatives (term-by-term differentiation of the power series) have the same ROC.
  • Power series will always be continuous functions; that is, any power series converges to a continuous function on the ROC.
• Fourier series is not so nice as power series.   Fourier series may not converge uniformly in its ROC.   So interchanging integration/differentiation and infinite summation is not legitimate.
• Problems:
  • Integrable functions whose Fourier series do not converge
  • Convergent trigonometric series whose sum is not integrable
  • Trigonometric series that converge only at rational values of x or are unbounded in any open interval
  • Functions that are continuous at every point but that lack a derivative at any point
    • Weierstrass found a trigonometric series, called Weierstrass function, satisfies this property.
  • There exists a trigonometric series, which converges in an interval but is not continuous at any point in the interval.  (?)
• convergence of a series, uniform convergence of a series, continuity of the sum, uniform continuity of the sum.  What's the condition?
  • Tests for convergence of a series
    • Ratio test
    • Limit ratio test
    • Root test
    • Gauss's test
    • Abel's best
  • Tests for uniform convergences of a series
    • Weierstrass's M test
    • Dirichlet's test: uniformly bounded partial sums
  • Sufficient condition for continuity of the sum

Integration: 

A more archaic term for integration is quadrature.  Note quadrature has three different meanings in mathematics. Integration by quadrature either means solving an integral analytically (i.e., symbolically in terms of known functions), or solving of an integral numerically (e.g., Gaussian quadrature, quadrature formulas). Ueberhuber (1997, p. 71) uses the word "quadrature" to mean numerical computation of a univariate integral, and "cubature" to mean numerical computation of a multiple integral.   The word quadrature is also used to mean squaring: the construction of a square using only compass and straightedge which has the same area as a given geometric figure. If quadrature is possible for a plane figure, it is said to be quadrable.

• Cauchy integral: taking value at the left end-points of the subintervals.
• Riemann integral: 
  • Defined for a bounded function f(x) over a closed and bounded interval [a,b].
  • Ideas of Riemann integral: 
    • divide the domain of the function (usually a closed, bounded interval) into finitely many subintervals (the partition)
    • construct two simple functions that have a constant value (supremum/infimum) on each of the subintervals of the partition (approximate the original function by two simple functions)
    • construct the upper and lower sums
    • take the limit of these upper and lower sums as you add more and more points to the partition. 
      
      If the limit exists it is called the Riemann integral and the function is called Riemann integrable.
  • Two definitions: 
    • using the lower and upper integrals to define Riemann integral. (the above one)
    • taking value at an arbitrary point on each subinterval.
  • A function is Riemann-integrable if and only if the lower and upper Riemann integrals are equal.
  • A bounded function f defined on [a,b] is Riemann integrable if and only if it is continuous almost everywhere.  (i.e., the sets of points of  discontinuity of the function is to be of measure zero.)
  • Cauchy integral and Riemann integral are equivalent.
  • What is the space of Riemann integrable functions R[a,b]?
    • If f is piecewise continuous, i.e., f only has a finite number of discontinuities, then f is Riemann integrable.  (The objective of defining Cauchy Integral is to deal with piecewise continuous functions.) 
    • If f is continuous on [a, b] except possibly at countably many points (continuous almost everywhere), then f is Riemann integrable.  (The objective of defining Riemann Integral is to deal with continuous-almost-everywhere functions.) 
    • R[a,b] is not a complete space w.r.t. the metric defined as \int_a^b |f(x)-g(x)| dx.
    • If a function is Riemann integrable over [a,b], the set of the points, at which the function is continuous, is dense w.r.t. the interval [a,b].
  • Using limit-taking technique, Riemann integral can be extended to unbounded functions (with singular points in a finite interval) and unbounded intervals.  
    • (Singular point) For unbounded functions with a singular point x_0 in a finite interval, first compute the Riemann intergral over [x_0+\epsilon, b] and then take the limit as \epsilon goes to 0.
    • Improper Riemann integral:  For unbounded intervals (-infinity, +infinity), first compute the Riemann intergral over [-x_1, x_1] and then take the limit as x_1 goes to infinity.
• Riemann-Stieltjes (R-S) integral
  • The Riemann-Stieltjes integral of f(x) w.r.t. a monotonically increasing function g(x), over [a,b] 
    • If the lower integral and upper integral are equal, then R-S integral can be defined and is equal to the lower/upper integral.
    • Notation: \int_a^b f(x) d g(x)
• Lebesgue integral: an integral over a set 
  • Motivation of defining measure: since Lebesgue integral is a integral over a set which is not a real number, we have to assign a real number to a set, which is called a measure.  A measure is a function that maps a set to a real number.   Specifically, we use Lebesgue measure to define Lebesgue integral.

    The Riemann integral is defined on interval in R.  It can certainly be defined for functions whose domain are "intervals" in n-dimensional Euclidean space. But a Riemann integral is dependent on partitions, which depend on the structure of the real line. Therefore, you can not define a Riemann integrable for functions defined on more abstract sets (even the set of natural numbers).         

    One of the limitations of the Riemann integral is that it is based on the concept of an "interval", or rather on the length of subintervals
    [x_{j-1}, x_j]. We therefore need to find a generalization of the "length" concept of a set in the real line. That new "length" concept, which we will call "measure", should satisfy two key conditions: 
    
    1.The new "measure" concept should be applicable to intervals, unions of intervals, and to more general sets (such as a Cantor set). Ideally, it should be defined for all sets. 

    2.The new "measure" concept should share as many properties as possible with the standard length of an interval, such as: 
    the 'measure' of a set should be non-negative 
    the 'measure' of an interval should be the length of that interval 
    the 'measure' of countably many disjoint sets should be the sum of the 'measures' of the individual sets 
    
    To define what will eventually be called Lebesgue Measure, we follow a two-stage strategy: 
    
    Stage One: 
    We will define a concept extending length that is defined for all sets (to satisfy condition 1 above) 

    Stage Two: 
    We will modify that concept so that it looks as close as possible to the standard length concept (to satisfy condition 2 above) 
    
    The stage-one concept is called outer measure, defined as follows: 
    
    If A is any subset of R, define the (Lebesgue) outer measure of A as: 
    
    m*(A) = inf { l(A_n) } 
    
    where the infimum is taken over all countable collections of open intervals A_n such that A \in union of A_n and l(An) is the standard length of the interval An. 

    Outer measure is defined for all sets in R and has some of the properties we wanted our concept of measure to have.   It has subadditivity property but does not have additive property for union of disjoint sets.

  • Integral of f(A) w.r.t. a measure m(A) over a set \omega.  
  • Ideas of Lebesgue integral:
    • divide the range of the function into finitely many pieces (sub-ranges) and select a value from each of the sub-ranges.
    • construct a simple function whose values are the selected finitely many numbers 
    • construct a sum (of product of each values of this simple function and the Lebesgue measure of the corresponding set) 
    • take the limit of the sums as you add more and more points in the range of the original function   

    If the limit exists it is called the Lebesgue integral and the function is called Lebesgue integrable.

• Lebesgue-Stieltjes integral 
  • Similar to Riemann-Stieltjes integral, Stieltjes extended Lebesgue integral to Lebesgue-Stieltjes integral by introducing a monotonically increasing function g(A), which maps a set A to another set B.
• Riemann integral vs. Lebesgue integral
  • A function f is L-integrable on [a,b] if and only if |f| is L-integrable on [a,b].
  • There is a function f that is R-integrable but |f| is not R-integrable
  • Riemann integral for bounded functions only, Lebesgue for bounded and unbounded functions.  For an unbounded function to be Lebesgue-integrable, the set of the points where a function are unbounded must have measure zero. (?) 
  • Riemann integral can be extended to improper Riemann integrals, but can not allow functions that are extended real valued. 
  • An improper R-integral may exist without the function being L-integrable (see f(x) = sin(x) / x for x > 0. 
  • If f is L-integrable and the improper R-integral exists, then both agree. 
  • Mention sequences of R-integrable function vs. L-integrable 
  • Lebesgue's Theorem: a bounded function f on [a, b] is Riemann integrable if and only if the set of points where f is not continuous has
    measure zero. 
• First Fundamental Theorem of Calculus (Integral Evaluation Shortcut):
  • To evaluate an integral over an interval [a, b], find an antiderivative (primitive) F of the integrand f and compute F(b) - F(a). 
• A repeated integral is an integral taken multiple times over a single variable (as distinguished from a multiple integral, which consists of a
  number of integrals taken with respect to different variables).  A repeated integral consists of multiple independent integrations; a multiple integral is only one integral over a high dimensional space, characterized by several variables.
• Differentiability implies continuity; continuity implies L-integrability; L-integrability implies L-measurability.

• Real-valued function: maps R into R.
• Real-valued set function: maps a collection of sets into R
  • Lebesgue integral maps a collection of measurable sets on [a,b] into R.   Lebesgue integral is a continuous function.
• Set-valued function: maps a collection of sets into a set.
• Operator: maps a set of functions into a set of functions; maps a function space into another function space.  If the range is on the real line or in the complex plane, the mapping is usually called a functional instead. 
  • A functional is a real-valued or complex-valued function on a vector space V, usually of functions.  It maps a vector space into R or C.               
• A pattern is symmetric if there is at least one symmetry (rotation, translation, reflection, glide reflection) that leaves the pattern unchanged. There are four types of symmetry in the plane:
  • Rotation: To rotate an object means to turn it around. Every rotation has a center and an angle. 
  • Reflection: To reflect an object means to produce its mirror image. Every reflection has a mirror line. 
  • Translation:  To translate an object means to move it without rotating or reflecting it. Every translation has a direction and a distance. 
  • Glide Reflection: A glide reflection combines a reflection with a translation along the direction of the mirror line. Glide reflections are the only type of symmetry that involve more than one step.
• Symmetry of a real-valued function
  • A function is odd if its graph is symmetric about the origin (180 degree rotation-symmetric), i.e., f(x)= - f(-x).
  • A function is even if its graph is symmetric about the Y-axis, i.e., f(x)= f(-x).