March 19, 2012 --- Class 19 --- Simpson's Rule Rederived, Hit or Miss Method,
Sample Mean Method
We indicated how Simpson's rule can simply
be derived from removing the leading 1/N^2 error in the trapeziod
rule with N and 2N points. This technique can be extended in
the method known as Romberg integration. The previous two points are
expanded upon in Numerical Recipes. Practical codes may be found
there.
Simpson's Rule Rederived
The leading error in the trapeziod rule is 1/N^2. We can use an
idea similar to what we did to remove sample size bias, but taking
into account the fact that dependence of the error on N is different.
Starting from the trapezoid rule, we used the formulae below
for correcting the leading 1/N^2 errors.
S_N = A + E/N^2
S_2N = A + E/(2N)^2
so A = 4/3 S_2N -1/3 S_N
Applying this to a one-dimensional integral, we will easily find that
the weighting of the sum of two expressions on the right hand side
exactly coincides with the Simpson's rule formula. We previously
found that Simpson's Rule has a leading error of order 1/N^4. Since
the approach above removes the leading 1/N^2 error and since the
error of the trapezoid rule only has even powers of 1/N (asserted, but
not proved), this is not surprising.
Higher Dimensional Integrals
Monte Carlo methods like hit or miss and the sample mean methods
have errors that fall like n^{-1/2}. This is not very good. To
cut the error in half you need to do 4 times a much work. For
Simpson's rule the error goes like n^{-4}, so if you do twice a
much work, the error falls by a factor of 16. However, for
multidimensional integrals, a method with an error like h^a,
where h is the spacing between grid points, will have an error like
n^{-a/d} because the number of points on the grid goes like h^{-d}.
As an example, if a=4 and d=10, a/d=0.4 so the error is falling
more slowly than n^{-1/2}. So, for high dimension integrals,
Monte Carlo methods may be bad, but other methods are worse.
Hit-or-Miss and Sample Mean Methods
The Hit-or-Miss method and Sample Mean methods are described in CSM
Sec. 11.2. I introduced the hit-or-miss method first. We also looked
briefly at the sample mean method.
Testing Hit-or-Miss Method
Program ~sg/src/misc/pi3.c implements the hit-or-miss method using a
feedback shift register random number generator. Compile it with
this command:
cc -o pi3 pi3.c newfsr.c
You have to tell the program how many "pebbles you want to throw."
This is the first command line argument. You may also have it
repeat the experiment multiple times. This is the optional second
argument. Look at the code to see how this is done if you are
unfamiliar with this type of feature. Also note how the random
number generator gets its seed through a call to pid(). This is
the "process id" of the calling routine. It is essentially a
random number. If you like, you can modify the code so that you
specify the seed yourself. This can be very useful when you are
trying to debug the code and want to see if results are
reproducible. Sometimes I set up my code so that it asks for a
seed, but if the seed I give is negative, it calls pid() to get a
random seed. That way, I can do many runs without having to think
up new seeds, and when I want to reproduce runs with a known seed I
can also do that.
I ran the pi3 code a few times with this command
pi3 10
The output ranged from 2.4 to 3.6. Clearly the output is a random
variable. I used the second argument to repeat the experiment
multiple time and made historgrams of the result. With larger sample
sizes, the histograms were narrower.
We have to answer several questions.
Do we get the right answer on average?
Can we estimate the error?
How does the error depend on n, the number of trials?
We expect the error to fall like 1/sqrt(n), but can we show that?
I asked everyone to run the program to see of the error falls like
1/sqrt(n). I also showed a simple shell script and introduced a
program aver that is very useful for data analysis. We will explore
its use in the next class.