HEAD

PREVIOUS

---

Footnotes:

${}^1$Throughout this book, matrices and vectors in abstract multidimensional space are denoted by upright bold font. Vectors in physical three-dimensional space are instead denoted by italic bold font. ${}^2$This problem with polynomial fitting is sometimes called Runge's phenomenon. ${}^3$Sometimes called simply "orthogonal", the real version of "unitary". ${}^4$A highly abbreviated outline of the SVD is as follows. The $M\times M$ matrix ${\mathbf{S}}^T\mathbf{S}$ is symmetric. Therefore it has real eigenvalues $d_i^2$ which because of its form are non-negative. Its eigenvectors ${\mathit{v}}_i$, satisfying ${\mathbf{S}}^T\mathbf{S}{\mathit{v}}_i=d_i^2{\mathit{v}}_i$, can be arranged into an orthonormal set in order of decreasing magnitude of $d_i^2$. The $M$ eigenvectors can be considered the columns of an orthonormal matrix $\mathbf{V}$, which diagonalizes ${\mathbf{S}}^T\mathbf{S}$ so that ${\mathbf{V}}^T{\mathbf{S}}^T\mathbf{S}\mathbf{V}={\mathbf{D}}^2$ is an $M\times M$ non-negative, diagonal matrix with diagonal values $d_i^2$. Since $(\mathbf{S}\mathbf{V}{)}^T\mathbf{S}\mathbf{V}$ is diagonal, the columns of the $N\times M$ matrix $\mathbf{SV}$ are orthogonal. Its columns corresponding to $d_i=0$ are zero and are not useful to us. The useful columns corresponding to non-zero $d_i$ ($i=1,...,L$, say, $L\le M$) can be normalized by dividing by $d_i$. Then by appending $N-L$ normalized column $N$-vectors that are orthogonal to all the previous ones, we can construct a complete $N\times N$ orthonormal matrix $\mathbf{U}=[\mathbf{S}\mathbf{V}{\mathbf{D}}_L^{-1},{\mathbf{U}}_{N-L}]$. Here ${\mathbf{D}}_L^{-1}$ denotes the $M\times L$ diagonal matrix with elements $1/d_i$ and ${\mathbf{U}}_{N-L}$ denotes the appended extra columns. Now consider $\mathbf{U}\mathbf{D}{\mathbf{V}}^T$. The appended ${\mathbf{U}}_{N-L}$ make zero contribution to this product because the lower rows of $\mathbf{D}$ which they multiply are always zero. The rest of the product is $\mathbf{S}\mathbf{V}{\mathbf{D}}_L^{-1}{\mathbf{D}}_L{\mathbf{V}}^T=\mathbf{S}$. Therefore we have constructed the singular value decomposition $\mathbf{S}=\mathbf{U}\mathbf{D}{\mathbf{V}}^T$. ${}^5$If $M>N$, the combination $\mathbf{D}{\mathbf{D}}^{-1}$, which arises from forming $\mathbf{S}{\mathbf{S}}^{-1}$ is not an $M\times M$ identity matrix. Instead it has ones only at most for the first $N$ of the diagonal positions, and zeros thereafter. It is an $N\times N$ identity matrix with extra zero rows and columns padding it out to $M\times M$. So the pseudo inverse is a funny kind of inverse which works only one way. If $\mathbf{S}$ is square and nonsingular, then the pseudo inverse is exactly the same as the (normal) inverse. If $\mathbf{S}$ were a singular square matrix, for example (and possibly in other situations) then at least one of the original $d_j$ would be zero. We usually consider the singular values ($d_j$) to be arranged in descending order of size; so that the zero values come at the end. ${\mathbf{D}}^{-1}$ would then have an element $1/d_j$ that is infinite, and the formal manipulations would be unjustified. What the pseudo-inverse does in these tricky cases is put the value of the inverse $1/d_j$ equal to zero instead of infinity. In that case, once again, an incomplete identity matrix is produced, with extra diagonal zeros at the end. And it actually doesn't completely "work" as an inverse in either direction. For those who know some linear algebra, what's happening is that the pseudo-inverse projects vectors in the range of the original matrix back into the complement of its nullspace. ${}^6$See for example, Numerical Recipes, W H Press, B P Flannery, S A Teukolsky, and W T Vettering, Cambridge University Press (first edition, 1989) (henceforth cited simply as Numerical Recipes) Section 2.9. ${}^7$by QR decomposition ${}^8$This notation means the first $N$ rows of the compound matrix consist of $\mathbf{S}$, and the next $N_R$ rows are $\mathit{\lambda }\mathbf{R}$. ${}^9$This regularization is equivalent to what is sometimes called a "smoothing spline". In the limit of large smoothing parameter $\mathit{\lambda }$, the function $f$ is a straight line (zero second derivative everywhere). In the limit of small $\mathit{\lambda }$, it is a cubic spline interpolation through all the values $(x_i,y_i)$. ${}^{10}$This topic and many others in data fitting is addressed for example in Siegmund Brandt (2014) Data Analysis Statistical and Computational Methods for Scientists and Engineers, Fourth Ed., Springer, New York. ${}^{11}$Of course Fourier analysis is usually approached differently, as briefly discussed in the final chapter. We are here just using sine as an example function, in part because it is familiar. ${}^{12}$There are lots of other correct but more verbose ways of doing it. ${}^{13}$If $f$ is independent of $y$, then we should use $f(x_{n+1/2})$ where $x_{n+1/2}=(x_{n+1}+x_n)/2$, and we then have a formula accurate to second-order for performing simple integration ${\int }_0^{x_n}f(x)\mathit{dx}=y_n-y_0=\sum _{k=0}^{n-1}f(x_{k+1/2})(x_{k+1}-x_k)$. There are fancier, higher-order, ways to do integration, but this is by far the most straightforward. ${}^{14}$The notation $O({\mathit{\epsilon }}^n)$ denotes additional terms whose magnitude is proportional to a (usually small) parameter $\mathit{\epsilon }$ raised to the power $n$ or higher. ${}^{15}$The proof is rather hard work. Here's an outline. Use notation $F(\mathit{\delta }x)=f(y(x_n+\mathit{\delta }x),\mathit{\delta }x)$ to refer to values of the derivative function along the exact orbit. Suppose we compose $\mathit{\Delta }y=(\frac{1}{6}F(0)+\frac{1}{3}F(\frac{\mathit{\Delta }}{2})+\frac{1}{3}F(\frac{\mathit{\Delta }}{2})+\frac{1}{6}F(\mathit{\Delta }))\mathit{\Delta }$. This is the form corresponding to eqs. (2.16) and (2.17), but using the exact $f$ on the orbit, rather than the intermediate $f^{(n)}$ estimates. By substituting from eq. (2.11) it is easy but tedious to show that this $\mathit{\Delta }y$ is equal to $y(x_n+\mathit{\Delta })-y_n+O({\mathit{\Delta }}^5)$; that is, it is an accurate representation of the $y$-step to fourth-order. Actually it is instructive to realize that the symmetry of the $\mathit{\delta }x$ positions, $0$, $\frac{\mathit{\Delta }}{2}$, and $\mathit{\Delta }$ ensures that all even order errors in this $\mathit{\Delta }y$ are zero while the first-order error is zero because the coefficients sum to unity. The factor of 2 difference between the center and the end coefficients is the choice that makes the third-order error zero. In itself this has not proved the scheme to be fourth-order accurate, because there are non-zero differences approximately proportional to $\frac{\partial f}{\partial y}$ between the $f$ values and the $F$ values: $f^{(1)}-F(\frac{\mathit{\Delta }}{2})=\frac{\partial f}{\partial y}(y^{(1)}-y(\frac{\mathit{\Delta }}{2}))$, $f^{(2)}-F(\frac{\mathit{\Delta }}{2})=\frac{\partial f}{\partial y}(y^{(2)}-y(\frac{\mathit{\Delta }}{2}))$, and $f^{(3)}-F(\mathit{\Delta })=\frac{\partial f}{\partial y}(y^{(3)}-y(\mathit{\Delta }))$. (Because of the order of the $y$-differences one can quickly see that $\frac{{\partial }^2}{\partial y^2}$ terms don't need to be kept.) The successive differences such as $(y^{(1)}-y(\frac{\mathit{\Delta }}{2}))$ can be expressed in terms of the Taylor expansion eq. (2.12), and then the $f$-differences are gathered in the combination of eq.  (2.17): $\frac{1}{6},\frac{1}{3},\frac{1}{3},\frac{1}{6}$. One can then evaluate the $\frac{\partial f}{\partial y}$ terms and demonstrate that they cancel to fourth order. ${}^{16}$A more general form of bisection in which the interval is divided in two unequal parts weighted by the value of the function, $s_n=[f(s_l)s_u-f(s_u)s_l]/[f(s_l)-f(s_u)]$ has comparable robustness, requires no additional function evaluations per step but a couple of extra multiplications, and converges quicker than plain bisection except in special cases. This is sometimes called the "false-position" root-finding method. It is generally possible to detect automatically when one of those slowly converging special cases is encountered and take additional measures. ${}^{17}$A shorthand way to remember the result for an order $N$ derivative is that the numerator is the sum of the neighboring $y$ values times the coefficients of a binomial expansion to the power $N$, and the denominator is $\mathit{\Delta }{x}^N$. ${}^{18}$An alternative approach to implementing boundary conditions is to regard the boundary positions as outside and not operated upon by the difference matrix. Then its indices runs only from $2$ to $N-1$, and it has dimensions $(N-2)\times (N-2)$. For Dirichlet conditions, the term in the difference stencils of positions $2$ and $N-1$ that contains the boundary value is moved into the right-hand-side source vector. Generically the matrix equation is then

${\displaystyle \left(\begin{array}{ccccc}\hfill -2\hfill & \hfill 1\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill \ddots \hfill & \hfill \ddots \hfill & \hfill \ddots \hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 1\hfill & \hfill -2\hfill & \hfill 1\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill \ddots \hfill & \hfill \ddots \hfill & \hfill \ddots \hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 1\hfill & \hfill -2\hfill \end{array}\right)\left(\begin{array}{c}\hfill y_2\hfill \\ \hfill :\hfill \\ \hfill y_n\hfill \\ \hfill :\hfill \\ \hfill y_{N-1}\hfill \end{array}\right)=\left(\begin{array}{c}\hfill g_2\mathit{\Delta }{x}^2-y_L\hfill \\ \hfill :\hfill \\ \hfill g_n\mathit{\Delta }{x}^2\hfill \\ \hfill :\hfill \\ \hfill g_{N-1}\mathit{\Delta }{x}^2-y_R\hfill \end{array}\right).}$

This form keeps the matrix symmetric, which is advantageous for some inversion algorithms. ${}^{19}$In some circumstances one can deliberately choose the mesh so as to put the boundary at $x_{3/2}$. Then the implementation represented by eq.  (3.15) does put the value at the correct place. This mesh choice is appropriate if the boundary condition is known to be of purely derivative form. It then lends itself to the alternative approach of the previous note, excluding the boundary from $\mathbf{D}$. The top left coefficient becomes $D_{22}=-1$, and $\mathit{\Delta }x\hspace{0.5em}\mathit{dy}/\mathit{dx}{|}_L$ is added to the source vector, to implement the boundary condition. Mixed boundary conditions are not handled so easily like this, which is why a second-order accurate form with the boundary at $x_1$ has been given. ${}^{20}$The error in the first derivative $y\text{'}$ is approximately $y"{}_2.\frac{1}{2}\mathit{\Delta }x$, first-order in $\mathit{\Delta }x$, but the error in $\mathit{\Delta }y$ is second-order in $\mathit{\Delta }x$ giving first order accuracy. ${}^{21}$With periodic boundary conditions the homogeneous equation ($\frac{d^{2}y}{{\mathit{dx}}^2}=0$) is satisfied by any constant $y$. An additional condition must therefore be applied to make the solution unique. Moreover, there exists no solution of the differential equation with continuous derivative unless $\int g\mathit{dx}=0$. These requirements are reflected in the fact that the matrix of this system is singular. If (3.19) is solved by pseudo-inverse, it gives the solution having zero mean: $\sum y_n/N=0$, using as right-hand-side instead of $\mathit{\Delta }{x}^2\mathbf{g}$ the quantity $\mathit{\Delta }{x}^2(\mathbf{g}-\sum g_n/N)$. ${}^{22}$On a structured mesh, a finite volume method is identical to the finite difference method provided this conservative differencing is used. ${}^{23}$The subscript $q$ on ${\mathit{\rho }}_q$ reminds us this is charge density, not mass density $\mathit{\rho }$ here. ${}^{24}$Vector dependent variable problems are hyperbolic if the matrix of the coefficients of their differentials is diagonizable with real eigenvalues, as we'll see later. ${}^{25}$Constructed with the distmesh routines from http://persson.berkeley.edu/distmesh/ ${}^{26}$For object-oriented purists, this could be a "method". ${}^{27}$In effect, we linearize about an exact solution of the equations and the $\mathit{\psi }$ we then analyse is the (presumed small) difference between the solution we obtain and the exact solution. In the text we avoid encumbrance of the notation by not drawing explicit attention to this fact. ${}^{28}$The Nyquist limit. ${}^{29}$Or, for example, an LU decomposition. ${}^{30}$See for example Numerical Recipes section 2.6 ${}^{31}$Octave and Matlab have a simple built-in function to do reordering, called reshape(). ${}^{32}$and of LU decomposition and backsubstitution ${}^{33}$We could alternatively have used integral positions and the second order scheme of eq. 3.16) ${}^{34}$Removing the (first-order) time derivative changes the classification of the equation because it is no longer a differential equation in $t$ at all. ${}^{35}$Jacobi's method actually can be used to solve a rather general matrix equation $\mathbf{B}\mathit{\psi }=\mathbf{s}$, and is implemented as ${\mathit{\psi }}^{(n+1)}={\mathbf{D}}^{-1}(\mathbf{s}-\mathbf{R}{\mathit{\psi }}^{(n)})$, where $\mathbf{D}$ is the diagonal part of $\mathbf{B}$ and $\mathbf{R}=\mathbf{B}-\mathbf{D}$ is the rest. Its convergence is guaranteed if $\mathbf{B}$ is diagonally dominant, meaning that the absolute value of the diagonal term exceeds the sum of the absolute values of all the other coefficients of the corresponding row. ${}^{36}$For the matrix expert, the advancing matrix is then explicitly "two-cyclic, and consistently ordered". ${}^{37}$A Fourier treatment is not rigorously justified in general, but serves to illustrate simply the most important characteristics. ${}^{38}$See the enrichment section for an outline if you are interested. G D Smith Numerical Soution of Partial Differential Equations, Oxford University Press, 1985, p275ff gives an accessible detailed treatment. ${}^{39}$If we apply the boundary conditions, then the eigenmode is actually $\sin (\mathit{jp}\mathit{\pi }/N_x)\sin (\mathit{kq}\mathit{\pi }/N_y)$ but the complex Von Neumann analysis gives the same result. ${}^{40}$In fact, if $N_x=N_y$ and $\mathit{\Delta }x=\mathit{\Delta }y$, then $M=N_x$, or alternatively if $\mathit{\Delta }y>>\mathit{\Delta }x$, and $N_x$ is not far smaller than $N_y$, then $M\approx N_x$. ${}^{41}$the difference between the "converged" iterative $\mathit{\psi }(0)$ and the analytic $\mathit{\psi }(0)$ normalized to the analytic value. ${}^{42}$If the Jacobian is position-dependent, then the modes are only approximately uncoupled, because our analysis is effectively assuming that $\mathbf{J}$ and $\partial /\partial x$ commute, which is not exact if $\mathbf{J}$ is spatially-varying. The stability analysis is then local, and only approximate. But in any case Von Neumann stability analysis is only approximate in non-uniform cases. ${}^{43}$provided the eigenvectors are linearly independent ${}^{44}$Although this can be proved, it is complicated. ${}^{45}$There is considerable subtlety in the question "how abruptly". After all any velocity change really takes a finite time and involves an acceleration. Why isn't it included in $\mathit{a}$? For present purposes we'll just finesse this question by saying that the source contains any velocity changes that are not included in the acceleration. ${}^{46}$(not multiple dependent variables that have to be arranged into a vector representation whose eigenvalue might not be real) ${}^{47}$The transport of radiation, of photons, can also be treated in this way, but their treatment has to be expressed in terms of energy (or momentum) rather than speed, since all photons travel at the same speed. ${}^{48}$Also there may be other spontaneous decays or fissions (delayed neutrons) but we'll ignore them for now. ${}^{49}$Most reactor physics texts express the velocity distribution differently, in terms of flux density $\mathit{\phi }\equiv vf$, as the dependent variable. To retain the more general relevance to solving Boltzmann's equation, we here keep $f$ as the dependent variable. The result is still equivalent to standard reactor physics equations. ${}^{50}$Although there we might have to include self-collisions. ${}^{51}$In practice delayed neutrons introduce a severe complication. ${}^{52}$Computational size reduction, retaining spatial dependence, might call for a very small number of speed groups. Perhaps even a single group $N_G=1$. Then the neutron transport is reduced to a single diffusion equation. ${}^{53}$And the boundary conditions are also homogeneous. Of course "homogeneous" does not here mean uniform in space; it means having no constant terms. ${}^{54}$Strictly speaking, the inverse of the eigenvalues of the matrix $(\mathit{\nu }{\mathit{\Sigma }}_f\mathbf{V}{)}^{-1}(-\mathbf{L}+{\mathit{\Sigma }}_t\mathbf{V}-{\mathit{\Sigma }}_s\mathbf{V})$. ${}^{55}$A generalized eigenvalue is the solution to $(\mathbf{A}-\mathit{\lambda }\mathbf{B})\mathbf{v}=0$ where both $\mathbf{A}$ and $\mathbf{B}$ are general matrices. ${}^{56}$Actually if it were the largest eigenvalue of $\mathbf{M}$, for a single group or diagonal $\mathbf{G}$ we could directly use the "power method", which is eq. 9.16 with the $F$-indices ($n$, $n+1$) exchanged. However, because we want the smallest eigenvalue $1/k$, we must effectively invert $\mathbf{M}$ (which requires interior iteration) because $\mathbf{M}$ is never diagonal. ${}^{57}$e.g.  SOR ${}^{58}$See e.g. Applied Reactor Physics Alain Hébert, Presses internationales Polytechnique, www.polymtl.ca, (2009). Actually many other choices of weighting will work as well, not just $(\mathbf{G}{\mathbf{F}}^{(n+1)}{)}^T$. ${}^{59}$The zeroes in the non-diagonal matrices arise because scatterings hardly ever move neutrons to higher energy or reduce the energy by more than one group, and fissions give rise only to energetic neutrons. The values given are roughly appropriate for a pressurized water reactor. ${}^{60}$In reactor physics $B$ is called the "Buckling". ${}^{61}$Alder and Wainwright, "Phase transition for a hard sphere system", J. Chem. Phys. 27, 1208-1209, (1957). Liquid behavior is an important phenomenon for which Molecular Dynamics is useful. ${}^{62}$For fixed $\mathit{\Delta }t$, the Verlet and leap-frog schemes are equivalent, with the identification ${\mathit{v}}_n=({\mathit{v}}_{n-1/2}+{\mathit{v}}_{n+1/2})/2$. The Verlet scheme, implemented in terms of velocity as in eq. (10.1), requires more storage or more acceleration evaluations: because two values of $\mathit{a}$ are needed. It can be implemented in terms of the position advance alone as ${\mathit{x}}_{n+1}=2{\mathit{x}}_n-{\mathit{x}}_{n-1}+\mathit{a}\mathit{\Delta }{t}^2$, which requires the same storage as leap-frog. ${}^{63}$Since the step cost for a neighboring domain of size $\propto N_l$ is $\propto N_p{N}_l^3$, and the step cost for the neighbor update is $\propto N_p^2/N_l$, a formal optimum occurs when these are equal $N_l\propto N_p^{1/4}$. So formally, the per-step cost of this $N_l$-optimized algorithm would be $\propto N_p{N}_p^{3/4}$: slightly better than $N_p^2$. ${}^{64}$See for example C K Birdsall and A B Langdon (1991) Plasma Physics via Computer Simulation, IOP Publishing, Bristol; or R W Hockney and J W Eastwood (1988) Computer Simulation using Particles, Taylor and Francis, New York ${}^{65}$See for example the SOR estimates. ${}^{66}$Siegmund Brandt (2014) Data Analysis Statistical and Computational Methods for Scientists and Engineers, Fourth Ed., Springer, New York, gives a much more expansive introduction to statistics and Monte Carlo techniques. ${}^{67}$Statisticians use the generic word "sample" to refer to the particular result of a single test or measurement. ${}^{68}$Division by the factor $N-1$ rather than $N$ makes this formula an unbiassed estimate of the distribution variance. One way to understand this is to recognize that the number of degrees of freedom of $\sum _1^N(v_i-{\mathit{\mu }}_N{)}^2$ is $N-1$, not $N$. Using $N-1$ is sometimes called "Bessel's correction". ${}^{69}$Often called the "expectation" of $v$ ${}^{70}$Central Limit Theorem. It is not straightforward to prove that the distribution becomes Gaussian. But it is fairly easy to show that the variance of the sample mean is the variance of the distribution divided by $N$. From the definition of ${\mathit{\mu }}_N$ one can immediately deduce that

${\displaystyle ({\mathit{\mu }}_N-\mathit{\mu }{)}^2={(\frac{1}{N}\sum _1^N(v_i-\mathit{\mu }))}^2=\frac{1}{N^2}\sum _{i,j}^N(v_i-\mathit{\mu })(v_j-\mathit{\mu }).}$

Take the expectation $⟨\dots ⟩$ of this quantity to obtain the variance of the distribution of sample means:

${\displaystyle ⟨({\mathit{\mu }}_N-\mathit{\mu }{)}^2⟩=\frac{1}{N^2}\sum _{i,j}^N⟨(v_i-\mathit{\mu })(v_j-\mathit{\mu })⟩=\frac{1}{N^2}\sum _i^N⟨(v_i-\mathit{\mu }{)}^2⟩=\frac{S^2}{N}.}$

The first equality, taking the expectation inside the sum, is a simple property of taking the expectation: the expectation of a sum is the sum of the expectations. The second equality uses the fact that $⟨(v_i-\mathit{\mu })(v_j-\mathit{\mu })⟩=0$ for $i\ne j$ because the quantities $(v_i-\mathit{\mu })$ are statistically independent and have zero mean. That is sufficient to yield the required result. Our estimate for the distribution variance is $S^2=S_N^2$. So the unbiassed estimate for the variance of ${\mathit{\mu }}_N$ is $⟨({\mathit{\mu }}_N-\mathit{\mu }{)}^2⟩=S_N^2/N$. The standard error is the square root of this quantity. ${}^{71}$An un-normalized Gaussian distribution has three, including the height. ${}^{72}$Linear interpolation is then equivalent to representing $p_v$ as a histogram. So adequate resolution may require a fairly large number $N_t$. ${}^{73}$Quiet start and quasi-random selection. When starting a particle-in-cell simulation, the initial positions of the particles might be chosen using random numbers to decide their location. However, they then will have density fluctuations of various wavelengths that in plasmas may be bigger than are present after running the simulation for many steps. The reason for this discrepancy is that the feedback effect of the self-consistent electric potential tends to smooth out density fluctuations, so that in the fully developed simulation the noise level is lower than purely random. A simple way of saying the same thing is that individual particles repel others of the same type, preventing clumping of the particles. It is therefore often physically reasonable to start a PIC simulation with positions that are chosen to be more evenly spaced than purely random. Indeed, for some calculations it is advantageous (but non-physical) to start with density fluctuations even lower than the final level that would be present for a steady plasma. In either case, what is called a "Quiet Start" can be obtained by using what are called "quasi-random" numbers [see e.g. Numerical Recipes] instead of (pseudo) random numbers. Quasi-random numbers are somewhat random, but much smoother in their distribution because each new number takes account of the already used numbers and tries to avoid being close to them. Successive numbers are thus correlated rather than uncorrelated. For Monte-Carlo integration, such smoother distributions in space are also often highly appropriate and can give lower-noise results for the same number of samples, beating the weak, $1/\sqrt{N}$, decrease of fractional error. ${}^{74}$Or if we wish to do "quiet injection" that is smoother than purely random ${}^{75}$The Discrete Poisson distribution. Suppose there is a very large number $N$ of similar, uncorrelated, events (for example radioactive decays of atoms) waiting to occur. In a time duration far shorter than the waiting time (e.g. the half-life) of an individual event the probability that any one of them occurs is small, say $p=r/N$. Then $r$ is the average total number of events occuring in this time duration. The total number of events actually occuring in a particular time sample is then an integer, and we want to find the probability of each possible integer. Any sample consists of $N$ choices about the individual events: yes or no. The number of yes events is distributed as a Binomial distribution, in which the probability of $n$ yes events is given by the number of different ways to choose $n$ out of $N$, which is $\frac{N!}{n!(N-n)!}$, times the probability of each specific arrangement of $n$ yes events and $N-n$ no events, $p^n(1-p{)}^{N-n}$. The total is

${\displaystyle p_n=\frac{N!}{n!(N-n)!}{p}^n(1-p{)}^{N-n}=\frac{r^n}{n!}{(1-\frac{r}{N})}^N\left[\frac{1}{N^n(1-r/N{)}^n}\frac{N!}{(N-n)!}\right].}$

Now recognize that the limit for large $N$ (but constant $n$) of the square bracket term is 1; while the limit of the term ${(1-\frac{r}{N})}^N$ is $\exp (-r)$. Therefore the probability of obtaining $n$ total events, of a type that are completely uncorrelated ($N\to \mathit{\infty }$), when their average rate of occurrence is $r$ is

${\displaystyle p_n=\frac{r^n}{n!}\exp (-r).}$

This is the discrete Poisson distribution. ${}^{76}$Or double-precision if $N$ is very large. ${}^{77}$The random generators I used here are both portable. The good generator is the RANLUX routine described by M. Luscher, Computer Physics Communications 79 (1994) 100, and F. James, Computer Physics Communications 79 (1994) 111, used with the lowest luxury level. The bad generator is the RAN1 routine from the first (FORTRAN) edition of Numerical Recipes (1989). It was replaced by better routines in later editions. ${}^{78}$For photons, the "velocity" should be interpreted as a combination of energy (frequency or wavelength) and propagation direction, since the particle's speed is always the speed of light. ${}^{79}$The velocity distribution $f_k$ in discrete velocity bins $d^3{v}_k$ may also be desired. It is given by summing only those passages that occur in the velocity bin: $f_k{d}^3{v}_k=\sum _{v_i\in d^3{v}_k}\mathit{\Delta }{t}_i$. It will have greater statistical uncertainty because of fewer samples per bin. ${}^{80}$One such book with lots of engineering examples is Steven C Chapra and Raymond P Canale (2006) (5th Ed, or later) "Numerical Methods for Engineers" McGraw-Hill, NY. ${}^{81}$The volume of a tetrahedron can be written as a $4\times 4$ determinant using the cartesian coordinates $(x_k,y_k,z_k)$ of its four corners ($k=1,4$) as

${\displaystyle V=\frac{1}{6}\left|\begin{array}{cccc}\hfill 1\hfill & \hfill x_1\hfill & \hfill y_1\hfill & \hfill z_1\hfill \\ \hfill 1\hfill & \hfill x_2\hfill & \hfill y_2\hfill & \hfill z_2\hfill \\ \hfill 1\hfill & \hfill x_3\hfill & \hfill y_3\hfill & \hfill z_3\hfill \\ \hfill 1\hfill & \hfill x_4\hfill & \hfill y_4\hfill & \hfill z_4\hfill \end{array}\right|}$

${}^{82}$These important details are discussed in texts devoted to finite elements, such as, Thomas J R Hughes (1987) "The Finite Element Method" Prentice Hall, Englewood Cliffs, NJ. The classic text on the background mathematical theory is, G Strang and G J Fix (1973, 2008) "An Analysis of the Finite Element Method", Reissued by Wellesley-Cambridge, Press, USA. ${}^{83}$Since $f$ is periodic, its value at $j=0$ is the same as at $j=N$, so the range is equivalent to $1,...,N$. ${}^{84}$If $N$ is odd, then the highest frequency is obviously $(N-1)/2$, but we'll avoid reiterating this alternative. ${}^{85}$$\mathit{\epsilon }$ is a small adjustment of the end point to avoid it falling on a delta function. ${}^{86}$An extensive discussion is given in W H Press et al, "Numerical Recipes", chapter 12. Various open source implementations are available. ${}^{87}$A non-zero ${\mathbf{x}}_0$ can be removed by reexpressing the equation in terms of $\mathbf{x}\text{'}=\mathbf{x}-{\mathbf{x}}_0$. ${}^{88}$The present derivation draws on C P Jackson and P C Robinson (1985) "A numerical study of various algorithms related to the preconditioned conjugate gradient method", International Journal for Numerical Methods in Engineering, Vol 21, pp 1315-1338, and G Markham (1990) "Conjugate Gradient Type Methods for Indefinite, Asymmetric, and Complex Systems" IMA Journal of Numerical Analysis, Vol 10, pp 155-170. ${}^{89}$This is a type of Gramm-Schmidt basis-set construction. ${}^{90}$Succinct descriptions of a wide range of iterative algorithms are given by R. Barrett, M. Berry, T. F. Chan, J. Demmel, J. Donato, J. Dongarra, V. Eijkhout, R. Pozo, C. Romine, H. Van der Vorst (1994) "Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods" 2nd Edition, SIAM, Philadelphia, which is available at http://www.netlib.org/linalg/html_templates/report.html. Open source libraries of implementations of Krylov schemes are widely available. One of the more comprehensive is "SLATEC" at http://www.netlib.org/slatec/. ${}^{91}$ Bi-conjugate gradient iterations are implemented using two residual and two search direction vectors requiring just one multiplication by $\mathbf{A}$ and one by ${\mathbf{A}}^T$ per step in this way:

1 Initialize:	choose ${\mathbf{x}}_0$, ${\mathbf{r}}_0=\mathbf{b}-\mathbf{A}{\mathbf{x}}_0$, ${\mathbf{p}}_0={\mathbf{r}}_0$, choose ${\bar{\mathbf{r}}}_0$, ${\bar{\mathbf{p}}}_0={\bar{\mathbf{r}}}_0$, set $k=1$.
2 Calculate $\mathit{\alpha }$:	${\mathit{\alpha }}_{k-1}={\mathbf{r}}_{k-1}^T{\bar{\mathbf{r}}}_{k-1}/{\mathbf{p}}_{k-1}^T\mathbf{A}{\bar{\mathbf{p}}}_{k-1}$
3 Update $\mathbf{r}$s, $\mathbf{x}$:	${\mathbf{r}}_k={\mathbf{r}}_{k-1}-{\mathit{\alpha }}_{k-1}\mathbf{A}{\mathbf{p}}_{k-1}$,    ${\bar{\mathbf{r}}}_k={\bar{\mathbf{r}}}_{k-1}-{\mathit{\alpha }}_{k-1}{\mathbf{A}}^T{\bar{\mathbf{p}}}_{k-1}$,
	${\mathbf{x}}_k={\mathbf{x}}_{k-1}-{\mathit{\alpha }}_{k-1}\mathbf{A}{\mathbf{p}}_{k-1}.$
4 Calculate $\mathit{\beta }$:	${\mathit{\beta }}_{k,k-1}={\bar{\mathbf{r}}}_k^T{\mathbf{r}}_k/{\bar{\mathbf{r}}}_{k-1}^T{\mathbf{r}}_{k-1}.$
5 Update $\mathbf{p}$s:	${\mathbf{p}}_k={\mathbf{r}}_k-{\mathit{\beta }}_{k,k-1}{\mathbf{p}}_{k-1}$    and     ${\bar{\mathbf{p}}}_k={\bar{\mathbf{r}}}_k-{\mathit{\beta }}_{k,k-1}{\bar{\mathbf{p}}}_{k-1}.$
6 Convergence?	If not converged, increment $k$ and repeat from 2.

The conjugate gradient scheme is the same except no barred quantities are calculated and unbarred quantities replace them. ${}^{92}$A compact mathematical description is given, for example, by Stephen Jardin (2010) "Computational Methods for Plasma Physics" CRC press, Taylor and Francis, Boca Raton. Section 3.7.2. ${}^{93}$The number of iterations required to converge to a given accuracy for separated eigenvalues is approximately proportional to the square root of the matrix condition number. The condition number is the ratio of the maximum to the minimum eigenvalue. That is unity for the unit matrix because all eigenvalues are 1. For a matrix representing an isotropic finite-difference Laplacian, the eigenvalues are the squared wave numbers of the spatial Fourier modes. The condition number is therefore the sum of the squares of the number of mesh points in each dimension. The number of iterations required is therefore proportional to the mesh count of the largest dimension. That is the same as optimized SOR. Finding the optimum relaxation parameter for SOR is relatively easy for a rectangular mesh on a cubical domain. Consequently un-preconditioned Krylov methods for finite-difference equations in simple domains are generally not much faster than SOR. Krylov methods come into their own when the matrices are more complicated and optimizing SOR is impossible, for example for finite element methods on unstructured grids. ${}^{94}$There are some problems in liquids where sound waves are important; just a minority. ${}^{95}$This equation arises from substituting $\mathit{\rho }=\mathit{constant}$ into the (sourceless) continuity equation, so $0=\frac{\partial \mathit{\rho }}{\partial t}=-\mathit{\nabla }.(\mathit{\rho }\mathit{v})$. If $\nabla \mathit{\rho }$ is non-zero, so $\mathit{\nabla }.(\mathit{\rho }\mathit{v})$ is non-zero even though $\mathit{\nabla }.\mathit{v}=0$, then the pressure equation acquires an additional term. ${}^{96}$See, e.g. J H Ferziger and M Perić (2002) "Computational Methods for Fluid Dynamics" third edition, Springer, Berlin. ${}^{97}$See, e.g. S Jardin (2010) op cit. ${}^{98}$A comprehensive introduction to limiter methods is given in Randall J Leveque (2002) "Finite Volume Methods for Hyperbolic Problems", Cambridge University Press. ${}^{99}$A large jet aircraft may reach ${10}^9$. ${}^{100}$See for example U. Piomelli (1999) "Large-eddy simulation: achievements and challenges" Progress in Aerospace Sciences, Vol 35, pp 335-362 ${}^{101}$or the product of two different fluctuating quantities ${}^{102}$See for example B E Launder, G J Reece, and W Rodi (1975) "Progress in development of a Reynolds-stress turbulence closure", Journal of Fluid Mechanics, Vol 68, pp 537-566. ${}^{103}$Of course there are deeper ways to think about vector spaces, but we here use the simplest approach. ${}^{104}$So a column vector can be considered a $J\times 1$ matrix and a row vector a $1\times J$ matrix. ${}^{105}$Consider recursively expanding the determinant and subsequent cofactors, always choosing not to expand by the rows that are identical. Eventually one ends up with all $2\times 2$ cofactors composed of the identical rows. They are all zero. ${}^{106}$A left inverse ${\mathbf{A}}^L$ such that ${\mathbf{A}}^L\mathbf{A}=\mathbf{I}$ must always be equal to a right inverse ${\mathbf{A}}^R$ such that $\mathbf{A}{\mathbf{A}}^R=\mathbf{I}$ because ${\mathbf{A}}^L={\mathbf{A}}^L\mathbf{I}={\mathbf{A}}^L(\mathbf{A}{\mathbf{A}}^R)=({\mathbf{A}}^L\mathbf{A}){\mathbf{A}}^R=\mathbf{I}{\mathbf{A}}^R={\mathbf{A}}^R$.

---

HEAD

---

File translated from T_EX by T_TM, version 4.12.