## Practical Numerical Methods with C#

### About this Book

This book provides an in-depth exposition of the various numerical methods used in real-world scientific and engineering computations. It emphasizes the practical aspects of C# numerical methods and mathematical functions programming, and discusses in detail various techniques that will enable you to implement these numerical methods in your own .NET applications.

Ideal for scientists, engineers, and students who would like to become more adept with numerical methods,
**Practical Numerical Methods with C#** familiarizes you with:

- The basics of C# programming.
- The mathematical background and fundamentals of numerical methods.
- Math libraries for complex numbers and functions, real and complex vector and matrix operations, and special functions.
- Numerical methods for generating random numbers and random distribution functions.
- Various numerical methods for solving linear and nonlinear equations.
- Numerical differentiation and integration.
- Interpolations and curve fitting.
- Optimization of single-variable and multi-variable functions using a variety of techniques, including advanced simulated annealing and evolutionary algorithms.
- Numerical techniques for solving ordinary differential equations.
- Numerical methods for solving boundary value problems.
- Eigenvalue problems.

In addition, this book provides code examples for every math function and numerical method to show you how to use these functions and methods in your own .NET applications.

### Table of Contents

**Introduction**
*
Overview
What This Book Includes
Is This Book for You
What Do You Need to Use This Book
How The Book Is Organized
Using Code Examples
Customer Support
*

**Chapter 1: Overview of C# Programming**

Your First C# program

Your First C# program

Data Types

Data Types

Value-Type Variables

Reference-Type Variables

Math Operators and Functions

Math Operators and Functions

Math Operators

Logical Operators

Built-In Math Functions

Math Operation Example

Selection Statements

Selection Statements

If Statement

Switch Statement

Loop Statements

Loop Statements

For Loop

Foreach Loop

While Loop

Do Loop

Methods

Methods

Creating a Method

Calling a Method

Properties

Properties

Field Members

Exposing a Property

Using Properties

Read-Only and Write-Only Properties

Auto-Implemented properties

**Chapter 2: Complex Numbers and Functions**

Complex Numbers and Operators

Complex Numbers and Operators

Complex Numbers in C#

Common Public Properties

Test Public Properties

Common Public Methods

Equality and Hashing

Complex Operators

Unary Operator

Addition Operator

Subtraction Operator

Multiplication Operator

Division Operator

Testing Math Operator

Complex Functions

Complex Functions

Basic Math Functions

Square Root Function

Exponential Function

Pow Function

Logarithm Function

Testing Math Functions

Trigonometric Functions

Sine Function

Cosine Function

Tangent Function

Testing Trigonometrix Functions

Inverse Trigonometric Functions

Inverse Sine Function

Inverse Cosine Function

Inverse Tangent Function

Testing Inverse Trigonometrix Functions

Hyperbolic Trigonometric Functions

Hyperbolic Sine Function

Hyperbolic Cosine Function

Hyperbolic Tangent Function

Testing Hyperbolic Trigonometrix Functions

Inverse Hyperbolic Trigonometric Functions

Inverse Hyperbolic Sine Function

Inverse Hyperbolic Cosine Function

Inverse Hyperbolic Tangent Function

Testing Hyperbolic Inverse Trigonometrix Functions

**Chapter 3: Vectors and Matrices**

Real Vector Structure

Real Vector Structure

basic Definitions

Mathematical Operators

Vector Multiplications

Scalar or Dot product

Cross Product

Triple Scalar Product

Triple Vector Product

Complex Vector Class

Complex Vector Class

Implementation

Testing the VectorC Class

Real Matrix Class

Real Matrix Class

basic Definitions

Matrix and Vector Multiplications

Vector from Matrix

Swap, Transpose, and Trace

Transformations

Matrix Multiplication

Mathematical Operators

Determinant

Inverse

Testing the Real Matrix

Complex matrix Class

Complex matrix Class

Implementation

Testing the Complex Matrix

**Chapter 4: Linear Algebraic Equations**

Gauss-Jordan Elimination

Gauss-Jordan Elimination

Algorithms

Implementation

Testing Gauss-Jordan Elimination

LU Decomposition

LU Decomposition

Algorithms

Implementation

Matrix Inverse

Testing LU Decomposition

Iteration Methods

Iteration Methods

Gauss-Jacobi Iteration

Gauss-Seidel Iteration

Testing Iteration Methods

**Chapter 5: Nonlinear Equations**

Incremental Method

Incremental Method

Implementation

Testing the Incremental Method

Fixed Point Method

Fixed Point Method

Implementation

Testing the Fixed Method

Bisection Method

Bisection Method

Implementation

Testing the Bisection Method

False Position Method

False Position Method

Implementation

Testing the False Position Method

Newton-Raphson Method

Newton-Raphson Method

Implementation

Testing the Newton-Raphson Method

Secant Method

Secant Method

Implementation

Testing the Secant Method

Newton Multiroot Method

Newton Multiroot Method

Implementation

Testing the Newton Multiroot Method

Birge-Vieta Method

Birge-Vieta Method

Implementation

Testing the Birge-Vieta Method

System of Equations

System of Equations

Algorithm

Implementation

Testing the NewtonMultiEquations Method

**Chapter 6: Special Functions**

Gamma and Beta Functions

Gamma and Beta Functions

Gamma Function

Implementation

testing the Gamma Method

Beta Function

Implementation

testing the Beta Method

Error Function

Error Function

Implementation

Testing the Erf and Erfc Methods

Sine and Cosine Integral Function

Sine and Cosine Integral Function

Implementation

Testing the Si and Ci Methods

Laguerre Function

Laguerre Function

Implementation

Testing the Laguerre Method

Hermite Polynomials

Hermite Polynomials

Implementation

Testing the Hermite Method

Chebyshev Polynomials

Chebyshev Polynomials

Implementation

Testing the Chebyshev Method

Legendre Polynomials

Legendre Polynomials

Implementation

Testing the Legendre Method

Bessel Functions

Bessel Functions

Implementation

Testing the Bessel Method

**Chapter 7: Random Numbers and Distribution Functions**

Built-in Random Number Generators

Built-in Random Number Generators

Normal Distribution

Normal Distribution

Probability Density Function

Normal Random Number Generator

Testing the Normal Distribution

Exponential Distribution

Exponential Distribution

Probability Density Function

Exponential Random Number Generator

Testing the Exponential Distribution

Chi and Chi-Square Distributions

Chi and Chi-Square Distributions

Probability Density Function

Chi and Chi-Square Random Number Generators

Testing the Chi and Chi-Square Distributions

Cauchy Distribution

Cauchy Distribution

Probability Density Function

Cauchy Random Number Generator

Testing the Cauchy Distribution

Student T Distribution

Student T Distribution

Probability Density Function

Student T Random Number Generator

Testing the Student T Distribution

Gamma Distribution

Gamma Distribution

Probability Density Function

Gamma Random Number Generator

Testing the Gamma Distribution

Beta Distribution

Beta Distribution

Probability Density Function

Beta Random Number Generator

Testing the Beta Distribution

Poisson Distribution

Poisson Distribution

Probability Density Function

Poisson Random Number Generator

Testing the Poisson Distribution

Binomial Distribution

Binomial Distribution

Probability Density Function

Binomial Random Number Generator

Testing the Binomial Distribution

**Chapter 8: Interpolation**

Linear Interpolation

Linear Interpolation

Algorithm

Implementation

Testing the Linear Interpolation

Lagrange Interpolation

Lagrange Interpolation

Algorithm

Implementation

Testing the Lagrange Interpolation

*arycentric*

B Interpolation

B Interpolation

Algorithm

Implementation

Testing the Barycentric Interpolation

Newton Divided Difference Interpolation

Newton Divided Difference Interpolation

Algorithm

Implementation

Testing the Newton Divided Difference Interpolation

Cubic Spline Interpolation

Cubic Spline Interpolation

Algorithm

Implementation

Testing the Spline Interpolation

Bilinear Interpolation

Bilinear Interpolation

Algorithm

Implementation

Testing the Bilinear Interpolation

**Chapter 9: Curve Fitting**

Least Squares Fit

Least Squares Fit

Straight Line Fit

Straight Line Fit

Implementation

Testing the Straight Line Fit

Linear Regression

Linear Regression

Implementation

Testing the Linear Regression

Polynomial Fit

Polynomial Fit

Implementation

Testing the Polynomial Fit

Weighted Linear Regression

Weighted Linear Regression

Implementation

Exponential Function Fit

Simple Moving Average

Simple Moving Average

Implementation

Testing the Simple Moving Average

Weighted Moving Average

Weighted Moving Average

Implementation

Testing the Weighted Moving Average

Exponential Moving Average

Exponential Moving Average

Implementation

Testing the Exponential Moving Average

**Chapter 10: Optimization**

Bisection Method

Bisection Method

Implementation

Testing the Bisection Method

Golden Search Method

Golden Search Method

Implementation

Testing the Golden Search Method

Newton Method

Newton Method

Implementation

Testing the Newton Method

Brent Method

Brent Method

Implementation

Testing the Brent Method

Newton Method for Multi-variable Functions

Newton Method for Multi-variable Functions

Implementation

Testing the MultiNewton Method

Simplex Method

Simplex Method

Implementation

Testing the Simplex Method

Simulated Annealing Method

Simulated Annealing Method

Implementation

Testing the Simulated Annealing Method

Differential Evolution

Differential Evolution

Algorithm

Implementation

Testing Differential Evolution

**Chapter 11: Numerical Differentiation**

Finite Difference Formulas

Finite Difference Formulas

Forward Difference Method

Forward Difference Method

Implementation

Testing the Forworward Difference Method

Backward Difference Method

Backward Difference Method

Implementation

Testing the Backworward Difference Method

Central Difference Method

Central Difference Method

Implementation

Testing the Central Difference Method

Extended Central Difference Method

Extended Central Difference Method

Implementation

Testing the Extended central Difference Method

Richardson Extrapolation

Richardson Extrapolation

Implementation

Testing the Richardson Extrapolation

Derivatives by Interpolation

Derivatives by Interpolation

Implementation

Testing the Derivatives by Interpolation

**Chapter 12: Numerical Integration**

Newtown-Cotes Formulas

Newtown-Cotes Formulas

Trapezoidal Rule

Implementation

Testing the Trapezoidal Method

Simpson's Rule

Implementation

Testing the Simpson Method

Higher Order Rules

Romberg Integration

Romberg Integration

Implementation

Testing the Romberg Method

Gaussian Integration

Gaussian Integration

Gauss-Legendre Integration

Implementation

Testing Gauss-Legendre Integration

Gauss-Laguerre Integration

Implementation

Testing Gauss-Laguerre Integration

Gauss-Hermite Integration

Implementation

Testing Gauss-hermite Integration

Gauss-Chebyshev Integration

Implementation

Testing Gauss-Chebyshev Integration

**Chapter 13: Ordinary Differential Equations**

Euler Method

Euler Method

Implementation

testing Euler's Method

Second-Order Runge-Kutta Method

Second-Order Runge-Kutta Method

Implementation

Testing the Second-Order Runge-Kutta Method

Fourth-Order Runge-Kutta Method

Fourth-Order Runge-Kutta Method

Implementation

Testing the Fourth-Order Runge-Kutta Method

Adaptive Runge-Kutta Method

Adaptive Runge-Kutta Method

Implementation

Testing the Adaptive Runge-Kutta Method

Runge-Kutta Method for Systems

Runge-Kutta Method for Systems

Implementation

Testing the MultiRungeKutta4 Method

Adaptive Runge-Kutta Method for Systems

Adaptive Runge-Kutta Method for Systems

Implementation

Testing the MultiRungeKuttaFehlberg Method

**Chapter 14: Boundary Value Problems**

Shooting Method

Shooting Method

Implementation

Testing the Shooting2 Method

Finite Difference for Linear Equation

Finite Difference for Linear Equation

Algorithm

Implementation

Testing FiniteDifferenceLinear2 Method

Finite Difference for Nonlinear Equation

Finite Difference for Nonlinear Equation

Algorithm

Implementation

Testing FiniteDifferenceNonlinear2 Method

Finite Difference for Higher-Order Equation

Finite Difference for Higher-Order Equation

Algorithm

Implementation

Testing FiniteDifferenceLinear4 Method

**Chapter 15: Eigenvalue Problems**

Jacobi Method

Jacobi Method

Algorithm

Implementation

Testing the Jacobi Method

Power Interation

Power Interation

Algorithm

Implementation

Testing the Power Method

Inverse Interation

Inverse Interation

Algorithm

Implementation

Testing the Inverse Method

Rayleigh Method

Rayleigh Method

Algorithm

Implementation

Testing the Rayleigh Method

Rayleigh-Quotient Method

Rayleigh-Quotient Method

Algorithm

Implementation

Testing the Rayleigh-Quotient Method

Matrix Tridiagonalization

Matrix Tridiagonalization

Algorithm

Implementation

Testing the Tridiagonalize Method

Eigenvalues of Symmetric Tridiagonal Matrices

Eigenvalues of Symmetric Tridiagonal Matrices

LU Decomposition of Tridiagonal Matrices

Implementation

Examples

**Index**

### Introduction

Overview

What This Book Includes

Is This Book for You

What Do You Need to Use This Book

How This Book is Organized

Using Code Examples

**Overview**

Welcome to *Practical Numerical Methods with C#*. This book is intended for scientists, engineers,
and .NET developers who want to create scientific and engineering applications using C# and the .NET
Framework. For many years, FORTRAN has been the dominant language of scientific and engineering
computation. But as Microsoft C# and .NET Framework gain popularity, you may find that they are also
suitable for technical computing. This book presents C#-based procedures that perform fundamental
mathematical and numerical computations critical to scientists and engineers.

The power of the C# programming language, combined with the simplicity of implementing Windows Forms, WPF desktop applications, and Silverlight Web applications based on Visual Studio .NET framework, makes real-world .NET program development faster and easier than ever before. Visual C# is a versatile and flexible tool that allows users with even the most elementary programming abilities to not only perform complicated computations, but also to display the calculated results in a variety of graphical representations. In this regard, C# is more powerful than FORTRAN, because it is hard to show results graphically using FORTRAN.

The main advantage of using FORTRAN in scientific and engineering computing is its rich math libraries. These libraries implement a complete collection of mathematical, statistical, and numerical algorithms, which have been evolving steadily for several decades. Each subroutine and algorithm in these libraries has undergone rigorous testing and quality assurance, providing users with more time to focus on their applications. On the other hand, the C# programming language is relatively new to the scientific and engineering community. The lack of C# math libraries prevents many researchers from using the C# programming language in their applications. In this book, I will show you that it is fairly easy to develop math libraries in C#, and that it is worth developing scientific and engineering applications using C# due to its computing power and graphical representations capability.

The aim of this book is to provide scientists and engineers with a comprehensive explanation of scientific computing using C#. Much of the work in this book is original, based on my own programming experience in developing commercial Computer Aided Design (CAD) packages, which involve intensive scientific computations and sophisticated graphical representations. With FORTRAN, developing advanced graphics and chart applications is a difficult and time-consuming task. To add even simple charts or graphs to your applications, you have to waste either effort creating a chart program, or money buying commercial graphics and chart add-on packages. Visual C# and its rich graphics features make it possible to easily implement both powerful math libraries and professional graphics using entirely managed C# codes.

Practical Numerical Methods with C# provides an in-depth introduction to performing complicated scientific computations using C# applications. In this book, I will begin with an overview of the C# and .NET Framework, and then present procedural descriptions of linear algebra, numerical solution of nonlinear and ordinary differential equations, optimization, parameter estimation, and special functions of mathematical physics. I will show you how to create useful C# mathematical and numerical libraries that you can use in real-world scientific and engineering problems. I will try my best to introduce the C# program to scientists and engineers in a simple way--simple enough for C# beginners to follow with ease. From this book, you will learn how to perform complicated scientific computations and create your own math libraries based on C# and the .NET Framework.

Practical Numerical Methods with C# is not simply a book, but a powerful C# math library. You will find that you can immediately use some of the examples in this book to solve your real-world problems, and that you can use others to give you inspiration for adding more advanced math libraries to your applications.

**What This Book Includes**

This book and its sample code listings, which are available for download at this website, provide you with:

- Complete, in-depth instruction on practical scientific computing and programming using C#. After reading this book and running the example programs contained within, you will be able to create various math and numerical libraries in your own C# applications.
- Ready-to-run example programs that allow scientists and engineers to explore the numerical methods described in the book. You can use these examples to better understand how the mathematical models and algorithms work. You can also modify the code or add new features to them to form the basis of your own programs. Some of the example code listings provided in this book are already sophisticated math libraries; these can be used directly in your own real-world applications.
- Many C# classes in the sample code listings that you will find useful in solving your real-world scientific and engineering problems. These classes include linear algebra, matrix manipulation, numerical approaches, and other useful utility classes. You can extract these classes and plug them into your own applications.

**Is This Book for You**

You do not have to be an experienced C# developer or expert to use this book. I designed this book to be useful to scientists and engineers of all levels of C# programming experience. In fact, I believe that if you have some experience with programming languages other than C#, you will be able to sit down in front of your computer, start up the Microsoft .NET Framework SDK or Visual Studio .NET, follow the examples provided with this book, and quickly become familiar with C# programming in scientific computing. For those of you who are already experienced C# developers, I believe this book has plenty to offer to you as well. The information in this book about creating C# math libraries is not available in any other C# tutorial and reference book. In addition, most of the example programs provided in this book can be used directly in your own real-world application development. This book will provide you with a level of detail, explanation, instruction, and sample program code that will enable you to do just about anything scientific and engineering computing-related with Visual C#.

This book is specifically designed for scientists and engineers. In fact, my own background is in theoretical physics, a field involving extensive numerical calculations as well as graphical representations of calculated data. I have been dedicated to this field for many years. My first computer experience was with FORTRAN. Later on, I gained programming experience in Basic, C, C++, and MATLAB. I still remember how hard it was in those early days to present computational results graphically. I often spent hours creating a publication-quality chart by hand, using a ruler, graph paper, and rub-off lettering. During that time, I started to pay attention to various development tools that could be used to create integrated applications. I tried to find an ideal development tool that would allow me to not only easily generate data (computation capability) but also easily represent data graphically (graphics and chart power). The C# and Microsoft Visual Studio .NET development environment makes it possible to develop such integrated applications. Ever since Microsoft .NET 1.0 came out, I have been in love with the C# language, and have used this tool to successfully create powerful scientific and plotting applications, including commercial CAD packages.

The majority of the example programs in this book can be used routinely by scientists and engineers. Throughout this book, I will emphasize the usefulness of C# programming in solving real-world scientific and engineering problems. If you follow this book closely, you will be able to easily develop various math and numerical libraries. At the same time, I will not spend too much time discussing program style, execution speed, and code optimization, because there is already a plethora of books out there that deal with these topics. Most of the example programs in this book omit error handling. This makes the code easier to understand by focusing on the key concepts.

Note that this book focuses on numerical computing methods and math library development using C#. It will not address the graphical representations of calculation results, even though the real power of the .NET Framework is its ability to create graphics and user interfaces. If you are interested in graphics and user interface programming in C#, you can read my other books:

*Practical C# Charts and Graphics* - This book is a perfect guide to learning all the basics for creating
advanced chart and graphics applications in C#, GDI+, and Windows Form. It clearly explains
practical chart and graphics methods and their underlying algorithms. The 2D and 3D chart packages
contained in the book can be directly used in your C# applications.

*Practical WPF Graphics Programming* - This book provides all the information you need to add advanced
graphics to your .NET applications using C# and Windows Presentation Foundation (WPF), which
comes with the new version (3.0 or later) of .NET framework. From 2D shapes and charts to complex
interactive 3D models, this book uses code examples to explain every step it takes to build a variety of
WPF graphics applications.

*Practical Silverlight Programming* - This book shows you how to develop rich interactive applications
(RIAs) for Web using C# and Silverlight. Silverlight is a subset of WPF and enables you to create
advanced graphics and user interfaces for Web applications. You will learn from this book how to
display your computation results graphically and interactively over the internet.

**What Do You Need to Use This Book**

You will need no special equipment to make the best use of this book and understand the algorithms. To run and modify the sample programs, you will need a computer capable of running either the Windows Vista or XP operating system. The software installed on your computer should include .NET Framwork SDK 2.0 or later, which is available for free at the Microsoft Website. It is best to have Visual Studio 2005 or 2008 standard edition or higher. Please note that all of the example programs and math libraries were created and tested in Visual Studio 2008. However, the example code should be independent of the platform you use.

**How This Book is Organized**

This book is organized into fifteen chapters, each of which covers a different topic of numerical computing. The following summaries of each chapter should give you an overview of the book?s contents:

*Chapter 1, Overview of C# Programming*

This chapter introduces the basics of C# programming, including the basic types, properties, methods,
mathematical operations, and how to create branches and loops.

*Chapter 2, Complex Numbers and Functions*

This chapter demonstrates how to implement a complex structure, which contains the definition of
complex numbers, complex operators, and commonly used complex functions. This structure allows
you to perform various computations using complex numbers and functions.

*Chapter 3, Vectors and Matrices*

This chapter introduces a more general n-dimensional vector class and a general matrix class with the
n×m dimension, which can be used in many scientific and engineering computations involving the
solution of linear equations with multiple variables. Matrix analysis is a basic theory of these linear
operations.

*Chapter 4, LinearAlgebraic Equations*

This chapter introduces various numerical methods for solving linear equations with an arbitrary
number of unknowns. Solving linear equations is one of the most commonly used operations in
numerical analysis and scientific and engineering applications.

*Chapter 5, Nonlinear Equations*

This chapter describes several numerical methods for solving nonlinear equations. These numerical
methods are all iterative in nature, and may be used for equations that contain one or several variables.

*Chapter 6, Special Functions*

This chapter discusses a special function class, which contains popular special functions such as the
gamma function, beta function, error function, elliptic integral, Laguerre function, Hermit function,
Chebyshev function, Legendre function, and Bessel function, among others.

*Chapter 7, Random Numbers and Distribution Functions*

This chapter covers a variety of random number generators and different probability distribution
functions, which can be used to simulate the different chaotic circumstances that can be found in the
real world.

*Chapter 8, Interpolation*

This chapter explains the implementation of several interpolation methods, which can be used to
construct new data points within the range of a discrete set of known data points. Interpolation can be regarded as a special case of curve fitting,
in which the function must go exactly through the data points.

*Chapter 9, Curve Fitting*

This chapter explains a variety of curve fitting approaches that can be applied to data containing noise,
usually due to measurement errors. Curve fitting tries to find the best fit to a set of given data. Thus,
the curve does not necessarily pass through all of the given data points.

*Chapter 10, Optimization*

This chapter covers several popular methods for optimizing functions with multiple variables, including
the golden search, Newton, simplex, simulated annealing, and differential evolution techniques. In
particular, simulated annealing and differential evolution can deal with highly nonlinear models,
chaotic and noisy data, and constraints.

*Chapter 11, Numerical Differentiation*

This chapter discusses several methods of numerical differentiation, such as forward and backward
difference, central difference, extended central difference, Richardson extrapolation, and derivatives
by interpolation. These methods provide you with different tools for estimating the derivative of a
function.

*Chapter 12, Numerical Integration*

This chapter covers a variety of methods for numerical integration, including methods based on
Newton-Cotes formulas, Romberg integration, and Gaussian quadrature methods. These methods can
be used to estimate the finite and infinite integrals of functions.

*Chapter 13, Ordinary Differential Equations*

This chapter focuses on solving ordinary differential equations numerically. It presents several popular
methods including the Euler method, second- and fourth-order Runge-Kutta methods, the adaptive
Runge-Kutta method, and the Runge-Kutta methods that can be used to solve a system of ordinary
differential equations.

*Chapter 14, Boundary Value Problems*

This chapter discusses two methods for solving boundary value problems: the shooting
method and the finite differences method. The shooting method involves guessing the missing values, and the
resulting solution is very unlikely to satisfy boundary conditions at the other end. The finite difference
method involves approximating the differential equations by finite differences at evenly spaced mesh
points.

*Chapter 15, Eigenvalue Problems*

This chapter presents several popular methods for solving eigenvalue problems, including the Jacobi
method, power iteration, Rayleigh method, Rayleigh-quotient method, and matrix tridiagonalization
method. These methods offer you nontrivial tools for calculating eigenvalues and eigenvectors of
real symmetric matrix systems.

**Using Code Examples**

You may use the code in this book in your applications and documentation. You do not need to contact me or the publisher for permission unless you are reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing the example code listings does require permission. Incorporating a significant amount of example code from this book into your applications and documentation does require permission. Integrating the example code from this book into your commercial products is not allowed without the written permission from the author.