Introduction to Modern Scientific Programming and Numerical Methods

This is the official companion website for Introduction to Modern Scientific Programming and Numerical Methods by L. Brieda, J. Wang, and R. Martin, CRC Press. The book is currently in production and should be available for purchase in the second half of 2024.

After working in the field of numerical simulations, mainly as applicable to aerospace gas kinetic and plasma applications, for over a decade, we have come to realize that the existing undergraduate engineering curriculum does not adequately prepare the students for such a career. Specifically, the typical undergraduate may have one or two classes introducing programming with Matlab or Python, and perhaps another class on linear algebra and numerical methods. Then perhaps in graduate school the student takes a class on computational fluid dynamics, or finite element methods. While these fundamentals are indeed necessary, they fail to introduce the student to other important topics such as object oriented programming, software engineering, code parallelization, or even the use of non-traditional computational technologies such as microcontrollers, field programmable gate arrays (FPGAs), or web-based numerical analysis. As such, a new course was created at the USC Department of Astronautical Engineering where some of us work, to provide such fundamentals. This book summarizes the material covered therein. While no single course can fully prepare new students for the work in a computational field, we hope that the included material provides the sufficient background "to know what I did not know", and thus open doors to additional university course work or independent self-study with dedicated references.

We hope you find the text useful and do not hesitate to reach out with questions.

Dr. Lubos Brieda, Prof. Joseph Wang, and Dr. Robert Martin


Introduction to Modern Scientific Programming and Numerical Methods can be purchased from popular book sellers such as Routledge, Amazon, and Barnes & Noble.

Example Codes

Sample codes can be downloaded here (16.6 Mb).


Chapter 1: Scientific Computing Basics

We begin by introducing the concept of numerical integration by developing an algorithm for calculating the trajectory of a tennis ball. The algorithm is first evaluated using "pen and paper". It is then adapted into a spreadsheet, before finally being ported to Python. This example also allows us to introduce basic programming concepts such as functions, variables, as well as random number generators. Here we also compare around a dozen of frequently encountered programming languages.

Chapter 2: Finite Difference and Linear Algebra

Chapter 2 introduces Taylor Series and the Finite Difference Method. This method is used to discretize the 2D steady-state heat diffusion equation. Direct and iterative matrixes solvers are introduced. The solver is developed in Python.

Chapter 3: Numerical Analysis

This chapter covers various numerical analysis topics such as data filtering, interpolation, quadrature, Newton-Raphson linearization, distribution function sampling, and multigrid solvers.

Chapter 4: Introduction to C++

Chapter 4 is a crash course on C++. C++ is a language commonly found in high-performance codes and software libraries and hence it is imperative to become familiar with concepts such as references, polymorphism, operator overloading, and template arguments. A C++ version of the heat equation solver from Chapter 2 is developed. Results are visualized using Paraview.

Chapter 5: Kinetic Methods

Chapter 5 introduces particle methods. Although we describe these methods in the context of gas dynamics, topics covered here are applicable to other disciplines as well. We develop a sample code for free molecular, then collisional, and finally plasma flow around an infinitely long cylinder. The chapter covers mesh-to-particle interpolation, and the computation of macroscopic flow properties such as density, velocity, and temperature from particle data.

Chapter 6: Eulerian Methods

This chapter cover mesh-based Eulerian approaches. It discusses stability analysis in the context of several model equations including advection-diffusion, wave, Burger's, and Maxwell's equations. An unsteady version of the heat equation solver is developed. Fluid modeling is demonstrated with the streamfunction-vorticity method. A Vlasov solver for the Boltzmann equation governing evolution of velocity distribution function is also included.

Chapter 7: Interactive Applications

Chapter 7 changes gears and covers development of HTML and Javascript codes that run in a web browser. Such codes not only offer interactivity not easily incorporated into C++ codes, but also allow one to develop programs on lab machines that otherwise may lack a programming environment. We describe rendering using the CPU-based 2D context as well as the GPU accelerated webgl interface.

Chapter 8: Software Engineering

Chapter 8 covers various topics related to software engineering, including debugging, use of test suites, version control, build systems, libraries, documentation, and coding practices. The LaTeX document setting environment is also discussed.

Chapter 9: High Performance Computing

Chapter 9 introduces parallel programming. It starts by covering profiling for identifying code parts most applicable to parallelization. Multithreading is then introduced. Here we cover the race condition and the use of mutexes for serialization of sensitive code sections. Next, domain decomposition with MPI is discussed. Here we cover deadlock, as well as the use of ghost cells for data transfer. The chapter closes with an overview of graphics card (GPU) processing using CUDA, as well as the use of OpenGL for rendering.

Chapter 10: Optimization and Machine Learning

Chapter 10 starts by introducing approaches such as gradient descent and genetic algorithms for finding input parameters for improving agreement with expected results. Machine learning is then introduced and a small dense neural network is constructed for classifying real values. The back propagation method is used to find the optimal neural net weights and biases.

Chapter 11: Embedded Systems

Finally, Chapter 11 covers development of codes running on microcontrollers and FPGAs. Such devices allow for the so-called edge computing in which data analysis is done close to the source of data. We see how to integrate an Arduino microcontroller with a 3rd party sensor. We then cover FPGA programming using the Intel Cyclone development kit and also learn how to utilize Arduino microcontroller that comes with a built-in FPGA.