3 Credit Hours

Artificial Intelligence (AI) tries to build smart machines to automate intellectual tasks normally performed by humans, in a way that we consider intelligent. Machine Learning (ML) is a subset of AI that studies computer algorithms which improve automatically through experience (data). ML algorithms typically build a mathematical model based on training data and then make predictions without being explicitly programmed to do so. Its performance increases with experience (data), in other words, the machine learns. Scientific machine learning (SciML) is a burgeoning discipline in AI/ML that blends scientific computing and ML. SciML is a core component of ML. It consists of computational technologies that can be trained with scientific data to augment or automate human skills.

This course aims at augmenting the applications of AI/ML in scientific computing especially in the nuclear engineering area, and preparing the students for transformative solutions across various DOE missions, and for driving the next wave of data-driven scientific discovery in nuclear engineering.

After this course, the students will understand most of the fundamentals of supervised ML, including neural networks and other algorithms for regression/classification, as well as dimensionality reduction for unsupervised ML. The students will be able to implement ML algorithms in nuclear engineering scientific computing applications, with a heavy focus on uncertainty quantification, sensitivity analysis, calibration and inverse problems.

Prerequisites

• Background in calculus, linear algebra, scientific computing. Undergraduate level courses in these areas are sufficient.
• Background in Probability and Statistics, equivalent to ST 311 (Introduction to Statistics) and ST 371 (Introduction to Probability and Distribution Theory).
• Prior experience in Machine Learning is not required.
• Python is the recommended programming language.
• In the first lecture, instructions and resources to learn Python will be provided, as well as Python ecosystems and libraries that will be useful for this course.

Course Objectives

The students completing this course will be able to :
– Analyze common regression/classification problems using classical supervised ML techniques;
– Demonstrate understanding of the fundamentals of dimensionality reduction for unsupervised ML;
– Evaluate the performance of ML models;
– Build ML models and write numerical codes to solve regression, classification and dimensionality reduction problems;
– Use open source ML libraries such as scikit-learn, keras, TensorFlow and PyTorch.
– Apply SciML methods on uncertainty quantification, sensitivity analysis, calibration and inverse problems;
– Implement advanced techniques for UQ of neural networks;
– Solve research problems using the SciML techniques learned in this class;
– Develop research skills that combine the complementary perspectives of computational science and computer science to craft a new generation of ML methods for complex applications across nuclear engineering.

Course Outline

Part 1: Artificial Neural Network (ANN)

1. Introduction
2. Perceptrons, Sigmoid Neurons and ANNs
3. Classifying Handwritten Digits and Gradient Descent
4. Backpropagation
5. Cross-entropy, Softmax and Negative Log-likelihood Cost
6. Overfitting and Regularization
7. Bias, Variance, Hyperparameters and Weight Initialization
8. Vanishing Gradient, Activation Functions and SGD Variations

Part 2: Fundamental Techniques of Machine Learning

9. Generalized Linear Regression, Subset Selection and Shrinkage
10. Logistic Regression
11. K-Nearest Neighbor (KNN) and Feature Scaling
12. Naive Bayes (NB)
13. Decision Tree (DT)
14. Ensemble Learning – Bagging, Boosting and Stacking
15. Support Vector Machine (SVM)
16. Dimensionality Reduction with Principal Component Analysis (PCA)
17. Gaussian Processes (GP) Modeling (2 lectures)

Part 3: Uncertainty Quantification (UQ) and Sensitivity Analysis (SA) with ML

18. Forward UQ Methods (3 lectures)
19. SA Methods
20. Inverse UQ Methods
21. UQ of Deep Neural Networks (DNNs)

Course Requirements

Homework problems (40%): 4-5 assignments based on the course lectures.

Computational projects (60%): The research-based project should use ML techniques covered in the lecture materials. The computer project will be evaluated by a proposal, a progress report, a final report and a final presentation. Each of the milestones will be submitted through Moodle.

There will be no exams.

Textbook

None.

Software Requirements

Programming with Python.

Created: 03/28/2025