Skip to content
Dr. Ram KumarDiabetes & metabolic health

Medical AI · Deep learning & imaging · Lesson 1 of 5

Neural-network basics

Reviewed by Dr. Ram Kumar G, MBBS, MD (Internal Medicine), MPH (USA), on · Last updated

  • Intermediate
  • Technical focus
  • About 30 minutes
  • Includes exercise

Before this lesson: A logistic-regression risk score, Setting up Python and Colab for clinical data work

In short

A neural network is a chain of simple layers: each multiplies its inputs by learned weights, adds them up and passes the result through a non-linear function. Stacking layers lets the network learn curved, complex relationships that a linear model such as logistic regression cannot. It learns by repeatedly comparing its predictions with the correct answers and nudging every weight to reduce the error, a process called gradient descent with backpropagation. Frameworks such as PyTorch do the calculus automatically.

You will learn to

  • Explain layers, weights, activation functions and loss in plain terms
  • Show why a neural network can separate data that logistic regression cannot
  • Implement forward and backward passes of a two-layer network in NumPy
  • Build and train the same network in PyTorch

From logistic regression to a neural network

Logistic regression is a one-layer network: it weights the inputs, adds them up and squashes the total into a probability. Its decision boundary is a straight line (or flat plane). A neural network adds 'hidden' layers in between, each applying a non-linear activation such as tanh or ReLU, so the boundary can bend.

TermMeaning
LayerA set of units that each compute a weighted sum of the previous layer's outputs
Weights and biasesThe numbers the network learns
Activation functionA non-linear function (tanh, ReLU) applied after each weighted sum
LossA measure of how wrong the predictions are, such as cross-entropy for classification
Gradient descentRepeatedly adjusting weights in the direction that reduces the loss
BackpropagationThe method for computing how much each weight contributed to the error
EpochOne pass through the training data

Data a straight line cannot separate

python
import numpy as np
from sklearn.datasets import make_moons
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

X, y = make_moons(n_samples=2000, noise=0.25, random_state=0)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)
lr = LogisticRegression().fit(X_train, y_train)
print(f"Logistic regression accuracy: {lr.score(X_test, y_test):.3f}")

# Logistic regression accuracy: 0.852
Two interleaved, curved classes: logistic regression as the baseline.

A neural network from scratch

The network below has 2 inputs, 16 hidden units with tanh activation and one output. Each loop runs a forward pass (predict), a backward pass (compute gradients with the chain rule) and a gradient-descent step.

python
rng = np.random.default_rng(0)
W1, b1 = rng.normal(0, 1, (2, 16)), np.zeros(16)          # input -> 16 hidden units
W2, b2 = rng.normal(0, 1, (16, 1)), np.zeros(1)           # hidden -> output
sigmoid = lambda z: 1 / (1 + np.exp(-z))

for epoch in range(2001):
    h = np.tanh(X_train @ W1 + b1)                         # forward pass
    p = sigmoid(h @ W2 + b2).ravel()
    grad_out = (p - y_train)[:, None] / len(y_train)       # backward pass (cross-entropy loss)
    grad_W2, grad_b2 = h.T @ grad_out, grad_out.sum(0)
    grad_h = grad_out @ W2.T * (1 - h ** 2)
    grad_W1, grad_b1 = X_train.T @ grad_h, grad_h.sum(0)
    for param, grad in [(W1, grad_W1), (b1, grad_b1), (W2, grad_W2), (b2, grad_b2)]:
        param -= 1.0 * grad                                # gradient descent step
    if epoch % 500 == 0:
        loss = -np.mean(y_train * np.log(p + 1e-9) + (1 - y_train) * np.log(1 - p + 1e-9))
        print(f"epoch {epoch:>4}  training loss {loss:.3f}")

p_test = sigmoid(np.tanh(X_test @ W1 + b1) @ W2 + b2).ravel()
print(f"Neural network accuracy: {np.mean((p_test > 0.5) == y_test):.3f}")

# epoch    0  training loss 0.915
# epoch  500  training loss 0.125
# epoch 1000  training loss 0.122
# epoch 1500  training loss 0.121
# epoch 2000  training loss 0.120
# Neural network accuracy: 0.938
Continuing from the previous block: a two-layer network in NumPy.
  • The hidden layer lets the network bend its boundary around the curved classes: 93.8% accuracy against 85.2% for logistic regression.
  • The loss falls fast and then levels off; continuing longer would mainly fit noise in the training data.

The same network in PyTorch

Writing gradients by hand does not scale. PyTorch records the operations in the forward pass and computes all gradients with one call to backward(). The Adam optimiser adapts the step size for each weight.

python
import torch
from torch import nn

torch.manual_seed(0)
model = nn.Sequential(nn.Linear(2, 16), nn.Tanh(), nn.Linear(16, 1))
optimiser = torch.optim.Adam(model.parameters(), lr=0.01)
loss_fn = nn.BCEWithLogitsLoss()
Xt, yt = torch.tensor(X_train, dtype=torch.float32), torch.tensor(y_train, dtype=torch.float32)

for epoch in range(500):
    optimiser.zero_grad()
    loss = loss_fn(model(Xt).squeeze(1), yt)
    loss.backward()                                          # PyTorch computes the gradients
    optimiser.step()

with torch.no_grad():
    logits = model(torch.tensor(X_test, dtype=torch.float32)).squeeze(1)
print(f"PyTorch network accuracy: {((logits > 0).numpy() == y_test).mean():.3f}")
print(f"Parameters: {sum(p.numel() for p in model.parameters())}")

# PyTorch network accuracy: 0.945
# Parameters: 65
Continuing from the first block: the network in PyTorch (pip install torch).
  • This network has 65 parameters. Image models have thousands to billions, but they learn in exactly the same way.
  • For tabular clinical data, as earlier lessons showed, neural networks rarely beat well-built regression or gradient boosting. Their advantage is on images, signals and text.

About the numbers

  • The code uses fixed random seeds, PyTorch 2.8 and a laptop CPU. Other versions or hardware can give slightly different numbers; the pattern of results is what matters.
  • Images in this track are synthetic, so they run anywhere in seconds. The exercises point to MedMNIST, an openly licensed collection of small medical-image datasets, for real images.

Exercise

Build, train and probe a small neural network, and see when it helps over logistic regression.

Data: Synthetic data generated in the notebook (scikit-learn make_moons) (Generated in the notebook)

  1. Reproduce the NumPy and PyTorch networks and plot their decision boundaries.
  2. Change the hidden layer to 2, 4 and 64 units and compare test accuracy and boundaries.
  3. Add more noise to the data and show how a large network can overfit; add early stopping on a validation set.
  4. Train the PyTorch network on the synthetic diabetes-risk data from the logistic-regression lesson and compare AUROC and calibration with logistic regression.

Open the notebook in ColabView on GitHubWorked solution coming soon

Key takeaways

  • A neural network stacks weighted sums and non-linear activations.
  • Hidden layers let it learn curved boundaries that linear models cannot.
  • It learns by gradient descent, with backpropagation computing the gradients.
  • Neural networks shine on images, signals and text more than on tabular data.

Terms used: Backpropagation, Logistic regression, Neural network

Related: CNNs for chest X-ray and retinal images, A logistic-regression risk score, How large language models work

FAQ

What is a neural network?

A model made of layers of simple units, each computing a weighted sum of its inputs followed by a non-linear function. Stacking layers lets it learn complex relationships from data.

What is backpropagation?

The method for working out how much each weight contributed to the prediction error, using the chain rule from calculus, so that every weight can be adjusted to reduce the error.

Is deep learning better than logistic regression?

For images, signals and text, usually yes. For typical tabular clinical data, well-built regression or gradient boosting often performs as well and is easier to check.

What does 'deep' mean in deep learning?

Having many layers. Deeper networks can represent more complex patterns, but need more data and care to train well.

Do I need a GPU?

Not for this lesson. Large image models train much faster on GPUs, which services such as Colab provide.

Sources

Educational content only. This material teaches methods; it is not clinical advice. Code, models and examples here are not validated medical devices and must not be used to make decisions about real patients without proper clinical validation, governance and any required regulatory approval. Examples use synthetic or openly licensed de-identified data only. Never paste identifiable patient information into public AI tools. Mentions of models, tools or vendors are for identification, not endorsement.