Hidden Markov Model: Baum-Welch [UAlgo]

Hidden Markov Model: Baum-Welch is a regime detection and reversal signaling indicator that applies a 3 state Hidden Markov Model to normalized log returns and continuously adapts its parameters using an online Baum Welch expectation maximization routine. The script is designed to classify the market into three latent regimes, then express that classification as real time probabilities for Bull, Range, and Bear conditions.

The indicator runs in its own pane (overlay=false) and outputs:
Probability curves for the three regimes
A dominant regime score scaled to 0 to 1
A regime strip visualization for quick bias reading
Adaptive background coloring based on the dominant regime and confidence
Optional regime shift markers
Optional buy and sell reversal markers driven by strict multi condition logic

The core idea is that price behavior can be modeled as transitions between hidden states that each have their own return distribution. The script fits a Gaussian emission model for each state, estimates state transition probabilities, and updates the posterior probability of each state on every bar. It retrains the full model at fixed intervals, while using a faster one step forward update between retrains for efficiency.

This implementation is not a simple threshold oscillator. It is a full mini HMM engine built in Pine with:
Scaled forward and backward algorithms
Expectation step producing gamma and xi posteriors
Maximization step updating initial distribution, transition matrix, state means, and state variances
Safeguards such as variance floors and transition floors to maintain numerical stability

The output is a regime aware probability system that can be used for bias, context, and reversal confirmation rather than simple entry signals.

Educational tool only. Not financial advice.

🔹 Features

🔸 1) Three State Hidden Markov Model Regime Engine

The model uses three hidden states and continuously estimates the probability of being in each state:
Bull regime
Range regime
Bear regime

This gives a probabilistic regime map rather than a single hard classification.

🔸 2) Baum Welch Training with Scheduled Retraining

The script retrains its parameters using an EM routine at a user defined interval in bars. Each retrain runs a configurable number of EM iterations. Between retrains, the indicator performs a one step forward Bayesian update of the posterior state probabilities.

This structure balances adaptability with performance.

🔸 3) Normalized Log Return Observations

The observation series is a z score normalized log return:
Log returns convert price changes into additive units
An EMA and rolling standard deviation normalize the series to stabilize the HMM fit

This helps the HMM learn regimes based on relative return behavior rather than raw price scale.

🔸 4) Automatic Bull, Range, and Bear Role Assignment

The model learns state means. The script then assigns roles by ranking those learned means:
The state with the lowest mean becomes the Bear state
The state with the highest mean becomes the Bull state
The remaining state is treated as Range

This keeps regime labeling consistent even as the internal state ordering shifts during training.

🔸 5) Probabilities and Dominant Regime Visualization

The script plots:
Bull probability curve
Range probability curve
Bear probability curve

It also plots an area for the dominant probability and a regime strip that makes it easy to see the dominant regime quickly without reading the full curves.

🔸 6) Regime Score Line (Bull minus Bear)

A continuous score is calculated as Bull probability minus Bear probability, then scaled to a 0 to 1 range. This score becomes the main regime momentum signal used for rebound and reversal logic.

🔸 7) Adaptive Background Coloring by Regime and Confidence

The pane background color changes based on the dominant regime. Transparency adapts according to confidence, so strong regime certainty produces a more visible background while low certainty remains subtle.

🔸 8) Strict Signal Filters for Bias and Reversal

The indicator provides bias filters:
Bull bias when Bull probability and confidence exceed thresholds and the dominant regime is Bull
Bear bias when Bear probability and confidence exceed thresholds and the dominant regime is Bear

It also provides reversal style buy and sell signals based on a multi condition framework described in the calculations section.

🔸 9) Reversal Logic Combining Extremes, Rebounds, and Transition Edge

Reversal signals are not generated by a single crossover. The script requires:
An extreme score pivot
An extreme regime probability at that pivot
A rebound trigger through predefined rebound levels
A minimum probability and confidence filter
A transition asymmetry and edge condition that favors switching toward the target regime
A momentum condition requiring Bull probability rising and Bear probability falling for buys, and the inverse for sells
A time window limit so reversals must occur within a limited number of bars after the extreme

This creates a high selectivity reversal engine.

🔸 10) Transition Matrix Insight and Switch Edge Metrics

The script computes predicted transition probabilities toward Bull and Bear using the current posterior and the transition matrix. It also measures transition asymmetry between Bull to Bear and Bear to Bull and uses these values as part of reversal confirmation.

This adds structural information that classic oscillators do not capture.

🔸 11) Anti Duplicate Reversal Signals

Once a pivot extreme has been used to generate a reversal signal, it is marked as consumed so the same pivot cannot repeatedly trigger additional buy or sell signals. This helps avoid signal repetition.

🔸 12) Full Informational Label Output

A live info label prints:
Current regime
Current signal text
Confidence
Bull, Range, Bear probabilities
Log likelihood
Key trigger thresholds
Reversal settings and edge settings

This provides transparency into what the model is currently seeing and why signals are or are not appearing.

🔹 Calculations

1) Observation Series: Normalized Log Returns

The script uses log returns:

logRet = math.log(close / nz(close[1], close))

Then normalizes them with an EMA mean and rolling standard deviation:

retMean = nz(ta.ema(logRet, normLength), 0.0)
retStd = math.max(nz(ta.stdev(logRet, normLength), 0.0), 1e-6)
obs = (logRet - retMean) / retStd

This creates an observation series with more stable scale properties across time.

2) Rolling Observation Window

The HMM is trained on a rolling window of length windowLen. Only the most recent processRecentBars are processed to control load:

startBar = last_bar_index - processRecentBars
activeRange = bar_index >= (startBar < 0 ? 0 : startBar)

If active, the observation is appended and the oldest one is removed:

if array.size(obsWindow) < windowLen
array.push(obsWindow, obs)
else
array.shift(obsWindow)
array.push(obsWindow, obs)

The model is ready only when the window is full.

3) Model Initialization

The script initializes a 3 state model with:
Uniform initial state probabilities
A transition matrix seeded with high persistence and equal small jump probabilities
State means initialized around zero with a configured separation
State variances initialized to a configured starting value

Key logic:
Stay probability equals initialPersistence
Jump probability equals the remaining probability split across other states

This gives the HMM a stable starting point before training.

4) Emission Model: Gaussian per State

Each state emits observations using a Gaussian density:

math.exp(-0.5 * d * d / varS) / math.sqrt(TWO_PI * varS)

Variance uses a floor:

float varS = math.max(array.get(this.vr, s), varMin)

This prevents variance collapse and numeric instability.

5) Forward Algorithm with Scaling

The script computes the forward probabilities alpha and applies scaling coefficients c[t] to prevent underflow. It then recovers log likelihood from the scaling coefficients:

this.logLik := -sum(log(c[t]))

This is essential because HMM sequences quickly underflow without scaling.

6) Backward Algorithm with Scaling

The backward probabilities beta are computed using the scaling values from the forward pass, ensuring alpha and beta remain numerically stable across the entire window.

7) Expectation Step: Gamma and Xi

Gamma represents posterior probability of being in state i at time t. Xi represents posterior probability of transitioning from i to j between t and t+1.

Xi is normalized per time step:

xij = xi_raw / denom

Gamma is computed as the sum of xi across outgoing transitions for each state:

gamma(t, i) = sum_j xi(t, i, j)

8) Maximization Step: Updating Parameters

Initial probabilities update from gamma at time 0:

pi = gamma(0, i)

Transition probabilities update from xi sums divided by gamma sums, with a transition floor and row normalization:
Each transition is clamped to transitionFloor
Each row is normalized to sum to 1

Means update as weighted averages of observations using gamma weights.
Variances update as weighted squared deviation sums with a variance floor.

9) Retraining Schedule and Online Updates

The model retrains when:
It is not initialized yet
Or the bar index matches the retrain interval

shouldRetrain = ready and (not modelInitialized or bar_index % retrainEveryBars == 0)

On retrain, Baum Welch is run for emIterations.

Between retrains, the script performs a one step forward update of the posterior:

hmm.forwardOne(posterior, obs, varianceFloor, posteriorTmp)

This provides continuous posterior updates without full retraining on every bar.

10) Role Mapping to Bull, Range, Bear

The script assigns which internal state corresponds to Bear and Bull by looking at the learned means:
Bear state is the state with the minimum mean
Bull state is the state with the maximum mean
Range is the remaining state index

This mapping updates dynamically as the model learns.

11) Regime Score and Confidence

The regime score is:

score = pBull - pBear

It is then scaled to 0 to 1:

score01 = 0.5 + 0.5 * score

Confidence is:

confidence = max(pBull, pRange, pBear)

This confidence drives background alpha and signal gating.

12) Probability Filters for Bias

Bull filter requires:
Bull probability above bullProbTrigger
Confidence above signalConfidenceMin

Bear filter requires similar conditions for Bear probability.

Bias validity adds the requirement that the dominant regime role matches the direction:
Bull bias requires dominantRole equals 1
Bear bias requires dominantRole equals minus 1

13) Extreme Pivot Logic for Reversal Candidates

The script looks for pivots in the score line:

ta.pivotlow(score01, pivotStrength, 1)
ta.pivothigh(score01, pivotStrength, 1)

It stores the most recent pivot low and pivot high along with the associated Bull or Bear probability at the pivot bar.

A low extreme is valid if:
Score at pivot is below dipScoreLevel
Bear probability at pivot exceeds extremeProbMin

A high extreme is valid if:
Score at pivot is above topScoreLevel
Bull probability at pivot exceeds extremeProbMin

14) Rebound Triggers

After an extreme, the script waits for rebound triggers:
Up rebound:

ta.crossover(score01, reboundUpLevel)

Down rebound:

ta.crossunder(score01, reboundDownLevel)

Rebound must occur within the reversal window bars from the extreme pivot.

15) Transition Edge and Asymmetry Logic

The script computes predicted probabilities of switching toward Bull or Bear using the transition matrix and current posterior. It also computes transition asymmetry between the Bull to Bear and Bear to Bull transitions.

A bullish switch condition requires:
Switch edge greater than hmmEdgeMin
Transition asymmetry favoring Bear to Bull at or above transitionAsymMin
Bull probability greater than Bear probability

A bearish switch condition uses the mirrored logic.

This adds a model based confirmation that a regime switch is plausible, not only that the score bounced.

16) Momentum Confirmation

Bull momentum requires:
Bull probability rising
Bear probability falling

Bear momentum requires the opposite.

These conditions prevent signals when probabilities are flat or conflicting.

17) Final Reversal Signal Construction

Buy reversal requires:
Valid low extreme
Not consumed
Inside reversal window
Rebound up
Bull probability and confidence filter
Bullish HMM switch condition
Bull momentum

Sell reversal requires the mirrored set of conditions.

The sell is suppressed if a buy is simultaneously true so conflicting signals do not print on the same bar.

18) Visualization Output

The script plots:
Probability curves for each regime
A dominant probability area
A thick score line colored by regime
A regime strip column plot
Fills between Bull and Bear curves and between rebound levels
Adaptive background
Optional markers for regime shifts
Reversal markers as glow plus label style plots

The info label consolidates the most important current state and threshold data for transparency.