There are thousands of tutorials showing an LSTM Bitcoin prediction chart and almost none showing whether the resulting decisions survive costs, chronology and repeated training. Our conclusion after testing recurrent and hybrid branches is narrower than ‘deep learning does not work’: on the daily Bitcoin problem we actually operate, the recurrent layer did not earn its complexity.
The current production system is the Frontier Model, a frozen BTCUSD D1 XGBoost signal model with seven engineered inputs, no LSTM inference and no automatic production retraining.
The setup: compare model classes, not marketing claims
The research programme compared recurrent models, hybrid LSTM-plus-tree designs and gradient-boosted trees under the same broad validation discipline: chronological splits, transaction costs, validation-led selection and later robustness checks.
Older versions of this article mixed that lesson with a separate equity-index feature set. That was misleading for the current site. The production system is Bitcoin-only, and the feature list below is the actual seven-input Frontier signal set.
Published Frontier inputs
dist_swing_high_12dist_swing_high_24dist_swing_low_12rs_plusrsi_norm_10rv_ratiotsmom_norm
We publish the input names because they carry the methodological lesson. We do not publish their complete formulas, the target/prediction horizon, target barriers, decision threshold or XGBoost parameter values.
What the LSTM experiments actually taught us
1. Training variance can dominate a model comparison
Repeated neural training runs showed materially more seed and hardware sensitivity than the fixed trained tree model used for production inference. That does not mean every LSTM is irreproducible; it means a single neural training run is not enough evidence for promotion.
The practical consequence is important: if a research score changes substantially across retrains, ranking thousands of neural trials can become a ranking of training luck. Robust comparison requires repeated seeds or ensembles, which increases both compute cost and the number of ways a researcher can accidentally overfit the process.
2. Engineered tabular features already contain much of the useful sequence summary
Daily Bitcoin offers only a few thousand observations over many years. In that regime, explicit features describing structure, realised volatility, momentum and oscillator state give a tree model a compact representation of the market without asking a recurrent network to rediscover those summaries from raw sequences.
3. A frozen production binary is easier to audit
The current Frontier does not retrain itself in production. The exact signal binary is frozen, which makes changes in the public record easier to attribute: if execution changes, that can be studied separately from a changing model fit.
What beat the LSTM
The production winner is not an abstract claim that ‘XGBoost always beats deep learning’. It is a project-specific result: a compact XGBoost model on the seven published BTC inputs survived the validation process better than the recurrent alternatives tested for this use case.
The strongest newer finding came after the model-class comparison: repeated rolling refits were not strong enough to replace the exact frozen signal binary. That reinforced the same preference for a simpler, auditable production object.
Current Frontier evidence
| Metric | In-sample | Strict model-weight OOS |
|---|---|---|
| Period | Jan 2018 – Jul 2022 | 30 Jul 2022 – Aug 2026 |
| Trades | 151 | 210 |
| SQN | 5.348 | 2.129 |
| Simple return | +733.7 pt | +253.9 pt |
| Win rate | 63.6% | 50.5% |
| Maximum drawdown | -31.04 pt | -25.34 pt |
| Profit / drawdown | 23.63 | 10.02 |
| Research leverage | 1:1 equivalent | 1:1 equivalent |
The uninterrupted full-history replay contains 361 exact trades and ends at +992.65 simple percentage points. The isolated IS/OOS table above remains the cleaner comparison for research chronology; the continuous replay is a separate state-preserving diagnostic.
Why this is not a universal anti-LSTM conclusion
Sequence models can be useful when the dataset is much larger, the sequence itself carries information that engineered features cannot summarise, or the model can be updated and evaluated online without contaminating the research process.
Our conclusion is specific: for the Bitcoin D1 production problem measured here, the LSTM branch did not provide enough stable incremental evidence to justify deployment over the simpler tree baseline.
FAQ
Can LSTM non-determinism be fixed with seeds?
It can be reduced, but practical reproducibility depends on the software and hardware stack. Repeated-seed evaluation and ensembling are more reliable than assuming one seeded run is representative.
Is an LSTM-plus-XGBoost hybrid worth trying?
It is worth testing if the incremental neural features are evaluated by ablation against the same tree model without them. Complexity is justified only when the added component survives that comparison.
What replaced the LSTM in production?
A frozen XGBoost BTCUSD D1 signal model using the seven published engineered inputs above. The model is evaluated through chronological research blocks and a separate forward record.
Publication boundary. We publish aggregate research results, chronology, validation methodology and the names of the production inputs. We do not publish the target/prediction horizon, label/barrier calibration, decision threshold, exact XGBoost hyperparameters, trained model weights, or the full production execution recipe. Those details are deliberately withheld because they would make the production model substantially easier to reproduce.
Read the companion article XGBoost for Trading and the walk-forward analysis for the validation context.
Risk notice. Historical and simulated results are not guarantees of future performance. This article is research, not investment advice.
1 thought on “Why Our LSTM Bitcoin Model Failed (and What Beat It)”
Comments are closed.