---
title: "Calibrain Harness: An Open-Source Crypto Bot Tester on Bybit Testnet"
date: "2026-11-14"
video_id: "7707967435034397958"
tiktok_url: "https://www.tiktok.com/@github.signals/video/7707967435034397958"
creator: "@github.signals"
category: "Finance"
tags: ["calibrain harness", "crypto trading bot", "bybit testnet", "forward testing", "open source"]
excerpt: "Calibrain Harness grades every model forecast against real Bybit market data, but orders go only to the Bybit testnet. How it works and what the TikTok skips."
---
Calibrain Harness is an open-source, MIT-licensed tool that tests whether a fast AI classification model has a real edge in crypto perpetual futures. It grades every forecast the model makes against what the market actually did, using live Bybit market data. Orders go only to the Bybit testnet with simulated funds, a limit a TikTok from @github.signals never states aloud when it describes the tool as running on "live crypto markets."

## What is Calibrain Harness?

The project's README calls it "A forward-testing harness for trading with fast classification models: plug a model in, run it on live perpetual markets, and find out honestly whether it has an edge." Its badges read "status under development," "venue Bybit testnet only," "models Kairos | Jev," "python 3.12," "next.js 16" and "MIT."

A warning box says the project is under active development, trades on Bybit testnet only, and is not financial advice. The testnet host is hard-coded, with no switch for mainnet.

## Does Calibrain trade real money?

No. The video says "It plugs fast classification models into live crypto markets," and the description repeats "live crypto markets." Most viewers would take that to mean a bot trading real money.

The repository says signals use Bybit mainnet market data, while orders are placed on the Bybit testnet with simulated funds. Observe-only is the default, and auto-trading has to be switched on by hand. "Live" is accurate for the price feed and misleading for execution. The description's hashtag #PaperTrading is closer to the truth than the narration.

## How does it grade predictions?

On each closed price bar, the harness asks the model a small set of questions that can be checked later. The README's example is "which is touched first in the next 2 h: +1.0% or -0.6%?" Code turns the model's probabilities into expected value and places trades that clear its gates. It then "scores every prediction against what the market actually did, traded or not."

That last phrase matters. Forecasts that never became trades are still graded, which supports the video's line that it "grades every single decision." Grading covers the model's answers to the harness's own question set.

The video gives no results. Neither it nor the repository shows a hit rate, profit figure or evidence that any model has an edge. The tool tests for an edge without claiming one.

## Does it stop trading when the model is guessing?

Broadly, yes. The video says "If the model is just guessing, the system learns and stops trading." "Learns" here means calibration rather than retraining.

The repository describes the mechanism. Model outputs are blended with the instrument's historical base rates, and the blend weight is fitted on labelled outcomes. A model that adds nothing stops driving trades. Weights are refitted every 15 minutes once a group has 30 independent labelled windows. Drift detection can also set a model's weight to zero or pause trading.

Two details the video skips:

- In "strict" mode, a symbol must be statistically validated first. Until then the system logs decisions and trades only at small size, so small trades can happen before a model has beaten base rates.
- 30 windows is a small sample for a noisy market, so a short run proves little.

## Which models does it support?

The video does not name any. The README lists two: Kairos (DMind, served through Minara Cloud) and Jev (TypeSafe). The Kairos row shows model id "DMIND/Kairos" and key "KAIROS_API_KEY," so you bring your own model access and credentials. You can switch models while it runs, and results are tracked per model.

## Safety features

The repository describes a KILL button and a KILL file that close bot positions, cancel orders and halt the bot. A daily loss limit applies, set to 3% by default. The narration mentions none of this.

## Is it a benchmark?

The video calls it "the benchmark you need to know you are not fooling yourself." A benchmark usually has fixed tasks so results compare across models and over time. The README says interfaces, question sets and defaults change between commits, so scores from different versions may not line up. "Forward-testing harness," the README's own term, is the better description.

When fetched, the repository showed 236 stars, 51 forks, 3 watchers and 218 commits under an MIT license.

## Bottom line

Calibrain Harness grades every forecast against live Bybit market data, including forecasts it did not trade, and it can turn a useless model's influence down to zero. It trades only on the Bybit testnet with simulated funds. It is an early, changing project with no published performance results.

## Sources

- [zadescoxp/calibrain-harness on GitHub](https://github.com/zadescoxp/calibrain-harness): testnet-only venue, models, calibration and kill mechanisms, license and repository counts.


## Video description and verification notes

**The clip runs 40.07 s (audio stream 40.02 s) at 576x1024, 9:16, in an MP4/MOV container (isom). Video is H.264 High at 30 fps, 333 kb/s. Audio is HE-AAC v2 stereo at 44.1 kHz, 32 kb/s. Overall bitrate is 372 kb/s and the file is 1,863,876 bytes. It was posted at epoch 1794651091, which is 2026-11-14 10:11:31 UTC, by @github.signals (nickname "Github Signals"). The audio credit is "original sound - Github Signals". The description is about 760 characters and links github.com/zadescoxp/calibrain-harness. Engagement at capture (2026-11-14T13:01:15Z) was 3,595 views, 203 likes, 0 comments, 53 reposts and 262 saves. I read 5 frames: 001, 006, 012, 018 and 020. At 2-second sampling these fall at roughly 0, 10, 22, 34 and 38 s. Frames 001 to 018 show a screen recording of the GitHub README for zadescoxp/calibrain-harness. The recording has a browser-style bar with the URL, a "GithubSignals" watermark and a slow downward scroll. The README shows the title "Calibrain Harness" and the tagline "A forward-testing harness for trading with fast classification models: plug a model in, run it on live perpetual markets, and find out honestly whether it has an edge." Badges read "status under development", "venue Bybit testnet only", "models Kairos | Jev", "python 3.12", "next.js 16" and "MIT". A warning box says the project is under active development, trades on Bybit testnet only (the host is hard-coded, with no mainnet switch), and is not financial advice. A dark trading-terminal screenshot sits under the warning. Its caption reads "Live terminal: monitor, Kairos signals, evaluation and news." Yellow-highlight burned-in captions read "CALIBRAIN HARNESS" (frame 001), "DATA TO SEE" (006), "THE SYSTEM LEARNS" (012) and "FOOLING YOURSELF." (018). Frame 020 is an end card: "Follow for more open-source projects" and "@GithubSignals". No code, terminal run, profit figure, backtest result or calibration chart appears in any frame I read. The transcript has 6 segments and 100 words. The speaker argues that most trading bots hide their mistakes, that this one scores every prediction against market data, and that it stops trading when a model is just guessing. The last segment starts at 30.0 s and ends at 35.1 s, inside the 40.02 s audio, so no segment is fabricated. Segment 4 ends at 29.4 s. The remaining 5 s have no speech.**

## "Live crypto markets": market data is live, but orders go only to a testnet

At 12.88 s the video says "It plugs fast classification models into live crypto markets." The description repeats "live crypto markets." A viewer would probably read this as a bot trading real money. The README frames in the video say otherwise. The badge reads "venue Bybit testnet only", and the warning says the host is hard-coded with no mainnet switch. Frame 001 also shows the tagline "run it on live perpetual markets."

I fetched the repository page. It says signals use Bybit mainnet market data, while orders are placed on the Bybit testnet with simulated funds. Observe-only is the default, and auto-trading has to be switched on by hand. So "live" is accurate for the price feed. It is misleading if it suggests live-money execution. The description's hashtag #PaperTrading is closer to the truth than the spoken script. The video never says "testnet" or "paper" aloud. The only statement of that limit is the on-screen README text, which sits on screen for only part of the 40 seconds.

The repository page also shows a hard kill path. A KILL button or a KILL file closes bot positions, cancels orders and halts the bot. A daily loss limit applies, set to 3% by default. None of these safety features is mentioned in the narration.

## "Scores every prediction against real market data": confirmed by the README

At 5.24 s the video says "this one scores every prediction against real market data to see if it truly has an edge." The README text visible in frame 006 matches this. It says the harness connects a high-speed classification model to perpetual futures and treats every answer as a forecast that will be checked. On each closed bar it asks the model a small set of labellable questions. The README gives this example: "which is touched first in the next 2 h: +1.0% or -0.6%?" It turns the probabilities into expected value in code and trades those that clear the gates. Then it "scores every prediction against what the market actually did, traded or not." The last phrase matters: predictions that never became trades are still graded. That supports the video's wording "grades every single decision," with one qualification. Grading applies to the model's forecasts on the question set, and the question set is the harness's own design. A reader should not take it to mean the system grades discretionary trades from outside sources.

The video says nothing about which questions are asked, what horizon is used, or how many predictions have been scored. I found no results table, hit rate or profit figure in the frames I read or on the repository page. The video shows no evidence that any model has been shown to have an edge. It says the tool can test for one.

## "If the model is just guessing, the system learns and stops trading": broadly right, but the mechanism is calibration blending

At 20.84 s the video says "If the model is just guessing, the system learns and stops trading." The word "learns" could suggest the system trains a neural network or retrains the model. According to the repository page, the mechanism is narrower. Model outputs are blended with the instrument's historical base rates. The blend weight is fitted on labelled outcomes, and the page says a model that adds nothing stops driving trades. Weights are refitted every 15 minutes once a group has 30 independent labelled windows. Drift detection can also set a model's weight to zero or pause trading.

So the claim holds in substance. Fitting a weight is a statistical calibration step, not the model learning to trade better, and it can only turn a model's influence down. The repository page also describes a "strict" mode in which a symbol must be statistically validated first. Until then the system logs decisions and trades only at small size. That means the bot can place small trades before a model has been shown to beat base rates. The video's phrasing of "stops trading" skips this warm-up period. The 30-window minimum also means a short run proves little, and 30 windows is a small sample for a noisy market. I could not find that sample-size caveat in the video.

## Models, status and traction: what the video leaves out

The video never names a model. The README badges and the repository page name two: Kairos (DMind, served through Minara Cloud) and Jev (TypeSafe). Frame 018 shows the Kairos row with provider "DMind via Minara Cloud," model id "DMIND/Kairos" and key "KAIROS_API_KEY." The README says you can switch between the models live and that each model's results are tracked separately. Kairos needs an API key, so the user brings an external model and its credentials. The spoken claim "it plugs fast classification models" is accurate.

The README status badge reads "under development," and the warning says interfaces, question sets and defaults change between commits. The video presents the tool as "the benchmark you need" (at 30.0 s, "this is the benchmark you need to know you are not fooling yourself"). A benchmark normally has fixed tasks so that results can be compared across models and over time. The README says the question sets change between commits, so scores from different versions may not be comparable. I would call it a forward-testing harness, which is the README's own term, not a standard benchmark.

When I fetched the page it showed 236 stars, 51 forks, 3 watchers and 218 commits, under an MIT license. The page did not show creation or commit dates, so I cannot say how old the project is or how recently it changed. The description's claim "Open source" is correct, since the license is MIT.

## Key Takeaways

- **Verified:** The harness grades each model forecast against what the market did, including forecasts that were not traded. The README in frame 006 and the repository page both say so.
- **Correction:** At 12.88 s the video says it plugs models "into live crypto markets." The README says orders go only to the Bybit testnet with simulated funds, and the host is hard-coded. Only the market data is live, from Bybit mainnet.
- **Partial correction:** At 20.84 s the video says "the system learns and stops trading." The repository describes a calibration weight, refitted every 15 minutes after 30 labelled windows, that can fall to zero. It is not model retraining, and small-size trading can occur before a symbol is validated.
- **Clarification:** The video calls the tool "the benchmark you need." The README says question sets and defaults change between commits, so it is a forward-testing harness whose scores may not compare across versions.
- **Unverified:** No frame or source I read shows any performance result, profit figure or evidence that either model has an edge. The repository's creation date and recent commit dates also could not be established.
- **Context:** The repository showed 236 stars, 51 forks and an MIT license when fetched. The video had 3,595 views and 262 saves at capture.

## Resources

- [zadescoxp/calibrain-harness, GitHub (fetched 2026-11-14)](https://github.com/zadescoxp/calibrain-harness) establishes the testnet-only venue, the Kairos and Jev models, the calibration and kill mechanisms, the MIT license and the star, fork and commit counts I cite.
