A small shelf.
Three different jobs.
Lawrence asked whether an agent could literally fine-tune a model on an old writer. We chose Stevenson, prepared six books and gave each a role. Four supplied prose for learning; two whole works stayed outside the weight updates.
“Held out” means held out from this training. The starting models may already have encountered these books. The test work was reused across both experiments.
Most of the model stays still.
A small part learns.
The agent downloaded a public base model, prepared the text, saved its first responses and ran bounded training on the Mac. It learned a small adapter while keeping the original weights frozen. This adapted Qwen; it did not alter Astra or train a frontier model from scratch.
Learning, then a plateau.
At intervals, the agent checked the model against An Inland Voyage. For the 3B run, the best result on that book chose the saved checkpoint. The separate test book improved less. Better prediction and better storytelling are different questions.
Validation book · lower loss is better
Read the exact points
| Completed updates | Validation loss |
|---|
The 3B run · a real 90-minute training allowance
Two attempts stopped before any weight update because disk space was low. Cleanup and a restart made room. The clock includes worker startup, training and validation; setup, downloads and separate before/after measurements are outside it. Peak model-runtime allocation: 3.07 GB, not total Mac memory.
A voice is easier
than a world.
The bigger model sounds more consistently like literary prose after training. Four baseline continuations that became exam questions now stay in prose. But the story still loses the locked box, the ferryman’s son and the reason for stopping the engine. Here are three short, unedited glimpses.
Read this complete continuation
Inspect any of the twelve larger-model pairs
Before training
After training
These are reading judgments, not scored coherence tests. Identical prompts, seeds and generation settings make changes inspectable. They do not establish a general improvement, a specifically Stevenson-like voice, or originality.
Ardnamurchan.
A new opening, after the measured trials · unedited excerpt · full generation took about 14 seconds
05 / What opens up
We can run the experiment.
Then run a better one.
The surprising result is practical. A person could set the question, writer and resource limit, then work with an agent through corpus preparation, real weight updates, evaluation and a reusable saved model. The prose was uneven; the experimental capability was real.
That creates a way to follow advances in open models, architectures and training tools—even with a lag. When a useful development becomes accessible on affordable hardware, we can try it against a known baseline. Compatibility, cost and better results still have to be demonstrated each time.
Enough detail to inspect it.
The page reads a bundled evidence file. Its graphs and switches display saved results; they do not run a model or accept prompts. All twelve 3B pairs are included. Source hashes and preparation details identify the underlying records.
Download the curated evidence JSON ↓
No model weights are distributed here. The base may have seen Stevenson during pretraining; there was no matched other-author control. The repeated test book and small sample set limit the conclusion.
Books, models and method.
- Qwen2.5-3B model card and the four-bit MLX version.
- MLX’s LoRA training method: small learned adapters on a pretrained model.
- Qwen research licence; this is a noncommercial experiment.
- The six original Gutenberg texts are linked from the book selector above.