Goodbye Krake & Clio

When our older text and image models reach retirement, we like to give them a second life by open sourcing them whenever we can. Now two more of our legacy models are heading toward retirement, so we’re giving them the same send-off. You can still use them for now, but they will eventually be removed from novelai.net.

Krake

Krake is our finetune of EleutherAI’s GPT-NeoX-20B. Due to his size and somewhat temperamental behaviour, he was only ever available to our Opus subscribers.

Nowadays, barely anyone uses Krake anymore, so before long it will be time to retire him.

For ease of use, Krake has been converted from our internal format to Huggingface Safetensors, and his modules have been converted to Huggingface PEFT format. He is released under the GPL-3.0 license.

You can find your very own Krake here!

Original Krake Art by @Ainiwaffles

Clio

Clio is the first fully original large language model we pretrained from scratch on our Shoggy H100 cluster. At 3 billion parameters she runs very fast, but thanks to her very large training volume for the time, her writing outperforms much bigger models like Euterpe and Krake. She was trained on our custom Nerdstash dataset with our custom Nerdstash Tokenizer V1.

Nowadays, much stronger models like Kayra, Erato and Xialong are available on NovelAI, so before long Clio will get to enjoy a peaceful retirement.

We found an existing model class inside Huggingface Transformers that fits Clio’s architecture once an additional config flag is set, so no custom code is needed to run her. She and her modules are released under the GPL-2.0 license (not “or later”).

You can find your very own Clio here!

Original Clio Art by @Ainiwaffles

Why we’re doing this

For the sake of nostalgia, posterity, and historic preservation, we want these models to stay available long after they leave NovelAI. Enjoy!