Skip to content

SewoongLab/byte-sampler

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

11 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Byte sampling

This is the implementation of Sampling from Your Language Model One Byte at a Time.

Installation

Clone the repository. Install uv and then do:

uv run ipython

You can access this project's functionality via the byte_sampler module.

Usage

You can then sample from a model byte-wise:

from byte_sampling import *

# load in a model
bc = ByteConditioning("meta-llama/Llama-3.1-8B")

# sample a continuation with a QA formatted prompt.
generate_batched(
    BytewiseQAFactory(bc),
    ["What is your favorite flavor of ice cream?"],
    stop_strings=('\n',),
    display=True
)

About

Convert your language model into a byte-level one!

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages