Skip to content

Commit df993c4

Browse files
committed
Update the README
1 parent ded2690 commit df993c4

File tree

1 file changed

+177
-115
lines changed

1 file changed

+177
-115
lines changed

README.md

Lines changed: 177 additions & 115 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,8 @@
77
Fast and reliable neural text generation.
88

99
[Install](#installation)
10-
[Prompting primitives](#prompting)
1110
[Guided generation](#guided-generation)
11+
[Prompting primitives](#prompting)
1212
[Examples](#examples)
1313
[Stay tuned](#stay-tuned-for)
1414

@@ -19,7 +19,9 @@ more flexible replacement for the `generate` method in the
1919
[transformers](https://github.com/huggingface/transformers) library.
2020

2121
**Outlines** 〰 helps developers *guide text generation* to build robust
22-
interfaces with external systems.
22+
interfaces with external systems. Provides generation methods that
23+
guarantee that the output will match a regular expressions, or follow
24+
a JSON schema.
2325

2426
**Outlines** 〰 provides *robust prompting primitives* that separate the prompting
2527
from the execution logic and lead to simple implementations of few-shot
@@ -37,16 +39,17 @@ via the next-token logits. It can be used with API-based models as well.
3739

3840
- [x] 🖍️Simple and powerful prompting primitives based on the [Jinja templating engine](https://jinja.palletsprojects.com/)
3941
- [x] 🚄 Guided generation, including multiple choice, type constraints and dynamic stopping
40-
- [x] ⚡ Fast regex-guided generation
42+
- [x] ⚡ Fast [regex-guided generation](#efficient-regex-guided-generation)
43+
- [x] 🔥 Fast [JSON generation](#efficient-json-generation-following-a-pydantic-model) following a JSON schema or a Pydantic model
4144
- [x] 🐍 Interleave completions with loops, conditionals, and custom Python functions
4245
- [x] 💾 Caching of generations
4346
- [x] 🤗 Integration with HuggingFace's `transformers` models
4447

48+
Outlines 〰 has new releases and features coming every week! Make sure to ⭐ star and 👀 watch this repository to stay up to date.
4549

4650
## Stay tuned for
4751

4852
- Context-Free Grammar guided generation ([#178](https://github.com/normal-computing/outlines/pull/178));
49-
- Generate JSON with a defined structure ([#140](https://github.com/normal-computing/outlines/pull/140))
5053
- Prompt-token alignment so you don't have to think about tokenization details ([#201](https://github.com/normal-computing/outlines/pull/201))
5154
- An infilling DSL ([#182](https://github.com/normal-computing/outlines/issues/182))
5255

@@ -61,6 +64,172 @@ You can follow [@NormalComputing](https://twitter.com/NormalComputing), [@remilo
6164
pip install outlines
6265
```
6366

67+
68+
## Guided generation
69+
70+
The first step towards reliability of systems that include large language models
71+
is to ensure that there is a well-defined interface between their output and
72+
user-defined code. **Outlines** provides ways to control the generation of
73+
language models to make their output more predictable.
74+
75+
### Early stopping
76+
77+
You can stop the generation after a given sequence has been found:
78+
79+
``` python
80+
import outlines.text.generate as generate
81+
import outlines.models as models
82+
83+
model = models.transformers("gpt2")
84+
answer = generate.continuation(model, stop=["."])("Tell me a one-sentence joke.")
85+
```
86+
87+
### Multiple choices
88+
89+
You can reduce the completion to a choice between multiple possibilities:
90+
91+
``` python
92+
import outlines.text.generate as generate
93+
import outlines.models as models
94+
95+
model = models.transformers("gpt2")
96+
97+
prompt = labelling("Just awesome", examples)
98+
answer = generate.choice(model, ["Positive", "Negative"])(prompt)
99+
```
100+
101+
### Type constraint
102+
103+
You can instruct the model to only return integers or floats:
104+
105+
106+
``` python
107+
import outlines.text.generate as generate
108+
import outlines.models as models
109+
110+
model = models.transformers("gpt2")
111+
112+
prompt = "1+1="
113+
answer = generate.integer(model)(prompt)
114+
115+
prompt = "sqrt(2)="
116+
answer = generate.float(model)(prompt)
117+
```
118+
119+
### Efficient regex-guided generation
120+
121+
Outlines also comes with fast regex-guided generation. In fact, the `choice`,
122+
`integer` and `float` functions above all use regex-guided generation under the
123+
hood:
124+
125+
``` python
126+
import outlines.models as models
127+
import outlines.text.generate as generate
128+
129+
130+
model = models.transformers("gpt2-medium")
131+
132+
prompt = "Is 1+1=2? "
133+
unguided = generate.continuation(model, max_tokens=30)(prompt)
134+
guided = generate.regex(model, r"\s*([Yy]es|[Nn]o|[Nn]ever|[Aa]lways)", max_tokens=30)(
135+
prompt
136+
)
137+
138+
print(unguided)
139+
# Is 1+1=2?
140+
#
141+
# This is probably the most perplexing question.
142+
# As I said in one of my articles describing how
143+
# I call 2 and 1, there isn't
144+
145+
print(guided)
146+
# Is 1+1=2? Always
147+
```
148+
149+
``` python
150+
import outlines.models as models
151+
import outlines.text.generate as generate
152+
153+
154+
model = models.transformers("gpt2-medium")
155+
156+
prompt = "What is the IP address of the Google DNS servers? "
157+
unguided = generate.continuation(model, max_tokens=30)(prompt)
158+
guided = generate.regex(
159+
model,
160+
r"((25[0-5]|2[0-4]\d|[01]?\d\d?)\.){3}(25[0-5]|2[0-4]\d|[01]?\d\d?)",
161+
max_tokens=30,
162+
)(prompt)
163+
164+
print(unguided)
165+
# What is the IP address of the Google DNS servers?
166+
#
167+
# Passive DNS servers are at DNS servers that are private.
168+
# In other words, both IP servers are private. The database
169+
# does not contain Chelsea Manning
170+
171+
print(guided)
172+
# What is the IP address of the Google DNS servers?
173+
# 2.2.6.1
174+
```
175+
176+
Unlike other libraries, regex-guided generation in Outlines is almost as fast
177+
as non-guided generation.
178+
179+
### Efficient JSON generation following a Pydantic model
180+
181+
Outlines 〰 allows to guide the generation process so the output is *guaranteed* to follow a [JSON schema](https://json-schema.org/) or [Pydantic model](https://docs.pydantic.dev/latest/):
182+
183+
```python
184+
from typing import List
185+
from enum import Enum
186+
from pydantic import BaseModel, constr
187+
188+
import outlines.models as models
189+
import outlines.text.generate as generate
190+
191+
192+
class Weapon(str, Enum):
193+
sword = "sword"
194+
axe = "axe"
195+
mace = "mace"
196+
spear = "spear"
197+
bow = "bow"
198+
crossbow = "crossbow"
199+
200+
201+
class Armor(str, Enum):
202+
leather = "leather"
203+
chainmail = "chainmail"
204+
plate = "plate"
205+
206+
207+
class Character(BaseModel):
208+
name: constr(max_length=10)
209+
age: int
210+
armor: Armor
211+
weapon: Weapon
212+
strength: int
213+
214+
215+
model = models.transformers("gpt2")
216+
sequence = generate.json(model, Character)("Give me a character description")
217+
print(sequence)
218+
# {
219+
# "name": "ranbelt",
220+
# "age": 26,
221+
# "armor": "chainmail",
222+
# "weapon": "bow",
223+
# "strength": 5
224+
# }
225+
226+
parsed = Character.model_validate_json(sequence)
227+
print(parsed)
228+
# name='ranbelt' age=26 armor=<Armor.chainmail: 'chainmail'> weapon=<Weapon.bow: 'bow'> strength=5
229+
```
230+
231+
The method works with union types, optional types, arrays, nested schemas, etc. Some field constraints are [not supported yet](https://github.com/normal-computing/outlines/issues/215), but everything else should work.
232+
64233
## Prompting
65234

66235
Writing prompts by concatenating strings in pure Python quickly becomes
@@ -184,117 +353,6 @@ With these prompting primitives **Outlines** makes building agents like
184353
Agent](https://huggingface.co/docs/transformers/transformers_agents) easier by
185354
removing boilerplate prompting code.
186355

187-
## Guided generation
188-
189-
The first step towards reliability of systems that include large language models
190-
is to ensure that there is a well-defined interface between their output and
191-
user-defined code. **Outlines** provides ways to control the generation of
192-
language models to make their output more predictable.
193-
194-
### Early stopping
195-
196-
You can stop the generation after a given sequence has been found:
197-
198-
``` python
199-
import outlines.text.generate as generate
200-
import outlines.models as models
201-
202-
model = models.transformers("gpt2")
203-
answer = generate.continuation(model, stop=["."])("Tell me a one-sentence joke.")
204-
```
205-
206-
### Multiple choices
207-
208-
You can reduce the completion to a choice between multiple possibilities:
209-
210-
``` python
211-
import outlines.text.generate as generate
212-
import outlines.models as models
213-
214-
model = models.transformers("gpt2")
215-
216-
prompt = labelling("Just awesome", examples)
217-
answer = generate.choice(model, ["Positive", "Negative"])(prompt)
218-
```
219-
220-
### Type constraint
221-
222-
You can instruct the model to only return integers or floats:
223-
224-
225-
``` python
226-
import outlines.text.generate as generate
227-
import outlines.models as models
228-
229-
model = models.transformers("gpt2")
230-
231-
prompt = "1+1="
232-
answer = generate.integer(model)(prompt)
233-
234-
prompt = "sqrt(2)="
235-
answer = generate.float(model)(prompt)
236-
```
237-
238-
### Efficient regex-guided generation
239-
240-
Outlines also comes with fast regex-guided generation. In fact, the `choice`,
241-
`integer` and `float` functions above all use regex-guided generation under the
242-
hood:
243-
244-
``` python
245-
import outlines.models as models
246-
import outlines.text.generate as generate
247-
248-
249-
model = models.transformers("gpt2-medium")
250-
251-
prompt = "Is 1+1=2? "
252-
unguided = generate.continuation(model, max_tokens=30)(prompt)
253-
guided = generate.regex(model, r"\s*([Yy]es|[Nn]o|[Nn]ever|[Aa]lways)", max_tokens=30)(
254-
prompt
255-
)
256-
257-
print(unguided)
258-
# Is 1+1=2?
259-
#
260-
# This is probably the most perplexing question.
261-
# As I said in one of my articles describing how
262-
# I call 2 and 1, there isn't
263-
264-
print(guided)
265-
# Is 1+1=2? Always
266-
```
267-
268-
``` python
269-
import outlines.models as models
270-
import outlines.text.generate as generate
271-
272-
273-
model = models.transformers("gpt2-medium")
274-
275-
prompt = "What is the IP address of the Google DNS servers? "
276-
unguided = generate.continuation(model, max_tokens=30)(prompt)
277-
guided = generate.regex(
278-
model,
279-
r"((25[0-5]|2[0-4]\d|[01]?\d\d?)\.){3}(25[0-5]|2[0-4]\d|[01]?\d\d?)",
280-
max_tokens=30,
281-
)(prompt)
282-
283-
print(unguided)
284-
# What is the IP address of the Google DNS servers?
285-
#
286-
# Passive DNS servers are at DNS servers that are private.
287-
# In other words, both IP servers are private. The database
288-
# does not contain Chelsea Manning
289-
290-
print(guided)
291-
# What is the IP address of the Google DNS servers?
292-
# 2.2.6.1
293-
```
294-
295-
Unlike other libraries, regex-guided generation in Outlines is almost as fast
296-
as non-guided generation.
297-
298356
## Contributing
299357

300358
### What contributions?
@@ -333,3 +391,7 @@ Do not hesitate to open a draft PR before your contribution is ready, especially
333391
year={2023}
334392
}
335393
```
394+
395+
## License
396+
397+
Outlines is open-source and licensed under the [Apache License 2.0](LICENSE).

0 commit comments

Comments
 (0)