Skip to content

Commit 3bd219a

Browse files
authored
README: Subcommand presentation and improved Server instructions (pytorch#936)
1 parent 83676a3 commit 3bd219a

File tree

1 file changed

+43
-32
lines changed

1 file changed

+43
-32
lines changed

README.md

Lines changed: 43 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -84,24 +84,35 @@ python3 torchchat.py download llama3
8484
<details>
8585
<summary>Additional Model Inventory Management Commands</summary>
8686

87+
### List
88+
This subcommands shows the available models
89+
```bash
90+
python3 torchchat.py list
8791
```
88-
# View available models
92+
93+
### Where
94+
This subcommands shows location of a particular model.
95+
```bash
8996
python3 torchchat.py list
97+
```
98+
This is useful in scripts when you do not want to hard-code paths
9099

91-
# Query the location of a particular model
92-
# This is useful in scripts when you do not want to hard-code paths
93-
python3 torchchat.py where llama3
94100

95-
# Remove downloaded models
101+
### Remove
102+
This subcommands removes the specified model
103+
```bash
96104
python3 torchchat.py remove llama3
97105
```
106+
98107
More information about these commands can be found by adding the `--help` option.
99108

100109
</details>
101110

102111

103112
## Running via PyTorch / Python
104-
[Follow the installation steps if you haven't already.](#installation)
113+
114+
> [!TIP]
115+
> For more information about these commands, please refer to the `--help` menu.
105116
106117
### Chat
107118
This mode allows you to chat with an LLM in an interactive fashion.
@@ -119,11 +130,29 @@ This mode generates text based on an input prompt.
119130
python3 torchchat.py generate llama3 --prompt "write me a story about a boy and his bear"
120131
```
121132

133+
### Browser
134+
This mode allows you to chat with the model using a UI in your browser
135+
Running the command automatically open a tab in your browser.
136+
137+
[skip default]: begin
138+
139+
```
140+
streamlit run torchchat.py -- browser llama3
141+
```
142+
143+
[skip default]: end
122144

123145
### Server
124146
**Note: This feature is still in progress and not all endpoints are working ATM**
125-
126-
This mode gives a REST API that matches the OpenAI API spec for interacting with a model
147+
148+
149+
<details>
150+
<summary>This mode gives a REST API that matches the OpenAI API spec for interacting with a model</summary>
151+
152+
To test out the REST API, **you'll need 2 terminals**: one to host the server, and one to send the request.
153+
154+
155+
In one terminal, kick off the server
127156

128157
[skip default]: begin
129158

@@ -132,15 +161,9 @@ python3 torchchat.py server llama3
132161
```
133162
[skip default]: end
134163

135-
136-
Open another terminal window to interact with the API using curl
137-
138-
> [!NOTE]
139-
> Depending on the model configuration, this query might take a few minutes
140-
> to respond
141-
164+
In the other terminal window, interact with the API using curl. Depending on the model configuration, this query might take a few minutes to respond
142165

143-
**Sample Input + Output**
166+
**Example Input + Output**
144167

145168
```
146169
curl http://127.0.0.1:5000/chat \
@@ -158,24 +181,12 @@ curl http://127.0.0.1:5000/chat \
158181
}
159182
]
160183
}'
161-
162-
{"response":" I'm a software developer with a passion for building innovative and user-friendly applications. I have experience in developing web and mobile applications using various technologies such as Java, Python, and JavaScript. I'm always looking for new challenges and opportunities to learn and grow as a developer.\n\nIn my free time, I enjoy reading books on computer science and programming, as well as experimenting with new technologies and techniques. I'm also interested in machine learning and artificial intelligence, and I'm always looking for ways to apply these concepts to real-world problems.\n\nI'm excited to be a part of the developer community and to have the opportunity to share my knowledge and experience with others. I'm always happy to help with any questions or problems you may have, and I'm looking forward to learning from you as well.\n\nThank you for visiting my profile! I hope you find my information helpful and interesting. If you have any questions or would like to discuss any topics, please feel free to reach out to me. I"}
163184
```
164-
165-
### Browser
166-
This mode allows you to chat with the model using a UI in your browser
167-
Running the command automatically open a tab in your browser.
168-
169-
[skip default]: begin
170-
171185
```
172-
streamlit run torchchat.py -- browser llama3
186+
{"response":" I'm a software developer with a passion for building innovative and user-friendly applications. I have experience in developing web and mobile applications using various technologies such as Java, Python, and JavaScript. I'm always looking for new challenges and opportunities to learn and grow as a developer.\n\nIn my free time, I enjoy reading books on computer science and programming, as well as experimenting with new technologies and techniques. I'm also interested in machine learning and artificial intelligence, and I'm always looking for ways to apply these concepts to real-world problems.\n\nI'm excited to be a part of the developer community and to have the opportunity to share my knowledge and experience with others. I'm always happy to help with any questions or problems you may have, and I'm looking forward to learning from you as well.\n\nThank you for visiting my profile! I hope you find my information helpful and interesting. If you have any questions or would like to discuss any topics, please feel free to reach out to me. I"}
173187
```
174188

175-
[skip default]: end
176-
177-
> [!TIP]
178-
> For more information about these commands, please refer to the `--help` menu.
189+
</details>
179190

180191

181192
## Desktop/Server Execution
@@ -449,7 +460,7 @@ you can perform the example commands with any of these models.
449460
450461
## Design Principles
451462
452-
torchchat embodies PyTorch’s design philosophy [[details](https://pytorch.org/docs/stable/community/design.html)], especially "usability over everything else".
463+
torchchat embodies PyTorch’s design philosophy [details](https://pytorch.org/docs/stable/community/design.html), especially "usability over everything else".
453464
454465
### Native PyTorch
455466
@@ -490,7 +501,7 @@ link provided in the error to get access.
490501
If `./scripts/install_et.sh` fails with an error like `Building wheel for executorch (pyproject.toml) did not run successfully` It's possible that it's linking to an older version of pytorch installed some other way like via homebrew. You can break the link by uninstalling other versions such as `brew uninstall pytorch` Note: You may break something that depends on this, so be aware.
491502
492503
493-
### Disclaimer
504+
## Disclaimer
494505
The torchchat Repository Content is provided without any guarantees
495506
about performance or compatibility. In particular, torchchat makes
496507
available model architectures written in Python for PyTorch that may

0 commit comments

Comments
 (0)