Anyone was able to run it succesfully?

[yarodevuci]

I want to know what is look like when running? What you can do with it? Checkpoints are not finetuned or anything.. so what to do next with this code and trained data?

[LagPixelLOL]

ran it on 8x a100 80g with the code in this repo(no modification, i just added a loop to get input from terminal), using 524GB of vram during single batch inference with nearly no context(10~100 tokens input), speed is only 7 tokens per second ish
even tho they said its a base model, but funny it can get into a role of assistant pretty well with just User: xxx\n\nAssistant: format, which isnt like how llama 2 base model behaved, still the dialogue ability is very very bad, so i think for this model to be useful it still needs more fine tunes
also if asked what the knowledge cut off is, it will say 2021/9, and sometimes it thinks it's made by openai, idk if this is because of the sheer amount of shared chatgpt chatlog on the internet or its because they trained grok on openai's models' outputs

[iayub02]

@LagPixelLOL We are trying with RunPod Pytorch 2.1, 8 x A100 SXM 80GB and 256 vCPU 2002 GB RAM.

[iayub02]

What kind of performance are you getting now on PyTorch?

[LagPixelLOL]

@iayub02 I just found someone who made a full huggingface transformers implementation and it's way better than mine, I'll test on this instead https://huggingface.co/keyfan/grok-1-hf

[Cpmcharlesandrew]

pip install dm-haiku
pip install jax jaxlib
On Windows Run the following first to install the prerequisites.

[iamdeepakgit]

Yes it's running good and prefect 😉

[yarodevuci]

Has anyone tried on Mac M3 with 128GB shared memory?

[MrRaja23]

Assuming it needs 500gb vram in its current state that would go the same for ram no?

[dominiquegarmier]

@LagPixelLOL

need to port it to pytorch somehow

I created a pytorch implementation: https://github.com/DominiqueGarmier/grok-pytorch, you can't really use it at this point but maybe it would be a good starting point.

[LagPixelLOL]

apparently someone already made a full huggingface implementation of grok, here: https://huggingface.co/keyfan/grok-1-hf

[yarodevuci]

will it work on one video card?

[dominiquegarmier]

will it work on one video card?

no, not even maybe

[dominiquegarmier]

apparently someone already made a full huggingface implementation of grok, here: https://huggingface.co/keyfan/grok-1-hf

hmm I looked at the code and it appears to just be copy of Mixtral with "Mixtral" replaced with "Grok", but maybe they edited some things to make it compatible.

[LagPixelLOL]

@dominiquegarmier Yeah but they did modify some of the code to adapt it to Grok-1, I'm currently trying to convert it to safentensors but IDK why it's failing nvm figured it out.

[andy108369]

I finally was able to run it (though PyTorch version of it https://huggingface.co/hpcai-tech/grok-1 ) on Akash Network.
Details akash-network/awesome-akash#507 (comment)

[iayub02]

Is it consuming all 8 GPU when you ask a question? We got it working but it seems super inefficient. George

…

On Fri, Mar 22, 2024 at 1:25 PM Andrey Arapov ***@***.***> wrote: I finally was able to run it (though PyTorch version of it https://huggingface.co/hpcai-tech/grok-1 ) on Akash Network. Details akash-network/awesome-akash#507 (comment) <akash-network/awesome-akash#507 (comment)> image.png (view on web) <https://github.com/xai-org/grok-1/assets/107317698/70909402-a77b-41cd-bcc6-21b9a571344a> — Reply to this email directly, view it on GitHub <#168 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AR5KMMHZLTEJR5RPZGLIXXLYZRSQZAVCNFSM6AAAAABE4O2YYGVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DQOBRGAZTE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[ateeq-pk]

I have made the model successfully working on 8 A100 80GB, but the response from model was not satisfactory. I am going to try to do second time and more properly set query.

Instructions: #246

[yh-yao]

https://www.fedml.ai/models/941?owner=xai-org Try here. I just use it for the completion task. The current Grok-1 does not support chat template. It is much better now.

[Xmabel112Lubi]

Hello can me someone explain what is that all thanks Odoslané z aplikácie Outlook pre Android<https://aka.ms/AAb9ysg>

…

________________________________ From: iamdeepakgit ***@***.***> Sent: Tuesday, June 25, 2024 6:06:09 AM To: xai-org/grok-1 ***@***.***> Cc: Subscribed ***@***.***> Subject: Re: [xai-org/grok-1] Anyone was able to run it succesfully? (Discussion #168) Yes it's running good and prefect 😉 — Reply to this email directly, view it on GitHub<#168 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BITFOVUQYN7AOITQJKFDKWLZJDUDDAVCNFSM6AAAAABE4O2YYGVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TQNRWG4ZTE>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>