INSAIT-Institute
/

MamayLM-Gemma-2-9B-IT-v0.1

Text Generation

text-generation-inference

Model card Files Files and versions

hannayukhymenko commited on Jun 17

Commit

c6f0a6d

·

verified ·

1 Parent(s): b094bbe

remove vllm eager attn

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -149,7 +149,7 @@ sampling_params = SamplingParams(
 llm = LLM(
     model="INSAIT-Institute/MamayLM-Gemma-2-9B-IT-v0.1",
     dtype="bfloat16",
-    enforce_eager=True
 )
 messages = [
     {"role": "user", "content": "Хто такий Козак Мамай?"},

 llm = LLM(
     model="INSAIT-Institute/MamayLM-Gemma-2-9B-IT-v0.1",
     dtype="bfloat16",
+    # enforce_eager=True
 )
 messages = [
     {"role": "user", "content": "Хто такий Козак Мамай?"},