Update README.md
Browse files
README.md
CHANGED
|
@@ -17,10 +17,12 @@ Weight compression was performed using `nncf.compress_weights` with the followin
|
|
| 17 |
* mode: **int8_asym**
|
| 18 |
* ratio: **1**
|
| 19 |
For more information on quantization, check the [OpenVINO model optimization guide](https://docs.openvino.ai/2024/openvino-workflow/model-optimization-guide/weight-compression.html).
|
|
|
|
| 20 |
## Compatibility
|
| 21 |
The provided OpenVINO™ IR model is compatible with:
|
| 22 |
* OpenVINO version 2024.5.0 and higher
|
| 23 |
* Optimum Intel 1.21.0 and higher
|
|
|
|
| 24 |
## Running Model Inference with [Optimum Intel](https://huggingface.co/docs/optimum/intel/index)
|
| 25 |
1. Install packages required for using [Optimum Intel](https://huggingface.co/docs/optimum/intel/index) integration with the OpenVINO backend:
|
| 26 |
```
|
|
@@ -30,7 +32,7 @@ pip install optimum[openvino]
|
|
| 30 |
```
|
| 31 |
from transformers import AutoProcessor
|
| 32 |
from optimum.intel.openvino import OVModelForSpeechSeq2Seq
|
| 33 |
-
model_id = "
|
| 34 |
tokenizer = AutoProcessor.from_pretrained(model_id)
|
| 35 |
model = OVModelForSpeechSeq2Seq.from_pretrained(model_id)
|
| 36 |
dataset = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation", trust_remote_code=True)
|
|
@@ -44,6 +46,7 @@ outputs = model.generate(input_features)
|
|
| 44 |
text = processor.batch_decode(outputs)[0]
|
| 45 |
print(text)
|
| 46 |
```
|
|
|
|
| 47 |
## Running Model Inference with [OpenVINO GenAI](https://github.com/openvinotoolkit/openvino.genai)
|
| 48 |
1. Install packages required for using OpenVINO GenAI.
|
| 49 |
```
|
|
@@ -54,8 +57,8 @@ pip install -U --pre --extra-index-url https://storage.openvinotoolkit.org/simpl
|
|
| 54 |
|
| 55 |
```
|
| 56 |
import huggingface_hub as hf_hub
|
| 57 |
-
model_id = "
|
| 58 |
-
model_path = "whisper-
|
| 59 |
hf_hub.snapshot_download(model_id, local_dir=model_path)
|
| 60 |
```
|
| 61 |
3. Run model inference:
|
|
|
|
| 17 |
* mode: **int8_asym**
|
| 18 |
* ratio: **1**
|
| 19 |
For more information on quantization, check the [OpenVINO model optimization guide](https://docs.openvino.ai/2024/openvino-workflow/model-optimization-guide/weight-compression.html).
|
| 20 |
+
|
| 21 |
## Compatibility
|
| 22 |
The provided OpenVINO™ IR model is compatible with:
|
| 23 |
* OpenVINO version 2024.5.0 and higher
|
| 24 |
* Optimum Intel 1.21.0 and higher
|
| 25 |
+
|
| 26 |
## Running Model Inference with [Optimum Intel](https://huggingface.co/docs/optimum/intel/index)
|
| 27 |
1. Install packages required for using [Optimum Intel](https://huggingface.co/docs/optimum/intel/index) integration with the OpenVINO backend:
|
| 28 |
```
|
|
|
|
| 32 |
```
|
| 33 |
from transformers import AutoProcessor
|
| 34 |
from optimum.intel.openvino import OVModelForSpeechSeq2Seq
|
| 35 |
+
model_id = "bweng/whisper-large-v3-turbo-int8"
|
| 36 |
tokenizer = AutoProcessor.from_pretrained(model_id)
|
| 37 |
model = OVModelForSpeechSeq2Seq.from_pretrained(model_id)
|
| 38 |
dataset = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation", trust_remote_code=True)
|
|
|
|
| 46 |
text = processor.batch_decode(outputs)[0]
|
| 47 |
print(text)
|
| 48 |
```
|
| 49 |
+
|
| 50 |
## Running Model Inference with [OpenVINO GenAI](https://github.com/openvinotoolkit/openvino.genai)
|
| 51 |
1. Install packages required for using OpenVINO GenAI.
|
| 52 |
```
|
|
|
|
| 57 |
|
| 58 |
```
|
| 59 |
import huggingface_hub as hf_hub
|
| 60 |
+
model_id = "bweng/whisper-large-v3-turbo-int8"
|
| 61 |
+
model_path = "whisper-large-v3-turbo-int8"
|
| 62 |
hf_hub.snapshot_download(model_id, local_dir=model_path)
|
| 63 |
```
|
| 64 |
3. Run model inference:
|