Instructions to use sapinsapin/llama31-8b-balitanlp-cpt with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use sapinsapin/llama31-8b-balitanlp-cpt with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="sapinsapin/llama31-8b-balitanlp-cpt")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("sapinsapin/llama31-8b-balitanlp-cpt")
model = AutoModelForCausalLM.from_pretrained("sapinsapin/llama31-8b-balitanlp-cpt", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use sapinsapin/llama31-8b-balitanlp-cpt with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "sapinsapin/llama31-8b-balitanlp-cpt"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sapinsapin/llama31-8b-balitanlp-cpt",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/sapinsapin/llama31-8b-balitanlp-cpt

SGLang

How to use sapinsapin/llama31-8b-balitanlp-cpt with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "sapinsapin/llama31-8b-balitanlp-cpt" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sapinsapin/llama31-8b-balitanlp-cpt",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "sapinsapin/llama31-8b-balitanlp-cpt" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sapinsapin/llama31-8b-balitanlp-cpt",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use sapinsapin/llama31-8b-balitanlp-cpt with Docker Model Runner:
```
docker model run hf.co/sapinsapin/llama31-8b-balitanlp-cpt
```

llama31-8b-balitanlp-cpt

This model is a continuously pretrained version of meta-llama/Llama-3.1-8B on an unknown dataset.

Model description

Continuous pretraining on Filipino news articles from https://huggingface.co/datasets/LanceBunag/BalitaNLP

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 3e-05
train_batch_size: 1
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 8
total_train_batch_size: 8
optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 10
training_steps: 100

Training results

Example

Before continuous pretraining:

After:

Test #	Prompt	Max Tokens	Model	Completion (Truncated)	Finish Reason	Prompt Tokens	Completion Tokens	Total Tokens
1	`Kumusta ka?`	50	internetoftim/llama31-8b-balitanlp-cpt	"Ano ang isang magandang kanta na maaaring maging..."	length	6	50	56
2	`P900-M RELIEF GOODS INIHANDA NG NDRRMC PARA SA BAGYO`	100	internetoftim/llama31-8b-balitanlp-cpt	"NA TUMATAAS SA HAWAAN NG BISAYA – NDRRMC ..."	length	22	100	122
3	`Tony Labrusca`	500	internetoftim/llama31-8b-balitanlp-cpt	"gagahin ang kanyang mga fan sa Instagram kung paano na raw siya ngayon..."	stop	6	168	174

📌 API Test Notes

1️⃣ Request 1

curl -X POST localhost:8080/v1/completions
-H "Content-Type: application/json"
-d '{ "model": "internetoftim/llama31-8b-balitanlp-cpt", "prompt": "Kumusta ka?", "max_tokens": 50, "temperature": 0.7 }'

Result 1 { "id": "cmpl-f10e26c5290a460b89156bc373b5d6f6", "object": "text_completion", "created": 1765715251, "model": "internetoftim/llama31-8b-balitanlp-cpt", "choices": [ { "index": 0, "text": " Ano ang isang magandang kanta na ang maaaring maging ang ikawang single na sisimulan mo ngayong taon? Sa halip na manood ka lang ng mga kanta ng ibang artista,", "logprobs": null, "finish_reason": "length", "stop_reason": null, "token_ids": null, "prompt_logprobs": null, "prompt_token_ids": null } ], "service_tier": null, "system_fingerprint": null, "usage": { "prompt_tokens": 6, "total_tokens": 56, "completion_tokens": 50, "prompt_tokens_details": null }, "kv_transfer_params": null }

2️⃣ Request 2

curl -X POST localhost:8080/v1/completions
-H "Content-Type: application/json"
-d '{ "model": "internetoftim/llama31-8b-balitanlp-cpt", "prompt": "P900-M RELIEF GOODS INIHANDA NG NDRRMC PARA SA BAGYO", "max_tokens": 100, "temperature": 0.7 }'

Result 2 { "id": "cmpl-106c32f72d464409a9f7d8597bded8b6", "object": "text_completion", "created": 1765715647, "model": "internetoftim/llama31-8b-balitanlp-cpt", "choices": [ { "index": 0, "text": " NA TUMATAAS SA HAWAAN NG BISAYA – NDRRMC (NATIONAL DISASTER RISK REDUCTION AND MANAGEMENT COUNCIL) ay inihanda na ang P900-Million worth ng relief goods na gagamitin para sa mga nasalanta ng bagyo sa Eastern Visayas, Central Visayas at Western Visayas, na tumataas sa Dagat sa Timog China at patungong Luzon sa loob ng Linggo,", "logprobs": null, "finish_reason": "length", "stop_reason": null, "token_ids": null, "prompt_logprobs": null, "prompt_token_ids": null } ], "service_tier": null, "system_fingerprint": null, "usage": { "prompt_tokens": 22, "total_tokens": 122, "completion_tokens": 100, "prompt_tokens_details": null }, "kv_transfer_params": null }

🔹 Request 3

🔹 Result

{ "id": "cmpl-106c32f72d464409a9f7d8597bded8b6", "object": "text_completion", "created": 1765715647, "model": "internetoftim/llama31-8b-balitanlp-cpt", "choices": [ { "index": 0, "text": " NA TUMATAAS SA HAWAAN NG BISAYA – NDRRMC (NATIONAL DISASTER RISK REDUCTION AND MANAGEMENT COUNCIL) ay inihanda na ang P900-Million worth ng relief goods na gagamitin para sa mga nasalanta ng bagyo sa Eastern Visayas, Central Visayas at Western Visayas, na tumataas sa Dagat sa Timog China at patungong Luzon sa loob ng Linggo,", "logprobs": null, "finish_reason": "length", "stop_reason": null, "token_ids": null, "prompt_logprobs": null, "prompt_token_ids": null } ], "service_tier": null, "system_fingerprint": null, "usage": { "prompt_tokens": 22, "total_tokens": 122, "completion_tokens": 100, "prompt_tokens_details": null }, "kv_transfer_params": null }

####TL;DR This is pre-trained base model, it's not quite useful but you can see it learned some Filipino. Next would be to do instruction tuning.

Framework versions

Transformers 4.57.3
Pytorch 2.10.0a0+b558c986e8.nv25.11
Datasets 4.4.1
Tokenizers 0.22.1

Downloads last month: 5

Safetensors

Model size

8B params

Tensor type

BF16

Model tree for sapinsapin/llama31-8b-balitanlp-cpt

Base model

meta-llama/Llama-3.1-8B

Finetuned

(1,501)

this model

sapinsapin
/

llama31-8b-balitanlp-cpt