Hugging Face's logo

flexitok
/
maddition_hun_Latn_360

Hungarian
tokenizer
bpe
flexitok
fineweb2
Model card Files
xet
 Community
maddition_hun_Latn_360

Commit History

  1. Upload maddition_360_held_out/flexitok--maddition_hun_Latn_360_overlap.json with huggingface_hub

    gsaltintas
    Verified signature
    d48d896

  2. Upload maddition_360_held_out/participating_tokenizers.json with huggingface_hub

    gsaltintas
    Verified signature
    84b451c

  3. Upload maddition_360_held_out/flexitok--maddition_hun_Latn_360.yaml with huggingface_hub

    gsaltintas
    Verified signature
    b546f06

  4. Upload maddition_360_held_out/flexitok--maddition_hun_Latn_360_info.json with huggingface_hub

    gsaltintas
    Verified signature
    e1da19f

  5. Upload maddition_360_held_out/flexitok--maddition_hun_Latn_360_vocab.json with huggingface_hub

    gsaltintas
    Verified signature
    076421e

  6. Upload maddition_360_held_out/flexitok--maddition_hun_Latn_360_super_mapping.json with huggingface_hub

    gsaltintas
    Verified signature
    6430449

  1. Upload maddition_360_v2/flexitok--maddition_hun_Latn_360_overlap.json with huggingface_hub

    gsaltintas
    Verified signature
    ae22e40

  2. Upload maddition_360_v2/participating_tokenizers.json with huggingface_hub

    gsaltintas
    Verified signature
    b26294d

  3. Upload maddition_360_v2/flexitok--maddition_hun_Latn_360.yaml with huggingface_hub

    gsaltintas
    Verified signature
    6713a58

  4. Upload maddition_360_v2/flexitok--maddition_hun_Latn_360_info.json with huggingface_hub

    gsaltintas
    Verified signature
    8156032

  5. Upload maddition_360_v2/flexitok--maddition_hun_Latn_360_vocab.json with huggingface_hub

    gsaltintas
    Verified signature
    62912ec

  6. Upload maddition_360_v2/flexitok--maddition_hun_Latn_360_super_mapping.json with huggingface_hub

    gsaltintas
    Verified signature
    0b52987

  1. Upload maddition_360/flexitok--maddition_hun_Latn_360_overlap.json with huggingface_hub

    gsaltintas
    Verified signature
    c2137ac

  2. Upload maddition_360/participating_tokenizers.json with huggingface_hub

    gsaltintas
    Verified signature
    ea6b859

  3. Upload maddition_360/flexitok--maddition_hun_Latn_360.yaml with huggingface_hub

    gsaltintas
    Verified signature
    3597788

  4. Upload maddition_360/flexitok--maddition_hun_Latn_360_info.json with huggingface_hub

    gsaltintas
    Verified signature
    549d93c

  5. Upload maddition_360/flexitok--maddition_hun_Latn_360_vocab.json with huggingface_hub

    gsaltintas
    Verified signature
    02fb709

  6. Upload maddition_360/flexitok--maddition_hun_Latn_360_super_mapping.json with huggingface_hub

    gsaltintas
    Verified signature
    f9adda8

  7. Upload folder using huggingface_hub

    gsaltintas
    Verified signature
    74cbac3

  8. initial commit

    gsaltintas
    Verified signature
    78f0e80