--- language: - en license: apache-2.0 tags: - sentence-transformers - sparse-encoder - sparse - splade - generated_from_trainer - dataset_size:99000 - loss:SpladeLoss - loss:SparseMultipleNegativesRankingLoss - loss:FlopsLoss base_model: distilbert/distilbert-base-uncased widget: - text: 'The term emergent literacy signals a belief that, in a literate society, young children even one and two year olds, are in the process of becoming literate”. ... Gray (1956:21) notes: Functional literacy is used for the training of adults to ''meet independently the reading and writing demands placed on them''.' - text: Rey is seemingly confirmed as being The Chosen One per a quote by a Lucasfilm production designer who worked on The Rise of Skywalker. - text: are union gun safes fireproof? - text: Fruit is an essential part of a healthy diet — and may aid weight loss. Most fruits are low in calories while high in nutrients and fiber, which can boost your fullness. Keep in mind that it's best to eat fruits whole rather than juiced. What's more, simply eating fruit is not the key to weight loss. - text: Treatment of suspected bacterial infection is with antibiotics, such as amoxicillin/clavulanate or doxycycline, given for 5 to 7 days for acute sinusitis and for up to 6 weeks for chronic sinusitis. datasets: - sentence-transformers/gooaq pipeline_tag: feature-extraction library_name: sentence-transformers metrics: - dot_accuracy@1 - dot_accuracy@3 - dot_accuracy@5 - dot_accuracy@10 - dot_precision@1 - dot_precision@3 - dot_precision@5 - dot_precision@10 - dot_recall@1 - dot_recall@3 - dot_recall@5 - dot_recall@10 - dot_ndcg@10 - dot_mrr@10 - dot_map@100 - query_active_dims - query_sparsity_ratio - corpus_active_dims - corpus_sparsity_ratio co2_eq_emissions: emissions: 13.27850984546157 energy_consumed: 0.03416115647838594 source: codecarbon training_type: fine-tuning on_cloud: false cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K ram_total_size: 31.777088165283203 hours_used: 0.142 hardware_used: 1 x NVIDIA GeForce RTX 3090 model-index: - name: splade-distilbert-base-uncased trained on GooAQ results: - task: type: sparse-information-retrieval name: Sparse Information Retrieval dataset: name: NanoMSMARCO type: NanoMSMARCO metrics: - type: dot_accuracy@1 value: 0.3 name: Dot Accuracy@1 - type: dot_accuracy@3 value: 0.48 name: Dot Accuracy@3 - type: dot_accuracy@5 value: 0.62 name: Dot Accuracy@5 - type: dot_accuracy@10 value: 0.72 name: Dot Accuracy@10 - type: dot_precision@1 value: 0.3 name: Dot Precision@1 - type: dot_precision@3 value: 0.16 name: Dot Precision@3 - type: dot_precision@5 value: 0.124 name: Dot Precision@5 - type: dot_precision@10 value: 0.07200000000000001 name: Dot Precision@10 - type: dot_recall@1 value: 0.3 name: Dot Recall@1 - type: dot_recall@3 value: 0.48 name: Dot Recall@3 - type: dot_recall@5 value: 0.62 name: Dot Recall@5 - type: dot_recall@10 value: 0.72 name: Dot Recall@10 - type: dot_ndcg@10 value: 0.4923029036903167 name: Dot Ndcg@10 - type: dot_mrr@10 value: 0.4212222222222222 name: Dot Mrr@10 - type: dot_map@100 value: 0.4327140646742442 name: Dot Map@100 - type: query_active_dims value: 129.6999969482422 name: Query Active Dims - type: query_sparsity_ratio value: 0.9957506062201611 name: Query Sparsity Ratio - type: corpus_active_dims value: 389.0450134277344 name: Corpus Active Dims - type: corpus_sparsity_ratio value: 0.9872536198994911 name: Corpus Sparsity Ratio - type: dot_accuracy@1 value: 0.32 name: Dot Accuracy@1 - type: dot_accuracy@3 value: 0.48 name: Dot Accuracy@3 - type: dot_accuracy@5 value: 0.62 name: Dot Accuracy@5 - type: dot_accuracy@10 value: 0.7 name: Dot Accuracy@10 - type: dot_precision@1 value: 0.32 name: Dot Precision@1 - type: dot_precision@3 value: 0.16 name: Dot Precision@3 - type: dot_precision@5 value: 0.124 name: Dot Precision@5 - type: dot_precision@10 value: 0.07 name: Dot Precision@10 - type: dot_recall@1 value: 0.32 name: Dot Recall@1 - type: dot_recall@3 value: 0.48 name: Dot Recall@3 - type: dot_recall@5 value: 0.62 name: Dot Recall@5 - type: dot_recall@10 value: 0.7 name: Dot Recall@10 - type: dot_ndcg@10 value: 0.497755451311688 name: Dot Ndcg@10 - type: dot_mrr@10 value: 0.4339126984126984 name: Dot Mrr@10 - type: dot_map@100 value: 0.44682157418631346 name: Dot Map@100 - type: query_active_dims value: 121.5999984741211 name: Query Active Dims - type: query_sparsity_ratio value: 0.9960159885173278 name: Query Sparsity Ratio - type: corpus_active_dims value: 397.7707824707031 name: Corpus Active Dims - type: corpus_sparsity_ratio value: 0.9869677353230226 name: Corpus Sparsity Ratio - task: type: sparse-information-retrieval name: Sparse Information Retrieval dataset: name: NanoNFCorpus type: NanoNFCorpus metrics: - type: dot_accuracy@1 value: 0.36 name: Dot Accuracy@1 - type: dot_accuracy@3 value: 0.44 name: Dot Accuracy@3 - type: dot_accuracy@5 value: 0.46 name: Dot Accuracy@5 - type: dot_accuracy@10 value: 0.54 name: Dot Accuracy@10 - type: dot_precision@1 value: 0.36 name: Dot Precision@1 - type: dot_precision@3 value: 0.2733333333333333 name: Dot Precision@3 - type: dot_precision@5 value: 0.21600000000000003 name: Dot Precision@5 - type: dot_precision@10 value: 0.182 name: Dot Precision@10 - type: dot_recall@1 value: 0.016695854443026216 name: Dot Recall@1 - type: dot_recall@3 value: 0.03106456335603726 name: Dot Recall@3 - type: dot_recall@5 value: 0.039511512430362564 name: Dot Recall@5 - type: dot_recall@10 value: 0.05434224491570579 name: Dot Recall@10 - type: dot_ndcg@10 value: 0.21590295298422338 name: Dot Ndcg@10 - type: dot_mrr@10 value: 0.41285714285714287 name: Dot Mrr@10 - type: dot_map@100 value: 0.07349060309007502 name: Dot Map@100 - type: query_active_dims value: 221.5399932861328 name: Query Active Dims - type: query_sparsity_ratio value: 0.9927416292088942 name: Query Sparsity Ratio - type: corpus_active_dims value: 851.602783203125 name: Corpus Active Dims - type: corpus_sparsity_ratio value: 0.9720987227834637 name: Corpus Sparsity Ratio - type: dot_accuracy@1 value: 0.32 name: Dot Accuracy@1 - type: dot_accuracy@3 value: 0.42 name: Dot Accuracy@3 - type: dot_accuracy@5 value: 0.48 name: Dot Accuracy@5 - type: dot_accuracy@10 value: 0.54 name: Dot Accuracy@10 - type: dot_precision@1 value: 0.32 name: Dot Precision@1 - type: dot_precision@3 value: 0.25999999999999995 name: Dot Precision@3 - type: dot_precision@5 value: 0.22399999999999998 name: Dot Precision@5 - type: dot_precision@10 value: 0.184 name: Dot Precision@10 - type: dot_recall@1 value: 0.011195854443026216 name: Dot Recall@1 - type: dot_recall@3 value: 0.0305213668117608 name: Dot Recall@3 - type: dot_recall@5 value: 0.04137763819101165 name: Dot Recall@5 - type: dot_recall@10 value: 0.054323362586038954 name: Dot Recall@10 - type: dot_ndcg@10 value: 0.21092767617388347 name: Dot Ndcg@10 - type: dot_mrr@10 value: 0.388079365079365 name: Dot Mrr@10 - type: dot_map@100 value: 0.07135787831660151 name: Dot Map@100 - type: query_active_dims value: 199.74000549316406 name: Query Active Dims - type: query_sparsity_ratio value: 0.9934558677185911 name: Query Sparsity Ratio - type: corpus_active_dims value: 879.7700805664062 name: Corpus Active Dims - type: corpus_sparsity_ratio value: 0.9711758705010679 name: Corpus Sparsity Ratio - task: type: sparse-information-retrieval name: Sparse Information Retrieval dataset: name: NanoNQ type: NanoNQ metrics: - type: dot_accuracy@1 value: 0.24 name: Dot Accuracy@1 - type: dot_accuracy@3 value: 0.5 name: Dot Accuracy@3 - type: dot_accuracy@5 value: 0.56 name: Dot Accuracy@5 - type: dot_accuracy@10 value: 0.7 name: Dot Accuracy@10 - type: dot_precision@1 value: 0.24 name: Dot Precision@1 - type: dot_precision@3 value: 0.16666666666666663 name: Dot Precision@3 - type: dot_precision@5 value: 0.11200000000000002 name: Dot Precision@5 - type: dot_precision@10 value: 0.07 name: Dot Precision@10 - type: dot_recall@1 value: 0.23 name: Dot Recall@1 - type: dot_recall@3 value: 0.46 name: Dot Recall@3 - type: dot_recall@5 value: 0.52 name: Dot Recall@5 - type: dot_recall@10 value: 0.64 name: Dot Recall@10 - type: dot_ndcg@10 value: 0.43543954013123615 name: Dot Ndcg@10 - type: dot_mrr@10 value: 0.3874365079365079 name: Dot Mrr@10 - type: dot_map@100 value: 0.37672138136649846 name: Dot Map@100 - type: query_active_dims value: 103.12000274658203 name: Query Active Dims - type: query_sparsity_ratio value: 0.9966214532879044 name: Query Sparsity Ratio - type: corpus_active_dims value: 603.1914672851562 name: Corpus Active Dims - type: corpus_sparsity_ratio value: 0.9802374855092997 name: Corpus Sparsity Ratio - type: dot_accuracy@1 value: 0.28 name: Dot Accuracy@1 - type: dot_accuracy@3 value: 0.48 name: Dot Accuracy@3 - type: dot_accuracy@5 value: 0.56 name: Dot Accuracy@5 - type: dot_accuracy@10 value: 0.72 name: Dot Accuracy@10 - type: dot_precision@1 value: 0.28 name: Dot Precision@1 - type: dot_precision@3 value: 0.16 name: Dot Precision@3 - type: dot_precision@5 value: 0.11599999999999999 name: Dot Precision@5 - type: dot_precision@10 value: 0.07400000000000001 name: Dot Precision@10 - type: dot_recall@1 value: 0.26 name: Dot Recall@1 - type: dot_recall@3 value: 0.45 name: Dot Recall@3 - type: dot_recall@5 value: 0.52 name: Dot Recall@5 - type: dot_recall@10 value: 0.67 name: Dot Recall@10 - type: dot_ndcg@10 value: 0.45351079818093326 name: Dot Ndcg@10 - type: dot_mrr@10 value: 0.40330158730158727 name: Dot Mrr@10 - type: dot_map@100 value: 0.3886037227739609 name: Dot Map@100 - type: query_active_dims value: 97.54000091552734 name: Query Active Dims - type: query_sparsity_ratio value: 0.996804272298161 name: Query Sparsity Ratio - type: corpus_active_dims value: 625.50048828125 name: Corpus Active Dims - type: corpus_sparsity_ratio value: 0.9795065694161179 name: Corpus Sparsity Ratio - task: type: sparse-nano-beir name: Sparse Nano BEIR dataset: name: NanoBEIR mean type: NanoBEIR_mean metrics: - type: dot_accuracy@1 value: 0.3 name: Dot Accuracy@1 - type: dot_accuracy@3 value: 0.47333333333333333 name: Dot Accuracy@3 - type: dot_accuracy@5 value: 0.5466666666666667 name: Dot Accuracy@5 - type: dot_accuracy@10 value: 0.6533333333333333 name: Dot Accuracy@10 - type: dot_precision@1 value: 0.3 name: Dot Precision@1 - type: dot_precision@3 value: 0.19999999999999998 name: Dot Precision@3 - type: dot_precision@5 value: 0.1506666666666667 name: Dot Precision@5 - type: dot_precision@10 value: 0.108 name: Dot Precision@10 - type: dot_recall@1 value: 0.18223195148100876 name: Dot Recall@1 - type: dot_recall@3 value: 0.32368818778534575 name: Dot Recall@3 - type: dot_recall@5 value: 0.39317050414345417 name: Dot Recall@5 - type: dot_recall@10 value: 0.47144741497190196 name: Dot Recall@10 - type: dot_ndcg@10 value: 0.3812151322685921 name: Dot Ndcg@10 - type: dot_mrr@10 value: 0.4071719576719577 name: Dot Mrr@10 - type: dot_map@100 value: 0.2943086830436059 name: Dot Map@100 - type: query_active_dims value: 151.45333099365234 name: Query Active Dims - type: query_sparsity_ratio value: 0.9950378962389864 name: Query Sparsity Ratio - type: corpus_active_dims value: 576.610088197042 name: Corpus Active Dims - type: corpus_sparsity_ratio value: 0.9811083779504279 name: Corpus Sparsity Ratio - type: dot_accuracy@1 value: 0.3591522762951334 name: Dot Accuracy@1 - type: dot_accuracy@3 value: 0.5289795918367346 name: Dot Accuracy@3 - type: dot_accuracy@5 value: 0.6121507064364207 name: Dot Accuracy@5 - type: dot_accuracy@10 value: 0.7046153846153848 name: Dot Accuracy@10 - type: dot_precision@1 value: 0.3591522762951334 name: Dot Precision@1 - type: dot_precision@3 value: 0.2408895866038723 name: Dot Precision@3 - type: dot_precision@5 value: 0.19369544740973316 name: Dot Precision@5 - type: dot_precision@10 value: 0.14295133437990584 name: Dot Precision@10 - type: dot_recall@1 value: 0.20232124722865225 name: Dot Recall@1 - type: dot_recall@3 value: 0.31785179276573494 name: Dot Recall@3 - type: dot_recall@5 value: 0.3843850663750471 name: Dot Recall@5 - type: dot_recall@10 value: 0.4668945859961475 name: Dot Recall@10 - type: dot_ndcg@10 value: 0.4149846764819604 name: Dot Ndcg@10 - type: dot_mrr@10 value: 0.46609898831327395 name: Dot Mrr@10 - type: dot_map@100 value: 0.34193786183886477 name: Dot Map@100 - type: query_active_dims value: 207.32511528026893 name: Query Active Dims - type: query_sparsity_ratio value: 0.9932073548496079 name: Query Sparsity Ratio - type: corpus_active_dims value: 571.211992111076 name: Corpus Active Dims - type: corpus_sparsity_ratio value: 0.9812852371367842 name: Corpus Sparsity Ratio - task: type: sparse-information-retrieval name: Sparse Information Retrieval dataset: name: NanoClimateFEVER type: NanoClimateFEVER metrics: - type: dot_accuracy@1 value: 0.18 name: Dot Accuracy@1 - type: dot_accuracy@3 value: 0.32 name: Dot Accuracy@3 - type: dot_accuracy@5 value: 0.4 name: Dot Accuracy@5 - type: dot_accuracy@10 value: 0.52 name: Dot Accuracy@10 - type: dot_precision@1 value: 0.18 name: Dot Precision@1 - type: dot_precision@3 value: 0.12 name: Dot Precision@3 - type: dot_precision@5 value: 0.08800000000000001 name: Dot Precision@5 - type: dot_precision@10 value: 0.06199999999999999 name: Dot Precision@10 - type: dot_recall@1 value: 0.09166666666666666 name: Dot Recall@1 - type: dot_recall@3 value: 0.16333333333333333 name: Dot Recall@3 - type: dot_recall@5 value: 0.19 name: Dot Recall@5 - type: dot_recall@10 value: 0.24733333333333335 name: Dot Recall@10 - type: dot_ndcg@10 value: 0.20663969061747678 name: Dot Ndcg@10 - type: dot_mrr@10 value: 0.2785238095238095 name: Dot Mrr@10 - type: dot_map@100 value: 0.16465402486405106 name: Dot Map@100 - type: query_active_dims value: 243.5399932861328 name: Query Active Dims - type: query_sparsity_ratio value: 0.9920208376487081 name: Query Sparsity Ratio - type: corpus_active_dims value: 704.9307250976562 name: Corpus Active Dims - type: corpus_sparsity_ratio value: 0.976904176492443 name: Corpus Sparsity Ratio - task: type: sparse-information-retrieval name: Sparse Information Retrieval dataset: name: NanoDBPedia type: NanoDBPedia metrics: - type: dot_accuracy@1 value: 0.54 name: Dot Accuracy@1 - type: dot_accuracy@3 value: 0.74 name: Dot Accuracy@3 - type: dot_accuracy@5 value: 0.82 name: Dot Accuracy@5 - type: dot_accuracy@10 value: 0.9 name: Dot Accuracy@10 - type: dot_precision@1 value: 0.54 name: Dot Precision@1 - type: dot_precision@3 value: 0.4666666666666666 name: Dot Precision@3 - type: dot_precision@5 value: 0.43200000000000005 name: Dot Precision@5 - type: dot_precision@10 value: 0.41200000000000003 name: Dot Precision@10 - type: dot_recall@1 value: 0.055010013413515094 name: Dot Recall@1 - type: dot_recall@3 value: 0.11580740263202617 name: Dot Recall@3 - type: dot_recall@5 value: 0.15730927980831147 name: Dot Recall@5 - type: dot_recall@10 value: 0.2871622888330674 name: Dot Recall@10 - type: dot_ndcg@10 value: 0.49104405587222694 name: Dot Ndcg@10 - type: dot_mrr@10 value: 0.6685238095238095 name: Dot Mrr@10 - type: dot_map@100 value: 0.34192873131869617 name: Dot Map@100 - type: query_active_dims value: 180.3000030517578 name: Query Active Dims - type: query_sparsity_ratio value: 0.994092785431762 name: Query Sparsity Ratio - type: corpus_active_dims value: 549.1513671875 name: Corpus Active Dims - type: corpus_sparsity_ratio value: 0.9820080149666633 name: Corpus Sparsity Ratio - task: type: sparse-information-retrieval name: Sparse Information Retrieval dataset: name: NanoFEVER type: NanoFEVER metrics: - type: dot_accuracy@1 value: 0.54 name: Dot Accuracy@1 - type: dot_accuracy@3 value: 0.68 name: Dot Accuracy@3 - type: dot_accuracy@5 value: 0.76 name: Dot Accuracy@5 - type: dot_accuracy@10 value: 0.84 name: Dot Accuracy@10 - type: dot_precision@1 value: 0.54 name: Dot Precision@1 - type: dot_precision@3 value: 0.22666666666666666 name: Dot Precision@3 - type: dot_precision@5 value: 0.15600000000000003 name: Dot Precision@5 - type: dot_precision@10 value: 0.08599999999999998 name: Dot Precision@10 - type: dot_recall@1 value: 0.53 name: Dot Recall@1 - type: dot_recall@3 value: 0.6466666666666666 name: Dot Recall@3 - type: dot_recall@5 value: 0.7266666666666666 name: Dot Recall@5 - type: dot_recall@10 value: 0.8066666666666665 name: Dot Recall@10 - type: dot_ndcg@10 value: 0.6669183841429774 name: Dot Ndcg@10 - type: dot_mrr@10 value: 0.6358571428571428 name: Dot Mrr@10 - type: dot_map@100 value: 0.6246746886992789 name: Dot Map@100 - type: query_active_dims value: 255.33999633789062 name: Query Active Dims - type: query_sparsity_ratio value: 0.9916342311664409 name: Query Sparsity Ratio - type: corpus_active_dims value: 861.5084228515625 name: Corpus Active Dims - type: corpus_sparsity_ratio value: 0.9717741818081527 name: Corpus Sparsity Ratio - task: type: sparse-information-retrieval name: Sparse Information Retrieval dataset: name: NanoFiQA2018 type: NanoFiQA2018 metrics: - type: dot_accuracy@1 value: 0.18 name: Dot Accuracy@1 - type: dot_accuracy@3 value: 0.36 name: Dot Accuracy@3 - type: dot_accuracy@5 value: 0.42 name: Dot Accuracy@5 - type: dot_accuracy@10 value: 0.52 name: Dot Accuracy@10 - type: dot_precision@1 value: 0.18 name: Dot Precision@1 - type: dot_precision@3 value: 0.15333333333333332 name: Dot Precision@3 - type: dot_precision@5 value: 0.12800000000000003 name: Dot Precision@5 - type: dot_precision@10 value: 0.086 name: Dot Precision@10 - type: dot_recall@1 value: 0.10933333333333334 name: Dot Recall@1 - type: dot_recall@3 value: 0.23572222222222222 name: Dot Recall@3 - type: dot_recall@5 value: 0.2821111111111111 name: Dot Recall@5 - type: dot_recall@10 value: 0.35740476190476195 name: Dot Recall@10 - type: dot_ndcg@10 value: 0.2731926938394576 name: Dot Ndcg@10 - type: dot_mrr@10 value: 0.28719047619047616 name: Dot Mrr@10 - type: dot_map@100 value: 0.22109942326282112 name: Dot Map@100 - type: query_active_dims value: 88.83999633789062 name: Query Active Dims - type: query_sparsity_ratio value: 0.9970893127469402 name: Query Sparsity Ratio - type: corpus_active_dims value: 414.41741943359375 name: Corpus Active Dims - type: corpus_sparsity_ratio value: 0.9864223373490076 name: Corpus Sparsity Ratio - task: type: sparse-information-retrieval name: Sparse Information Retrieval dataset: name: NanoHotpotQA type: NanoHotpotQA metrics: - type: dot_accuracy@1 value: 0.66 name: Dot Accuracy@1 - type: dot_accuracy@3 value: 0.72 name: Dot Accuracy@3 - type: dot_accuracy@5 value: 0.78 name: Dot Accuracy@5 - type: dot_accuracy@10 value: 0.9 name: Dot Accuracy@10 - type: dot_precision@1 value: 0.66 name: Dot Precision@1 - type: dot_precision@3 value: 0.33333333333333326 name: Dot Precision@3 - type: dot_precision@5 value: 0.23199999999999996 name: Dot Precision@5 - type: dot_precision@10 value: 0.13799999999999998 name: Dot Precision@10 - type: dot_recall@1 value: 0.33 name: Dot Recall@1 - type: dot_recall@3 value: 0.5 name: Dot Recall@3 - type: dot_recall@5 value: 0.58 name: Dot Recall@5 - type: dot_recall@10 value: 0.69 name: Dot Recall@10 - type: dot_ndcg@10 value: 0.6058525828428769 name: Dot Ndcg@10 - type: dot_mrr@10 value: 0.7181269841269841 name: Dot Mrr@10 - type: dot_map@100 value: 0.5210794772668003 name: Dot Map@100 - type: query_active_dims value: 147.39999389648438 name: Query Active Dims - type: query_sparsity_ratio value: 0.9951706967467242 name: Query Sparsity Ratio - type: corpus_active_dims value: 622.540283203125 name: Corpus Active Dims - type: corpus_sparsity_ratio value: 0.9796035553632422 name: Corpus Sparsity Ratio - task: type: sparse-information-retrieval name: Sparse Information Retrieval dataset: name: NanoQuoraRetrieval type: NanoQuoraRetrieval metrics: - type: dot_accuracy@1 value: 0.4 name: Dot Accuracy@1 - type: dot_accuracy@3 value: 0.58 name: Dot Accuracy@3 - type: dot_accuracy@5 value: 0.72 name: Dot Accuracy@5 - type: dot_accuracy@10 value: 0.82 name: Dot Accuracy@10 - type: dot_precision@1 value: 0.4 name: Dot Precision@1 - type: dot_precision@3 value: 0.19333333333333333 name: Dot Precision@3 - type: dot_precision@5 value: 0.15200000000000002 name: Dot Precision@5 - type: dot_precision@10 value: 0.08599999999999998 name: Dot Precision@10 - type: dot_recall@1 value: 0.3866666666666667 name: Dot Recall@1 - type: dot_recall@3 value: 0.5466666666666667 name: Dot Recall@3 - type: dot_recall@5 value: 0.6906666666666667 name: Dot Recall@5 - type: dot_recall@10 value: 0.7906666666666666 name: Dot Recall@10 - type: dot_ndcg@10 value: 0.5774223027465384 name: Dot Ndcg@10 - type: dot_mrr@10 value: 0.5223333333333333 name: Dot Mrr@10 - type: dot_map@100 value: 0.5159175593795703 name: Dot Map@100 - type: query_active_dims value: 52.779998779296875 name: Query Active Dims - type: query_sparsity_ratio value: 0.9982707555606023 name: Query Sparsity Ratio - type: corpus_active_dims value: 59.61296081542969 name: Corpus Active Dims - type: corpus_sparsity_ratio value: 0.9980468854984789 name: Corpus Sparsity Ratio - task: type: sparse-information-retrieval name: Sparse Information Retrieval dataset: name: NanoSCIDOCS type: NanoSCIDOCS metrics: - type: dot_accuracy@1 value: 0.34 name: Dot Accuracy@1 - type: dot_accuracy@3 value: 0.52 name: Dot Accuracy@3 - type: dot_accuracy@5 value: 0.64 name: Dot Accuracy@5 - type: dot_accuracy@10 value: 0.78 name: Dot Accuracy@10 - type: dot_precision@1 value: 0.34 name: Dot Precision@1 - type: dot_precision@3 value: 0.22 name: Dot Precision@3 - type: dot_precision@5 value: 0.18 name: Dot Precision@5 - type: dot_precision@10 value: 0.14 name: Dot Precision@10 - type: dot_recall@1 value: 0.07266666666666667 name: Dot Recall@1 - type: dot_recall@3 value: 0.13666666666666666 name: Dot Recall@3 - type: dot_recall@5 value: 0.18566666666666662 name: Dot Recall@5 - type: dot_recall@10 value: 0.2876666666666667 name: Dot Recall@10 - type: dot_ndcg@10 value: 0.2663751860700744 name: Dot Ndcg@10 - type: dot_mrr@10 value: 0.453611111111111 name: Dot Mrr@10 - type: dot_map@100 value: 0.17985066244400083 name: Dot Map@100 - type: query_active_dims value: 197.97999572753906 name: Query Active Dims - type: query_sparsity_ratio value: 0.9935135313633596 name: Query Sparsity Ratio - type: corpus_active_dims value: 729.0889892578125 name: Corpus Active Dims - type: corpus_sparsity_ratio value: 0.9761126731781072 name: Corpus Sparsity Ratio - task: type: sparse-information-retrieval name: Sparse Information Retrieval dataset: name: NanoArguAna type: NanoArguAna metrics: - type: dot_accuracy@1 value: 0.02 name: Dot Accuracy@1 - type: dot_accuracy@3 value: 0.16 name: Dot Accuracy@3 - type: dot_accuracy@5 value: 0.22 name: Dot Accuracy@5 - type: dot_accuracy@10 value: 0.24 name: Dot Accuracy@10 - type: dot_precision@1 value: 0.02 name: Dot Precision@1 - type: dot_precision@3 value: 0.05333333333333333 name: Dot Precision@3 - type: dot_precision@5 value: 0.044000000000000004 name: Dot Precision@5 - type: dot_precision@10 value: 0.024000000000000004 name: Dot Precision@10 - type: dot_recall@1 value: 0.02 name: Dot Recall@1 - type: dot_recall@3 value: 0.16 name: Dot Recall@3 - type: dot_recall@5 value: 0.22 name: Dot Recall@5 - type: dot_recall@10 value: 0.24 name: Dot Recall@10 - type: dot_ndcg@10 value: 0.1308713212722807 name: Dot Ndcg@10 - type: dot_mrr@10 value: 0.0955 name: Dot Mrr@10 - type: dot_map@100 value: 0.10586268132966514 name: Dot Map@100 - type: query_active_dims value: 798.3800048828125 name: Query Active Dims - type: query_sparsity_ratio value: 0.9738424741208698 name: Query Sparsity Ratio - type: corpus_active_dims value: 696.6200561523438 name: Corpus Active Dims - type: corpus_sparsity_ratio value: 0.9771764610395012 name: Corpus Sparsity Ratio - task: type: sparse-information-retrieval name: Sparse Information Retrieval dataset: name: NanoSciFact type: NanoSciFact metrics: - type: dot_accuracy@1 value: 0.44 name: Dot Accuracy@1 - type: dot_accuracy@3 value: 0.58 name: Dot Accuracy@3 - type: dot_accuracy@5 value: 0.64 name: Dot Accuracy@5 - type: dot_accuracy@10 value: 0.68 name: Dot Accuracy@10 - type: dot_precision@1 value: 0.44 name: Dot Precision@1 - type: dot_precision@3 value: 0.20666666666666667 name: Dot Precision@3 - type: dot_precision@5 value: 0.14 name: Dot Precision@5 - type: dot_precision@10 value: 0.078 name: Dot Precision@10 - type: dot_recall@1 value: 0.415 name: Dot Recall@1 - type: dot_recall@3 value: 0.555 name: Dot Recall@3 - type: dot_recall@5 value: 0.615 name: Dot Recall@5 - type: dot_recall@10 value: 0.67 name: Dot Recall@10 - type: dot_ndcg@10 value: 0.5536377846083319 name: Dot Ndcg@10 - type: dot_mrr@10 value: 0.5218888888888888 name: Dot Mrr@10 - type: dot_map@100 value: 0.5200703583453583 name: Dot Map@100 - type: query_active_dims value: 270.4800109863281 name: Query Active Dims - type: query_sparsity_ratio value: 0.9911381950400915 name: Query Sparsity Ratio - type: corpus_active_dims value: 766.060302734375 name: Corpus Active Dims - type: corpus_sparsity_ratio value: 0.9749013726907028 name: Corpus Sparsity Ratio - task: type: sparse-information-retrieval name: Sparse Information Retrieval dataset: name: NanoTouche2020 type: NanoTouche2020 metrics: - type: dot_accuracy@1 value: 0.4489795918367347 name: Dot Accuracy@1 - type: dot_accuracy@3 value: 0.8367346938775511 name: Dot Accuracy@3 - type: dot_accuracy@5 value: 0.8979591836734694 name: Dot Accuracy@5 - type: dot_accuracy@10 value: 1.0 name: Dot Accuracy@10 - type: dot_precision@1 value: 0.4489795918367347 name: Dot Precision@1 - type: dot_precision@3 value: 0.5782312925170067 name: Dot Precision@3 - type: dot_precision@5 value: 0.5020408163265306 name: Dot Precision@5 - type: dot_precision@10 value: 0.4183673469387756 name: Dot Precision@10 - type: dot_recall@1 value: 0.028637012782604658 name: Dot Recall@1 - type: dot_recall@3 value: 0.11168898095521147 name: Dot Recall@3 - type: dot_recall@5 value: 0.168207833765178 name: Dot Recall@5 - type: dot_recall@10 value: 0.26840587129271604 name: Dot Recall@10 - type: dot_ndcg@10 value: 0.46065286658673993 name: Dot Ndcg@10 - type: dot_mrr@10 value: 0.652437641723356 name: Dot Mrr@10 - type: dot_map@100 value: 0.34327142171812286 name: Dot Map@100 - type: query_active_dims value: 37.918365478515625 name: Query Active Dims - type: query_sparsity_ratio value: 0.9987576710085015 name: Query Sparsity Ratio - type: corpus_active_dims value: 507.71820068359375 name: Corpus Active Dims - type: corpus_sparsity_ratio value: 0.9833655002724724 name: Corpus Sparsity Ratio --- # splade-distilbert-base-uncased trained on GooAQ This is a [SPLADE Sparse Encoder](https://www.sbert.net/docs/sparse_encoder/usage/usage.html) model finetuned from [distilbert/distilbert-base-uncased](https://huggingface.co/distilbert/distilbert-base-uncased) on the [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) dataset using the [sentence-transformers](https://www.SBERT.net) library. It maps sentences & paragraphs to a 30522-dimensional sparse vector space and can be used for semantic search and sparse retrieval. ## Model Details ### Model Description - **Model Type:** SPLADE Sparse Encoder - **Base model:** [distilbert/distilbert-base-uncased](https://huggingface.co/distilbert/distilbert-base-uncased) - **Maximum Sequence Length:** 256 tokens - **Output Dimensionality:** 30522 dimensions - **Similarity Function:** Dot Product - **Training Dataset:** - [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) - **Language:** en - **License:** apache-2.0 ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Documentation:** [Sparse Encoder Documentation](https://www.sbert.net/docs/sparse_encoder/usage/usage.html) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sparse Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=sparse-encoder) ### Full Model Architecture ``` SparseEncoder( (0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'DistilBertForMaskedLM'}) (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': 30522}) ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SparseEncoder # Download from the 🤗 Hub model = SparseEncoder("tomaarsen/splade-distilbert-base-uncased-gooaq-peft-r64") # Run inference queries = [ "how many days for doxycycline to work on sinus infection?", ] documents = [ 'Treatment of suspected bacterial infection is with antibiotics, such as amoxicillin/clavulanate or doxycycline, given for 5 to 7 days for acute sinusitis and for up to 6 weeks for chronic sinusitis.', 'Most engagements typically have a cocktail dress code, calling for dresses at, or slightly above, knee-length and high heels. If your party states a different dress code, however, such as semi-formal or dressy-casual, you may need to dress up or down accordingly.', 'The average service life of a gas furnace is about 15 years, but the actual life span of an individual unit can vary greatly. There are a number of contributing factors that determine the age a furnace reaches: The quality of the equipment.', ] query_embeddings = model.encode_query(queries) document_embeddings = model.encode_document(documents) print(query_embeddings.shape, document_embeddings.shape) # [1, 30522] [3, 30522] # Get the similarity scores for the embeddings similarities = model.similarity(query_embeddings, document_embeddings) print(similarities) # tensor([[90.6936, 25.1699, 31.3902]]) ``` ## Evaluation ### Metrics #### Sparse Information Retrieval * Datasets: `NanoMSMARCO`, `NanoNFCorpus`, `NanoNQ`, `NanoClimateFEVER`, `NanoDBPedia`, `NanoFEVER`, `NanoFiQA2018`, `NanoHotpotQA`, `NanoMSMARCO`, `NanoNFCorpus`, `NanoNQ`, `NanoQuoraRetrieval`, `NanoSCIDOCS`, `NanoArguAna`, `NanoSciFact` and `NanoTouche2020` * Evaluated with [SparseInformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseInformationRetrievalEvaluator) | Metric | NanoMSMARCO | NanoNFCorpus | NanoNQ | NanoClimateFEVER | NanoDBPedia | NanoFEVER | NanoFiQA2018 | NanoHotpotQA | NanoQuoraRetrieval | NanoSCIDOCS | NanoArguAna | NanoSciFact | NanoTouche2020 | |:----------------------|:------------|:-------------|:-----------|:-----------------|:------------|:-----------|:-------------|:-------------|:-------------------|:------------|:------------|:------------|:---------------| | dot_accuracy@1 | 0.32 | 0.32 | 0.28 | 0.18 | 0.54 | 0.54 | 0.18 | 0.66 | 0.4 | 0.34 | 0.02 | 0.44 | 0.449 | | dot_accuracy@3 | 0.48 | 0.42 | 0.48 | 0.32 | 0.74 | 0.68 | 0.36 | 0.72 | 0.58 | 0.52 | 0.16 | 0.58 | 0.8367 | | dot_accuracy@5 | 0.62 | 0.48 | 0.56 | 0.4 | 0.82 | 0.76 | 0.42 | 0.78 | 0.72 | 0.64 | 0.22 | 0.64 | 0.898 | | dot_accuracy@10 | 0.7 | 0.54 | 0.72 | 0.52 | 0.9 | 0.84 | 0.52 | 0.9 | 0.82 | 0.78 | 0.24 | 0.68 | 1.0 | | dot_precision@1 | 0.32 | 0.32 | 0.28 | 0.18 | 0.54 | 0.54 | 0.18 | 0.66 | 0.4 | 0.34 | 0.02 | 0.44 | 0.449 | | dot_precision@3 | 0.16 | 0.26 | 0.16 | 0.12 | 0.4667 | 0.2267 | 0.1533 | 0.3333 | 0.1933 | 0.22 | 0.0533 | 0.2067 | 0.5782 | | dot_precision@5 | 0.124 | 0.224 | 0.116 | 0.088 | 0.432 | 0.156 | 0.128 | 0.232 | 0.152 | 0.18 | 0.044 | 0.14 | 0.502 | | dot_precision@10 | 0.07 | 0.184 | 0.074 | 0.062 | 0.412 | 0.086 | 0.086 | 0.138 | 0.086 | 0.14 | 0.024 | 0.078 | 0.4184 | | dot_recall@1 | 0.32 | 0.0112 | 0.26 | 0.0917 | 0.055 | 0.53 | 0.1093 | 0.33 | 0.3867 | 0.0727 | 0.02 | 0.415 | 0.0286 | | dot_recall@3 | 0.48 | 0.0305 | 0.45 | 0.1633 | 0.1158 | 0.6467 | 0.2357 | 0.5 | 0.5467 | 0.1367 | 0.16 | 0.555 | 0.1117 | | dot_recall@5 | 0.62 | 0.0414 | 0.52 | 0.19 | 0.1573 | 0.7267 | 0.2821 | 0.58 | 0.6907 | 0.1857 | 0.22 | 0.615 | 0.1682 | | dot_recall@10 | 0.7 | 0.0543 | 0.67 | 0.2473 | 0.2872 | 0.8067 | 0.3574 | 0.69 | 0.7907 | 0.2877 | 0.24 | 0.67 | 0.2684 | | **dot_ndcg@10** | **0.4978** | **0.2109** | **0.4535** | **0.2066** | **0.491** | **0.6669** | **0.2732** | **0.6059** | **0.5774** | **0.2664** | **0.1309** | **0.5536** | **0.4607** | | dot_mrr@10 | 0.4339 | 0.3881 | 0.4033 | 0.2785 | 0.6685 | 0.6359 | 0.2872 | 0.7181 | 0.5223 | 0.4536 | 0.0955 | 0.5219 | 0.6524 | | dot_map@100 | 0.4468 | 0.0714 | 0.3886 | 0.1647 | 0.3419 | 0.6247 | 0.2211 | 0.5211 | 0.5159 | 0.1799 | 0.1059 | 0.5201 | 0.3433 | | query_active_dims | 121.6 | 199.74 | 97.54 | 243.54 | 180.3 | 255.34 | 88.84 | 147.4 | 52.78 | 197.98 | 798.38 | 270.48 | 37.9184 | | query_sparsity_ratio | 0.996 | 0.9935 | 0.9968 | 0.992 | 0.9941 | 0.9916 | 0.9971 | 0.9952 | 0.9983 | 0.9935 | 0.9738 | 0.9911 | 0.9988 | | corpus_active_dims | 397.7708 | 879.7701 | 625.5005 | 704.9307 | 549.1514 | 861.5084 | 414.4174 | 622.5403 | 59.613 | 729.089 | 696.6201 | 766.0603 | 507.7182 | | corpus_sparsity_ratio | 0.987 | 0.9712 | 0.9795 | 0.9769 | 0.982 | 0.9718 | 0.9864 | 0.9796 | 0.998 | 0.9761 | 0.9772 | 0.9749 | 0.9834 | #### Sparse Nano BEIR * Dataset: `NanoBEIR_mean` * Evaluated with [SparseNanoBEIREvaluator](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseNanoBEIREvaluator) with these parameters: ```json { "dataset_names": [ "msmarco", "nfcorpus", "nq" ] } ``` | Metric | Value | |:----------------------|:-----------| | dot_accuracy@1 | 0.3 | | dot_accuracy@3 | 0.4733 | | dot_accuracy@5 | 0.5467 | | dot_accuracy@10 | 0.6533 | | dot_precision@1 | 0.3 | | dot_precision@3 | 0.2 | | dot_precision@5 | 0.1507 | | dot_precision@10 | 0.108 | | dot_recall@1 | 0.1822 | | dot_recall@3 | 0.3237 | | dot_recall@5 | 0.3932 | | dot_recall@10 | 0.4714 | | **dot_ndcg@10** | **0.3812** | | dot_mrr@10 | 0.4072 | | dot_map@100 | 0.2943 | | query_active_dims | 151.4533 | | query_sparsity_ratio | 0.995 | | corpus_active_dims | 576.6101 | | corpus_sparsity_ratio | 0.9811 | #### Sparse Nano BEIR * Dataset: `NanoBEIR_mean` * Evaluated with [SparseNanoBEIREvaluator](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseNanoBEIREvaluator) with these parameters: ```json { "dataset_names": [ "climatefever", "dbpedia", "fever", "fiqa2018", "hotpotqa", "msmarco", "nfcorpus", "nq", "quoraretrieval", "scidocs", "arguana", "scifact", "touche2020" ] } ``` | Metric | Value | |:----------------------|:----------| | dot_accuracy@1 | 0.3592 | | dot_accuracy@3 | 0.529 | | dot_accuracy@5 | 0.6122 | | dot_accuracy@10 | 0.7046 | | dot_precision@1 | 0.3592 | | dot_precision@3 | 0.2409 | | dot_precision@5 | 0.1937 | | dot_precision@10 | 0.143 | | dot_recall@1 | 0.2023 | | dot_recall@3 | 0.3179 | | dot_recall@5 | 0.3844 | | dot_recall@10 | 0.4669 | | **dot_ndcg@10** | **0.415** | | dot_mrr@10 | 0.4661 | | dot_map@100 | 0.3419 | | query_active_dims | 207.3251 | | query_sparsity_ratio | 0.9932 | | corpus_active_dims | 571.212 | | corpus_sparsity_ratio | 0.9813 | ## Training Details ### Training Dataset #### gooaq * Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c) * Size: 99,000 training samples * Columns: question and answer * Approximate statistics based on the first 1000 samples: | | question | answer | |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------| | type | string | string | | details | | | * Samples: | question | answer | |:-----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | what are the 5 characteristics of a star? | Key Concept: Characteristics used to classify stars include color, temperature, size, composition, and brightness. | | are copic markers alcohol ink? | Copic Ink is alcohol-based and flammable. Keep away from direct sunlight and extreme temperatures. | | what is the difference between appellate term and appellate division? | Appellate terms An appellate term is an intermediate appellate court that hears appeals from the inferior courts within their designated counties or judicial districts, and are intended to ease the workload on the Appellate Division and provide a less expensive forum closer to the people. | * Loss: [SpladeLoss](https://sbert.net/docs/package_reference/sparse_encoder/losses.html#spladeloss) with these parameters: ```json { "loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score')", "document_regularizer_weight": 3e-05, "query_regularizer_weight": 5e-05 } ``` ### Evaluation Dataset #### gooaq * Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c) * Size: 1,000 evaluation samples * Columns: question and answer * Approximate statistics based on the first 1000 samples: | | question | answer | |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------| | type | string | string | | details | | | * Samples: | question | answer | |:-----------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | should you take ibuprofen with high blood pressure? | In general, people with high blood pressure should use acetaminophen or possibly aspirin for over-the-counter pain relief. Unless your health care provider has said it's OK, you should not use ibuprofen, ketoprofen, or naproxen sodium. If aspirin or acetaminophen doesn't help with your pain, call your doctor. | | how old do you have to be to work in sc? | The general minimum age of employment for South Carolina youth is 14, although the state allows younger children who are performers to work in show business. If their families are agricultural workers, children younger than age 14 may also participate in farm labor. | | how to write a topic proposal for a research paper? | ['Write down the main topic of your paper. ... ', 'Write two or three short sentences under the main topic that explain why you chose that topic. ... ', 'Write a thesis sentence that states the angle and purpose of your research paper. ... ', 'List the items you will cover in the body of the paper that support your thesis statement.'] | * Loss: [SpladeLoss](https://sbert.net/docs/package_reference/sparse_encoder/losses.html#spladeloss) with these parameters: ```json { "loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score')", "document_regularizer_weight": 3e-05, "query_regularizer_weight": 5e-05 } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: steps - `per_device_train_batch_size`: 32 - `per_device_eval_batch_size`: 32 - `learning_rate`: 2e-05 - `num_train_epochs`: 1 - `bf16`: True - `load_best_model_at_end`: True - `batch_sampler`: no_duplicates #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: steps - `prediction_loss_only`: True - `per_device_train_batch_size`: 32 - `per_device_eval_batch_size`: 32 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 1 - `eval_accumulation_steps`: None - `torch_empty_cache_steps`: None - `learning_rate`: 2e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1.0 - `num_train_epochs`: 1 - `max_steps`: -1 - `lr_scheduler_type`: linear - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.0 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: True - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: True - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: None - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `include_for_metrics`: [] - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `eval_on_start`: False - `use_liger_kernel`: False - `eval_use_gather_object`: False - `average_tokens_across_devices`: False - `prompts`: None - `batch_sampler`: no_duplicates - `multi_dataset_batch_sampler`: proportional - `router_mapping`: {} - `learning_rate_mapping`: {}
### Training Logs | Epoch | Step | Training Loss | Validation Loss | NanoMSMARCO_dot_ndcg@10 | NanoNFCorpus_dot_ndcg@10 | NanoNQ_dot_ndcg@10 | NanoBEIR_mean_dot_ndcg@10 | NanoClimateFEVER_dot_ndcg@10 | NanoDBPedia_dot_ndcg@10 | NanoFEVER_dot_ndcg@10 | NanoFiQA2018_dot_ndcg@10 | NanoHotpotQA_dot_ndcg@10 | NanoQuoraRetrieval_dot_ndcg@10 | NanoSCIDOCS_dot_ndcg@10 | NanoArguAna_dot_ndcg@10 | NanoSciFact_dot_ndcg@10 | NanoTouche2020_dot_ndcg@10 | |:----------:|:--------:|:-------------:|:---------------:|:-----------------------:|:------------------------:|:------------------:|:-------------------------:|:----------------------------:|:-----------------------:|:---------------------:|:------------------------:|:------------------------:|:------------------------------:|:-----------------------:|:-----------------------:|:-----------------------:|:--------------------------:| | 0.0323 | 100 | 132.3714 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.0646 | 200 | 17.6927 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.0970 | 300 | 2.8291 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.1293 | 400 | 0.9365 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.1616 | 500 | 0.5878 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.1939 | 600 | 0.4041 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.1972 | 610 | - | 0.2847 | 0.3951 | 0.1965 | 0.3212 | 0.3043 | - | - | - | - | - | - | - | - | - | - | | 0.2262 | 700 | 0.3624 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.2586 | 800 | 0.295 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.2909 | 900 | 0.282 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.3232 | 1000 | 0.2947 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.3555 | 1100 | 0.2664 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.3878 | 1200 | 0.2387 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.3943 | 1220 | - | 0.2030 | 0.4818 | 0.2086 | 0.4084 | 0.3663 | - | - | - | - | - | - | - | - | - | - | | 0.4202 | 1300 | 0.2512 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.4525 | 1400 | 0.2117 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.4848 | 1500 | 0.2367 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.5171 | 1600 | 0.2085 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.5495 | 1700 | 0.1745 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.5818 | 1800 | 0.1958 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | **0.5915** | **1830** | **-** | **0.1449** | **0.4978** | **0.2109** | **0.4535** | **0.3874** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | | 0.6141 | 1900 | 0.1946 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.6464 | 2000 | 0.1801 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.6787 | 2100 | 0.1854 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.7111 | 2200 | 0.232 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.7434 | 2300 | 0.1798 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.7757 | 2400 | 0.1725 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.7886 | 2440 | - | 0.1437 | 0.4885 | 0.2206 | 0.4230 | 0.3774 | - | - | - | - | - | - | - | - | - | - | | 0.8080 | 2500 | 0.1598 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.8403 | 2600 | 0.1586 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.8727 | 2700 | 0.172 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.9050 | 2800 | 0.1875 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.9373 | 2900 | 0.1691 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.9696 | 3000 | 0.1473 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.9858 | 3050 | - | 0.1358 | 0.4923 | 0.2159 | 0.4354 | 0.3812 | - | - | - | - | - | - | - | - | - | - | | -1 | -1 | - | - | 0.4978 | 0.2109 | 0.4535 | 0.4150 | 0.2066 | 0.4910 | 0.6669 | 0.2732 | 0.6059 | 0.5774 | 0.2664 | 0.1309 | 0.5536 | 0.4607 | * The bold row denotes the saved checkpoint. ### Environmental Impact Carbon emissions were measured using [CodeCarbon](https://github.com/mlco2/codecarbon). - **Energy Consumed**: 0.034 kWh - **Carbon Emitted**: 0.013 kg of CO2 - **Hours Used**: 0.142 hours ### Training Hardware - **On Cloud**: No - **GPU Model**: 1 x NVIDIA GeForce RTX 3090 - **CPU Model**: 13th Gen Intel(R) Core(TM) i7-13700K - **RAM Size**: 31.78 GB ### Framework Versions - Python: 3.11.6 - Sentence Transformers: 4.2.0.dev0 - Transformers: 4.52.4 - PyTorch: 2.7.1+cu126 - Accelerate: 1.5.1 - Datasets: 2.21.0 - Tokenizers: 0.21.1 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### SpladeLoss ```bibtex @misc{formal2022distillationhardnegativesampling, title={From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective}, author={Thibault Formal and Carlos Lassance and Benjamin Piwowarski and Stéphane Clinchant}, year={2022}, eprint={2205.04733}, archivePrefix={arXiv}, primaryClass={cs.IR}, url={https://arxiv.org/abs/2205.04733}, } ``` #### SparseMultipleNegativesRankingLoss ```bibtex @misc{henderson2017efficient, title={Efficient Natural Language Response Suggestion for Smart Reply}, author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, year={2017}, eprint={1705.00652}, archivePrefix={arXiv}, primaryClass={cs.CL} } ``` #### FlopsLoss ```bibtex @article{paria2020minimizing, title={Minimizing flops to learn efficient sparse representations}, author={Paria, Biswajit and Yeh, Chih-Kuan and Yen, Ian EH and Xu, Ning and Ravikumar, Pradeep and P{'o}czos, Barnab{'a}s}, journal={arXiv preprint arXiv:2004.05665}, year={2020} } ```