README.md

---
license: other
license_name: lowdown-labs-lovely-license-1.0
license_link: LICENSE
tags:
  - fela
  - fourier-neural-operator
  - fno
  - cpu
  - on-device
  - chemistry
  - cheminformatics
  - smiles
  - molecular-property-prediction
  - retrosynthesis
  - reaction-yield
  - gated-deltanet
library_name: transformers
pipeline_tag: text-classification
model-index:
- name: fela-chemistry
  results:
  - task:
      type: text-classification
      name: Molecular property prediction
    dataset:
      type: moleculenet-bace
      name: MoleculeNet BACE
    metrics:
    - type: auroc
      value: 0.872
      name: ROC AUC (scaffold split)
---

# Disclaimer

This model is a research preview. Some of the datasets it was trained on carry copyleft or
attribution terms (in particular ChEMBL is ShareAlike, see "Training data" below), so review
those terms before any commercial use. This model has not been validated in a laboratory
environment, and any inference it produces is for demonstration only. Lowdown Labs built it in
the interest of advancing public science.

# FELA-Chem: a small chemistry model that runs on a plain CPU

FELA-Chem reads a molecule written as a SMILES string and gives you a number back. Hand it
aspirin and ask whether it looks toxic, how soluble it is, or how a reaction might score for
yield, and it answers. It also ranks which known reaction templates could build a target you
want to make, and that ranking drives its route planner.

The whole thing runs on your own laptop or a CPU box. No GPU, no cloud. A screening pass or a
route search calls the model thousands of times, so keeping it small and on your machine keeps
it cheap. One thing to be clear about up front: it scores and ranks chemistry that already
exists. It does not invent new molecules, so nothing new comes out as SMILES, only numbers and
rankings.

# What goes in, what comes out

- Input: a molecule as a SMILES string, the standard text form of a molecular graph. Aspirin,
  for instance, is `CC(=O)Oc1ccccc1C(=O)O`. For the reaction tasks (yield, retrosynthesis) you
  pass a full reaction SMILES: a dot between co reactants, and `>>` between the reactant and
  product sides. Inside, the model reads the string one atom or bond at a time over a window of
  up to 128 tokens.
- Output depends on which task head you load:
  - property classification: one probability per endpoint, say toxic or not, or blood brain
    barrier permeable or not. Take the probability, or the argmax for a label.
  - property regression: a real value in the property's own units, like log solubility in mol/L
    for ESOL, or predicted yield in percent for a reaction.
  - retrosynthesis template ranking: a ranked list of reaction templates with probabilities,
    which the route planner uses to decide what to try next.
- In practice that covers three things you might do without touching the cloud: triage a
  compound library for toxicity and ADMET style endpoints, estimate a reaction's yield from its
  full context, and drive a retrosynthesis planner that works backward from a target to buyable
  building blocks.

# Why we built it this way

A normal attention model builds a table that grows with the square of the input length, which
is what makes big sequence models hungry. The Fourier operator has no such table. Its working
memory stays a fixed size no matter how long the molecule, so the model stays small, tens of
millions of parameters, and stays cheap to call. That matters most for a route planner, which
scores dozens of candidate steps per target and would run up a real bill on a per call API.

To train it, we first show the model a large pile of unlabeled SMILES and have it fill in
hidden pieces of each string, so it learns the grammar of chemistry on its own. We also show it
several valid ways to write the same molecule. Then we fine tune a light task head on top. It
is the same recipe that worked for the FELA genomics model.

# Performance

This model runs on CPU. The architecture is CPU native by design, with an FFT global convolution
plus a fixed size state recall layer, no GPU kernel needed for inference.

The retrosynthesis planner ran CPU only, calling the 33M parameter policy as its expansion
model. Mean search time was 10.9 seconds per target and median 3.5 seconds per target, at a
mean of 43.7 policy expansions per target. A planner is the heaviest CPU consumer of this model
(it scores many candidate states per target), so a single property or yield score is well below
those numbers.

Sizes (fp32 on disk, approximate):

| Variant | Params | fp32 disk | Device class |
|---|---|---|---|
| 10M (property, smallest) | 9.86M | about 39 MB | laptop or small CPU |
| 33M (property default) | 33.5M | about 134 MB | laptop or CPU server |
| 116M (best retro and yield) | 116M | about 464 MB | CPU server |
| 286M (flagship backbone) | 286.34M | about 1.1 GB | CPU server |

The FNO plus recall backbone could export to ONNX for smaller CPU or ARM targets. The 286M flagship
backbone was pretrained on 122M unique SMILES (PubChem plus ChEMBL) to a final masked token
accuracy of 0.9613. 

The downstream results are the reaction yield below (R2 0.960), the USPTO
full 18,240 class template ranker (top 1 0.94, top 10 0.99), and the planner re run on that
ranker. The flagship property scores are not reported here; the property tables cite the
validated 33M and 116M results.

# Accuracy

The short version - on property prediction FELA-Chem is competitive with the well known models
and wins outright on a couple of tasks, while trailing the largest pretrained models on the
hardest ones, with our current training budget.

On reaction yield it matches Yield-BERT. As a route planner it clears the standard
open tool AiZynthFinder but sits below the strong academic planner retro* on retro*'s own
targets.

All numbers are measured with the protocol stated per task. Property prediction uses the
MoleculeNet datasets (Wu et al. 2018) with scaffold splits, which hold out molecules built on
novel scaffolds and are the harder, more honest way to test generalization. 

We keep a validation
split for model selection and report test only for the best validation config, with no tuning on
test. Baselines are the published numbers for the named models.

## Property classification, ROC-AUC (higher is better)

| Benchmark | FELA-Chem (best variant) | Named baselines |
|---|---|---|
| BACE | 0.872 (FNO+GLA 33M) | D-MPNN 0.852, Uni-Mol 0.857, MolFormer-XL 0.882 |
| ClinTox | 0.894 (FNO+GLA 10M, mean of 4 seeds, best of 4 = 0.907) | Uni-Mol 0.919, ChemBERTa-2 0.563, MolFormer-XL 0.948 |
| Tox21 | 0.784 (GDN 116M) | ChemBERTa-2 0.749, Uni-Mol 0.796, MolFormer-XL 0.847 |
| SIDER | 0.629 (GDN 33M) | Uni-Mol 0.659, MolFormer-XL 0.690 |
| BBBP | 0.750 (GDN 33M) | D-MPNN 0.886, MolFormer-XL 0.937 |
| HIV | 0.768 (FNO 65M) | Uni-Mol 0.808, MolFormer-XL 0.822 |

## Property regression, RMSE (lower is better)

| Benchmark | FELA-Chem (best variant) | Named baselines |
|---|---|---|
| ESOL (log solubility) | 0.816 (FNO+GLA 33M) | ChemBERTa-2 0.889, Uni-Mol 0.788, MolFormer-XL 0.279 |
| Lipophilicity | 0.642 (GDN 33M) | ChemBERTa-2 0.758, Uni-Mol 0.603, MolFormer-XL 0.529 |
| FreeSolv | 2.248 (FNO+GLA 33M) | ChemBERTa-2 1.349, Uni-Mol 1.480, MolFormer-XL 0.231 |

## Retrosynthesis template ranking, USPTO-50k (the planner policy)

The model ranks which of the USPTO-50k reaction templates apply to a product. This is the
expansion policy of the route planner. Protocol: USPTO-50k template classification, top k
accuracy on the held out split, templates extracted with rdchiral.

| System | Top 1 | Top 10 |
|---|---|---|
| neuralsym (baseline) | 0.443 | 0.787 |
| GLN (graph baseline) | 0.526 | 0.838 |
| LocalRetro (context) | 0.534 | not reported |
| FELA-Chem FNO+GLA 33M | 0.485 | 0.782 |
| FELA-Chem GDN 33M | 0.529 | 0.833 |
| FELA-Chem GDN 116M | 0.542 | 0.848 |

The planner needs a wider policy than the USPTO-50k 1,472 class ranker. The flagship policy
scales this to the full USPTO template library, an 18,240 class ranker (305M), evaluated on its
held out validation split (top k accuracy, templates extracted with rdchiral):

| System | Top 1 | Top 10 |
|---|---|---|
| FELA-Chem USPTO full ranker (18,240 class, 305M) | 0.94 | 0.99 |

This is the policy the planner below uses.

## Reaction yield, Buchwald-Hartwig HTE (full reaction context)

Protocol: the Ahneman et al. 2018 Buchwald-Hartwig high throughput experimentation set, the
standard FullCV_01 70/30 split, full reaction context input (aryl halide, amine, ligand, base,
additive as one reaction SMILES), the same feature set Yield-BERT uses.

| Benchmark | FELA-Chem (GDN 286M) | FELA-Chem (GDN 116M) | Baseline |
|---|---|---|---|
| Buchwald-Hartwig HTE yield, test R2 | 0.960 | 0.956 | Yield-BERT about 0.95 |
| Buchwald-Hartwig HTE yield, test RMSE (yield %) | 5.486 | 5.73 | Yield-BERT band about 5 to 8 |

The 286M flagship edges Yield-BERT on the same full reaction context input and FullCV_01 split.

## Retrosynthesis planning (system level), USPTO held out products

The template ranker drops in as the expansion policy of a retro star style AND-OR tree search;
rdchiral applies each ranked template as the operator (enforcing valid chemistry); the search
composes routes and stops when every leaf is a molecule in the public AiZynthFinder ZINC in
stock set (17.4M buyable molecules). 

The protocol we adopted was to run 500 USPTO held out products that the policy never saw,
ZINC stock, CPU only. We report two policies on the identical planner and the identical 500
targets, changing only the policy, which isolates the effect of policy strength.

| Policy | Solved rate | Context baseline |
|---|---|---|
| Strong 18,240 class ranker (top 10 0.99) | about 0.89 with search budget; about 0.80 (393 of 500) at a conservative 30 second per target limit | retro* about 0.86 to 0.88 (own hard target set); AiZynthFinder about 0.55 to 0.70 |
| This planner on retro*'s own 190 hard targets (retro*'s stock) | 0.058 (11 of 190) | retro* about 0.86 to 0.88 (same 190 targets) |
| Weak frozen probe policy (top 10 0.659, ablation) | 0.456 strict / 0.553 lenient | AiZynthFinder about 0.55 to 0.70 (same ZINC stock) |

With the strong 18,240 class ranker the planner solves about 0.89 of held out
USPTO targets with a search budget, and holds about 0.80 (393 of 500) at a conservative 30
second per target limit, both on our held out USPTO set with the ZINC stock. This clears the
standard open planner AiZynthFinder (about 0.55 to 0.70) on the same ZINC stock, CPU only. We do
not claim a same benchmark tie with the strong academic planner retro* (an ICML 2020 neural
guided A* planner at about 0.86 to 0.88 on its own hard target benchmark): run like for like on
retro*'s own 190 hard targets with retro*'s own stock, our planner solves only 5.8 percent (11
of 190), well below retro*, and the gap is a missing learned value function, which is a scoped
next step if we can continue work on FELA-chem in the future.

The solved rate is sensitive to search budget, the 30 second per target limit is a
reproducible operating point and not a hard ceiling, the rate rises with more compute per target,
which is itself the point. A cheap policy buys more search per dollar. 

The jump from 0.456
to about 0.80 comes from changing only the policy (weak probe to strong 18,240 class ranker) on
the same planner and test set, which confirms that policy template recall, not search budget,
was the binding constraint. The real advantage, as with other FELA models, seems to be the solution rate delivered CPU only and on
premises.

# Loading with standard tooling

The repo ships `config.json` (the architecture hyperparameters and per variant metadata) and a
self contained `modeling.py` with `load_model` / `from_pretrained` and scoring helpers. A few
lines load the model from a local directory or a Hugging Face repo:

```python
from modeling import load_model, score_smiles
m = load_model("/path/to/weights_dir")                   # dir with the safetensors + config.json + tokenizer.json
label, prob = score_smiles(m, "CC(=O)Oc1ccccc1C(=O)O")   # aspirin -> (label, probability)
```

For the retrosynthesis policy checkpoint:

```python
from modeling import load_model, rank_templates
policy = load_model("/path/to/weights_dir", variant="ranker")
ranked = rank_templates(policy, "CCC(C)N(C)C(=O)c1cc2ccccc2c(-c2ccccc2F)n1", top_k=10)
# -> [(template_class_id, probability), ...] for the planner
```

For a property head (classification or regression) or the reaction yield head:

```python
from modeling import load_model, predict_property
tox = load_model("/path/to/weights_dir", variant="tox21")
print(predict_property(tox, "CC(=O)Oc1ccccc1C(=O)O"))   # per endpoint probabilities

esol = load_model("/path/to/weights_dir", variant="esol")
print(predict_property(esol, "CC(=O)Oc1ccccc1C(=O)O"))   # predicted log solubility (mol/L)

y = load_model("/path/to/weights_dir", variant="yield")
print(predict_property(y, "Brc1ccccc1.Nc1ccccc1>>c1ccc(Nc2ccccc2)cc1"))  # predicted yield (percent)
```

Every task ships its own safetensors weights and, where needed, its own tokenizer, all selected
by the variant name. The shipped variants are: `foundation` (encoder), `ranker` (retrosynthesis
template ranking), `synth` (synthesizability), `cond_heads` (reaction conditions), the nine
property heads (`bace`, `bbbp`, `clintox`, `sider`, `tox21`, `hiv`, `esol`, `freesolv`, `lipo`),
and `yield` (reaction yield). The property and yield heads are the exact fine tuned checkpoints
behind the accuracy tables above, so each reported number is reproducible from the weights in
this repo.

The weights ship as `safetensors` (not pickle).

# How to run it

FELA-Chem is a small suite, not a single file. Each capability (the nine property heads,
reaction yield, the retrosynthesis ranker, synthesizability, and reaction conditions) has its
own variant in `config.json`, and you load the one you need with `variant=`. Point
`FELA_CHEM_WEIGHTS` at the directory that holds the weights (with its `config.json` and the
tokenizers) and run. The quickstart default scores aspirin with the foundation encoder.

See `quickstart/` for a runnable, self contained example:

```bash
pip install -r quickstart/requirements.txt
export FELA_CHEM_WEIGHTS=/path/to/weights_dir
python quickstart/run.py
```

For an interactive playground, see the Hugging Face Space in `space/`. For a production serving
path, the models run under the FELA server, a separate CPU native serving project
(`https://github.com/Lowdown-Labs/fela_server`).

## Formats

- fp32: reference and CPU.
- int8: on device deployment format (AVX512-VNNI on x86, NEON dot product on ARM).
- bf16: server and GPU inference only; most commodity ARM and microcontroller CPUs lack native
  bf16, so it is not the on device format.

# Training data

- Pretraining corpus: ChEMBL 37 (`chembl_37_chemreps.txt`, the canonical_smiles field),
  2,897,819 raw molecules, reduced to 2,872,185 unique valid canonical SMILES after RDKit
  canonicalization (2 to 120 heavy atoms, up to 250 characters). The 116M backbone adds PubChem
  (the CID-SMILES release) for a deduplicated 6,539,567 SMILES corpus. The 286M flagship backbone
  is pretrained on the full PubChem plus ChEMBL corpus, 122,249,322 unique valid canonical SMILES
  (final masked token accuracy 0.9613). Masked token self supervised pretraining (BERT style 15
  percent masking) with SMILES enumeration augmentation (RDKit random SMILES, Bjerrum 2017).
  ChEMBL is released under CC BY-SA 3.0 (Zdrazil et al. 2024). PubChem records are public domain
  (NCBI).
- Tokenizer: the atom wise SMILES regex tokenizer (Schwaller et al. 2019), vocab 421 plus an
  appended mask id (the 116M backbone uses a 686 token vocab). Ship `tokenizer.json` with the
  weights so the token to id map matches.
- Property fine tuning and evaluation: MoleculeNet (Wu et al. 2018): BACE, BBBP, ClinTox, Tox21,
  SIDER, HIV (classification), ESOL, FreeSolv, Lipophilicity (regression), with Bemis-Murcko
  scaffold splits (Bemis and Murcko 1996), loaded from the DeepChem mirror.
- Retrosynthesis: USPTO-50k reaction templates (Lowe 2012; typed by Schneider et al. 2015),
  extracted and applied with rdchiral (Coley et al. 2019). The planner stock is the AiZynthFinder
  public ZINC in stock set, 17,422,831 buyable molecules (Genheden et al. 2020).
- Reaction yield: the Buchwald-Hartwig HTE dataset (Ahneman et al. 2018), FullCV_01 split.
- Reference baselines (we did not retrain them): D-MPNN / Chemprop (Yang et al. 2019), Uni-Mol
  (Zhou et al. 2023), MolFormer-XL (Ross et al. 2022), ChemBERTa-2 (Ahmad et al. 2022), neuralsym
  (Segler and Waller 2017), GLN (Dai et al. 2019), LocalRetro (Chen and Jung 2021), Yield-BERT
  (Schwaller et al. 2021), AiZynthFinder (Genheden et al. 2020), retro star (Chen et al. 2020).
- The GDN variant uses the Gated DeltaNet layer from the flash-linear-attention (fla) library;
  its Triton kernel is used for training, and CPU inference uses a pure torch recurrence.

# Splits and licensing

Per component: the training dataset, the split definition (method, seed, sizes, and the
reproducible split function in `train.py`), the dataset license, and the commercial verdict.
Split sizes are reproduced from the training data by `python train.py smoke` (bare size
assertions per component). One dataset carries a commercial use flag; see the note at the end.

- Backbone (masked LM pretrain). ChEMBL 37 (canonical_smiles field) plus, for the larger
  backbones, PubChem CID-SMILES. Corpus sizes: 33M and 65M backbones = ChEMBL only (2,872,185
  unique valid canonical SMILES, 2 to 120 heavy atoms, up to 250 characters); 116M = deduplicated
  ChEMBL plus PubChem 6,539,567 SMILES; 286M flagship = full PubChem plus ChEMBL 122,249,322
  unique valid canonical SMILES. No held out split: self supervised masked LM pretraining
  consumes the whole corpus (BERT style 15 percent masking, 80/10/10 corrupt, SMILES enumeration
  augmentation), so there is no train and test line to cite. Licenses: ChEMBL 37 is CC BY-SA 3.0
  (Zdrazil et al. 2024); PubChem records are U.S. NCBI public domain. Commercial verdict:
  permitted with attribution; ChEMBL's ShareAlike is flagged below.
- Property finetunes (MoleculeNet). BACE, BBBP, ClinTox, Tox21, SIDER, HIV (classification),
  ESOL, FreeSolv, Lipophilicity (regression), from MoleculeNet (Wu et al. 2018), via the DeepChem
  mirror CSVs. Split: deterministic Bemis-Murcko scaffold split, frac (0.8, 0.1, 0.1), largest
  scaffold groups to train, defined in `train.py` `scaffold_split`. Reproduced test sizes: BACE
  152, ClinTox 148, Tox21 783, SIDER 144, HIV 4112, ESOL 114, FreeSolv 65, Lipophilicity 420,
  BBBP 205. License: MoleculeNet via DeepChem is MIT. Commercial verdict: permitted (research
  benchmark; underlying assay data is public).
- Reaction yield head (Buchwald-Hartwig HTE). Ahneman et al., Science 2018 high throughput C-N
  coupling set, via the `BH_input.xlsx` feature file (Sandfort et al. 2020), sheet `FullCV_01`,
  n=3955. Split: the Yield-BERT protocol, a seed 0 70/30 random split (30 percent test), holding
  10 percent of train as validation, defined in `train.py` `yield_split`, reproduced test size
  1186. Commercial verdict: permitted as a published research dataset; confirm reuse terms of the
  Science supplementary feature file for redistribution (flagged as verify terms).
- Retrosynthesis template ranker (the planner policy). USPTO reaction corpus (Lowe 1976 to Sep
  2016 grants and applications, about 3.75M reactions), retro templates extracted with rdchiral.
  Flagship 18,240 class ranker: templates kept at min count 10, product classified, seed 0
  80/10/10 random split over n=1,027,751 reactions, defined in `train.py` `retro_full_split`;
  reproduced classes 18,240 and val size 102,775 (the held out val used to report top 1 0.94, top
  10 0.99). The USPTO-50k gate (1,472 class, min count 5) is the smaller variant. License: USPTO
  patent reaction data is U.S. public domain. Commercial verdict: permitted (public domain).
- Reaction condition heads (catalyst, solvent, reagent). The same USPTO grants and applications;
  the agent field is parsed into catalyst, solvent, and reagent by RDKit heuristics, agents
  stripped from the input so the model predicts conditions. Cap 400,000 reactions, top 50 label
  vocab per category plus none and other. Split: seed 0 shuffle, 80/10/10, defined in `train.py`
  `cond_split`; reproduced test size 40,000. License: USPTO public domain. Commercial verdict:
  permitted (public domain).
- Synthesizability head (SCScore style). Real molecules only: ChEMBL drug like SMILES plus USPTO
  reaction products; the head regresses the RDKit SAScore. Split: seed 0 permutation, 80/10/10,
  defined in `train.py` `synth_split`; on the 50,000 molecule build the test size is 5,000. The
  headline metric is the test Spearman of the FELA score against SAScore on real molecules: 0.970
  on a fresh 8,000 molecule ChEMBL holdout (and 0.9165 on the harder mixed ChEMBL plus USPTO
  test). License: ChEMBL CC BY-SA 3.0 plus USPTO public domain; SAScore labels are computed with
  RDKit (BSD). Commercial verdict: permitted with attribution; the ChEMBL ShareAlike flag
  applies.

Licensing flag (ChEMBL ShareAlike): ChEMBL 37 is released under CC BY-SA 3.0. The ShareAlike
term means a derivative of the ChEMBL data must itself be offered under CC BY-SA 3.0 (or a
compatible license). ChEMBL SMILES appear in the pretraining corpus (all backbones) and in the
synthesizability head's training molecules. Whether trained model weights constitute a
derivative work of the underlying data under CC BY-SA is legally unsettled and is a business
decision, not a technical one; it is flagged for legal review before any commercial
redistribution, and it interacts with this repo's own Lowdown Labs Lovely License 1.0 (CC BY-NC
4.0 plus Hippocratic 3.0). PubChem, MoleculeNet (DeepChem, MIT), and USPTO (public domain) carry
no such copyleft. The Buchwald-Hartwig `BH_input.xlsx` feature file's redistribution terms should
be confirmed against the Science 2018 supplementary material.

The reproducible split definitions and size assertions live in `train.py`; run
`python train.py smoke` (with `FELA_CHEM_DATA` pointing at the MoleculeNet CSVs and the optional
`FELA_CHEM_BH` / `FELA_CHEM_CONDDIR` / `FELA_CHEM_SYNTHDIR` env vars for the reaction datasets)
to rebuild each split and check the audited sizes.

# Intended use, limitations, and safety

What it is for: research use. CPU, no cloud scoring of molecular properties (toxicity and ADMET
style endpoints, solubility, lipophilicity), reaction yield estimation from a full reaction
context, ranking which reaction templates could make a target, and serving as the expansion
policy of a retrosynthesis planner that proposes routes to buyable building blocks.

What it is not for: it is not a molecule generator and does not invent new SMILES (it scores and
ranks existing chemistry). It is not a clinical, diagnostic, or safety decision tool, and it is
not a substitute for laboratory validation. Predicted properties, yields, and routes are
statistical and must be checked by a chemist and by experiment before any synthesis, dosing, or
safety decision.

Privacy: the model runs on the device. Proprietary structures and reactions do not have to leave
your machine, which matters for unpublished or competitively sensitive chemistry.

Evaluated conditions and known failure modes:

- Property prediction was evaluated on MoleculeNet with scaffold splits; on the hardest scaffold
  generalization tasks (BBBP, HIV) and on the very low RMSE regression endpoints (ESOL, FreeSolv,
  Lipophilicity) the model trails MolFormer-XL and Uni-Mol, which are much larger and use far
  more pretraining data or 3D conformers. FreeSolv (642 molecules) is the weakest case and is
  small and noisy.
- The retrosynthesis planner was evaluated on 500 held out USPTO products against the ZINC stock;
  that set and stock differ from the retro star 190 hard target benchmark and the AiZynthFinder
  sets, so the solved rate clears the standard open planner AiZynthFinder (0.55 to 0.70) on the
  same ZINC stock but is not a same benchmark tie with retro* (0.86 to 0.88). On retro*'s own 190
  hard targets with retro*'s own stock, our planner solves only 5.8 percent (11 of 190), the gap
  being a missing learned value function. The solved rate is a reproducible operating point at a
  30 second per target limit, not a ceiling.
- Inputs must be valid SMILES; for reaction tasks they must be full reaction SMILES with the
  reaction context co reactants present (a product only yield input gives an R2 near zero, a
  feature set error, not a model limit). The loader validates SMILES and fails clearly on
  malformed input.

# How to cite

```bibtex
@misc{lowdownlabs_felachem,
  title  = {FELA-Chem: a small CPU-runnable Fourier Neural Operator model for molecular
            property prediction, reaction scoring, and retrosynthesis planning},
  author = {Lowdown Labs},
  year   = {2026},
  note   = {Model card}
}
```

You must also cite the datasets, the reference benchmarks, and the libraries. Each entry below
gives the reference and, for datasets, the governing license.

# Datasets

- ChEMBL 37 (pretraining corpus and the synthesizability head's molecules). License: CC BY-SA
  3.0. Zdrazil, B., Felix, E., Hunter, F., et al. (2024). The ChEMBL Database in 2023. Nucleic
  Acids Research, 52(D1), D1180-D1192. https://doi.org/10.1093/nar/gkad1004 . License text:
  https://creativecommons.org/licenses/by-sa/3.0/ . Flag: ShareAlike is copyleft; see the
  licensing flag above.
- PubChem (CID-SMILES release, 116M and 286M backbones). Status: U.S. NCBI public domain. Kim,
  S., Chen, J., Cheng, T., et al. (2023). PubChem 2023 update. Nucleic Acids Research, 51(D1),
  D1373-D1380. https://doi.org/10.1093/nar/gkac956 .
- USPTO patent reactions (retro ranker and reaction condition heads). Status: derived from U.S.
  patent text, public domain. Lowe, D. M. (2012). Extraction of chemical structures and reactions
  from the literature. PhD thesis, University of Cambridge. https://doi.org/10.17863/CAM.16293 .
  Dataset: https://doi.org/10.6084/m9.figshare.5104873.v1 . Typing: Schneider, N., Stiefl, N.,
  Landrum, G. A. (2015). JCIM, 55(1), 39-53. https://doi.org/10.1021/ci5006614 .
- Buchwald-Hartwig HTE (reaction yield head). Ahneman, D. T., Estrada, J. G., Lin, S., Dreher, S.
  D., Doyle, A. G. (2018). Predicting reaction performance in C-N cross coupling using machine
  learning. Science, 360(6385), 186-190. https://doi.org/10.1126/science.aar5169 . Feature file:
  Sandfort, F., Strieth-Kalthoff, F., Kuhnemund, M., et al. (2020). Chem, 6(6), 1379-1390.
  https://doi.org/10.1016/j.chempr.2020.02.017 . Flag: confirm redistribution terms of the
  feature file (verify terms).
- MoleculeNet (property fine tuning and evaluation). Wu, Z., Ramsundar, B., Feinberg, E. N., et
  al. (2018). MoleculeNet: A Benchmark for Molecular Machine Learning. Chemical Science, 9,
  513-530. https://doi.org/10.1039/C7SC02664A . Loaded via the DeepChem mirror (MIT).
- SAScore basis (synthesizability head's label). Ertl, P., Schuffenhauer, A. (2009). Estimation
  of synthetic accessibility score. Journal of Cheminformatics, 1, 8.
  https://doi.org/10.1186/1758-2946-1-8 . Computed with RDKit.
- Scaffold split, buyable stock, augmentation: Bemis, G. W., Murcko, M. A. (1996). J. Med. Chem.,
  39(15), 2887-2893. https://doi.org/10.1021/jm9602928 . AiZynthFinder ZINC in stock set:
  Genheden, S., Thakkar, A., Chadimova, V., et al. (2020). J. Cheminformatics, 12, 70
  (Apache-2.0). Bjerrum, E. J. (2017). SMILES Enumeration as Data Augmentation. arXiv:1703.07076.

# Methods, architecture, and libraries

- Fourier Neural Operator (the FNO sequence mixer): Li, Z., Kovachki, N., Azizzadenesheli, K., et
  al. (2021). ICLR. arXiv:2010.08895 .
- Gated DeltaNet (the FNO plus GDN recall layer): Yang, S., Kautz, J., Hatamizadeh, A. (2024).
  Gated Delta Networks: Improving Mamba2 with Delta Rule. arXiv:2412.06464 .
- Gated Linear Attention (the FNO plus GLA recall layer): Yang, S., Wang, B., Shen, Y., Panda, R.,
  Kim, Y. (2024). arXiv:2312.06635 .
- flash-linear-attention (fla): the Gated DeltaNet Triton kernel used for GPU training of the GDN
  variant; CPU inference uses the pure torch recurrence in `modeling.py`.
  https://github.com/fla-org/flash-linear-attention (MIT).
- RDKit (SMILES canonicalization and validation, template extraction and application via rdchiral,
  SAScore labels): https://www.rdkit.org (BSD-3-Clause). rdchiral: Coley, C. W., Green, W. H.,
  Jensen, K. F. (2019). JCIM, 59(6), 2529-2537. https://doi.org/10.1021/acs.jcim.9b00286 .
- Atom wise SMILES tokenizer: Schwaller, P., Laino, T., Gaudin, T., et al. (2019). Molecular
  Transformer. ACS Cent. Sci., 5(9), 1572-1583. https://doi.org/10.1021/acscentsci.9b00576 .
- PyTorch: Paszke, A., Gross, S., Massa, F., et al. (2019). NeurIPS. Weights ship as safetensors.

Note: Landmark attention is not used in FELA-Chem. The SSSL layer pattern in `config.json`
interleaves the FNO mixer with a single recall layer (GLA or Gated DeltaNet) every fourth block;
there is no landmark or sliding window attention layer in `modeling.py`.

# Reference baselines (cited for comparison, not retrained by us)

- Yield-BERT: Schwaller, P., Vaucher, A. C., Laino, T., Reymond, J.-L. (2021). Mach. Learn.: Sci.
  Technol., 2, 015016. https://doi.org/10.1088/2632-2153/abc81d .
- LocalRetro: Chen, S., Jung, Y. (2021). JACS Au, 1(10), 1612-1620.
  https://doi.org/10.1021/jacsau.1c00246 .
- GLN: Dai, H., Li, C., Coley, C., Dai, B., Song, L. (2019). NeurIPS. arXiv:2001.01408 .
- neuralsym: Segler, M. H. S., Waller, M. P. (2017). Chem. Eur. J., 23(25), 5966-5971.
  https://doi.org/10.1002/chem.201605499 .
- retro star: Chen, B., Li, C., Dai, H., Song, L. (2020). ICML. arXiv:2006.15820 .
- D-MPNN / Chemprop: Yang, K., Swanson, K., Jin, W., et al. (2019). JCIM, 59(8), 3370-3388.
  https://doi.org/10.1021/acs.jcim.9b00237 .
- Uni-Mol: Zhou, G., Gao, Z., Ding, Q., et al. (2023). ICLR.
  https://openreview.net/forum?id=6K2RM6wVqKu .
- MolFormer-XL: Ross, J., Belgodere, B., Chenthamarakshan, V., et al. (2022). Nature Machine
  Intelligence, 4, 1256-1264. https://doi.org/10.1038/s42256-022-00580-7 .
- ChemBERTa-2: Ahmad, W., Simon, E., Chithrananda, S., Grand, G., Ramsundar, B. (2022).
  arXiv:2209.01712 .

# Model family

This is part of the FELA family from Lowdown Labs: one FNO architecture across many modalities,
all CPU native and subquadratic. This repo is pushed as `lowdown-labs/fela-chemistry`. The sibling
repos are independently trained per modality and share no weights, so none carries a `base_model`
link.

# License

Released under the Lowdown Labs Lovely License 1.0 (CC BY-NC 4.0 plus Hippocratic License 3.0).
See LICENSE. For most LL models, a commercial license may be available; contact Lowdown Labs.