Motif-Technologies
/

activation

Model card Files Files and versions

activation / tests

26.2 kB

8 contributors

History: 12 commits

wyldecat's picture

feat: support sequence parallel with fused_add_rms_norm

151bb5a 3 months ago

__init__.py
0 Bytes

feat(poly-norm): Add PolyNorm 7 months ago
allclose_default.py
512 Bytes

Add fusion (#3) 4 months ago
pytest.ini
45 Bytes

fix(poly-norm): calc param grad explicitly 7 months ago
test_fused_add_rms_norm.py
3.6 kB

refactor(activation): change fused_add_rms_norm and fused_add_rms_norm_backward to out-place operations 3 months ago
test_fused_add_rms_norm_sequence_parallel.py
6.3 kB

feat: support sequence parallel with fused_add_rms_norm 3 months ago
test_fused_mul_poly_norm.py
4.13 kB

Add fusion (#3) 4 months ago
test_poly_norm.py
2.55 kB

Add fusion (#3) 4 months ago
test_rms_norm.py
1.94 kB

feat: make rms_norm as out-place 3 months ago
test_rms_norm_sequence_parallel.py
4.58 kB

feat: support sequence parallel with fused_add_rms_norm 3 months ago
utils.py
2.56 kB

Add fusion (#3) 4 months ago