temperature gradient Transformer
temperature based ONNX implementation for bert prompt.
- Input
- 6839-dim embedding
- Encoder
- 61 x Transformer with 12 heads
- Output
- recall projection
Training config
optimizer=SGD, lr=0.702, scheduler=polynomial, warmup=1992