TriomphantV2-ECE-78B-SLERP-V1
This model has been produced by:
- ROBERGE Marial, engineering student at French Engineering School ECE
- ESCRIVA Mathis, engineering student at French Engineering School ECE
- LALAIN Youri, engineering student at French Engineering School ECE
- RAGE LILIAN, engineering student at French Engineering School ECE
- HUVELLE Baptiste, engineering student at French Engineering School ECE
Under the supervision of:
- Andre-Louis Rochet, Lecturer at ECE & Co-Founder of TW3 Partners
- Paul Lemaistre, CTO of TW3 Partners
With the contribution of:
- ECE engineering school as sponsor and financial contributor
- François STEPHAN as director of ECE
- Gérard REUS as acting director of iLAB
- Matthieu JOLLARD ECE Alumni
- Louis GARCIA ECE Alumni
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
slices:
- sources:
- model: MaziyarPanahi/calme-3.2-instruct-78b
layer_range: [0, 80]
- model: dfurman/CalmeRys-78B-Orpo-v0.1
layer_range: [0, 80]
merge_method: slerp
base_model: MaziyarPanahi/calme-3.2-instruct-78b
parameters:
t:
- filter: self_attn
value: [0, 0.25, 0.5, 0.75, 1]
- filter: mlp
value: [1, 0.75, 0.5, 0.25, 0]
- value: 0.5
dtype: bfloat16
Supervisory structure
The iLab (intelligence Lab) is a structure created by the ECE and dedicated to artificial intelligence
About ECE
ECE, a multi-program, multi-campus, and multi-sector engineering school specializing in digital engineering, trains engineers and technology experts for the 21st century, capable of meeting the challenges of the dual digital and sustainable development revolutions.
Le_Triomphant-ECE-TW3
TriomphantV2-ECE-78B-SLERP-V1 is a merge of the following models using mergekit:
MaziyarPanahi/calme-3.2-instruct-78bdfurman/CalmeRys-78B-Orpo-v0.1