Join our Discord! https://discord.gg/Nbv9pQ88Xb
2500+ members strong 💪 Now with more channels! A hub for users and makers alike!
Lazarus 2407 100B 🌖
To the brave men and women who gave their lives so we could begin again.
Links
- Original: https://huggingface.co/TheDrummer/Lazarus-2407-100B
- GGUF: https://huggingface.co/TheDrummer/Lazarus-2407-100B-GGUF
- iMatrix (Recommended): https://huggingface.co/bartowski/Lazarus-2407-100B
A pruned version of Mistral Large 2407 123B
- Theory: https://arxiv.org/abs/2403.17887
- Practice: https://github.com/arcee-ai/PruneMe
slices:
- sources:
- model: ./largestral
layer_range: [0, 54]
- sources:
- model: ./largestral
layer_range: [70,88]
merge_method: passthrough
dtype: bfloat16
block_start,block_end,average_distance
1,17,0.42018115234375
2,18,0.40477392578125
3,19,0.3955498046875
4,20,0.39202392578125
5,21,0.38614453125
6,22,0.380365234375
7,23,0.37550830078125
8,24,0.36995361328125
9,25,0.36802099609375
10,26,0.36337158203125
11,27,0.3570400390625
12,28,0.35895654296875
13,29,0.360716796875
14,30,0.36415234375
15,31,0.3656328125
16,32,0.3657451171875
17,33,0.36321337890625
18,34,0.36210791015625
19,35,0.3613828125
20,36,0.36101806640625
21,37,0.36395361328125
22,38,0.36294921875
23,39,0.36490673828125
24,40,0.36392578125
25,41,0.3616650390625
26,42,0.360994140625
27,43,0.357458984375
28,44,0.35728759765625
29,45,0.35514013671875
30,46,0.35244140625
31,47,0.34866162109375
32,48,0.3489375
33,49,0.3480888671875
34,50,0.34322607421875
35,51,0.3366806640625
36,52,0.331208984375
37,53,0.31834423828125
38,54,0.30483203125
39,55,0.29027587890625
40,56,0.2789296875
41,57,0.263868408203125
42,58,0.25043017578125
43,59,0.23816162109375
44,60,0.223565185546875
45,61,0.216619140625
46,62,0.212794189453125
47,63,0.205733642578125
48,64,0.200558837890625
49,65,0.196917236328125
50,66,0.1951201171875
51,67,0.192659423828125
52,68,0.191610595703125
53,69,0.1910341796875
54,70,0.190966064453125
55,71,0.1920712890625
56,72,0.1935439453125
57,73,0.1973837890625
58,74,0.200970703125
59,75,0.205339111328125
60,76,0.216475341796875
61,77,0.218605224609375
62,78,0.22363916015625
63,79,0.229986083984375
64,80,0.23776171875
65,81,0.24455322265625
66,82,0.2542783203125
67,83,0.2635498046875
68,84,0.27629443359375
69,85,0.29324267578125
70,86,0.32201220703125
71,87,0.36908154296875
72,88,0.4007587890625