2024-08-15: This is now the base model. The model with python RDKit training is created as Gemma_ChemWiz_rdkit_16bit. This model is now frozen, with a next model being created from this with "new" additional skills such as developing chemistry specific applications using rdkit. 2024-08-15: Splitting model today. This model will be the base ChemWiz Model. The first vintage that I will create today will be the RDKit coder, using my custom data set. Once I have this model, I will create a dev critic out of that model. Will then start a set of tests with Microsoft Autogen testing to test if the addition of a coding critic would improve the results. Still toying with the idea of creating a ChemWiz critic to see if it improves the outcomes and reduces halicinations. But lets see.
2024-08-13: Taking the model through second round of AI4Chem/ChemData700K, I am amazed how the model seem to converge and the suddenly it does not. I suspect in nextfew days that it will converge. I am quite keen to see this happen. The results of chemichal smiles are very low at this point.
2024-08-12: The medalpaca/medical_meadow_mediqa data set was also used, but the model converged on this in less than one epoch, only 1400 steps of training was concluded, in future versions and editions I might elect to exclude this data set, but it is included in this version.
2024-08-12: Model is being fine tuned on chemical memory, rather than chmistry reasoning. Using the AI4Chem/ChemData700K dataset. Model is still halucinating chemical formulas, I will fine tune model on a few more data sets to see if this affects/reduce the halicinations.
2024-08-09: Model is still being fine tuned for logical reasoning, the responses being recieved at this time seem to be in line with the training set, so the model at this time for instance do not jump straight into an answer, but started "unpacking" the instruction before persorming a task, such as coding. Nothing this model creates at this time should be used for any production purpose, highly experimental.
base_model: unsloth/codegemma-7b-bnb-4bit language:
- en license: apache-2.0 tags:
- text-generation-inference
- transformers
- unsloth
- gemma
- trl datasets:
- AI-MO/NuminaMath-CoT
Uploaded model
- Developed by: dbands
- License: apache-2.0
- Finetuned from model : unsloth/codegemma-7b-bnb-4bit
This gemma model was trained 2x faster with Unsloth and Huggingface's TRL library.