Large Language Models (LLMs) are good with code generations. Sometimes LLMs do make mistakes in code generation. How about if they can give detailed explanation along with the code. This is what I have tried over here. The base Llama-3-8B model was used for training purpose. It is trained on around 23000+ set of codes. Each set having 2 conversations. This data was generated using GPT-3.5, GPT-4 etc. This conversation is in Vicuna/ShareGPT format. Each set, along with code, has detailed explanation.
Training framework: axolotl
Training details:
- epochs: 3
- learning rate: 0.0002
- learning rate scheduler type: cosine
- warmup_steps: 10
- cutoff len (i.e. context length): 4096
- gradient_accumulation_steps: 4
- micro_batch_size: 4
- fine-tuning type: full parameters
- optimizer: adamw_bnb_8bit
Examples
The following are some examples generated by our CodeLlama3-8B-python model:
User:Based on the principle of the quicksort algorithm, implement a quicksort algorithm in Python without using any built-in library functions. The input should be a list where all elements are of float type, and it should return a sorted list. Remember to add comments to the code.
Assistant: quicksort algorithm: """ A sorting algorithm that works by selecting a pivot element and partitioning the array around this pivot element. Then, recursively applying the same process to the two partitions until the entire array is sorted. """
def quicksort(arr):
if len(arr) <= 1:
return arr
# Selecting a pivot element
pivot = arr[len(arr)//2]
# Partitioning the array around the pivot element
left = [x for x in arr if x > pivot]
middle = [x for x in arr if x == pivot]
right = [x for x in arr if x < pivot]
# Recursively applying the quicksort algorithm to the left and right partitions
return quicksort(left) + middle + quicksort(right)
The quicksort function takes an input list arr and returns a sorted list. It uses the quicksort algorithm to achieve this.
User:Based on the principle of the KMP algorithm, implement a KMP algorithm in Python and add comments to the code.
Assistant:
def kmp_algorithm(pattern, text):
n = len(pattern)
m = len(text)
# Build the table
table = [0] * m
length = 1
i = 1
while i < n:
if pattern[i] == text[length]:
table[length] = i
length += 1
i += 1
else:
if length > 1:
length = min(length, table[i - 1])
table[i] = length
i += 1
# Find all matches
matches = []
i = 0
j = 0
while i < n and j < m:
if pattern[i] == text[j]:
i += 1
j += 1
if i == n:
matches.append(text[j - i: j])
return matches