TLDR;
- Updated to V2:
- Now it can deal with Leetcode problems, (still far from being perfect, check the example).
- The model is quite fun to play with as it seems pretty competent:
- It's much better than bigger general models but not perfect.
- It shows a good understanding about coding:
- It can correct itself or improve the code if asked for it.
- It has a good knowledge about Python and Python libraries:
- You can for example, ask the model to create a tutorial about some python library.
- Getting what you want from the model can be a hit or miss:
- Sometimes you'll have to fight extensively with the way you write the prompt to get what you want.
- Sometimes you won't be able to make the model do what you want.
Objective:
This is a proof of concept to see how far LLM's on the smaller side can go when fine-tuned for code generation and understanding.
Foundation:
The base model is based on OpenLlama-3b and can be found here and it has been fine-tuned with LoRA using axolotl. Unlike the base model the prompt is based on Alpaca.
Examples:
Next there are some examples to show what the model is currently capable of.
(I have edited the quotes to be able to show the prompts correctly here the model outputs proper markdown.)
The first example shows a non-trivial example of the model generating code.
(Depending on the complexity of the required task the generated code might not work. For example, making the model write code to move the square
around has proven to be quite difficult, the latest version can produce valid code for this.)
Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
Write Python code to draw a square on the screen using Pygame.
### Response:
Sure, here's a Python code to draw a square on the screen using PyGame.
'''python
import pygame
pygame.init()
window = pygame.display.set_mode((800, 600))
def draw(x, y):
pygame.draw.rect(window, (255, 170, 170), [x, y, 50, 50])
running = True
while running:
for event in pygame.event.get():
if event.type == pygame.QUIT:
running = False
# Clear the screen
window.fill((0, 0, 0))
# Draw a square
x, y = 40, 30
draw(x, y)
# Update the display
pygame.display.flip()
'''
Next there are a few samples to show how the model understands the code.
In this first example the model is able to answer correctly what shape is being drawn and even tells the position correctly, it gets the color wrong in this case
Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
What shape does the following code draw on the screen?
import pygame
pygame.init()
window = pygame.display.set_mode((800, 600))
def draw(x, y):
pygame.draw.rect(window, (255, 170, 170), [x, y, 50, 50])
running = True
while running:
for event in pygame.event.get():
if event.type == pygame.QUIT:
running = False
# Clear the screen
window.fill((0, 0, 0))
# Draw a square
x, y = 40, 30
draw(x, y)
# Update the display
pygame.display.flip()
### Response:
The code will draw a red square at position (40, 30).
To the question about how to improve the code the model answers by adding code to limit the number of frames per second.
Below there are some samples generated by the model:
Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
How could you improve the following code?
'''python
import pygame
from pygame.locals import *
pygame.init()
window = pygame.display.set_mode((800, 600))
def main():
done = False
while not done:
for event in pygame.event.get():
if event.type == QUIT:
done = True
# Clear the screen
window.fill((0, 0, 0))
# Draw the square
x, y = 150, 140
pygame.draw.rect(window, (255, 0, 0), [x, y, 50, 50])
# Update the display
pygame.display.flip()
if __name__ == '__main__':
main()
'''
### Response:
You can use `pygame.time.Clock` to control the frames per second and limit the number of frames to prevent your program from freezing. Here's a modified version of your code with this addition:
'''python
import pygame
from pygame.locals import *
pygame.init()
window = pygame.display.set_mode((800, 600))
clock = pygame.time.Clock()
def main():
done = False
targetFPS = 30
currentFPS = clock.tick(targetFPS) / targetFPS
maxFrames = int(currentFPS * 60) + 1
while not done:
for event in pygame.event.get():
if event.type == QUIT:
done = True
# Limit frames to targetFPS
if currentFPS > targetFPS:
currentFPS = targetFPS
maxFrames = min(maxFrames, targetFPS * 60)
# Clear the screen
window.fill((0, 0, 0))
# Draw the square
x, y = 150, 140
pygame.draw.rect(window, (255, 0, 0), [x, y, 50, 50])
# Update the display
pygame.display.flip()
if __name__ == '__main__':
main()
'''
Finally an easy leetcode problem. The latest version of the model can solve the problem (with a O(n^2) solution). Given the implementation it shouldn't be hard for the model to turn nums into a set and get the O(n) solution, but it's always a game of try and error.
Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
### Instruction:
Solve the following problem using Python 3. Write only the code.
### Input:
Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.
You may assume that each input would have exactly one solution, and you may not use the same element twice.
You can return the answer in any order.
Example 1:
Input: nums = [2,7,11,15], target = 9
Output: [0,1]
Explanation: Because nums[0] + nums[1] == 9, we return [0, 1].
Example 2:
Input: nums = [3,2,4], target = 6
Output: [1,2]
Example 3:
Input: nums = [3,3], target = 6
Output: [0,1]
Constraints:
2 <= nums.length <= 10^4
-10^9 <= nums[i] <= 10^9
-10^9 <= target <= 10^9
Only one valid answer exists.
### Response:
Certainly, here is a potential way to implement a program in Python that addresses the problem at hand.
'''python
class Solution:
def twoSum(self, nums: List[int], target: int) -> List[int]:
l = []
for i in range(len(nums)):
if target-nums[i] in l:
return [l.index(target-nums[i]), i]
else:
l.append(nums[i])
return [-1,-1]
'''