Code LLM

Graphics Code Generation

Goal was to use LLMs for three.js code generation given the user prompts supporting further adjustments and a debugging feature to ensure executability (exploring unit tests with LLM’s emergent self refining and self evaluation capabilities).

Direct RAG could not be applied since code response does require a level of modification and customization as per the prompt. Fine Tuned 7B llama3 model (2x40GB A100) with three js dataset having carefully designed diverse prompts and commented code. Improved performance compared to llama3 7B, 70B, codestral for the task tackling problems of module imports, assets loading, canvas element etc.

Sample Outputs

Some of the user prompts from the test samples and corresponding outputs are shown below:

Prompt #1: Create a Three.js scene featuring a spotlight located at (5, 10, 5) casting shadows of a cube moving along the x-axis from -5 to 5 units onto a plane. The cube should have a Lambert material with a green color. Enable shadow reception and casting for both the cube and the plane.
Prompt #2: Create a 3D scene with a cube that has a texture and rotates on both the X and Y axes.
Prompt #3: Create a dynamic 3D scene using Three.js that features a rotating torus, with the ability to adapt to varying screen dimensions and display resolutions.