Recreating the Windows 3D Pipes Screensaver With React / THREE.js 

If you were a kid in the early 2000s, there’s a good chance you remember sitting in front of your family’s Windows XP computer, mesmerized by the seemingly endless combinations generated by the Windows 3D Pipes screensaver. If you're anything like me, it might have left a lasting curiosity as to how exactly those random pipes were being created, what technologies were required to make such a thing, and how we can make our own version of it. In this blog post we'll try to answer some of those questions and attempt to recreate a modern version of that beloved screensaver using Three.js, a powerful JavaScript library for rendering 3D graphics in the browser.

Before diving into the code, ensure you have the following prerequisites:

"Node.js environment" "NEXT.js (we're using 15 + App Router)" "Typescript" "THREE.js"


The first thing we need to do is create our project. We're using NEXT.js so this can be done pretty easily using npx create-next-app

You'll want to run that command in your terminal and follow the prompts on the screen. You should configure it so that the project is using the App Router, Typescript, and Tailwind CSS. The other settings won't matter for the sake of this tutorial.

Next, we need to install the THREE.js package and its Typescript types with the following commands:

npm i three

npm i @types/three

npm i --save-dev @types/three

Now that we have all of that installed, we are ready to jump into the project! If you ran npx create-next-app and ensured you chose the App Router for your project, you should be able to navigate to your project folder and find src/app/page.tsx. This file will be served at the index route of your project when you run npm run dev. By default, there is some example project code located in that file that we can get rid of. Once you've cleared that file, lets begin building out our project by defining a functional component and declaring this page to be client side only for the app router:   

"use client"
import Head from "next/head";

const PipesPage: React.FC = () => {
  return (
    <>
      <Head>
        <title>Lucas Miller - 3D Pipes</title>
        <meta name="description" content="3D Pipes" />
      </Head>
    </>
  );
}

export default PipesPage

This is the skeleton of the page that our 3D scene will be rendered on. Now that its established lets move on to creating our 3D scene by importing THREE.js and adding a camera, lighting, renderer, and orbit controls that will render our scene and each animation to a canvas on the page: 

"use client"
import React, { useEffect, useRef } from "react";
import Head from "next/head";
import * as THREE from "three";
import { OrbitControls } from "three/examples/jsm/controls/OrbitControls.js";

const PipesPage: React.FC = () => {
  const containerRef = useRef<HTMLDivElement | null>(null);
  useEffect(() => {
    if (!containerRef.current) return;

    // Initialize the 3D scene
    const scene = new THREE.Scene();
    const camera = new THREE.PerspectiveCamera(
      45,
      window.innerWidth / window.innerHeight,
      1,
      1000
    );
    camera.position.set(20, 20, 20);

    // Initialize WebGL renderer
    const renderer = new THREE.WebGLRenderer({ antialias: true });
    renderer.setSize(window.innerWidth, window.innerHeight);
    renderer.setClearColor(0x000000); // Set background color
    containerRef.current.appendChild(renderer.domElement);

    // Add orbit controls for user interaction
    const controls = new OrbitControls(camera, renderer.domElement);
    controls.enableDamping = true;
    controls.dampingFactor = 0.1;

    // Add ambient and directional lighting
    scene.add(new THREE.AmbientLight(0x111111));
    const directionalLight = new THREE.DirectionalLight(0xffffff, 0.9);
    directionalLight.position.set(-1.2, 1.5, 0.5);
    scene.add(directionalLight);

    // Animation loop
    const animate = () => {

      // Render and update controls
      controls.update();
      renderer.render(scene, camera);
      requestAnimationFrame(animate);
    };

    animate(); // Start the animation loop
  }, []);

  return (
    <>
      <Head>
        <title>Lucas Miller - 3D Pipes</title>
        <meta name="description" content="3D Pipes" />
      </Head>
      <div ref={containerRef} style={{ width: "100vw", height: "100vh" }} />
    </>
  );
}

export default PipesPage

At this point, we have utilized THREE.js to initialize a basic scene with a camera, lighting, renderer, and orbit controls which will allow us to see any 3D objects we add to the scene later on as well as rotate the camera and zoom around them. We've also added an animation loop which we'll expand later to control animating our pipes to grow and change direction. All of this is defined in a useEffect hook with an empty dependency array to ensure it runs only once on page load.

At this stage you should have no errors and be able to run your project with npm run dev and check it out at localhost:3000/ . You should simply see an empty black screen though because we have no objects in the scene! Next up, we'll begin building a Pipe class that encapsulates logic for handling pipe creation and behavior such as creating random starting points, handling random pipe growth, and ensuring pipes don't collide with themselves or others. We'll also implement a 3D Cube grid which the pipes will be placed within the bounds of. 

"use client"
import React, { useEffect, useRef } from "react";
import Head from "next/head";
import * as THREE from "three";
import { OrbitControls } from "three/examples/jsm/controls/OrbitControls.js";

// Generates a random integer between x1 and x2 (inclusive)
function randomInteger(x1: number, x2: number): number {
  return Math.round(Math.random() * (x2 - x1) + x1);
}

// Pipe Class: Represents a single pipe structure that grows within the scene
class Pipe {
  currentPosition: THREE.Vector3; // The current position of the pipe's endpoint
  material!: THREE.Material;      // Material used for the pipe's geometry
  object3D: THREE.Object3D;       // Group containing all the pipe's segments and joints
  lastDirection: THREE.Vector3 | null = null; // Tracks the last growth direction to avoid backtracking

  constructor(
    scene: THREE.Scene,
    setLocationOccupied: (pos: THREE.Vector3) => void,
    gridBounds: THREE.Box3
  ) {
    // Generate a random initial position within gridBounds
    const randomX = randomInteger(gridBounds.min.x, gridBounds.max.x);
    const randomY = randomInteger(gridBounds.min.y, gridBounds.max.y);
    const randomZ = randomInteger(gridBounds.min.z, gridBounds.max.z);
    this.currentPosition = new THREE.Vector3(randomX, randomY, randomZ);

    // Create a 3D object group for the pipe and add it to the scene
    this.object3D = new THREE.Object3D();
    scene.add(this.object3D);

    // Create a random material color for this pipe
    const color = randomInteger(0, 0xffffff);
    const emissive = new THREE.Color(color).multiplyScalar(0.3);
    this.material = new THREE.MeshPhongMaterial({
      specular: 0xa9fcff,
      color: color,
      emissive: emissive,
      shininess: 100,
    });

    // Add the starting joint to the pipe and mark the position as occupied
    this.createJoint(this.currentPosition);
    setLocationOccupied(this.currentPosition);
  }

  // Creates a spherical joint at the given position
  createJoint(position: THREE.Vector3) {
    const geometry = new THREE.SphereGeometry(0.3, 16, 16);
    const mesh = new THREE.Mesh(geometry, this.material);
    mesh.position.copy(position);
    this.object3D.add(mesh);
  }

  // Creates a cylindrical segment between two points
  createSegment(from: THREE.Vector3, to: THREE.Vector3) {
    const delta = new THREE.Vector3().subVectors(to, from);
    const length = delta.length();
    const geometry = new THREE.CylinderGeometry(0.15, 0.15, length, 16);

    const mesh = new THREE.Mesh(geometry, this.material);
    mesh.position.addVectors(from, delta.clone().multiplyScalar(0.5));
    mesh.lookAt(to);           // Aligns the segment to point towards its destination
    mesh.rotateX(Math.PI / 2); // Adjusts orientation to match cylinder's default alignment
    this.object3D.add(mesh);
  }

  // Attempts to grow the pipe in a random direction, returns true if successful
  update(
    gridBounds: THREE.Box3,
    setLocationOccupied: (pos: THREE.Vector3) => void,
    isLocationOccupied: (pos: THREE.Vector3) => boolean
  ): boolean {
    const directions = [
      new THREE.Vector3(1, 0, 0),
      new THREE.Vector3(-1, 0, 0),
      new THREE.Vector3(0, 1, 0),
      new THREE.Vector3(0, -1, 0),
      new THREE.Vector3(0, 0, 1),
      new THREE.Vector3(0, 0, -1),
    ];

    // Filter out the last direction to avoid backtracking
    const validDirections = directions.filter(
      (dir) => !this.lastDirection || !dir.equals(this.lastDirection)
    );

    // Randomize the direction order
    const shuffledDirections = validDirections.sort(() => Math.random() - 0.5);

    // Try each direction in random order
    for (const direction of shuffledDirections) {
      const segmentLength = Math.floor(Math.random() * 10) + 1; // Random segment length
      const scaledDirection = direction.clone().multiplyScalar(segmentLength);
      const newPosition = this.currentPosition.clone().add(scaledDirection);

      // Check if the path is valid
      if (!this.isPathValid(this.currentPosition, scaledDirection, gridBounds, isLocationOccupied))
        continue;

      // Reserve the path, create the segment and joint
      this.reservePath(this.currentPosition, scaledDirection, setLocationOccupied);
      this.createSegment(this.currentPosition, newPosition);
      this.createJoint(newPosition);

      setLocationOccupied(newPosition);   // Mark the endpoint as occupied
      this.currentPosition = newPosition; // Update the pipe's endpoint
      this.lastDirection = direction;     // Update the last growth direction

      return true; // Growth occurred
    }

    return false; // No growth possible
  }

  // Checks if a proposed path is valid (unoccupied and within bounds)
  isPathValid(
    start: THREE.Vector3,
    direction: THREE.Vector3,
    gridBounds: THREE.Box3,
    isLocationOccupied: (pos: THREE.Vector3) => boolean
  ): boolean {
    const steps = Math.abs(direction.x || direction.y || direction.z); // Path length
    const stepVector = direction.clone().normalize(); // Unit step vector

    // Check each intermediate point along the path
    for (let i = 1; i <= steps; i++) {
      const intermediate = start.clone().add(stepVector.clone().multiplyScalar(i));
      if (isLocationOccupied(intermediate) || !gridBounds.containsPoint(intermediate)) {
        return false; // Path is invalid
      }
    }

    return true; // Path is valid
  }

  // Reserves a path by marking intermediate points as occupied
  reservePath(
    start: THREE.Vector3,
    direction: THREE.Vector3,
    setLocationOccupied: (pos: THREE.Vector3) => void
  ) {
    const steps = Math.abs(direction.x || direction.y || direction.z);
    const stepVector = direction.clone().normalize();

    for (let i = 1; i <= steps; i++) {
      const intermediate = start.clone().add(stepVector.clone().multiplyScalar(i));
      setLocationOccupied(intermediate);
    }
  }
}
 

const PipesPage: React.FC = () => {
  const containerRef = useRef<HTMLDivElement | null>(null);
  
  const numOfPipes = 5; // Control how many pipes should be created

  const boxSize = 10;   // Control the dimension of the 3D Cube the pipes are bound by

  useEffect(() => {
    if (!containerRef.current) return;

    // Initialize the 3D scene
    const scene = new THREE.Scene();
    const camera = new THREE.PerspectiveCamera(
      45,
      window.innerWidth / window.innerHeight,
      1,
      1000
    );
    camera.position.set(20, 20, 20);

    // Initialize WebGL renderer
    const renderer = new THREE.WebGLRenderer({ antialias: true });
    renderer.setSize(window.innerWidth, window.innerHeight);
    renderer.setClearColor(0x000000); // Set background color
    containerRef.current.appendChild(renderer.domElement);

    // Add orbit controls for user interaction
    const controls = new OrbitControls(camera, renderer.domElement);
    controls.enableDamping = true;
    controls.dampingFactor = 0.1;

    // Add ambient and directional lighting
    scene.add(new THREE.AmbientLight(0x111111));
    const directionalLight = new THREE.DirectionalLight(0xffffff, 0.9);
    directionalLight.position.set(-1.2, 1.5, 0.5);
    scene.add(directionalLight);

    // Define the grid boundaries, this is the 3D Cube grid upon which the
    // pipes should be rendered within
    const gridBounds = new THREE.Box3(
      new THREE.Vector3(-boxSize, -boxSize, -boxSize),
      new THREE.Vector3(boxSize, boxSize, boxSize)
    );
   
    // Track which positions on the grid are occupied by some segment of a pipe
    const nodes: Record<string, number> = {};

    // Marks a position on the grid as occupied
    const setLocationOccupied = (position: THREE.Vector3) => {
      const key = `${Math.round(position.x)},${Math.round(position.y)},${Math.round(position.z)}`;
      nodes[key] = 1; // Mark the location as occupied
    };

    // Checks if a position on the grid is occupied
    const isLocationOccupied = (position: THREE.Vector3): boolean => {
      const key = `${Math.round(position.x)},${Math.round(position.y)},${Math.round(position.z)}`;
      return nodes[key] === 1; // Return true if occupied, false otherwise
    };

    // Animation loop
    const animate = () => {

      // Render and update controls
      controls.update();
      renderer.render(scene, camera);
      requestAnimationFrame(animate);
    };

    animate(); // Start the animation loop
  }, []);

  return (
    <>
      <Head>
        <title>Lucas Miller - 3D Pipes</title>
        <meta name="description" content="3D Pipes" />
      </Head>
      <div ref={containerRef} style={{ width: "100vw", height: "100vh" }} />
    </>
  );
}

export default PipesPage

The Pipe class is designed to encapsulate all the functionality necessary for generating and growing a 3D pipe structure within a scene. Its implementation focuses on managing the pipe's geometry, material, position tracking, and collision avoidance logic. At its core, the class relies on Three.js to create 3D objects such as joints (spheres) and segments (cylinders), which are dynamically added to the scene as the pipe grows. Each pipe starts at a random position within the defined grid bounds, making use of helper functions like randomInteger to generate coordinates within valid ranges.

The growth logic of the Pipe class is central to its functionality. The update method randomly selects a direction from the six possible axes while avoiding backtracking into its previous position. It validates potential paths using the isPathValid method, which checks each intermediate point along the proposed growth path to ensure it is within bounds and not already occupied. Once a valid path is found, the reservePath method marks all intermediate points as occupied, and the pipe's geometry is extended by creating a cylindrical segment and a spherical joint at the endpoint. This ensures that the pipes grow in a structured yet randomized way while staying within the 3D Cube's bounds and avoiding collision with other pipe segments.

With the pipe class and the 3D Cube grid in place, we now want to create instances of our pipes that have random starting positions on the grid, attempt to make the pipes grow, and animate their progress along the way! We also want to check if pipes have stopped growing at any point. The grid bounds and collision prevention makes it so that pipes have a natural limit to their ability to grow and when all pipes stop growing we should reset our scene and start over again, just like the original 3D Pipes screensaver! 

"use client"
import React, { useEffect, useRef } from "react";
import Head from "next/head";
import * as THREE from "three";
import { OrbitControls } from "three/examples/jsm/controls/OrbitControls.js";

// Generates a random integer between x1 and x2 (inclusive)
function randomInteger(x1: number, x2: number): number {
  return Math.round(Math.random() * (x2 - x1) + x1);
}

// Pipe Class: Represents a single pipe structure that grows within the scene
class Pipe {
  currentPosition: THREE.Vector3; // The current position of the pipe's endpoint
  material!: THREE.Material;      // Material used for the pipe's geometry
  object3D: THREE.Object3D;       // Group containing all the pipe's segments and joints
  lastDirection: THREE.Vector3 | null = null; // Tracks the last growth direction to avoid backtracking

  constructor(
    scene: THREE.Scene,
    setLocationOccupied: (pos: THREE.Vector3) => void,
    gridBounds: THREE.Box3
  ) {
    // Generate a random initial position within gridBounds
    const randomX = randomInteger(gridBounds.min.x, gridBounds.max.x);
    const randomY = randomInteger(gridBounds.min.y, gridBounds.max.y);
    const randomZ = randomInteger(gridBounds.min.z, gridBounds.max.z);
    this.currentPosition = new THREE.Vector3(randomX, randomY, randomZ);

    // Create a 3D object group for the pipe and add it to the scene
    this.object3D = new THREE.Object3D();
    scene.add(this.object3D);

    // Create a random material color for this pipe
    const color = randomInteger(0, 0xffffff);
    const emissive = new THREE.Color(color).multiplyScalar(0.3);
    this.material = new THREE.MeshPhongMaterial({
      specular: 0xa9fcff,
      color: color,
      emissive: emissive,
      shininess: 100,
    });

    // Add the starting joint to the pipe and mark the position as occupied
    this.createJoint(this.currentPosition);
    setLocationOccupied(this.currentPosition);
  }

  // Creates a spherical joint at the given position
  createJoint(position: THREE.Vector3) {
    const geometry = new THREE.SphereGeometry(0.3, 16, 16);
    const mesh = new THREE.Mesh(geometry, this.material);
    mesh.position.copy(position);
    this.object3D.add(mesh);
  }

  // Creates a cylindrical segment between two points
  createSegment(from: THREE.Vector3, to: THREE.Vector3) {
    const delta = new THREE.Vector3().subVectors(to, from);
    const length = delta.length();
    const geometry = new THREE.CylinderGeometry(0.15, 0.15, length, 16);

    const mesh = new THREE.Mesh(geometry, this.material);
    mesh.position.addVectors(from, delta.clone().multiplyScalar(0.5));
    mesh.lookAt(to);           // Aligns the segment to point towards its destination
    mesh.rotateX(Math.PI / 2); // Adjusts orientation to match cylinder's default alignment
    this.object3D.add(mesh);
  }

  // Attempts to grow the pipe in a random direction, returns true if successful
  update(
    gridBounds: THREE.Box3,
    setLocationOccupied: (pos: THREE.Vector3) => void,
    isLocationOccupied: (pos: THREE.Vector3) => boolean
  ): boolean {
    const directions = [
      new THREE.Vector3(1, 0, 0),
      new THREE.Vector3(-1, 0, 0),
      new THREE.Vector3(0, 1, 0),
      new THREE.Vector3(0, -1, 0),
      new THREE.Vector3(0, 0, 1),
      new THREE.Vector3(0, 0, -1),
    ];

    // Filter out the last direction to avoid backtracking
    const validDirections = directions.filter(
      (dir) => !this.lastDirection || !dir.equals(this.lastDirection)
    );

    // Randomize the direction order
    const shuffledDirections = validDirections.sort(() => Math.random() - 0.5);

    // Try each direction in random order
    for (const direction of shuffledDirections) {
      const segmentLength = Math.floor(Math.random() * 10) + 1; // Random segment length
      const scaledDirection = direction.clone().multiplyScalar(segmentLength);
      const newPosition = this.currentPosition.clone().add(scaledDirection);

      // Check if the path is valid
      if (!this.isPathValid(this.currentPosition, scaledDirection, gridBounds, isLocationOccupied))
        continue;

      // Reserve the path, create the segment and joint
      this.reservePath(this.currentPosition, scaledDirection, setLocationOccupied);
      this.createSegment(this.currentPosition, newPosition);
      this.createJoint(newPosition);

      setLocationOccupied(newPosition);   // Mark the endpoint as occupied
      this.currentPosition = newPosition; // Update the pipe's endpoint
      this.lastDirection = direction;     // Update the last growth direction

      return true; // Growth occurred
    }

    return false; // No growth possible
  }

  // Checks if a proposed path is valid (unoccupied and within bounds)
  isPathValid(
    start: THREE.Vector3,
    direction: THREE.Vector3,
    gridBounds: THREE.Box3,
    isLocationOccupied: (pos: THREE.Vector3) => boolean
  ): boolean {
    const steps = Math.abs(direction.x || direction.y || direction.z); // Path length
    const stepVector = direction.clone().normalize(); // Unit step vector

    // Check each intermediate point along the path
    for (let i = 1; i <= steps; i++) {
      const intermediate = start.clone().add(stepVector.clone().multiplyScalar(i));
      if (isLocationOccupied(intermediate) || !gridBounds.containsPoint(intermediate)) {
        return false; // Path is invalid
      }
    }

    return true; // Path is valid
  }

  // Reserves a path by marking intermediate points as occupied
  reservePath(
    start: THREE.Vector3,
    direction: THREE.Vector3,
    setLocationOccupied: (pos: THREE.Vector3) => void
  ) {
    const steps = Math.abs(direction.x || direction.y || direction.z);
    const stepVector = direction.clone().normalize();

    for (let i = 1; i <= steps; i++) {
      const intermediate = start.clone().add(stepVector.clone().multiplyScalar(i));
      setLocationOccupied(intermediate);
    }
  }
}

// Main Component: Renders the 3D pipes animation
const PipesPage: React.FC = () => {
  const containerRef = useRef<HTMLDivElement | null>(null);

  const numOfPipes = 5; // Control how many pipes should be created

  const boxSize = 10;   // Control the dimension of the 3D Cube the pipes are bound by

  useEffect(() => {
    if (!containerRef.current) return;

    // Initialize the 3D scene
    const scene = new THREE.Scene();
    const camera = new THREE.PerspectiveCamera(
      45,
      window.innerWidth / window.innerHeight,
      1,
      1000
    );
    camera.position.set(20, 20, 20);

    // Initialize WebGL renderer
    const renderer = new THREE.WebGLRenderer({ antialias: true });
    renderer.setSize(window.innerWidth, window.innerHeight);
    renderer.setClearColor(0x000000); // Set background color
    containerRef.current.appendChild(renderer.domElement);

    // Add orbit controls for user interaction
    const controls = new OrbitControls(camera, renderer.domElement);
    controls.enableDamping = true;
    controls.dampingFactor = 0.1;

    // Add ambient and directional lighting
    scene.add(new THREE.AmbientLight(0x111111));
    const directionalLight = new THREE.DirectionalLight(0xffffff, 0.9);
    directionalLight.position.set(-1.2, 1.5, 0.5);
    scene.add(directionalLight);

    // Define the grid boundaries, this is the 3D Cube grid upon which the
    // pipes should be rendered within
    const gridBounds = new THREE.Box3(
      new THREE.Vector3(-boxSize, -boxSize, -boxSize),
      new THREE.Vector3(boxSize, boxSize, boxSize)
    );

    // Track which positions on the grid are occupied by some segment of a pipe
    const nodes: Record<string, number> = {};

    // Marks a position on the grid as occupied
    const setLocationOccupied = (position: THREE.Vector3) => {
      const key = `${Math.round(position.x)},${Math.round(position.y)},${Math.round(position.z)}`;
      nodes[key] = 1; // Mark the location as occupied
    };

    // Checks if a position on the grid is occupied
    const isLocationOccupied = (position: THREE.Vector3): boolean => {
      const key = `${Math.round(position.x)},${Math.round(position.y)},${Math.round(position.z)}`;
      return nodes[key] === 1; // Return true if occupied, false otherwise
    };

    // Create multiple pipes
    const pipes: Pipe[] = [];
    for (let i = 0; i < numOfPipes; i++) {
      pipes.push(new Pipe(scene, setLocationOccupied, gridBounds));
    }

    // Reset the scene after inactivity
    let idleTime = 0;
    const resetScene = () => {
      pipes.forEach((pipe) => scene.remove(pipe.object3D));
      pipes.length = 0;
      Object.keys(nodes).forEach((key) => delete nodes[key]);
      for (let i = 0; i < numOfPipes; i++) {
        pipes.push(new Pipe(scene, setLocationOccupied, gridBounds));
      }
      idleTime = 0;
      console.log("Scene reset!");
    };

    // Animation loop
    const animate = () => {
      let pipesUpdated = false; // Set to true if any pipe has grown during the last animation frame

      // Update each pipe
      pipes.forEach((pipe) => {
        if (pipe.update(gridBounds, setLocationOccupied, isLocationOccupied)) {
          pipesUpdated = true;
        }
      });

      // Check if pipes haven't grown in over 3 seconds, if so reset the scene
      if (!pipesUpdated) {
        idleTime += 1 / 60;
        if (idleTime > 3) resetScene();
      } else {
        idleTime = 0;
      }

      // Render and update controls
      controls.update();
      renderer.render(scene, camera);
      requestAnimationFrame(animate);
    };

    animate(); // Start the animation loop
  }, []);

  return (
    <>
      <Head>
        <title>Lucas Miller - 3D Pipes</title>
        <meta name="description" content="3D Pipes" />
      </Head>
      <div ref={containerRef} style={{ width: "100vw", height: "100vh" }} />
    </>
  );
};

export default PipesPage;


As you can see in the previous code snippet, we create an array of pipes and instantiate numOfPipes and push them into the array. In our animation loop, we iterate over the pipes in the array and call pipes.update to attempt to grow the pipes. If its successful we set the pipesUpdated flag to true. If it is not successful we set it to false. If pipesUpdated is false, later on in the animation loop we check every iteration if that flag has changed at all. If it remains false, we begin counting how long the program is idle for without any pipe growth. If enough time passes without any pipe growth, we call the resetScene() function which will clear the current existing pipes, instantiate new ones, reset our idle time to zero and start the process over again!

That's it! At this point our program is complete and you should be able to run it and relive those memories of the coolest thing Microsoft has ever produced whenever you want!

The idea for this project came around after a coworker was tasked with adding 3D models and animations to our company website. I joked with him how cool it would be to build something like this using THREE.js on the browser. Admittedly, I have no clue how close this is to the actual 3D Pipes screensaver implementation but I am super happy with the results! It is definitely not a faithful recreation and there are some things missing that I'd really like to add in the future like the Utah teapot or the candy cane material easter eggs but perhaps I'll come back to those at a later date! For now, I'll be enjoying my recreation over at https://lucasmiller.dev/pipes and you can check out the source code over at https://github.com/Lucas-Miller/3d-pipes

A recreation of the Microsoft Windows 3D Pipes Screensaver that you might remember from your family PC in the late 90's or early 2000's! Built with React/Typescript/Nextjs/THREE.js

Natural Language Queries for Magic The Gathering Cards Using BERT + FAISS

A coworker has been working on a project to assist Magic The Gathering (MTG) players with building and enhancing their decks. Full disclosure, I know nothing about MTG but I do have a little bit of experience working with Natural Language Query (NLQ) processing thanks to a capstone project I worked on in 2020 while completing my Computer Science Degree at Kent State University. My coworker was struggling with some of the AI aspects of his project so I decided to create a simple example to demonstrate how how to utilize the BERT model and Facebook's FAISS to create an efficient similarity search to help MTG players quickly find cards that suit their needs. 

We're utilizing BERT for our language processing to create embeddings of our MTG card descriptions (these are referred to as oracle texts) and our users Natural Language Query. We will also utilize FAISS (Facebook Artificial Intelligence Similarity Search) to efficiently find the most similar results to our users NLQ. 

"Python 3.x"
"PyTorch"
"Hugging Face's Transformers library"
"FAISS (Facebook AI Similarity Search) library"
"Numpy"

You can install the necessary libraries using pip:

pip install torch transformers faiss-cpu numpy

First we need to import the packages we installed and set up our database of MTG Card oracle texts. For this example we're just going to have an array of description strings.

import torch
from transformers import BertTokenizer, BertModel
import faiss 
import numpy as np

cards = [
    # Creature Cards
    "Flying. When Angel of Serenity leaves the battlefield, return the exiled cards to their owners' hands.",
    "Trample, haste. If you would draw a card, instead reveal the top three cards of your library, then put all creature cards revealed this way into your hand and the rest on the bottom of your library in any order.",
    "Deathtouch. Lifelink. Other creatures you control have deathtouch and lifelink.",
    "Whenever you cast an Elf spell, you may create a 1/1 green Elf Warrior creature token.",
    "At the beginning of your upkeep, sacrifice a creature. You gain life equal to that creature's toughness.",

    # Instant Cards
    "Counter target spell. Its controller pays 3 life.",
    "Target creature gets +3/+3 and gains flying until end of turn.",
    "Destroy target artifact. It can't be regenerated.",
    "Target player draws two cards.",
    "Exile target creature. Its controller gains life equal to its power.",

    # Sorcery Cards
    "Destroy all creatures. They can't be regenerated.",
    "Search your library for a card, then shuffle your library and put that card on top of it.",
    "Each player discards their hand, then draws seven cards.",
    "Deal 3 damage to any target. You gain 3 life.",
    "Return target creature card from your graveyard to the battlefield.",

    # Enchantment Cards
    "At the beginning of each player's draw step, that player draws an additional card.",
    "Creatures you control get +1/+1.",
    "Whenever a creature enters the battlefield under your control, you may gain 1 life.",
    "At the beginning of your upkeep, you may put a charge counter on target artifact.",
    "Each creature assigns combat damage equal to its toughness rather than its power.",

    # Artifact Cards
    "At the beginning of your upkeep, you may put a charge counter on this artifact.",
    "Tapped: Add one mana of any color.",
    "Equipped creature gets +2/+2 and has trample and lifelink.",
    "Whenever a player casts a spell, you may put a charge counter on this artifact.",
    "Tapped: Draw a card, then discard a card."
]

Next we need to get a natural language query from the user. In our example this is just a hardcodes string but you could easily set this up to accept actual user input!

query = "I'm looking for a card that can destroy all creatures."

The BERT (Bidirectional Encoder Representations from Transformers) is a powerful Natural Language Processing model. We will use it to convert the card texts and the query into meaningful numerical representations called embeddings that let us preform mathematical operations on them like similarity searches. In the following code, we're setting up the BERT model that will do our processing as well as the tokenizer which will split our natural language query and card descriptions into chunks of text which are required for model to process them into embeddings.

# Load the pre-trained BERT model and tokenizer
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertModel.from_pretrained('bert-base-uncased')
model.eval()

Now that we have our model and tokenizer set up, we can create our embeddings for our cards. Embeddings are vector representations of words, sentences, or documents. In NLP, they are used to capture the semantic meanings of texts. Each dimension of an embedding vector represents a latent feature of the text, often capturing aspects of its syntax and semantics. In simpler terms, they capture the aspect of text that is implied in a given context such as positive or negative sentiments, general tone like anger, excitement, or formality, etc.

In the code below we're utilizing PyTorch and its tensors which are multi dimensional arrays representing our embeddings to interact with the BERT model. We're also utilizing NumPy to covert the embeddings from the PyTorch tensors into NumPy arrays because FAISS operates on NumPy arrays, not PyTorch tensors. The embeddings are stored as a matrix for efficient computation and manipulation.

# Create a matrix to hold our embeddings
embeddings_matrix = []

# Encode each card and store in the matrix
for card in cards:
    input_ids = torch.tensor(tokenizer.encode(card, add_special_tokens=True)).unsqueeze(0)
    with torch.no_grad():
        embeddings = model(input_ids)[0].squeeze(0)
        vector = torch.mean(embeddings, dim=0)
    embeddings_matrix.append(vector.numpy())

# Convert the list of embeddings into a numpy array for use with FAISS
embeddings_matrix = np.array(embeddings_matrix).astype('float32')

We're utilizing Facebook Artificial Intelligence Similarity Search (FAISS) to be able to efficiently perform similarity searches between our users NLQ and the embeddings generated from our MTG card descriptions. FAISS is very efficient at performing similarity searches on high dimensional data like those of the embeddings created by the BERT model.

In the following code, we are converting the embeddings from pytorch into a format that FAISS can utilize which is a numpy array. Next, we grab the dimensionality. The dimensionality is simply the number of distinct pieces of information each vector holds. For BERT models, this is typically 768 which means that for each word or text segment there is a vector consisting of 768 distinct values. For any given natural language query, that's a lot of values! That's why BERT and similar models are considered 'high dimensionality' models. That's also why we need to utilize FAISS as it is specifically designed for fast and efficient similarity searches when working with high dimensional vectors.

Once we have the dimensionality of our vectors we can actually create our FAISS index. In our example we are using the L2 (Euclidean) distance index which is a basic and efficient index that computes the distance between vectors as a straight line in Euclidean space. Don't worry if that sounds super complicated, we're essentially using the Euclidean Distance Formula (that you can find online if you're curious) and using it to compare the numerical values/points of our NLQ vector to those of each vector in our card descriptions embeddings. The one with the smallest value from the results of that formula computation are considered to be the most similar while the ones that have a larger value are less similar.

Finally, now that we have created our index we can insert our embeddings matrix into the index.

# Convert the list of embeddings into a numpy array for use with FAISS
embeddings_matrix = np.array(embeddings_matrix).astype('float32')

# Build the FAISS index
d = embeddings_matrix.shape[1]  # dimensionality of the vectors
index = faiss.IndexFlatL2(d)    # Use the L2 norm for the distance metric
index.add(embeddings_matrix)    # Add vectors to the index

Next up, we need to create embeddings for the users natural language query in a similar manner to how we made them for the card descriptions using BERT. This process is nearly identical to the card description embeddings except we only have one string instead of multiple.

# Encode the query using BERT
input_ids = torch.tensor(tokenizer.encode(query, add_special_tokens=True)).unsqueeze(0)
with torch.no_grad():
    embeddings = model(input_ids)[0].squeeze(0)
    query_embedding = torch.mean(embeddings, dim=0)

Now that we have our embeddings created and our FAISS index set up, we can actually perform our similarity search using our index on our embeddings. In the code below, we're using k to represent the number of results we want to show. We have it set to 5 which will show us the 5 descriptions that are most similar to our natural language query.

We will again need to convert our PyTorch tensor representation of our NLQ into a NumPy array for use with FAISS. FAISS also requires that our NumPy array for our query has the same dimensionality as our description embeddings. Currently our query embeddings have 1 dimension but our description embeddings have 2 so we need to reshape our query embeddings to match. We're passing (1, -1) to the reshape method which will create 1 row and as many columns as needed to make our reshape operation work. This effectively turns our 1D array into a 2D array with only 1 row and x number of columns (since we're using BERT, x will be 768 columns).

Once we have our embeddings as a NumPy array with the correct dimensionality, we can now utilize it as the input for our FAISS search.

# Perform the search
k = 5  # number of nearest neighbors (how many 'similar' cards we want to return)
query_embedding_np = query_embedding.numpy().reshape(1, -1).astype('float32')
distances, indices = index.search(query_embedding_np, k)

Our last step is to take the result of our index search that we returned to the variable called indicies and use these indicies to access the corresponding element in our cards array. The indicies variable is a 2D array returned by FAISS so in order to access each index value, we access the first element of the indicies variable. This is an array of index values that we can iterate over. If we use each of these from the square brackets operator on the cards array and print it, we will see what our top k most similar card descriptions are.

print(f"The top {k} most similar cards to '{query}' are: \n")
for idx in indices[0]:
    print(cards[idx])

This was a fun project to implement and overall I'm happy with the results! Projects like this I feel are a really good introduction into the world of artificial intelligence for someone with some programming background. This whole project comes in at just under 100 lines of code and produces reasonable results without a lot of complexity. Leave a comment if you found this interesting! 

Utilizing AI to find Magic The Gathering cards that have a similar description to the users Natural Language Query

Blog

Recreating the Windows 3D Pipes Screensaver With React / THREE.js

Natural Language Queries for Magic The Gathering Cards Using BERT + FAISS

Want to stay updated?