Currently Working On...
SnakeGAN - Training AI to Recreate Snake from Gameplay
January 2025
- Created the classic game Snake in Python and used an RL agent to play it to record gameplay and key inputs.
- Currently training a GAN to recreate the dynamics and graphics of the game frame by frame.
- Reading up on the GameGAN paper to implement LSTMs for better performance.
- Inspired by Oasis AI Minecraft that generates Minecraft gameplay using a diffusion model.
Hollo - A "Holographic" Interactable Display
September 2024
- Building a "Holographic" display that you can talk to. Uses the Pepper's Ghost technique to project the screen onto a transparent surface, creating the illusion of a hologram.
- First built a V1 prototype on a breadboard which includes an avatar that you can talk to. Then I designed a 2-layer PCB which includes two ESP32 S3s that handle 50 fps video display, microphone input, and speaker output. Currently experimenting with TouchDesigner effects and improving the backend software for the AI assistant.
- Got a $1000 USD 1517 Medici Project grant to develop the idea further (Thank you Danielle!).
- Inspired by Bladerunner 2049.
V1 Prototype of Hollo
Streaming TouchDesigner effects over Wi-Fi
Hollo without the enclosure
Favourite Past Projects
Voice Assistant Camera Wearable
June 2024
Part 1 GitHub Repo | Part 2 GitHub Repo- Built a clip-on device to turn any pair of glasses into smart glasses with a voice assistant built with a XIAO ESP32 S3 Sense board. Originally started out as a separate device built with an ESP32-CAM and Muse 2 headset.
- Used a local multimodal large language model, Groq with RAG, and VOSK speech recognition model. First part only used text and image embeddings.
- Inspired by Meta Raybans and OpenGlass.
Part 1 Video
Part 2 Video
Wearing the first version
Ping Pong LED Walls
March 2024
Part 1 Writeup | Part 2 Writeup- Part 1 (Feb 2022): Built a lightweight LED Wall with 24 by 32 resolution. I designed the PCBs to be as modular, lightweight, and cheap as possible where I then painfully soldered them all. I also created a custom register-level parallel bit-banging protocol for the necessary speed to run it with an ATmega328P MCU without using a library. Made use of the Flash storage to display pictures, GIFs, and cool patterns.
- Part 2 (May 2022): Optimized my custom protocol by transitioning from register level code to assembly which can now display the camera output from a computer and allow screen sharing and drawing. This is also facilitated through several Processing sketches and UART communication at 2 million bits per second. I also stuck ping pong balls on top as diffusers!
- V2 (Mar 2024): Updated the previous Ping Pong LED Wall design to be a smaller and more modular IoT version with 3D printed clip-on pieces, solderable connector PCBs, and cool effects that can be controlled over Wi-Fi using the WLED app.
- Inspired by Bitluni's Ping Pong LED Wall.
V2 Video
V2 with WLED effects
Part 2 Video
Part 1 Video
LED Wall screen sharing
LED Wall camera output 👋
Nyan Cat GIF
Soldering each PCB and testing them (took forever)
EEG + EMG Measurement Device
December 2021
Writeup- Built a 2 channel electroencephalography (EEG) and electromyography (EMG) reader to "mind control" devices.
- Designed a PCB to amplify cognitive signals and attenuate noise, soldered SMT components, and wrote a Processing sketch to further filter the signal with an FFT algorithm and display the output.
- One channel was located by the pre-frontal lobe (Fp2) while the other channel was located by the occipital lobe (O1). The reference channel was also located at the pre-frontal lobe (Fp1).
YouTube video demonstration + explanation
Generating alpha waves when closing my eyes with a spike in the FFT spectrum at 8-14 Hz
The PCB Board (first time using SMT components!)
Baking the SMT components with a toaster oven repurposed as a reflow oven
The circuit block diagram
The 10-20 system for placing scalp electrodes to measure EEG
Custom 4-Bit Computer (CHUMP)
November 2021
Part 1 Writeup | Part 2 Writeup- Part 1: The beginning of a 4-bit breadboard computer called CHUMP (Cheap Homemade Understandable Minimal Processor). The first part goes over the custom "Chumpanese" code to program the computer, the adjustable clock circuit, and the counter circuit to keep track of and execute the current instruction.
- Part 2: The final custom 4-bit computer built from 1980's era TTL Integrated Circuits, coded in specific machine language. It executes 16 lines of code, capable of loading, adding, subtracting, storing, and reading memory, as well as a go-to command and conditional if zero commands.
Part 1 Video
Part 2 Video
The Code (stored in the EEPROM Program), Clock (555 timers) and Program Counter (PC) sections
The Completed CHUMP 4-bit Computer
The CHUMP block diagram
IR Nixie Clock
May 2021
Writeup- A 4-digit Nixie tube clock with alarm, timer, date, and temperature features controlled with an infrared remote.
- Programmed in C++ on a standalone ATmega328 IC. Designed a custom PCB in EAGLE and laser-cut acrylic in Adobe Illustrator.
- Designed a custom boost converter from 5V to 170V to drive the Nixie tubes.
- Inspired by Steins Gate.
YouTube video demonstration + explanation
Periodically cycling the digits to prevent cathode poisoning (flicker is from driving the tubes with Persistence of Vision)
The back PCB containing all the circuitry and sensors
PCB sandwich connecting the display PCB to the back PCB
My custom boost converter using the MAX1771 IC (left) and the commercially available Taylor 1364 Nixie power supply (right)
LiDAR Measurement and Mapping Device
March 2021
Writeup- A device that measures distance and models the surrounding room using LiDAR, servo motors, and an ESP32 programmed in C++.
- Coded a Processing program in Java to display a 3D point cloud, perform measurements, and control the ESP32 with Bluetooth.
YouTube video demonstration + explanation
Generated 3D point cloud of some objects with some distance measurements
Generated a 3D point cloud of myself!
Starting a real-time scan to create a 3D point cloud
The LiDAR Measurement Device consisting of the control unit and LiDAR unit