Writing EYESY Modes
The Critter & Guitari EYESY is a video synthesiser that generates visuals in response to
audio, and the programs it runs are called modes. A mode is a short Python file that draws one frame at a
time with pygame, reading the five panel knobs and the incoming audio off an object called
etc. This page is the reference for that object and for the contract a mode has
to meet, with working examples you can paste into a folder and run.
What a mode is
A mode is a directory with a file called main.py in it. The name of the directory
is the name that appears on the EYESY. That file defines two functions:
-
draw(screen, etc)is called once per frame, thirty times a second. This is required, and a mode without it will not load. -
setup(screen, etc)is called once, before the first frame. It is optional, and it is where you load images, precompute anything expensive, or turn off automatic clearing.
Both receive the same two arguments. screen is the pygame surface you draw onto,
1280 by 720. etc carries the state of the instrument: knob positions, the current
audio buffer, MIDI, and the colour palette.
A minimal mode
This is a complete, working mode. Put it in main.py inside a folder, point the
EYESY or the simulator at the folder above it, and it
will appear in the list.
import pygame
def setup(screen, etc):
pass
def draw(screen, etc):
screen.fill(etc.color_picker_bg(etc.knob5))
radius = int(etc.knob1 * 300) + 10
x = int(etc.knob2 * etc.xres)
y = int(etc.knob3 * etc.yres)
color = etc.color_picker(etc.knob4)
pygame.draw.circle(screen, color, (x, y), radius)
The import pygame line has to be at the top of the file, and every control is read
through etc, never as a bare knob1, which some tools accept and the
hardware does not. These are the two most common reasons a mode that works on a development
machine fails on the EYESY. math and random are already available
inside a mode without being imported, but importing them yourself costs nothing and makes the
file portable.
The knobs
Each knob is read as a float between 0.0 and 1.0. The five knobs are the whole control surface, so most of the work of designing a mode is deciding what those five numbers should mean and scaling them into useful ranges.
| Attribute | Type | Notes |
|---|---|---|
etc.knob1 | float | 0.0 to 1.0. Knob 1 on the panel. |
etc.knob2 | float | 0.0 to 1.0. |
etc.knob3 | float | 0.0 to 1.0. |
etc.knob4 | float | 0.0 to 1.0. |
etc.knob5 | float | 0.0 to 1.0. By convention it sets the background colour or palette. |
When a knob is mapped linearly onto a size or a speed, the small values that are most often
useful sit in the first centimetre of its travel. Using knob * knob or an
exponential curve spreads those values across more of the knob's range.
Audio
The EYESY supplies one hundred audio samples per frame. They arrive in 16-bit range, so roughly
-32767 to 32767, and dividing by 32767 gives you the more convenient -1 to 1. If all you want
is loudness rather than a waveform, etc.audio_peak is already normalised.
| Attribute | Type | Notes |
|---|---|---|
etc.audio_in | list[float] | 100 samples of the current frame, in 16-bit range, roughly -32767 to 32767. |
etc.audio_left | list[float] | Left channel. The simulator is mono, so this is a copy of audio_in. |
etc.audio_right | list[float] | Right channel. The simulator is mono, so this is a copy of audio_in. |
etc.audio_in_r | list[float] | This is the hardware's name for the right channel, with the same contents as audio_right. |
etc.audio_peak | float | Peak of the current buffer, normalised to 0.0 to 1.0. |
etc.audio_peak_r | float | Right-channel peak. Equal to audio_peak in the simulator. |
etc.audio_trig | bool | True when the peak passes 0.3. Use it for beat-driven changes. |
etc.trig | bool | Alias for audio_trig. |
Drawing the buffer as a waveform across the screen takes about six lines:
import pygame
def draw(screen, etc):
screen.fill(etc.bg_color)
color = etc.color_picker(etc.knob4)
gain = etc.knob1 * 2
step = etc.xres / len(etc.audio_in)
points = []
for i, sample in enumerate(etc.audio_in):
y = etc.yres / 2 + (sample / 32767.0) * gain * etc.yres / 2
points.append((i * step, y))
pygame.draw.lines(screen, color, False, points, 3) etc.audio_trig goes True whenever the peak crosses 0.3, which makes it a
serviceable beat detector for anything that should change on a hit rather than continuously.
Because it is a level test rather than onset detection, it stays True for as long as the sound
is loud, so if you want one event per hit you need to track the previous frame's value
yourself.
MIDI
Modes can read MIDI as well as audio. MIDI is most useful for driving discrete events, because a note arrives once at a defined moment, while an audio peak can cross a threshold several times or stay above it.
| Attribute | Type | Notes |
|---|---|---|
etc.midi_note_new | bool | True on the frame a new note arrives. |
etc.midi_note | int | Note number of the most recent note, 0 to 127. |
etc.midi_velocity | int | Velocity of the most recent note, 0 to 127. |
etc.midi_notes | list[int] | 128 entries, one per note number, holding the current state of each. |
etc.midi_clk | int | MIDI clock counter. |
etc.midi_notes is the only attribute that shows chords. It is a list of 128
entries, one per note number, so iterating over it gives every note held down at this instant
rather than only the most recent one.
Colour and the palette
EYESY OS 3 introduced a palette shared across modes, so that turning one knob recolours
whatever is running. A mode uses the shared palette by requesting its colours from it rather
than hard-coding them. Each picker takes a value from 0.0 to 1.0 and returns an
(r, g, b) tuple you can pass to any pygame call.
| Attribute | Type | Notes |
|---|---|---|
etc.color_picker(v) | → (r, g, b) | Takes 0.0 to 1.0 and returns a colour from the palette. |
etc.color_picker_bg(v) | → (r, g, b) | Returns a colour from the background end of the palette. |
etc.color_picker_fg(v) | → (r, g, b) | Returns a colour from the foreground end, offset 180 degrees from the background. |
etc.bg_color | (r, g, b) | Current background colour. Defaults to black. |
etc.fg_color | (r, g, b) | Current foreground colour. Defaults to white. |
A mode with hard-coded colours still runs. A mode that takes its colours from the palette follows the colour knob when someone changes it during a set, and a mode with hard-coded colours keeps its own scheme whatever the knob is set to.
The canvas
| Attribute | Type | Notes |
|---|---|---|
etc.xres | int | 1280. |
etc.yres | int | 720. |
etc.auto_clear | bool | True by default. Set it False in setup() to let frames accumulate and build trails. |
etc.fps | int | 30. |
etc.frame_count | int | Frames rendered since the mode started. It can be used as a clock. |
etc.mode | str | Name of the running mode. |
Read the resolution off etc.xres and etc.yres rather than writing
1280 and 720 into the file. It costs nothing and it means a mode keeps working if the
resolution ever changes under it.
etc.auto_clear sets whether the screen is wiped before every frame, which it is by
default. Set it False in setup() and frames pile up instead, which is how trails
and feedback are made:
import math
import pygame
def setup(screen, etc):
etc.auto_clear = False
def draw(screen, etc):
# Nothing wipes the screen now, so wipe it on a timer instead.
if etc.frame_count % int(etc.knob3 * 240 + 30) == 0:
screen.fill(etc.color_picker_bg(etc.knob5))
angle = etc.frame_count * (etc.knob2 * 0.2 + 0.01)
reach = etc.knob1 * etc.yres * 0.45
x = int(etc.xres / 2 + math.cos(angle) * reach)
y = int(etc.yres / 2 + math.sin(angle) * reach)
size = int(etc.audio_peak * 40) + 4
pygame.draw.circle(screen, etc.color_picker(etc.knob4), (x, y), size)
To make trails fade gradually, the usual method is to darken the old frame rather than wipe
it, by blitting a mostly transparent black surface over the screen. That works on hardware but
not in the simulator, whose surfaces are RGB and which does not implement
Surface.set_alpha, so the call raises. Clear on a timer or on
etc.audio_trig while you are developing, and try the fade on the device.
Drawing
Everything is drawn with ordinary pygame. The calls most modes use are
pygame.draw.circle, rect, line, lines,
polygon, ellipse and arc, along with
screen.fill, screen.blit, pygame.Surface,
pygame.image.load, pygame.transform.scale,
rotate and flip, and the font classes for text.
Build surfaces in setup() rather than in draw(). Anything
constructed inside draw() is constructed thirty times a second, and on the
hardware that extra work can be enough to make a mode drop frames.
What breaks when a mode reaches the hardware
Most modes that work in development and fail on the device fail for one of the three reasons below, and each is easier to avoid while writing than to diagnose afterwards.
Bare globals
Reading knob1 instead of etc.knob1 works in some tools and fails on
the device. Always go through etc.
A missing import
import pygame has to be at the top of main.py. Some environments
inject it for you, and a mode that relies on that will run in those environments and fail on
the EYESY.
Performance
The EYESY is considerably more constrained than the machine you are writing on, and a simulator running at your computer's speed does not reproduce that. A mode can hold thirty frames a second on a laptop and drop frames on the device. Get anything complex onto real hardware early rather than at the end.
Testing without the hardware
The EYESY Simulator is a free native app for macOS and Windows that runs real modes on your computer, with the five knobs on screen and simulated audio, so you can write and iterate with nothing plugged in. It implements everything in the tables above. The performance caveat still applies: the simulator shows whether a mode runs correctly, but not whether it runs fast enough on the device.
The modes on this site can be read as worked examples: Cubicles for audio bindings, Image Trip for loading assets, String Vibrations for per-frame drawing, and Tempestuous for a dense particle mode whose frame rate should be checked on the device.
Common questions
What is an EYESY mode?
A mode is a folder containing a file called main.py. That file defines a draw(screen, etc) function, which the EYESY calls once per frame at 30 frames a second, and optionally a setup(screen, etc) function, which is called once before the first frame. You draw onto screen with pygame, and you read the panel controls off etc.
Does an EYESY mode need a setup function?
No. Only draw is required, and a mode without it will fail to load. setup is optional, and it is where you load images, build lookup tables, or set etc.auto_clear to False.
Why does my mode work in the simulator but not on the EYESY?
The three usual causes are bare globals instead of etc.knob1 through etc.knob5, a missing import pygame line at the top of main.py, and performance. The hardware is considerably more constrained than a laptop, so a mode that holds 30 frames a second in a simulator can drop frames on the device. Test anything complex on real hardware early.
What range are the EYESY audio samples in?
etc.audio_in holds 100 samples per frame in 16-bit range, so roughly -32767 to 32767. Divide by 32767 to get a value between -1 and 1. If you want a single loudness number rather than a waveform, etc.audio_peak is already normalised to 0.0 to 1.0.
How do I use the EYESY colour palette in a mode?
Call etc.color_picker(v) with a value between 0.0 and 1.0 and it returns an (r, g, b) tuple from the palette, which you pass to any pygame drawing call. etc.color_picker_bg and etc.color_picker_fg give the background and foreground ends of the same palette. Driving them from etc.knob5 is the usual convention.
Can an EYESY mode respond to MIDI?
Yes. etc.midi_note_new is True on the frame a note arrives, etc.midi_note and etc.midi_velocity carry that note, etc.midi_notes holds the state of all 128 note numbers, and etc.midi_clk is a clock counter.
How do I make trails or feedback in an EYESY mode?
Set etc.auto_clear = False in setup(). The screen is then no longer wiped between frames, so each frame draws on top of the last, and you clear it yourself when you want to, on a timer or on etc.audio_trig. Fading the old frame by blitting a mostly transparent surface works on hardware but not in the EYESY Simulator, which does not implement Surface.set_alpha.
About this reference
The attribute tables are taken from the engine inside the EYESY Simulator, so they describe exactly what that app implements. Where the simulator approximates the hardware rather than reproducing it, the table says so: audio is mono, so the left and right buffers carry the same samples, and the colour pickers are an approximation of the palette rather than the device's own.
This is an independent reference and is not affiliated with Critter & Guitari. Their own documentation for EYESY OS 3 is at critterandguitari.github.io, and the stock modes that ship with the EYESY are at critterandguitari/EYESY_Modes_Pygame.