Get a free upgrade to WaveStack Modulate for iOS with code MODULATELAUNCH Free upgrade: MODULATELAUNCH

Music Guides

Writing EYESY Modes

The Critter & Guitari EYESY is a video synthesiser that generates visuals in response to audio, and the programs it runs are called modes. A mode is a short Python file that draws one frame at a time with pygame, reading the five panel knobs and the incoming audio off an object called etc. This page is the reference for that object and for the contract a mode has to meet, with working examples you can paste into a folder and run.

What a mode is

A mode is a directory with a file called main.py in it. The name of the directory is the name that appears on the EYESY. That file defines two functions:

Both receive the same two arguments. screen is the pygame surface you draw onto, 1280 by 720. etc carries the state of the instrument: knob positions, the current audio buffer, MIDI, and the colour palette.

A minimal mode

This is a complete, working mode. Put it in main.py inside a folder, point the EYESY or the simulator at the folder above it, and it will appear in the list.

import pygame

def setup(screen, etc):
    pass

def draw(screen, etc):
    screen.fill(etc.color_picker_bg(etc.knob5))

    radius = int(etc.knob1 * 300) + 10
    x = int(etc.knob2 * etc.xres)
    y = int(etc.knob3 * etc.yres)
    color = etc.color_picker(etc.knob4)

    pygame.draw.circle(screen, color, (x, y), radius)

The import pygame line has to be at the top of the file, and every control is read through etc, never as a bare knob1, which some tools accept and the hardware does not. These are the two most common reasons a mode that works on a development machine fails on the EYESY. math and random are already available inside a mode without being imported, but importing them yourself costs nothing and makes the file portable.

The knobs

Each knob is read as a float between 0.0 and 1.0. The five knobs are the whole control surface, so most of the work of designing a mode is deciding what those five numbers should mean and scaling them into useful ranges.

AttributeTypeNotes
etc.knob1float0.0 to 1.0. Knob 1 on the panel.
etc.knob2float0.0 to 1.0.
etc.knob3float0.0 to 1.0.
etc.knob4float0.0 to 1.0.
etc.knob5float0.0 to 1.0. By convention it sets the background colour or palette.

When a knob is mapped linearly onto a size or a speed, the small values that are most often useful sit in the first centimetre of its travel. Using knob * knob or an exponential curve spreads those values across more of the knob's range.

Audio

The EYESY supplies one hundred audio samples per frame. They arrive in 16-bit range, so roughly -32767 to 32767, and dividing by 32767 gives you the more convenient -1 to 1. If all you want is loudness rather than a waveform, etc.audio_peak is already normalised.

AttributeTypeNotes
etc.audio_inlist[float]100 samples of the current frame, in 16-bit range, roughly -32767 to 32767.
etc.audio_leftlist[float]Left channel. The simulator is mono, so this is a copy of audio_in.
etc.audio_rightlist[float]Right channel. The simulator is mono, so this is a copy of audio_in.
etc.audio_in_rlist[float]This is the hardware's name for the right channel, with the same contents as audio_right.
etc.audio_peakfloatPeak of the current buffer, normalised to 0.0 to 1.0.
etc.audio_peak_rfloatRight-channel peak. Equal to audio_peak in the simulator.
etc.audio_trigboolTrue when the peak passes 0.3. Use it for beat-driven changes.
etc.trigboolAlias for audio_trig.

Drawing the buffer as a waveform across the screen takes about six lines:

import pygame

def draw(screen, etc):
    screen.fill(etc.bg_color)
    color = etc.color_picker(etc.knob4)
    gain = etc.knob1 * 2

    step = etc.xres / len(etc.audio_in)
    points = []
    for i, sample in enumerate(etc.audio_in):
        y = etc.yres / 2 + (sample / 32767.0) * gain * etc.yres / 2
        points.append((i * step, y))

    pygame.draw.lines(screen, color, False, points, 3)

etc.audio_trig goes True whenever the peak crosses 0.3, which makes it a serviceable beat detector for anything that should change on a hit rather than continuously. Because it is a level test rather than onset detection, it stays True for as long as the sound is loud, so if you want one event per hit you need to track the previous frame's value yourself.

MIDI

Modes can read MIDI as well as audio. MIDI is most useful for driving discrete events, because a note arrives once at a defined moment, while an audio peak can cross a threshold several times or stay above it.

AttributeTypeNotes
etc.midi_note_newboolTrue on the frame a new note arrives.
etc.midi_noteintNote number of the most recent note, 0 to 127.
etc.midi_velocityintVelocity of the most recent note, 0 to 127.
etc.midi_noteslist[int]128 entries, one per note number, holding the current state of each.
etc.midi_clkintMIDI clock counter.

etc.midi_notes is the only attribute that shows chords. It is a list of 128 entries, one per note number, so iterating over it gives every note held down at this instant rather than only the most recent one.

Colour and the palette

EYESY OS 3 introduced a palette shared across modes, so that turning one knob recolours whatever is running. A mode uses the shared palette by requesting its colours from it rather than hard-coding them. Each picker takes a value from 0.0 to 1.0 and returns an (r, g, b) tuple you can pass to any pygame call.

AttributeTypeNotes
etc.color_picker(v)→ (r, g, b)Takes 0.0 to 1.0 and returns a colour from the palette.
etc.color_picker_bg(v)→ (r, g, b)Returns a colour from the background end of the palette.
etc.color_picker_fg(v)→ (r, g, b)Returns a colour from the foreground end, offset 180 degrees from the background.
etc.bg_color(r, g, b)Current background colour. Defaults to black.
etc.fg_color(r, g, b)Current foreground colour. Defaults to white.

A mode with hard-coded colours still runs. A mode that takes its colours from the palette follows the colour knob when someone changes it during a set, and a mode with hard-coded colours keeps its own scheme whatever the knob is set to.

The canvas

AttributeTypeNotes
etc.xresint1280.
etc.yresint720.
etc.auto_clearboolTrue by default. Set it False in setup() to let frames accumulate and build trails.
etc.fpsint30.
etc.frame_countintFrames rendered since the mode started. It can be used as a clock.
etc.modestrName of the running mode.

Read the resolution off etc.xres and etc.yres rather than writing 1280 and 720 into the file. It costs nothing and it means a mode keeps working if the resolution ever changes under it.

etc.auto_clear sets whether the screen is wiped before every frame, which it is by default. Set it False in setup() and frames pile up instead, which is how trails and feedback are made:

import math
import pygame

def setup(screen, etc):
    etc.auto_clear = False

def draw(screen, etc):
    # Nothing wipes the screen now, so wipe it on a timer instead.
    if etc.frame_count % int(etc.knob3 * 240 + 30) == 0:
        screen.fill(etc.color_picker_bg(etc.knob5))

    angle = etc.frame_count * (etc.knob2 * 0.2 + 0.01)
    reach = etc.knob1 * etc.yres * 0.45
    x = int(etc.xres / 2 + math.cos(angle) * reach)
    y = int(etc.yres / 2 + math.sin(angle) * reach)

    size = int(etc.audio_peak * 40) + 4
    pygame.draw.circle(screen, etc.color_picker(etc.knob4), (x, y), size)

To make trails fade gradually, the usual method is to darken the old frame rather than wipe it, by blitting a mostly transparent black surface over the screen. That works on hardware but not in the simulator, whose surfaces are RGB and which does not implement Surface.set_alpha, so the call raises. Clear on a timer or on etc.audio_trig while you are developing, and try the fade on the device.

Drawing

Everything is drawn with ordinary pygame. The calls most modes use are pygame.draw.circle, rect, line, lines, polygon, ellipse and arc, along with screen.fill, screen.blit, pygame.Surface, pygame.image.load, pygame.transform.scale, rotate and flip, and the font classes for text.

Build surfaces in setup() rather than in draw(). Anything constructed inside draw() is constructed thirty times a second, and on the hardware that extra work can be enough to make a mode drop frames.

What breaks when a mode reaches the hardware

Most modes that work in development and fail on the device fail for one of the three reasons below, and each is easier to avoid while writing than to diagnose afterwards.

Bare globals

Reading knob1 instead of etc.knob1 works in some tools and fails on the device. Always go through etc.

A missing import

import pygame has to be at the top of main.py. Some environments inject it for you, and a mode that relies on that will run in those environments and fail on the EYESY.

Performance

The EYESY is considerably more constrained than the machine you are writing on, and a simulator running at your computer's speed does not reproduce that. A mode can hold thirty frames a second on a laptop and drop frames on the device. Get anything complex onto real hardware early rather than at the end.

Testing without the hardware

The EYESY Simulator is a free native app for macOS and Windows that runs real modes on your computer, with the five knobs on screen and simulated audio, so you can write and iterate with nothing plugged in. It implements everything in the tables above. The performance caveat still applies: the simulator shows whether a mode runs correctly, but not whether it runs fast enough on the device.

The modes on this site can be read as worked examples: Cubicles for audio bindings, Image Trip for loading assets, String Vibrations for per-frame drawing, and Tempestuous for a dense particle mode whose frame rate should be checked on the device.

Common questions

What is an EYESY mode?

A mode is a folder containing a file called main.py. That file defines a draw(screen, etc) function, which the EYESY calls once per frame at 30 frames a second, and optionally a setup(screen, etc) function, which is called once before the first frame. You draw onto screen with pygame, and you read the panel controls off etc.

Does an EYESY mode need a setup function?

No. Only draw is required, and a mode without it will fail to load. setup is optional, and it is where you load images, build lookup tables, or set etc.auto_clear to False.

Why does my mode work in the simulator but not on the EYESY?

The three usual causes are bare globals instead of etc.knob1 through etc.knob5, a missing import pygame line at the top of main.py, and performance. The hardware is considerably more constrained than a laptop, so a mode that holds 30 frames a second in a simulator can drop frames on the device. Test anything complex on real hardware early.

What range are the EYESY audio samples in?

etc.audio_in holds 100 samples per frame in 16-bit range, so roughly -32767 to 32767. Divide by 32767 to get a value between -1 and 1. If you want a single loudness number rather than a waveform, etc.audio_peak is already normalised to 0.0 to 1.0.

How do I use the EYESY colour palette in a mode?

Call etc.color_picker(v) with a value between 0.0 and 1.0 and it returns an (r, g, b) tuple from the palette, which you pass to any pygame drawing call. etc.color_picker_bg and etc.color_picker_fg give the background and foreground ends of the same palette. Driving them from etc.knob5 is the usual convention.

Can an EYESY mode respond to MIDI?

Yes. etc.midi_note_new is True on the frame a note arrives, etc.midi_note and etc.midi_velocity carry that note, etc.midi_notes holds the state of all 128 note numbers, and etc.midi_clk is a clock counter.

How do I make trails or feedback in an EYESY mode?

Set etc.auto_clear = False in setup(). The screen is then no longer wiped between frames, so each frame draws on top of the last, and you clear it yourself when you want to, on a timer or on etc.audio_trig. Fading the old frame by blitting a mostly transparent surface works on hardware but not in the EYESY Simulator, which does not implement Surface.set_alpha.

About this reference

The attribute tables are taken from the engine inside the EYESY Simulator, so they describe exactly what that app implements. Where the simulator approximates the hardware rather than reproducing it, the table says so: audio is mono, so the left and right buffers carry the same samples, and the colour pickers are an approximation of the palette rather than the device's own.

This is an independent reference and is not affiliated with Critter & Guitari. Their own documentation for EYESY OS 3 is at critterandguitari.github.io, and the stock modes that ship with the EYESY are at critterandguitari/EYESY_Modes_Pygame.