Create Music with Estuary

By Clarissa Littler / Programming / August 2021 Issue

Learn how to make music and videos with your friends using Estuary!

Does making music and videos with code sound interesting? What about doing it with friends without needing anything more than a modern web browser? (Unfortunately, you can’t use all of the retrobrowsers we discuss elsewhere in this issue!) If that sounds fun, then you should read on to learn more about Estuary!

The main Estuary website is estuary.mcmaster.ca and if you go there you should see something like

The first thing you should do is click on “solo mode” and you’ll see a new screen with six regions, panels, that are all able to run code separately from each other. You pick the language to use by clicking on the dropdown menu above one of the panels. The different languages all do different multimedia things, but the three we’ll briefly introduce are MiniTidal, Punctual, and CineCer0.

Try selecting minitidal, typing s "bd*4" inside the panel, and then hitting the play button above the panel. You should hear a simple steady beat of bass drum hits.

The s means “take a sequence of sounds and turn them into something that can be played”. The part in between the quotation marks is the pattern itself. In this case, you can read it as “play four samples called ‘bd’” per cycle. Ah, cycle, that’s an important word here: a cycle is the basic unit of time in Tidal. It is, by default, a little more than two seconds. Tidal will fit the number of sound events you tell it to in each cycle, as fast or as slow as it needs to.

For example, if you type s "bd*8" instead you’ll get eight bass drum samples per cycle, played closer together. If you changed it to just two samples there’d be a lot more dead air between drum hits.

Now let’s make it more complicated, using square brackets [] to group events together. Another thing we’re going to use is that ~ represents silence, a rest in the music.

s "[~ bd*2] bd*4 ~ bd*8"

Now, there’s a lot more samples than just bass drums. To see them, you should open a new solo Estuary session and then type

!localview audiomap

You should see, then, a very very long list of samples. Roughly five hundred of them as of the time of writing this article!

That’s a very brief taste of music with MiniTidal, so let’s move on to graphics with Punctual. Don’t worry, though, there are more resources on learning Tidal linked in this article.

In a different panel, select punctual as the language and copy the following:
tex "https://upload.wikimedia.org/wikipedia/commons/6/63/Assassin_bug_(Rhynocoris_iracundus)_with_bee_(Apis_ssp)_prey.jpg" [fx,fy] >> video;

rect [sin cps,0] [0.5,0.5] * [0.6,0.3,0.8] >> video;

You should see a picture of a bug and a fuschia rectangle zipping back and forth on the screen. In fact, if you play a beat in MiniTidal you might notice that the rectangle and the beat are synced up. Here’s a more dynamic way to combine two things, where you have a circle and rectangle revealing the picture underneath.

pic << tex "https://upload.wikimedia.org/wikipedia/commons/6/63/Assassin_bug_(Rhynocoris_iracundus)_with_bee_(Apis_ssp)_prey.jpg" [fx,fy]; rect [sin 0.1,0] [0.3,0.3] * pic >> video; circle [sin 0.2, sin 0.3] 0.3*lo * pic >> video;

Even without pictures you can do tricks kind of like audioreactive shaders, like the following code that lights up when the bass gets loud:
[fx, (fx*fy*lo)**0.5, fy] >> video;

Finally, you can include videos with CineCer0 really easily: well, easily if you grab videos off of wikimedia commons, because unfortunately you can’t embed videos off of social media this way. You just paste the URL of the video into a CineCer0 panel inside quotation marks, like this

“https://upload.wikimedia.org/wikipedia/common/8/86/House_wren_in_JBWR_(50171).webm”

You can only include one video per panel, but you can have multiple CineCer0 panels!

Finally, I promised that you can do this with friends and thankfully that’s very simple! All you need to do is navigate to “Collaborate” instead of “Solo Mode” on the front page and create a new room. The room will be temporary, but you can make it last several hours, and you set a password to give your friends. Then, all they have to do is click on the Estuary room you made, enter the password, and then you can all work together!

Learn More

Clarissa Littler

Clarissa has worked in mathematics, physics, and computer science research but spends much of her time now trying to make computer science education accessible to a broader audience.

View all posts