Programming the Rev2 - in the python programming language!

[kris]

Hi friends,

following creativespirals work on the voice component modelling I was again intrigued by the massive capabilities hidden in the Rev2. I am not a great sound designer, so I am very interested to understand what others are doing as I want to learn.

I am though a computer programmer by profession, and might not be the only one here. I had an epiphany lately when I looked at the complexity of the VCM setup required, and the less than optimal accessibility of the gated sequencer: We should be able to program the Rev2 with a real programming language, with the help of a computer editor! This would make complex operations on patches a breeze, and could also allow for much quicker testing of VCM setups with different patches.

I repurposed some code I had written two years ago on a tool helping me with the Rev2 (clear poly sequencer, copy poly sequencer to gated sequencer, the like), and now have an environment where I can do real synthesizer programming!

An example on the python console:

Code: [Select]

    >>> from pytschirp import Rev2Patch, Rev2
    >>> rev2 = Rev2()
    >>> rev2.detect()
    Found DSI Prophet Rev2 on channel 6 replying on device Port 1 on MXPXT when sending to Port 1 on MXPXT

I know have a rev2 object, and it has connected to the real device automatically figuring out on which MIDI device it can talk to (round-trip) to the real Rev2.

Now you can do interesting things:

Code: [Select]

    >>> patches = rev2.loadSysex(R"ProphetRev2\VCM-Sound Set\VCM-Rev2-Soundset-U3-v1.syx")
    >>> patches[0].name
    'KEY ResoPluck VCM[+]'
    >>> seq1 = patches[0].attr('Gated Seq Track 1 Step 1,16').get()
    >>> seq1
    [17, 13, 18, 17, 25, 16, 126, 0, 0, 0, 0, 0, 0, 0, 0, 0]
So I can use the python console to load a patch bank from a sysex file, and inspect e.g. the track 1 of the gated sequencer of patch 0 (and the name of patch 0) in an easy way. But it doesn't stop there:

Code: [Select]

    >>> nowplaying = rev2.editBuffer()
    Got edit buffer from synth
    >>> nowplaying['Gated Seq Track 1 Step 1,16'] = seq1

and now I coped single-handedly (well, nearly single-line'dly) the gates sequencer track 1 loaded from a patch stored in a sysex file on disk into the one playing right now in my Rev2 - you get the gist! The assignment operation above will generate the right number of MIDI NRPN messages to modify the Rev2's edit buffer so the gated sequencer track is set to the variable's content.

I feel this is a big step and I can now use the full power of abstraction when talking to my rev2.

All of the code I made available as OpenSource on github (https://github.com/christofmuc/pytschirp), and I am working towards a little precompiled program to make this a bit more accessible to everybody. This is a work in progress and I will work more to document how it actually works.

I would be interested if this is of any interest to somebody of the great folks here?

[guyaguy]

Looks like it would be useful for people who know exactly what values they want for the sequence and are comfortable with entering the Python commands. I'm not sure if you're headed the same direction with this but the codeKnobs and Sound Tower Rev2 editors have a UI for editing the sequencer that offer similar control but with sliders. (See attached screenshot.) But entering the value by keypad might be easier or quicker for some people. The Sound Tower plugin doesn't let you do that, not sure about codeKnobs.

Personally I don't find the accessibility of the gated sequencer too bad but that's probably because I had the Poly Evolver for years. Moving from it to the single knob edit was an adjustment but I was already familiar with the general functionality. If I wasn't it probably would have been harder to learn. And I do miss having a knob per step.

[kris]

Looks like it would be useful for people who know exactly what values they want for the sequence and are comfortable with entering the Python commands. I'm not sure if you're headed the same direction with this but the codeKnobs and Sound Tower Rev2 editors have a UI for editing the sequencer that offer similar control but with sliders. (See attached screenshot.) But entering the value by keypad might be easier or quicker for some people. The Sound Tower plugin doesn't let you do that, not sure about codeKnobs.

Personally I don't find the accessibility of the gated sequencer too bad but that's probably because I had the Poly Evolver for years. Moving from it to the single knob edit was an adjustment but I was already familiar with the general functionality. If I wasn't it probably would have been harder to learn. And I do miss having a knob per step.

Thank you for your comment! I agree most changes you make you want to hear, and turning a knob is much more satisfying. I made actually a template a year ago to use a BCR2000 with its 32 knobs to control the gated sequencer: https://github.com/christofmuc/bcr2000_for_dsi_prophet_rev2.

What I am thinking of with the use of something like Python is more automatic - e.g. I made a function that would automatically patch in a specific gated sequence whenever I switch a program. The computer would get a program change message from the Rev2, and immediately send back the list of NRPN commands to setup the gated sequence again. This is cool e.g. if you have a bass line in the gated sequencer (remember the gated sequencer will auto-transpose unlike the poly sequencer!) and want to try it with different bass patches you got. Doing that manually would be a lot of work, but with that kind of automatic you don't even need to touch the computer, you can just keep playing.

Regarding voice component modelling, copying over the gated sequencer track isn't the end of it - you would actually also want to copy at least parts of the mod matrix, and the other gated tracks as well. This would be just as easy using python, while it gets tedious using a GUI software.

Of course I agree the learning curve is completely different, but I am currently thinking of ways to make this more accessible.

[guyaguy]

Thank you for your comment! I agree most changes you make you want to hear, and turning a knob is much more satisfying. I made actually a template a year ago to use a BCR2000 with its 32 knobs to control the gated sequencer: https://github.com/christofmuc/bcr2000_for_dsi_prophet_rev2.

What I am thinking of with the use of something like Python is more automatic - e.g. I made a function that would automatically patch in a specific gated sequence whenever I switch a program. The computer would get a program change message from the Rev2, and immediately send back the list of NRPN commands to setup the gated sequence again. This is cool e.g. if you have a bass line in the gated sequencer (remember the gated sequencer will auto-transpose unlike the poly sequencer!) and want to try it with different bass patches you got. Doing that manually would be a lot of work, but with that kind of automatic you don't even need to touch the computer, you can just keep playing.

Regarding voice component modelling, copying over the gated sequencer track isn't the end of it - you would actually also want to copy at least parts of the mod matrix, and the other gated tracks as well. This would be just as easy using python, while it gets tedious using a GUI software.

Of course I agree the learning curve is completely different, but I am currently thinking of ways to make this more accessible.

Ah yeah swapping the sequence from one patch to another could be quite useful since sequences are saved as part of a program rather than independently. It could be a way of finding the right patch like you say or you could save the same sequence to 2 patches then in a performance or recording switch from one patch to another, radically changing the timbre but keeping the sequence.

[jok3r]

I don‘t have time to play around with that at the moment, but being a computer scientist myself, I think a Python API for the Rev2 sounds very interesting.

I‘m working in a department for artificial intelligence at the university and for some time I think about doing some experiments with some of my synth. Since I have a lot to do with machine learning, neural networks and other fancy stuff, I think about implementing an AI that can (or could at least try to) program the Rev2 to sound as close to a given audio sample as it possibly can.

Since I‘m usually working with text data I don‘t have a clue if that could work at all. But if I find some spare time I will try that.

The problem so far was indeed that I didn‘t want to mess with the whole midi stuff myself. But having a good python API for the Rev2 would make things a lot easier.

Congratulations, you found a person who thinks your work is of great value! ;-)

[kris]

I don‘t have time to play around with that at the moment, but being a computer scientist myself, I think a Python API for the Rev2 sounds very interesting.

<snip>

The problem so far was indeed that I didn‘t want to mess with the whole midi stuff myself. But having a good python API for the Rev2 would make things a lot easier.

Congratulations, you found a person who thinks your work is of great value! ;-)

Whoo! You made my day! Now I take that extra motivation and I am going to document the stuff on github properly, and I also think of creating a little standalone program which gives you a faster start in trying that out!

P.S. On that patch creation thing, I have the same thoughts since a few years. I have already done quite a bit of research on what has been done so far - off topic for this thread, but mental note made!

[kris]

Ah yeah swapping the sequence from one patch to another could be quite useful since sequences are saved as part of a program rather than independently. It could be a way of finding the right patch like you say or you could save the same sequence to 2 patches then in a performance or recording switch from one patch to another, radically changing the timbre but keeping the sequence.

Thank you! I will try to make a program that demonstrates this can be achieved with what I got in mind.

[jok3r]

P.S. On that patch creation thing, I have the same thoughts since a few years. I have already done quite a bit of research on what has been done so far - off topic for this thread, but mental note made!

It's not so off topic at all I think. Basically all state-of-the-art machine learning libraries are python libraries by now. So to do what I was thinking of "Programming the Rev2 - in the python programming language!" is exactly the thing I was waiting for.

My first naiv idea was to auto record the current patch from the Rev2 and compare it with a given audio file sample by sample. Perhaps using more samples at once (higher batch size) and compare it with a default loss function for regression, e.g mean squared error or something. And then basically do a gradient descent (or some other optimizer) over the parameters of the synth. Don't know if the last point is possible, but since all operations of the Rev2 are somehow additions or multiplications I think it should be possible to express that as a derivable function over all parameters.

But perhaps that's a great non-sense... as said before: never worked with audio data so far.

[shiihs]

Great work!

I did something similar for supercollider (https://github.com/shimpe/sc-prophet-rev2) last year.

I've been able to use it with success in applications of two-way live editing of the synth's parameters, generative music and generative sound design, as well as loading scala tuning and keyboard mapping files, a 16-track simultaneously-polyphonic-and-gated sequencer (software based), even some simple auto-arranger functionality for rev2 (this last item is not working very well but it's still fun to play with), and I had started an "infinite undo and redo" for live-edited synth parameters (but that's still a work in progress, and I have other deadlines now).

As for sound design applications: if you use genetic programming techniques, you don't even need a gradient to descend. At some point I'd like to use it to automagically approximate a given timbre. Then I'd need a fitness function (a formula that tells me how well a given sound approximates another sound), perhaps something based of MFCCs in spectral domain and envelope extraction in the time domain? This is just a half-baked idea for now.

When you start programming the rev2, it enables applications that are simply impossible by using it manually. I'm really curious to see what applications you will come up with!

[creativespiral]

This is interesting Kris -

I've been using the SoundTower editor primarily for the UI - it definitely makes editing the gated seq and other parameters easier than trying to use the direct Rev2 interface.   Also the ST editor has the ability to save and load sequencer data independent of patches, which I sometimes use...   Although I've gotten pretty quick with just manually setting up voice modeling for a given patch (I can set it up in about a minute)

But, having a command line tool to quickly copy parameters could be interesting.    For the Voice Modeling, its a combination of gated seq data, and either two or four mod slots which define the voice-by-voice character... at least for the core setup controlling osc tuning (which makes the biggest difference)  Many of the patches in my VCM bank do use additional voice-by-voice modeling tied to filter cutoff, envelopes, or other macro parameters that I measured in classic synths that I recorded.

I suppose this would be a way to store some default VCM templates (or melodic sequence data) that could be quickly copied to new patches.   Actually, with the copying of melodic seq data, that would be interesting to have access to dozens, or potentially hundreds of melodic sequences that could be sampled for any given sound (or even mashed up possibly to create some semi-procedural sequences) 

Nice work!

Cheers, Jason

[guyaguy]

I don‘t have time to play around with that at the moment, but being a computer scientist myself, I think a Python API for the Rev2 sounds very interesting.

I‘m working in a department for artificial intelligence at the university and for some time I think about doing some experiments with some of my synth. Since I have a lot to do with machine learning, neural networks and other fancy stuff, I think about implementing an AI that can (or could at least try to) program the Rev2 to sound as close to a given audio sample as it possibly can.

Since I‘m usually working with text data I don‘t have a clue if that could work at all. But if I find some spare time I will try that.

The problem so far was indeed that I didn‘t want to mess with the whole midi stuff myself. But having a good python API for the Rev2 would make things a lot easier.

Congratulations, you found a person who thinks your work is of great value! ;-)

Ooh this sounds intriguing. I can imagine a lot of fun could be had trying to match samples of voices or tires screeching. The weird uncanny valley stuff could be just as interesting as the spot on emulations.

[kris]

My first naiv idea was to auto record the current patch from the Rev2 and compare it with a given audio file sample by sample. Perhaps using more samples at once (higher batch size) and compare it with a default loss function for regression, e.g mean squared error or something. And then basically do a gradient descent (or some other optimizer) over the parameters of the synth. Don't know if the last point is possible, but since all operations of the Rev2 are somehow additions or multiplications I think it should be possible to express that as a derivable function over all parameters.

But perhaps that's a great non-sense... as said before: never worked with audio data so far.

The gradient descent (or more probably simulated annealing) is a good first step! The trick to work with audio data is to first transform it into some higher level abstraction and not look at sample by sample thing. Think of what Shazam might be doing to compare a recorded audio with its database. I read a paper mentioning MFCC (Mel Frequency Cepstral Coefficients) as a good high level feature from the audio data - there are plenty of implementations to extract the MFCC features from audio.

These guys are on it as well: https://llllllll.co/t/machine-learning-for-synthesizer-parameters/16857/7.

If I would only have more time, this sounds like a lot of fun!

[kris]

Great work!

I did something similar for supercollider (https://github.com/shimpe/sc-prophet-rev2) last year.

I've been able to use it with success in applications of two-way live editing of the synth's parameters, generative music and generative sound design, as well as loading scala tuning and keyboard mapping files, a 16-track simultaneously-polyphonic-and-gated sequencer (software based), even some simple auto-arranger functionality for rev2 (this last item is not working very well but it's still fun to play with), and I had started an "infinite undo and redo" for live-edited synth parameters (but that's still a work in progress, and I have other deadlines now).
<snip>

Woah! I am impressed - I have never touched supercollider before, and I am not sure looking at your code that it does have a flat learning curve ;-) The syntax really is something different (my main workhorse is C++ and JUCE, a very nice framework for any musical application). I will definitely need to look at what you did, and try to understand!

Only now saw your comment about the MFCC - true, I found some papers from around ~2009 where they tried learning patch creation based on a pure-data synth, the genetic algorithm was designing the structure of the synth at the same time as its parameters, very interesting. Have to dig it up somewhere again, I lost the author's name.

[jok3r]

Thank you all for the nice ideas regarding machine learning and synthesizers.

I have SampleRobot (for those who don't know: an auto-sampling tool for creating multisamples) which I could abuse to generate lots of training data, e.g. all presets, whole keyrange, several velocity layers, all samples same length... for whole banks of sounds. With your tool I could extract the corresponding parameters of the synth. That would need quite some time, but since it's possibly full-automatic I could run this while I'm at work.

[shiihs]

Woah! I am impressed - I have never touched supercollider before, and I am not sure looking at your code that it does have a flat learning curve ;-) The syntax really is something different.

Sure thing, the learning curve is quite steep, but once you've climbed the initial stairs (it can be painful in the beginning) a few new worlds open up.

[lilakmonoke]

im all for it! ive been sequencing all my hardware from pure data patches for years. pure data is maybe better suited as it is a rather hands affai. BUT i want direct access to the clock rates of the oscillators and i want commands like "set freq osc 2 245" ... if thats possible at all. i mean we need access to the lowest level units in the dcos. OSC access preferably. is that an option for the future?

ps if you want to know what sequencing from pure data patches sounds like check soundcloud.com/lilakmonoke. the rev 2 btw is a sequencers dream come true even without the obove. full wide range spectrum sounds fully controllable and stackable into infinity!

[kris]

Ok, you guys motivated me to wrap this project up so it might have a future!

I built a little UI for everybody to try out (Windows only for now, sorry), you can build it yourself from the github project at https://github.com/christofmuc/PyTschirper, or for your convenience I created a Windows installer that you can download from https://github.com/christofmuc/PyTschirper/releases.

The "PyTschirper" has a little code editor and an already working Python environment. You can load one of the example files provided by me in the directory Documents\PyTschirperExamples, and run them with CTRL-ENTER or the button at the top. You can also select parts of it and run the selection only with ALT-ENTER. The output windows show stdout (print statements), stderr (errors from the python interpreter), and the MIDI log in case the Rev2 was detected and can communicate.

In the program directory you also find the standalone python module (the pyd file) that you can import into any other python interpreter, you don't have to use the UI tool. I just built it to be able to provide something that works for everybody much more easily than trying to support all the different python installations.

This is how it looks like:



The examples show you how to do some really nice things with just a few lines of code now. Given you ran

Code: [Select]

r = Rev2()
r.detect()
e = r.editBuffer()

you can now e.g. randomize the gated sequencer track 1 with 16 values between 30 and 80:

Code: [Select]

import random
e["Seq Track 1"] = [random.randrange(30, 80) for _ in range(16)]

or you can define yourself a little function to clear the poly sequencer (something I *absolutely* miss):

Code: [Select]

# Clear the poly sequencer, all 6 tracks both layers
def clearPolySequencer(patch):
    for l in range(0, 1):
        for t in range(1,6):
            patch.layer(l)['Poly Seq Note %d' % t] = [60] * 64
            patch.layer(l)['Poly Seq Vel %d' % t] = [0] * 64

and then use it to clear the live edit buffer:

Code: [Select]

clearPolySequencer(e)

You can also use it to load an existing sysex file of patches:

Code: [Select]

patches = r.loadSysex('Rev2_Programs_v1.0.syx')

and then write yourself a little function to show values that differ between layer A and B:

Code: [Select]

def printDiffLayers(patch):
    for param in patch.parameterNames():
        if patch.layer(0)[param].get() != patch.layer(1)[param].get():
            print("%s: %s vs. %s" % (param, patch.layer(0)[param], patch.layer(1)[param]))

and then you can get the difference between the two layers e.g. of the patch VeloStrings (which I adore!):

Code: [Select]

print(p[128].name)
printDiffLayers(p[128])

getting an output like:

Code: [Select]

VeloStrings[+]
Osc 1 Shape Mod: Pulse vs. Saw
Osc 2 Shape Mod: Pulse vs. Saw
Glide Mode: Fixed Rate vs. Fixed Time
LFO 1 Freq: 75 vs. 72
LFO 2 Freq: 72 vs. 71
LFO 3 Freq: 19 vs. 10
LFO 4 Freq: 1 vs. 3
Mod 1 Amount: 130 vs. 124
Mod 2 Amount: 129 vs. 125
Mod 4 Amount: 152 vs. 153
Mod 8 Amount: 0 vs. 254


[kris]

For what it's worth, I have created more programming documentation now online at the right spot. The readme over there shows a bit the concept of the parameter types and attribute lists, and how to load and save sysex etc.:

https://github.com/christofmuc/pytschirp

[Proton]

fantastic work Kris... though I´m on mac I´m impressed and hoping for a mac os version somewhere in the future. Would be very appreciated !

[kris]

fantastic work Kris... though I´m on mac I´m impressed and hoping for a mac os version somewhere in the future. Would be very appreciated !

Thank you, Proton!

The technology used is C++/JUCE/Python, all cross-platform, so with access to Mac hardware and the required knowledge, anybody could go and compile it.

I'd be happy to help in case of problems, I just don't have a Mac. I would expect the CMake files to be in need of some tweaking regarding libraries linked etc., so it would require somebody familiar with that kind of thing, it's certainly not trivial.

Oh, a Linux version I could do in case somebody needs that, but for UI programs I tend to stay on Windows.