Analyzing the REV2 SysEx structure

Razmo

  • ***
  • 2168
  • I am shadow...
    • Kaleidoscopic Artworks
Re: Analyzing the REV2 SysEx structure
« Reply #20 on: November 16, 2017, 02:47:53 AM »
I'm actually more worried about the weird behavior I'm experiencing with the LFO's right now...

Understandable! However, R2 a recent and current product so I would be very surprised if this bug is not resolved at some point in the future. Makes me wonder if its cause is algorithmic or voice CPU overload.

Hard to say, but I THINK it's just algorithmic... it works right on some keys, and switch around a few keys... the keys it happens on are connected, and it's about 2-4 keys in a row that will cause this... it does this even when only pressing one key at a time... so if the majority works ok this way and a few do not, I believe it cannot be a CPU power problem... but I cannot know... DSI will have to solve this, because it's darn annoying.
If you need me, follow the shadows...

dslsynth

  • ***
  • 1040
Re: Analyzing the REV2 SysEx structure
« Reply #21 on: November 16, 2017, 02:56:24 AM »
This is a perfect time for DSI staff to reassure their customers that this problem will be resolved eventually!
#!/bin/sh
cp -f $0 $HOME/.signature

Razmo

  • ***
  • 2168
  • I am shadow...
    • Kaleidoscopic Artworks
Re: Analyzing the REV2 SysEx structure
« Reply #22 on: November 20, 2017, 06:44:25 AM »
Current analysis here... there are quite a few indexes that I either have not populated yet, or they are simply "reserved" spaces because I think I've covered all of the parameters now (all of the NRPN parameters in the manual is covered here, so I doubt more are hiding in the non populated indexes in my list)... there may be some errors in there, as I've not made my editor yet, it's simply a "current state of analysis"... especially the poly sequencer steps bit fields may be wrong... need to experiment as the editor finishes...

Also, mod source and destination tables have not yet been made... they should match the lists in the manual I guess...

And also, note that I have NO IDEAR if there is a byte in one of the unused indexes, that designate a versioning of the dump as is the case with the Prophet 12... I do not even know if there are more than one version at this very moment... it has been made with the presets dumped thru OS 1.1.0.1.

-----------------------------------------

PROGRAM DATA DUMP (SYSEX):

LAYER A:
0      OSC1 Freq (0-120)
1      OSC1 Freq (0-120)
2      OSC1 Finetune (0-100)
3      OSC2 Finetune (0-100)
4      OSC1 Shape (0-4, off/Saw/SawTri/Tri/Pul)
5      OSC2 Shape (0-4, off/Saw/SawTri/Tri/Pul)
6      OSC1 ShapeMod (0-99)
7      OSC2 ShapeMod (0-99)
8      OSC1 Glide (0-127)
9      OSC2 Glide (0-127)
10      OSC1 Key on/off (0-1, off/on)
11      OSC2 Key on/off (0-1, off/on)
12      OSC1 Note Reset (0-1, off/on)
13      OSC2 Note Reset (0-1, off/on)
14      OSC Mix (0-127)
15      SubOct Level (0-127)
16      Noise Level (0-127)
17      Sync (0-1, off/on)
18      Glide Mode (0-3, Fixed Rate/Fixed Rate A/Fixed Time/Fixed Time A)
19      Glide on/off (0-1, off/on)
20      Pitch Bend Range (0-12)
21      OSC Slop (0-127)
22      Cutoff Freq (0-164)
23      Resonance (0-127)
24      Filter Key Amount (0-127)
25      Filter Audio Mod (0-127)
26      Filter Poles (0-1, 2pole/4pole)
27      VCA Level (0-127)
28      Program Volume (0-127)
29      Pan Spread (0-127)
30      Env3 Destination (0-52, *see table below)
31      Env3 Repeat (0-1, off/on)
32      EnvF Amount (0-254)
33      EnvA Amount (0-127)
34      Env3 Amount (0-254)
35      EnvF Velocity (0-127)
36      EnvA Velocity (0-127)
37      Env3 Velocity (0-127)
38      EnvF Delay (0-127)
39      EnvA Delay (0-127)
40      Env3 Delay (0-127)
41      EnvF Attack (0-127)
42      EnvA Attack (0-127)
43      Env3 Attack (0-127)
44      EnvF Decay (0-127)
45      EnvA Decay (0-127)
46      Env3 Decay (0-127)
47      EnvF Sustain (0-127)
48      EnvA Sustain (0-127)
49      Env3 Sustain (0-127)
50      EnvF Release (0-127)
51      EnvA Release (0-127)
52      Env3 Release (0-127)
53      LFO1 Rate (0-150)
54      LFO2 Rate (0-150)
55      LFO3 Rate (0-150)
56      LFO4 Rate (0-150)
57      LFO1 Shape (0-5, off/Tri/Saw/RevSaw/Square/Random)
58      LFO2 Shape (0-5, off/Tri/Saw/RevSaw/Square/Random)
59      LFO3 Shape (0-5, off/Tri/Saw/RevSaw/Square/Random)
60      LFO4 Shape (0-5, off/Tri/Saw/RevSaw/Square/Random)
61      LFO1 Amount (0-127)
62      LFO2 Amount (0-127)
63      LFO3 Amount (0-127)
64      LFO4 Amount (0-127)
65      LFO1 Destination (0-52, *see table below)
66      LFO2 Destination (0-52, *see table below)
67      LFO3 Destination (0-52, *see table below)
68      LFO4 Destination (0-52, *see table below)
69      LFO1 ClkSync (0-1, 0ff/on)
70      LFO2 ClkSync (0-1, 0ff/on)
71      LFO3 ClkSync (0-1, 0ff/on)
72      LFO4 ClkSync (0-1, 0ff/on)
73      LFO1 KeySync (0-1, off/on)
74      LFO2 KeySync (0-1, off/on)
75      LFO3 KeySync (0-1, off/on)
76      LFO4 KeySync (0-1, off/on)
77      Mod1 Source (0-22, *see table below)
78      Mod2 Source (0-22, *see table below)
79      Mod3 Source (0-22, *see table below)
80      Mod4 Source (0-22, *see table below)
81      Mod5 Source (0-22, *see table below)
82      Mod6 Source (0-22, *see table below)
83      Mod7 Source (0-22, *see table below)
84      Mod8 Source (0-22, *see table below)
85      Mod1 Amount (0-254)
86      Mod2 Amount (0-254)
87      Mod3 Amount (0-254)
88      Mod4 Amount (0-254)
89      Mod5 Amount (0-254)
90      Mod6 Amount (0-254)
91      Mod7 Amount (0-254)
92      Mod8 Amount (0-254)
93      Mod1 Destination (0-52, *see table below)
94      Mod2 Destination (0-52, *see table below)
95      Mod3 Destination (0-52, *see table below)
96      Mod4 Destination (0-52, *see table below)
97      Mod5 Destination (0-52, *see table below)
98      Mod6 Destination (0-52, *see table below)
99      Mod7 Destination (0-52, *see table below)
100      Mod8 Destination (0-52, *see table below)
101      Mod Wheel Amount (0-254)
102      Mod Wheel Destination (0-52, *see table below)
103      Pressure Mod Amount (0-254)
104      Pressure Mod Destination (0-52, *see table below)
105      Breath Mod Amount (0-254)
106      Breath Mod Destination (0-52, *see table below)
107      Velocity Mod Amount (0-254)
108      Velocity Mod Destination (0-52, *see table below)
109      MIDI Foot Mod Amount (0-254)
110      MIDI Foot Mod Destination (0-52, *see table below)
111      Gated Seq1 Destination (0-52, *see table below)
112      Gated Seq2 Destination (0-53, *see table below)
113      Gated Seq3 Destination (0-52, *see table below)
114      Gated Seq4 Destination (0-53, *see table below)
115      FX Select (0-13, Off/DM/DDS/BBD/Ch/PH/PL/PM/F1/F2/Rvb/RM/Dst/HPF)
116      FX on/off (0-1, off/on)
117      FX Mix (0-127)
118      FX Parameter 1 (0-255)
119      FX Parameter 2 (0-127)
120      FX Clock Sync on/off (0-1, off/on)
121      -
122      Key Mode (0-5, Low/Hi/Last/LowR/HiR/LastR)
123      Unison on/off (0-1, off/on)
124      Unison Mode (0-15/Chord)
125      -
126      -
127      -
128      -
129      -
130      BPM (30-250)
131      Divide (0-12, H/Q/8th/8H/8S/8T/16th/16H/16S/16T/32nd/32T/64T
132      Arp Mode (0-4, Up/Down/Up+Down/Random/assign)
133      Arp Range (0-2, 1Oct/2Oct/3Oct)
134      Arp Repeats (0-3)
135      Arp Relatch on/off (0-1, off/on)
136      Arp on/off (0-1, off/on)
137      -
138      Sequencer Mode (0-4, Normal/NoReset/NoGate/NoGateReset/KeyStep)
139      Sequencer Type (0-1, Gated/Poly)
140-155   Gated Seq1 Step 1-16 (0-125/reset/rest)
156-171   Gated Seq2 Step 1-16 (0-125/reset/rest)
172-187   Gated Seq3 Step 1-16 (0-125/reset/rest)
188-203   Gated Seq4 Step 1-16 (0-125/reset/rest)
204      -
205      -
206      -
207      -
208      Unison Detune (0-16)
209      Pan Mod Mode (0-1, Alternate/Fixed)
210      -
211      -
212      -
213      -
214      -
215      -
216      -
217      -
218      -
219      -
220      -
221      -
222      -
223      -
224      -
225      -
226      -
227      -
228      -
229      -
230      -
231      Layer Mode (0-2, LayerA/SplitAB/StackAB)
232      Split Point (0-120)
233      -
234      -
235-254   Layer A Name
255      -
256-319   Poly Seq Track1 Notes (bit0-7;note, bit8;tie)
320-383   Poly Seq Track1 Velocities (bit0-7;velocity, bit8;rest)
384-447   Poly Seq Track2 Notes (bit0-7;note, bit8;tie)
448-511   Poly Seq Track2 Velocities (bit0-7;velocity, bit8;rest)
512-575   Poly Seq Track3 Notes (bit0-7;note, bit8;tie)
576-639   Poly Seq Track3 Velocities (bit0-7;velocity, bit8;rest)
640-703   Poly Seq Track4 Notes (bit0-7;note, bit8;tie)
704-767   Poly Seq Track4 Velocities (bit0-7;velocity, bit8;rest)
768-831   Poly Seq Track5 Notes (bit0-7;note, bit8;tie)
832-895   Poly Seq Track5 Velocities (bit0-7;velocity, bit8;rest)
896-959   Poly Seq Track6 Notes (bit0-7;note, bit8;tie)
960-1023   Poly Seq Track6 Velocities (bit0-7;velocity, bit8;rest)

LAYER B:
Same as above, just add 1024 to the index value.
Last four bytes (2044-2047) does not exist in SysEx dump.
« Last Edit: November 20, 2017, 07:25:52 AM by Razmo »
If you need me, follow the shadows...

Razmo

  • ***
  • 2168
  • I am shadow...
    • Kaleidoscopic Artworks
Re: Analyzing the REV2 SysEx structure
« Reply #23 on: November 20, 2017, 07:12:37 AM »
a few errors in the manual regarding NRPNs:

NRPN no. 9 is missing (OSC2 KBD on/off)
Description of NRPN no. 37 is a copy of the above (VCA Env Release), should be LFO1 Frequency
NRPN 52 description... the number "4" is missing.
 
If you need me, follow the shadows...

shiihs

  • **
  • 103
  • phasing in and out of reality
Re: Analyzing the REV2 SysEx structure
« Reply #24 on: March 25, 2018, 03:10:27 PM »
Thanks for posting the sysex analysis. It has been tremendously helpful in my experiments so far.
It's a pity this is not documented officially, but I guess there's no guarantee for sysex to remain compatible in future OS revisions, so it might be too much work to maintain the documentation. Anyway I wanted to contribute my 5 cents by adding a sysex analysis of the global parameters in OS version 1.1.0.3. Note that the data must NOT be midi-unpacked in order to analyze the contents!

SYSEX_BYTEPOS   NRPN    NAME                    VALUES      DISPLAY_VALUES
0               4097    "Master Coarse Tune"    0-24        -12 - +12
1               4096    "Master Fine Tune"      0-100       -50 - +50
2               4098    "MIDI Channel"          0-16        0=ALL,1-16
3               4099    "MIDI Clock Mode"       0-4         0=OFF,1=MASTER,2=SLAVE,3=SLAVE THRU,4=SLAVE NO S/S
4               4100    "MIDI Clock Cable"      0-1         0=MIDI,1=USB
5               4101    "MIDI Param Send"       0-2         0=OFF,1=CC,2=NRPN
6               4102    "MIDI Param Receive"    0-2         0=OFF,1=CC,2=NRPN
7               4103    "MIDI Control Enable"   0-1         0=OFF,1=ON
8               4104    "MIDI SysEx Cable"      0-1         0=MIDI,1=USB
9               4105    "MIDI Out Select"       0-1         0=MIDI,1=USB
10               --     "UNKNOWN"
11               --     "UNKNOWN"
12              4107    "Local Control"         0-1         0=OFF,1=ON
13              4111    "Seq Pedal Mode"        0-3         0=NORMAL,1=TRIGGER,2=GATE,3=TRIG-GATE
14              4109    "Pot Mode"              0-2         0=RELATIVE,1=PASSTHRU,2=JUMP
15              4112    "Sustain Polarity"      0-1         0=NORMAL,1=REVERSE
16              4116    "Alt. Tunings"          0-16        (see list in manual)
17              4113    "Velocity Curve"        0-7         velocity curve 1,2,..8
18              4114    "Pressure Curve"        0-3         pressure curve 1,2,..4
19              4115    "Mono/Stereo"           0-1         0=stereo,1=mono
20              4120    "Screen Saver"          0-1         0=OFF,1=ON
21              4119    "Multi Mode"            0-1         0=OFF,1=ON
22              4118    "MIDI Prog Enable"      0-1         0=OFF,1=ON
23              4121    "Sustain/Arp"           0-1         0=ARP HOLD,1=SUSTAIN
24              4122    "Foot Assign"           0-5         0=Breath CC2,1=Foot CC4,2=Exp CC11,3=Volume,4=LPF Full,5=LPF Half
25               --     "UNKNOWN"
26               --     "UNKNOWN"
27               --     "UNKNOWN"
28               --     "UNKNOWN"

--
gear: prophet rev2 16 voice, kawai NV10, casio wk-7600, Roland Integra-7, supercollider, ardour

links:

https://www.youtube.com/stefaanhimpe
https://soundcloud.com/stefaanhimpe
https://technogems.blogspot.com
https://a-touch-of-music.blogspot.com/

dslsynth

  • ***
  • 1040
Re: Analyzing the REV2 SysEx structure
« Reply #25 on: March 26, 2018, 02:07:41 PM »
It's a pity this is not documented officially, but I guess there's no guarantee for sysex to remain compatible in future OS revisions, so it might be too much work to maintain the documentation.

Tooling exist to ease the documentation of the sysex format and and that tooling is quite robust to changes caused by software updates. So its indeed possible to document the sysex formats precisely. However, its also a question of resources and priorities.
#!/bin/sh
cp -f $0 $HOME/.signature

Re: Analyzing the REV2 SysEx structure
« Reply #26 on: April 20, 2018, 12:23:44 AM »
Although it not a critical issue it just gives a feeling the REV2 is not "complete". Apparently there are unused Sysex fields that  could be used to add the data for this step and such change would not break backward compatibility with the current patches.



Re: Analyzing the REV2 SysEx structure
« Reply #27 on: April 20, 2018, 11:48:09 AM »
In development you leave space for parameters you may or may not need. In the end there are bytes that don't get used. This in no way makes the instrument incomplete.
SEQUENTIAL

Re: Analyzing the REV2 SysEx structure
« Reply #28 on: July 26, 2018, 01:18:37 PM »
I've never used Sysex messages other than to dump or save stuff.
But does this mean that i can remotely change a Gated Pattern sequence by sending sysex messages ?

Re: Analyzing the REV2 SysEx structure
« Reply #29 on: July 26, 2018, 01:37:08 PM »
I've never used Sysex messages other than to dump or save stuff.
But does this mean that i can remotely change a Gated Pattern sequence by sending sysex messages ?

It seems like it. I would go for just using NRPN for that; much easier.

Re: Analyzing the REV2 SysEx structure
« Reply #30 on: July 26, 2018, 01:47:15 PM »
Great, thanks, that's exciting !
I'd like to send patterns from Max For Live to the Prophet, I don't know much about either Sysex or NRPN, but i'll check. There is a brand new free max for live NRPN message sender, so I can grab some knowledge from there. But Sysex might be more convenient to format with Max/MSP [sxformat] object ? I'm open to suggestions !

Re: Analyzing the REV2 SysEx structure
« Reply #31 on: July 26, 2018, 01:57:09 PM »
So, I you don't mind helping a Sysex noob, what would a sysex message to set up seq 1 step 1 look like ?

Razmo

  • ***
  • 2168
  • I am shadow...
    • Kaleidoscopic Artworks
Re: Analyzing the REV2 SysEx structure
« Reply #32 on: July 26, 2018, 01:58:03 PM »
I've never used Sysex messages other than to dump or save stuff.
But does this mean that i can remotely change a Gated Pattern sequence by sending sysex messages ?

Yes you can... you can change everything with the SysEx commands... With only NRPNs you do not have full control of the entire memory, you do with SysEx... you can store presets (even in the factory ones, and this only works via SysEx dumps). Also all global parameters can be changed... A SysEx for the REV2 has all the parameters of both layers in it, including the gated sequencers and the polysequencers data as well... unfortunately the SysEx of the REV2 is broken... the last four values of the sixth tracks velocity in layer B are not in the dump...

I've made an editor that control all with the VEry old SoundDiver program from Emagic, and it works fine:
If you need me, follow the shadows...

Re: Analyzing the REV2 SysEx structure
« Reply #33 on: July 26, 2018, 02:02:48 PM »
I've made an editor that control all with the VEry old SoundDiver program from Emagic, and it works fine:
Wow, looks nice ! Congrats ! Out of curiosity, do you share the editor ? (I'm on Mac).
As for the broken sysex messages, can we hope for a firmware update ?
Anyway, I'd be happy if someone could help me with how to format the messages to change the Seq values.

shiihs

  • **
  • 103
  • phasing in and out of reality
Re: Analyzing the REV2 SysEx structure
« Reply #34 on: July 26, 2018, 02:10:16 PM »
So, I you don't mind helping a Sysex noob, what would a sysex message to set up seq 1 step 1 look like ?

If you want to change only a few parameters you are probably much better off sending NRPN messages.
Every step has its own NRPN parameter. These are documented in the manual. In the gated sequencer you can also use NRPN to set up the destinations.

You cannot - to the best of my knowledge - use sysex to update only some parameters selectively, you always have to send all values for all parameters in bulk. Add to that that the sysex structure at times is a bit tricky, with e.g. sign bits of modulation values hiding inside bytes that encode other (enumeration) parameters. I assume this complexity was introduced to keep backward compatibility with prophet 8, but that's only a wild guess as I've never studied it.
--
gear: prophet rev2 16 voice, kawai NV10, casio wk-7600, Roland Integra-7, supercollider, ardour

links:

https://www.youtube.com/stefaanhimpe
https://soundcloud.com/stefaanhimpe
https://technogems.blogspot.com
https://a-touch-of-music.blogspot.com/

Razmo

  • ***
  • 2168
  • I am shadow...
    • Kaleidoscopic Artworks
Re: Analyzing the REV2 SysEx structure
« Reply #35 on: July 26, 2018, 02:19:12 PM »
I've made an editor that control all with the VEry old SoundDiver program from Emagic, and it works fine:
Wow, looks nice ! Congrats ! Out of curiosity, do you share the editor ? (I'm on Mac).
As for the broken sysex messages, can we hope for a firmware update ?
Anyway, I'd be happy if someone could help me with how to format the messages to change the Seq values.

Yes, i don't mind sharing the adaption if anyone want it, but i don't think that many are using SoundDiver anymore to be honest... if anyone want it, just write me a Private Message and I'll send it...

The broken SysEx is on the bug list at DSI... I've reported the bug twice, and the second time they asked me to submit a report again, to make sure it was put on the bug list... so i asume it'll be fixed soon... but it still does not work in the current beta, so I'm still waiting for the fix.

The NRPNs are just plain NRPNs... the values for the sequencer is all in the manual, but for Layer A, Sequence value one it is:

$B0, $63, $01 (top 7 bits of the parameter number)
$B0, $62, $40 (bottom 7 bits of the parameter number)

$B0, $06, $00 (top 7 bits of the parameter value)
$B0, $26, $00 (bottom 7 bits of the parameter value)

remember that MIDI bytes are 7 bits, not 8! ... both values are 14 bits in length, split over two 7bit MIDI bytes.
If you need me, follow the shadows...

Re: Analyzing the REV2 SysEx structure
« Reply #36 on: July 26, 2018, 02:21:22 PM »

If you want to change only a few parameters you are probably much better off sending NRPN messages.
Every step has its own NRPN parameter. These are documented in the manual. In the gated sequencer you can also use NRPN to set up the destinations.

You cannot - to the best of my knowledge - use sysex to update only some parameters selectively, you always have to send all values for all parameters in bulk. Add to that that the sysex structure at times is a bit tricky, with e.g. sign bits of modulation values hiding inside bytes that encode other (enumeration) parameters. I assume this complexity was introduced to keep backward compatibility with prophet 8, but that's only a wild guess as I've never studied it.

Ok that's super helpful. I'll try to figure out how to send NRPNs from Max. Thanks a lot, I appreciate the spirit of this place !

Razmo

  • ***
  • 2168
  • I am shadow...
    • Kaleidoscopic Artworks
Re: Analyzing the REV2 SysEx structure
« Reply #37 on: July 26, 2018, 02:21:37 PM »
So, I you don't mind helping a Sysex noob, what would a sysex message to set up seq 1 step 1 look like ?

If you want to change only a few parameters you are probably much better off sending NRPN messages.
Every step has its own NRPN parameter. These are documented in the manual. In the gated sequencer you can also use NRPN to set up the destinations.

You cannot - to the best of my knowledge - use sysex to update only some parameters selectively, you always have to send all values for all parameters in bulk. Add to that that the sysex structure at times is a bit tricky, with e.g. sign bits of modulation values hiding inside bytes that encode other (enumeration) parameters. I assume this complexity was introduced to keep backward compatibility with prophet 8, but that's only a wild guess as I've never studied it.

True... all single parameter changes must be done via NRPNs... even the global ones, plus all sequencer data... the advantage to the SysEx is that it allow you to control the program memories which is nice for creating librarians and managing your programs (and also overwrite the factory programs... as this is only doable via SysEx dumps)..
If you need me, follow the shadows...

Razmo

  • ***
  • 2168
  • I am shadow...
    • Kaleidoscopic Artworks
Re: Analyzing the REV2 SysEx structure
« Reply #38 on: July 26, 2018, 02:25:23 PM »
By the way... it's not totally true that you can only overwrite the factory programs via SysEx... you can copy all user banks to the factory banks and vice versa from the REV2 itself, you just have to enter service mode and choose that function... hold down the two transpose buttons, and at the same time press "global"... you can find the feature in the service mode.
If you need me, follow the shadows...

Re: Analyzing the REV2 SysEx structure
« Reply #39 on: July 26, 2018, 02:29:31 PM »
True... all single parameter changes must be done via NRPNs... even the global ones, plus all sequencer data... the advantage to the SysEx is that it allow you to control the program memories which is nice for creating librarians and managing your programs (and also overwrite the factory programs... as this is only doable via SysEx dumps)..
Yeah, but my point is just to be able to use any sound in the synth and use M4L to send a pre-made sequence to the sequencers.