The Official Sequential/DSI Forum

wav2evolver for the Pro3

chysn

  • ***
  • 1177
wav2evolver for the Pro3
« on: October 07, 2020, 07:28:27 PM »
I'm hoping that a Pro 3 owner can help a future Pro 3 owner out a bit.

Years before the Pro 3 allowed custom wavetables, there was the Evolver. When I had an Evolver, I developed a utility to convert uploaded WAV files to Evolver system exclusive files.

I have updated this utility so that it can also generate Pro 3 wavetables based on files uploaded to the utility or drawn in the waveform window. Right now, it simply loads the selected waveform into all 16 reference waves; but once this proof-of-concept works well, I'll add multi-wave morphing capability.

So, I wonder if somebody could give it a try; either select a wave from the Library, draw a wave, or upload one of your own, then click on the "Pro 3" tab to download the Pro 3 wavetable.

More to come!

http://wav2evolver.beigemaze.com/

If you don't want to mess around with the waveform editor, but are still willing to help out, please load the attached SysEx file into your Pro 3. It's a sawtooth wave sample from my old Little Phatty. At first I'd just like to know if the file successfully loads into a Pro 3, and sounds like a saw wave.
« Last Edit: October 07, 2020, 08:14:29 PM by chysn »
DSI: DSM03; previously: Mopho Keyboard, Desktop Mopho, Evolver, DSM01
Hardware: Eurorack, Arturia MicroBrute
Software: macOS, Ableton, MuseScore2
Modular Grid: https://www.modulargrid.net/e/racks/view/354385
GitHub: https://github.com/chysn

Re: wav2evolver for the Pro3
« Reply #1 on: October 07, 2020, 09:23:44 PM »
I'm hoping that a Pro 3 owner can help a future Pro 3 owner out a bit.

Years before the Pro 3 allowed custom wavetables, there was the Evolver. When I had an Evolver, I developed a utility to convert uploaded WAV files to Evolver system exclusive files.

I have updated this utility so that it can also generate Pro 3 wavetables based on files uploaded to the utility or drawn in the waveform window. Right now, it simply loads the selected waveform into all 16 reference waves; but once this proof-of-concept works well, I'll add multi-wave morphing capability.

So, I wonder if somebody could give it a try; either select a wave from the Library, draw a wave, or upload one of your own, then click on the "Pro 3" tab to download the Pro 3 wavetable.

I had also started looking at the format a while back but got sidetracked with the pandemic, energy levels, and client work. So it was great to see you also taking a look at this. I'll try to help out when I can.
Jeff Kellem—Typeface designer, Composer, Pianist, Analog synths, Dancer
https://1403.slantedhall.com/ | https://slantedhall.com/

Re: wav2evolver for the Pro3
« Reply #2 on: October 07, 2020, 09:48:07 PM »
If you don't want to mess around with the waveform editor, but are still willing to help out, please load the attached SysEx file into your Pro 3. It's a sawtooth wave sample from my old Little Phatty. At first I'd just like to know if the file successfully loads into a Pro 3, and sounds like a saw wave.

I did a quick test on my Pro 3 SE. The waveform didn't seem to load, still has the 33. Sequential wave form. A random waveform I already had did load properly. When I have a chance, I'll try to take a closer look.
Jeff Kellem—Typeface designer, Composer, Pianist, Analog synths, Dancer
https://1403.slantedhall.com/ | https://slantedhall.com/

chysn

  • ***
  • 1177
Re: wav2evolver for the Pro3
« Reply #3 on: October 08, 2020, 04:58:51 AM »
Thanks for trying it!

I was a bit concerned about the last two bytes of the format (just before end-of-sysex). They didn't seem to fit in as waveform data. I think they're some sort of checksum. I'll keep playing around with it.

Did your display indicate that it was loading a wavetable for a while?
« Last Edit: October 08, 2020, 05:20:33 AM by chysn »
DSI: DSM03; previously: Mopho Keyboard, Desktop Mopho, Evolver, DSM01
Hardware: Eurorack, Arturia MicroBrute
Software: macOS, Ableton, MuseScore2
Modular Grid: https://www.modulargrid.net/e/racks/view/354385
GitHub: https://github.com/chysn

Re: wav2evolver for the Pro3
« Reply #4 on: October 08, 2020, 08:02:08 AM »
Just downloaded a wavetable from your site and took a look.

The headers appear correct to me.  You mentioned it could be the final bytes.

Here's the last five bytes from the wavetable file I downloaded:
73
00
05
38
F7

Here's the last five bytes from a valid wavetable in the same position:
01
60
09
00
F7




Sequential Pro 3 and Rev 2 - Sound Sets and Patch Banks - Now Available on Sellfy: https://sellfy.com/sounddesign/

chysn

  • ***
  • 1177
Re: wav2evolver for the Pro3
« Reply #5 on: October 08, 2020, 08:31:57 AM »
The last word before the end-of-sysex $f7 is definitely a checksum, and it surely needs to be right before the wavetable is accepted.

I'll ask Sequential about it and see if they'll help me out. But I know they're busy with Prophet 5 stuff at the moment, so the plan is to pound on it for a little while before asking. It's not as simple as "count up all the values and mod 65536," but it appears to be linear behavior, so there's a good chance I figure it out myself.
« Last Edit: October 08, 2020, 08:36:20 AM by chysn »
DSI: DSM03; previously: Mopho Keyboard, Desktop Mopho, Evolver, DSM01
Hardware: Eurorack, Arturia MicroBrute
Software: macOS, Ableton, MuseScore2
Modular Grid: https://www.modulargrid.net/e/racks/view/354385
GitHub: https://github.com/chysn

Re: wav2evolver for the Pro3
« Reply #6 on: October 08, 2020, 10:14:03 AM »
Thanks for trying it!

I was a bit concerned about the last two bytes of the format (just before end-of-sysex). They didn't seem to fit in as waveform data. I think they're some sort of checksum. I'll keep playing around with it.

Did your display indicate that it was loading a wavetable for a while?

Yes, it did indicate that it was loading a wavetable.

The last byte is often a checksum, though sometimes ignored, e.g., on the MIDI tuning SysEx (at least for the Prophet X/XL). Try an XOR then of course mask the 7 bits (with & 0x7f). That's at least what has been done for some Sequential things, I believe, when the checksum is a single byte before the EOX final byte.

If that's the method being used, you may need to adjust the start point for calculating the checksum. I've seen some systems (not necessarily Sequential) start with the byte after the initial SYSEX start 0xf0 byte and others that skip the header bytes.
Jeff Kellem—Typeface designer, Composer, Pianist, Analog synths, Dancer
https://1403.slantedhall.com/ | https://slantedhall.com/

chysn

  • ***
  • 1177
Re: wav2evolver for the Pro3
« Reply #7 on: October 08, 2020, 10:51:16 AM »
Thanks!

Do you know anything about the MIDI tuning sysex for the Prophet X? Sequential always uses the same sysex packing method, so it's possible there'd be clues in other checksum algorithms. How do you get the tuning checksum to be ignored? Waldorf XT, for example, ignores the checksum and accepts the data if the checksum is $7f.

In this case, I'm fairly sure it's a two-byte checksum, or parity bytes, or whatever it is. And I'm confident that the checksum is applied to the waveform data and not to the 16-byte message header (it does not vary with wavetable name or number).

The thing that makes it challenging is that the original waveform is downsampled several times throughout the PCM. The non-orthogonality of making a bump to one value makes it tricky to observe the impact on the checksum.

But I do suspect that there's a non-zero starting point, as you suggested, and I think that starting point is $7f.
« Last Edit: October 08, 2020, 10:57:25 AM by chysn »
DSI: DSM03; previously: Mopho Keyboard, Desktop Mopho, Evolver, DSM01
Hardware: Eurorack, Arturia MicroBrute
Software: macOS, Ableton, MuseScore2
Modular Grid: https://www.modulargrid.net/e/racks/view/354385
GitHub: https://github.com/chysn

Re: wav2evolver for the Pro3
« Reply #8 on: October 08, 2020, 05:23:37 PM »
Do you know anything about the MIDI tuning sysex for the Prophet X? Sequential always uses the same sysex packing method, so it's possible there'd be clues in other checksum algorithms. How do you get the tuning checksum to be ignored?

It's simply the MIDI Tuning Spec. It's not that I do anything to get the Prophet X/XL to ignore the checksum, it just seems that it does. I'd created a few inverted tunings: https://forum.sequential.com/index.php/topic,3333.msg36056.html#msg36056
and noticed that the checksum was being ignored (and that all of the alternate tunings provided had a 0 checksum value. The MIDI Tuning Spec indicated to calculate the checksum based on the values between F0 and checksum F7 (so not including those three). I thought I remembered Sequential doing a similar checksum in the past but that might be my brain misremembering.

The 7 bytes in 8 bytes packing method is not used for the MIDI Tuning Spec.

Quote
In this case, I'm fairly sure it's a two-byte checksum, or parity bytes, or whatever it is. And I'm confident that the checksum is applied to the waveform data and not to the 16-byte message header (it does not vary with wavetable name or number).

The thing that makes it challenging is that the original waveform is downsampled several times throughout the PCM. The non-orthogonality of making a bump to one value makes it tricky to observe the impact on the checksum.

But I do suspect that there's a non-zero starting point, as you suggested, and I think that starting point is $7f.

I'd hope for a checksum byte(s) but also recall the Evolver didn't include a checksum for the Waveshape dump data. When I have a moment, I'll try to look at what you've dug up and see what comes to mind.

Jeff Kellem—Typeface designer, Composer, Pianist, Analog synths, Dancer
https://1403.slantedhall.com/ | https://slantedhall.com/

chysn

  • ***
  • 1177
Re: wav2evolver for the Pro3
« Reply #9 on: October 08, 2020, 07:54:48 PM »
When I have a moment, I'll try to look at what you've dug up and see what comes to mind.

What I've tried so far:

(1) Adding the sum of each byte in the PCM data (6172x16 bytes). Then & 0x7f7f the sum to mask away bits 7 for sysex.
(2) Taking the bytes of the PCM data two at a time, converting each pair to a big-endian 16-bit number and adding them. & 0x7f7f again.
(3) Doing the above stuff, but with packed data instead of raw PCM data.
(4) Alternately adding and subtracting to get the sum
DSI: DSM03; previously: Mopho Keyboard, Desktop Mopho, Evolver, DSM01
Hardware: Eurorack, Arturia MicroBrute
Software: macOS, Ableton, MuseScore2
Modular Grid: https://www.modulargrid.net/e/racks/view/354385
GitHub: https://github.com/chysn

chysn

  • ***
  • 1177
Re: wav2evolver for the Pro3
« Reply #10 on: October 09, 2020, 06:37:26 AM »
I did ask Sequential for some help on this. Puzzles are fun, to a certain extent, but I'd like to get on with the rest of the project.  :)
DSI: DSM03; previously: Mopho Keyboard, Desktop Mopho, Evolver, DSM01
Hardware: Eurorack, Arturia MicroBrute
Software: macOS, Ableton, MuseScore2
Modular Grid: https://www.modulargrid.net/e/racks/view/354385
GitHub: https://github.com/chysn

Re: wav2evolver for the Pro3
« Reply #11 on: October 09, 2020, 05:53:01 PM »
What I've tried so far:

(1) Adding the sum of each byte in the PCM data (6172x16 bytes). Then & 0x7f7f the sum to mask away bits 7 for sysex.

That PCM data number seems potentially off, slightly, though perhaps I'm missing something. Just looking at the numbers with Sequential packing that 6172x16 bytes would go from 98,752 bytes to 112,860 bytes which is larger than the 112,368 byte size of the wave table SysEx file.

Look forward to the response you get from your query. I hadn't yet reached out since I'd been busy with other projects.
Jeff Kellem—Typeface designer, Composer, Pianist, Analog synths, Dancer
https://1403.slantedhall.com/ | https://slantedhall.com/

chysn

  • ***
  • 1177
Re: wav2evolver for the Pro3
« Reply #12 on: October 09, 2020, 07:49:53 PM »
That PCM data number seems potentially off, slightly, though perhaps I'm missing something. Just looking at the numbers with Sequential packing that 6172x16 bytes would go from 98,752 bytes to 112,860 bytes which is larger than the 112,368 byte size of the wave table SysEx file.

Yes, the discrepancy is due to me being careless with my numbers above. It's actually 6144x16 bytes of PCM data, packed to 112,349 bytes. Then there's a 16-byte header, a 2-byte checksum (or parity) word, and then $f7, for a grand total of 112,368.
DSI: DSM03; previously: Mopho Keyboard, Desktop Mopho, Evolver, DSM01
Hardware: Eurorack, Arturia MicroBrute
Software: macOS, Ableton, MuseScore2
Modular Grid: https://www.modulargrid.net/e/racks/view/354385
GitHub: https://github.com/chysn

chysn

  • ***
  • 1177
Re: wav2evolver for the Pro3
« Reply #13 on: October 10, 2020, 06:28:01 AM »
17-byte header, not 16. 6144x16 packs to 112,348, etc.

But today's plans involved assuming the checksum was, itself, part of the packed data. This doesn't seem to work out, either.
DSI: DSM03; previously: Mopho Keyboard, Desktop Mopho, Evolver, DSM01
Hardware: Eurorack, Arturia MicroBrute
Software: macOS, Ableton, MuseScore2
Modular Grid: https://www.modulargrid.net/e/racks/view/354385
GitHub: https://github.com/chysn

chysn

  • ***
  • 1177
Re: wav2evolver for the Pro3
« Reply #14 on: October 10, 2020, 06:42:02 AM »
I got it!

The PCM data is big-endian, I knew that. But the checksum is a sum of little-endian pairs. Then this checksum is & 0x7f7f and stored as a little-endian pair. It's possible that the checksum is packed rather than simply & 0x7f7f, but that'll be easy to figure out when I have a Pro 3, now that I'm certain how it's generated.

Now I may continue. I'll have a new test file up shortly.
« Last Edit: October 10, 2020, 07:14:10 AM by chysn »
DSI: DSM03; previously: Mopho Keyboard, Desktop Mopho, Evolver, DSM01
Hardware: Eurorack, Arturia MicroBrute
Software: macOS, Ableton, MuseScore2
Modular Grid: https://www.modulargrid.net/e/racks/view/354385
GitHub: https://github.com/chysn