Anyone know how to unpackage the Tempest SYSEX MIDI?

Anyone know how to unpackage the Tempest SYSEX MIDI?
« on: January 30, 2021, 09:49:23 AM »
I'm trying to decode the patch name from the MIDI data when selecting "Export Sound over MIDI", in TypeScript.

The code I'm using to unpack the MIDI data is https://github.com/stuartkuentzel/sequential-utils/blob/master/src/index.ts, which is a TypeScript version of the great work by https://github.com/Chysn/sequential_lib/blob/main/sequential_packing.h#L84.

Once I have the unpackaged data, I'm running `String.fromCharCode(unpackaged)`, but I can never seen to be able to get a patch name from it.

Any pointers would be super appreciated!

chysn

  • *****
  • 1536
Re: Anyone know how to unpackage the Tempest SYSEX MIDI?
« Reply #1 on: January 30, 2021, 02:43:19 PM »
I don't think the traditional DSI/Sequential 7-into-8 packing scheme is used in the Tempest program data, at least not for the name. If it was, I would expect to see a 0x00 somewhere in the middle of packed ASCII data; since bit 7 is never set within ASCII data, there'd be a pack byte of all zeros. I never see that.

They are packed in some way, though. Every eight characters, the ASCII bit pattern is intact, but they seem to be pulled apart as the mod 8 of the index approaches 7. I'd need to see a lot of programs with short, and known, names with runs of adjacent characters and identical characters, like "ABC", "CBA", "AAA". I could probably then figure it out.

The way the spaces drift to the right, the format doesn’t care about byte boundaries like the DSI format does.

Maybe somebody already knows, though...  :)
« Last Edit: January 30, 2021, 02:58:40 PM by chysn »
Pro 3 #1640

Other: Make Noise modular, Circuit Tracks, MicroBrute
Software: macOS, Ableton, MuseScore2, Serum, Pianoteq, OPS7
www.wav2pro3.comwww.soundcloud.com/beige-mazewww.github.com/chysnwww.beigemaze.com

he/him/his

KoSv

Re: Anyone know how to unpackage the Tempest SYSEX MIDI?
« Reply #2 on: January 30, 2021, 06:40:56 PM »
yeah, I know.
since I wrote the sound randomizer (orig. in C/C++ then compiled to wasm) and I'm able to fully pack and unpack sound files from ram.
I looked in your code and my advise would be not wasting your time.
the tempest ram sound sysex format is really complicated and not trivial at all.
due to hardware limitations nearly every parameter has its own bit mask.
if I find some time I could extract a wasm file for you to only return the soundname.
whats your project all about?
greets!
Classical piano drilled.
Jazz disillusioned.
Technoid.

Re: Anyone know how to unpackage the Tempest SYSEX MIDI?
« Reply #3 on: January 30, 2021, 07:18:16 PM »
The Tempest Randomizer is new to me, very cool stuff! Out of curiosity, is it open source?
The project is a simple web interface for users to share Tempest sounds/kits using WebMidi. Was looking to abstract some of the functionality into an open source Sequential utils TypeScript library, since it seems like a few determined devs spend the time unpacking the sysex and mapping it to params.

But at the moment, the main thing I'm looking to do is get the patch name.
« Last Edit: January 30, 2021, 08:28:22 PM by tempest »

kris

  • **
  • 131
Re: Anyone know how to unpackage the Tempest SYSEX MIDI?
« Reply #4 on: July 28, 2022, 07:28:27 AM »
yeah, I know.
since I wrote the sound randomizer (orig. in C/C++ then compiled to wasm) and I'm able to fully pack and unpack sound files from ram.
I looked in your code and my advise would be not wasting your time.
the tempest ram sound sysex format is really complicated and not trivial at all.
due to hardware limitations nearly every parameter has its own bit mask.
if I find some time I could extract a wasm file for you to only return the soundname.
whats your project all about?
greets!
For what it's worth, I have been asked to add the Tempest to the KnobKraft Orm free sysex Librarian. For that, I want to parse the sysex data and extract the sound name so the Librarian displays it.

This is the code that works:

Code: [Select]
def nameFromDump(message):
    # Tempest names are 0 terminated and not fixed length
    dataBlock = unescapeSysex(message[6:-1])
    if len(dataBlock) > 0:
        sound_name = ''
        i = 0
        while dataBlock[i] != 0:
            sound_name += chr(dataBlock[i])
            i += 1
        return sound_name
    return "Invalid"

The structure is such you need to skip every 8th byte after discarding the sysex header of length 6. Normally, the 8th byte should contain the most significant bits of the previous 7 bytes, but that does not seem the case. Adding those in would garble the ASCII names, so it must be something different. Might be a checksum (mighty many then), but I didn't look further.

This is Python to do the discards and extract only the valid data block:

Code: [Select]
def unescapeSysex(sysex):
    # The Tempest has a different scheme for the high bit as all other DSIs, with a strange 8th byte transmitted after
    # 7 normal bytes. It does not seem to be the MSBs, because that would screw up the ASCII patch name, and
    # I don't see this is a checksum of some sort yet.
    result = []
    dataIndex = 0
    while dataIndex < len(sysex):
        for i in range(7):
            if dataIndex < len(sysex):
                result.append(sysex[dataIndex])
            dataIndex += 1
        dataIndex += 1
    return result
The names then are displayed then show that these are varying length, and as paths like /S are sounds and then /S/Toms are the Toms etc. As they are varying length, it looks they are 0-terminated. If I run the code above on the factory program sysex, I get:

Quote
/S/Toms/Zap~Mid
/S/Toms/Zap~~Lo
/S/Toms/Zap~~~Hi
/S/Toms/Trinculo Mid
/S/Toms/Trinculo Low
/S/Toms/Trinculo High
/S/Toms/Tom1

Which looks good to me. That's enough for the Librarian to show the names, to be able to edit the name on the computer I woud probably need to know how to reassemble the sysex in case I changed text in there. As I don't own a Tempest I can't easily test or reverse engineer, so maybe KoSv can give some hints on what he did?

An experimental Tempest adaptation will be part of the next KnobKraft release.
Check out the free Sysex Librarian for Sequential and more https://github.com/christofmuc/KnobKraft-orm