Recently, I spend some time parsing the undocumented Sysex patch dump of a nord drum 3p and thought I’d chime in here to share a real-world example.
As mentioned in this thread before, there are various ways how different manufacturers are storing data in their Sysex dump. The following steps are meant to give an idea how to approach deciphering an unknown patch dump. The concrete parsing might be different depending on the device.
Most values that we are collecting from a Sysex patch dump are numbers. So before diving into the details of the parsing process, I believe it is a good idea to recap how numbers are composed in a binary bit stream.
When a number is transmitted in more than one byte, you can use Electra’s parameterBitPosition to put the bits back together. When parsing unknown data, it is very helpful to understand how this calculation works.
Let’s start with a simple example. Assumed we have an 8-bit parameter that is transmitted in the first four bits of two bytes %0000yyyy %0000yyyy. Given, we are receiving a Hex value 03 in the first byte and 0A in the second byte, which is 3 in decimal and 111 in binary, or 10 and 1010 respectively. The result of putting the binary back together is really just putting them back together. So 111 and 1010 will be 1111010. But we can also calculate the decimal value from the details we have so far.
To put a parameter part to its corresponding bit position, we just need to multiply it by 2 power ‘length of first parameter part’ and add the multiplied result to the first value. In our example, the first parameter part has a length of 4. 2 power 4 equals 16.
So we can calculate the result like this: 3 * 16 + 10 = 58
Sometimes we also see other types of numbers, like signed, two’s complement or even fractions.
To put a (simple) float back together, we first follow the same steps mentioned above. Then we divide the result by 2 power ‘position of decimal point’.
The nord drum is storing its Delay Rate parameter as float in 16-bits, distributed across 3 bytes. The lowest value as complete binary looks like this 1010101010101001. That is 43689 as decimal. The decimal point position is 15 (from the right). 2 power 15 equals 32768.
Which means we can calculate the float like this : 43689 / 32768 = 1.333 (The correct binary notation would be 1.010101010101001)
Besides a power of 2 table as reference, personally I also find it helpful to have a list at hand to quickly look up the highest number that can be stored in a certain bit length:
2-bit: 3
3-bit: 7
4-bit: 15
5-bit: 31
6-bit: 63
7-bit: 127
Now, let’s dive into the actual parsing workflow.
For the most part, I’m using snoize’s MIDI monitor and the text editor BBEdit, but there are several other tools which get the job done as well.
Note: the following steps only work when the patch dump sends out the current parameter position. Otherwise, it might be necessary to save the patch on the device between sending the patch dumps.
-
Make sure MIDI monitor is running and listening to incoming MIDI messages
-
Turn a first parameter on the MIDI device to its minimal position
-
Trigger a patch dump
-
Turn the same parameter to its maximum position
-
Trigger a second patch dump
-
Create two empty documents in BBedit
-
In MIDI monitor, double click the first patch response message and copy its content into the first created BBEdit document
-
Copy the second patch dump message into the second BBEdit document
-
In the first BBEdit document press [cmd] + [F] to open the Find window
-
Put a space character into the Find field and a line return \n into the Replace field. Click Replace All. → This will convert all space characters between the Hex values into line returns, meaning that the line number is now equal to the bytes offset
-
Repeat the same process in BBEdit’s second document
-
In BBEdit’s sidebar, select both documents and right-click to open the context menu. Select Compare from the context menu.
-
In the resulting window, you should see the Hex values (bytes) highlighted that are different in the patch dump messages.
-
If the data type is unknown, I would continue with converting these highlighted Hex values into binary and decimal and check if that already makes sense. To do so, I’m using the following online calculator
-
When looking at the binary, I’m mostly interested in detecting the boundaries of the parameter. So I’m looking for the highest and lowest bit that are changing. If the parameter is distributed across more than one byte, I would also try putting the binary parts together in the aforementioned calculator and see if the result corresponds with the number of parameter values.
Example:
Many parameters on the nord drum range from 0 to 50. And most of them are distributed across two bytes.
Here is the binary of a parameter at its minimal and maximum position:
00: 0000 0010110
50: 1100 1010110
From the bit length list above, we know that a parameter ranging from 0 to 50 needs at least 6 bits.
In the example, we can see that the first five bits of the second byte (reading from right to right) did not change. The length of the remaining bits is 6. If we remove the unchanged bits and put only these remaining 6 bits of the maximum parameter position again into the online calculator, we will be happy to see that 110010 is the binary representation of 50.
The only missing piece of the puzzle is the byte number required in Electra’s JSON file. All we need to calculate that is the line number of the changed bytes in the BBEdit document and the length of the Sysex response header, defined in the JSON file.
The Sysex response header for the nord drum 3p looks like this:
{
"responses": [
{
"header": [
"33",
"7F",
"1A",
"08",
"05",
"06",
"00"
],
So it has a length of 7 bytes. To complete our calculation, we need to add 2 to the length of the response header. 1 for the unlisted start byte F0 and another 1 because Electra’s byte index is 0 based, but our line numbers in BBEdit start at 1.
Now, we can calculate the byte number by subtracting 9 (the header length + 2 for the start byte and 0 index) from the line number in BBEdit.
With that information, I would move to Electra’s JSON and add it like this:
{
"type": "sysex",
"parameterNumber": 1,
"parameterBitPosition": 2,
"byte": 41,
"byteBitPosition": 0,
"bitWidth": 4
},
{
"type": "sysex",
"parameterNumber": 1,
"parameterBitPosition": 0,
"byte": 42,
"byteBitPosition": 5,
"bitWidth": 2
}
Next, I would load the updated preset into Electra and send the same patch dump messages with the first parameter being at its minimum and maximum position and check if the control is responding correctly.
This is just scratching at the surface, but maybe it helps a bit when starting to parse a Sysex patch dump.
Happy New Year!
