Roland System-8 owners: an attempt to reverse engineer its SysEx

General discussion
21

could you share your findings? I mean, what adresses control what. thanks in advance. Google docs or whatever works for you.

22

Iā€™ll attach my excel in here when Iā€™m back home.

Thereā€™s a way to derive it from the E1 preset:

  • If an E1 parameter is virtual, its parameter is directly related to an address in the SysEx data. This is used for practically all continuous controls . On a System-7 they range 0..255 so SysEx provide a finer resolution than CC7.
  • If an E1 parameter is CC7 , itā€™s because CC as available Ć nd the controls was not contunuous or with a limited range (like a WaveForm control, or a coarse control). I first using SysEx for them but the System-8 is not 100% reliableinto listening to SysEx, so whenever possible (no losss in resolution) CC was used.
  • The values used in CC are diffrent form the values received is SysEx:

As for SysEx construction. The headers are as follows:

function rolandDevice(valueObject, value) 
  rolandDeviceID = value-1 -- the Roland Device ID as to be used in the sys-8 SysEx commands
  dataTransmitArray ={0x41, rolandDeviceID, 0x00, 0x00, 0x7E,0x12}
  dataRequestArray ={0x41, rolandDeviceID, 0x00, 0x00, 0x7E,0x11}
end

The checksum is calculated here:

function calcChecksum(payLoad)
  local result = 0
  for i= 1, #payLoad do
    result = result+payLoad[i]
  end
  result = (128-result)%128
  return result
end

And here is the code to transmit virtual parameter changes as Roland SysEx parameter changes:

function parameterMap.onChange(valueObjects, origin, midiValue)
  if origin ~= INTERNAL then return end
  for i, valueObject in ipairs(valueObjects) do
    local message = valueObject:getMessage ()
    local parameter = message:getParameterNumber ()
    local parameterType = message:getType ()
    if parameterType == PT_VIRTUAL and parameter < 1000 then -- cutoff
      local MSB = math.floor(midiValue/16)
      local LSB = midiValue%16
      local payLoad = {0x03, 0x00, math.floor((parameter-1)/128), (parameter-1)%128, MSB, LSB}
      local checkSum = {calcChecksum(payLoad)}
      midi.sendSysex (devPort, concat(dataTransmitArray,concat(payLoad,checkSum)))
    end 
  end
end

On receipt of any Sysex, we first check if indeed it comes from the System-8 as a data dump:
We have a help function for that:

local function isSystem8Reply(sysexBlock)
  return sysexBlock:peek(2) == 0x41 and
         sysexBlock:peek(3) == rolandDeviceID and
         sysexBlock:peek(4) == 0x00 and
         sysexBlock:peek(5) == 0x00 and
         sysexBlock:peek(6) == 0x7E and
         sysexBlock:peek(7) == 0x12
end

Another help function is to construct a Roland address based on its 4 bytes:

local function decodeRolandAddress(b0, b1, b2, b3)
  return ((b0-3) << 21) + (b1 << 14) + (b2 << 7) + b3
end

Parsing a received Sysex .
The par-array is used for two purposes:

  • what received parameters must be converted to what CC7 (instead of Virtual) parameter, and which are to be considered as virtual ones.
  • how to convert the received sysex value into its CC7 counterpartner value. The conversion is different according to the amount of options (sysex uses 0,1ā€¦n with n = amount of options, while CC7 spreads them from 0..127), or if the control is a System-8 variation control (in which case in CC7 all options are steps of 16).
  • the check against virtual paramter 1002 is for specific plugouts, that have a different number of options for the same parameter.
function midi.onSysex(midiInput, sysexBlock) 
  local HEADER_SIZE = 11
  local CHECKSUM_SIZE = 2
  -- define parameters for the System-8 in par array
  local par = {} 
  for i = 1,159,2 do par[i]=PT_VIRTUAL end -- sysEx values are the virtual parameter values
  for i = 271,319,2 do par[i]=PT_VIRTUAL end -- sysEx values ARP
  for i = 455,471,2 do par[i]=PT_VIRTUAL end -- sysEx values 
  -- for CC7 : value/1000 = amount of options between 0 and 255, value%1000 = CC number
  -- for Variations: number is between 1001 and 1999
  par[01]=6035 par[13]=2117 par[15]= 6046 par[19]=6060 par[21]=6047 par[025]=6061 par[029]= 6063 par[031]=6062 -- LFO OSC1 OSC2
  par[35]=2111 par[37]=2112 par[39]=23087 par[51]=2114 par[99]=2116 par[101]=2119 par[103]=24041 par[107]=2118 -- OSC2 Mixer General
  par[479]=4049 par[487]=256068 par[495]=256075 par[503]=256080 par[511]=256109 par[519]=  1014 par[527]=  2015 -- Bender LFO
  par[535]=1103 par[543]= 23020 par[551]=256021 par[559]=  1104 par[567]=  6107 par[575]=256025 par[583]=256048 -- OSC1 OSC2 OSC3
  par[591]=1051 par[599]=  6108 par[607]=  6070 par[615]=256071 par[623]=  6072 par[631]=  6073 par[639]=256074 -- Filter Effects
  if parameterMap.get (deviceId, PT_VIRTUAL, 1002) == 1 then par[39]=73087 end -- osc2 coarse for jupiter-8
  if parameterMap.get (deviceId, PT_VIRTUAL, 1002) == 3 then par[37]=3112 end -- x-mod for jx-3P
  local payloadLength = sysexBlock:getLength() - HEADER_SIZE - CHECKSUM_SIZE
  if not isSystem8Reply(sysexBlock) then return end -- System-8 data reply only
  local startAddress =  decodeRolandAddress(sysexBlock:peek(8),sysexBlock:peek(9),sysexBlock:peek(10),sysexBlock:peek(11))
  for i  = 2, payloadLength, 2 do  --
    local address = startAddress+i-1
    local highNibble = sysexBlock:peek(i+10)
    local lowNibble = sysexBlock:peek(i+11)    
    local value = highNibble*16 + lowNibble  
    if par[address] == PT_VIRTUAL then 
      parameterMap.set (deviceId, PT_VIRTUAL, address,value)
    elseif par[address] then 
      local step = 16 -- for variation controls
      local valueCC7 =value*step 
      if math.floor(par[address]/1000) ~= 1 then 
        step =127/ (math.floor(par[address]/1000)-1) -- CC7 ranging 0..127
        valueCC7 = math.floor(value*step + 0.5) -- correct rounding
      end 
      parameterMap.set (deviceId, PT_CC7, par[address]%1000,valueCC7)
    end 
  end 
end

Last but not least; the patch dump request is :

    local payLoad = {0x03, 0x00,0x00,0x00,0x00,0x00, 0x04, 0x7F} 
    -- first 4 bytes = start address
    -- next 4 bytes : length in bytes
    local checkSum = {calcChecksum(payLoad)}
    midi.sendSysex (devPort, concat(dataRequestArray,concat(payLoad,checkSum)))
1 Like
23

I noticed something really strange: On the Sys-8 , Roland uses specially crafted data dump, to which the destination replies witj a data dump as well. So the 12x00 is no longer for providing data from a specific memory location, but now is being used for a kind of structured chat between 2 devices. I havenā€™t reverse engineered those yet. They deem to deal a.o. with knowing what plugout (and version) is available in which slot.

For now Iā€™ve only found useable data in the hex03-hex04 range but there might be more in the hex02-hex03 range..

1 Like
24

System 8 HEX sysex addresses.zip (21.4 KB)
As promised @ambivalence here are my notes on what addresses in SysEx correspond to what system-8 parameters and whether they have a CC7 counterpartner.
If you find inaccuracies, do let me know.