# Not enough memory to get patch names?

**URL:** <https://forum.electra.one/t/not-enough-memory-to-get-patch-names/1988>\
**Category:** Developers\
**Tags:** lua\
**Created:** [April 2, 2023, 4:03pm UTC](https://forum.electra.one/t/not-enough-memory-to-get-patch-names/1988 "2023-04-02T16:03:49Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![urbanspaceman](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.electra.one/urbanspaceman/32/206_2.png) [@urbanspaceman](https://forum.electra.one/u/urbanspaceman)\
**Post date:** [April 2, 2023, 4:03pm UTC](https://forum.electra.one/t/not-enough-memory-to-get-patch-names/1988/1 "2023-04-02T16:03:49Z")

</div>

I’ve been exploring if I can get all the patch names from my synth on to the E1, but am having some problems working in Lua and running out of memory. Perhaps there’s a better way to do this. Here’s the code I’m using:

```auto
function dump(o)
   if type(o) == 'table' then
      local s = ''
      for k,v in pairs(o) do
         if type(k) ~= 'number' then k = '"'..k..'"' end
         s = s .. dump(v) .. ' '
      end
      return s
   else
      return tostring(o)
   end
end

function midi.onSysex (midiInput, sysexBlock)
    print ("Received sysex, length = " .. sysexBlock:getLength ())
    local theBlock = {}
    for i = 1, sysexBlock:getLength () do
        local sysexByte = string.format ("%02x", sysexBlock:peek(i))
        table.insert(theBlock, string.upper(sysexByte))
    end
    print (dump(theBlock))
end

```

There’s a dump request message (which I’m currently triggering with a pad in the preset) that asks for the entire user bank of 32 voices that returns response that’s supposed to be 4104 bytes long (according to the manual). The E1 doesn’t print out anything. So I tried another request that’s supposed to be 2450 bytes long, and I get the response…

```auto
lua: Received sysex, length = 12723.0
error running function 'onSysex': not enough memory

```

…which is over 5 times the expected size. So I expect the other one is just too big. Is there another way?

---

<div class="post-metadata">

**Author:** ![martin](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.electra.one/martin/32/145_2.png) [@martin](https://forum.electra.one/u/martin)\
**Post date:** [April 2, 2023, 4:31pm UTC](https://forum.electra.one/t/not-enough-memory-to-get-patch-names/1988/2 "2023-04-02T16:31:19Z")

</div>

yeah, this way of printing the sysex data is very memory hungry 🙂

with 12.7 kB long sysex message, `theBlock` will grow to at least 25kB, string `s` to around 40kB, with each concat (…) on `s` requiring minimum size of `s` \* 2. concat creates new strings.

The important thing is that sysexBlocks live outside of the regular RAM. `seek`, `peek`, `read` should be used to parse the data out.

What is strange is that you are getting longer size of the received message. I just tried to send a few larger sysex messages to your script (with `print (dump(theBlock))` commented out) and I always got what I sent in.

---

<div class="post-metadata">

**Author:** ![urbanspaceman](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.electra.one/urbanspaceman/32/206_2.png) [@urbanspaceman](https://forum.electra.one/u/urbanspaceman)\
**Post date:** [April 2, 2023, 4:42pm UTC](https://forum.electra.one/t/not-enough-memory-to-get-patch-names/1988/3 "2023-04-02T16:42:17Z")

</div>

I’ve put a ‘print’ line in the loop, and with the 2450 size SysEx dump I get a bunch of zeros before the ‘real’ response starting with F0 comes through. This might partially explain why I get a different size every time and the array gets full. The only other thing I can think of is to request each patch one by one to get the patch names, but I’m unsure how (or if) that can be done.

---

<div class="post-metadata">

**Author:** ![martin](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.electra.one/martin/32/145_2.png) [@martin](https://forum.electra.one/u/martin)\
**Post date:** [April 2, 2023, 4:51pm UTC](https://forum.electra.one/t/not-enough-memory-to-get-patch-names/1988/4 "2023-04-02T16:51:29Z")

</div>

Do you get those extra leading zeros with the 4k SysEx dump too? 4104 bytes sounds like tx7/dx7 bank dump. Is that correct? I work with these dumps often and it works just fine.

---

<div class="post-metadata">

**Author:** ![urbanspaceman](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.electra.one/urbanspaceman/32/206_2.png) [@urbanspaceman](https://forum.electra.one/u/urbanspaceman)\
**Post date:** [April 2, 2023, 4:55pm UTC](https://forum.electra.one/t/not-enough-memory-to-get-patch-names/1988/5 "2023-04-02T16:55:52Z")

</div>

That’s right, it’s the TX81Z. It’s the largest bulk dump response, and it’s the only one with nothing coming back, not even the `sysexBlock:getLength ()`. I’m sure the request from the E1 is right, because the response is getting to my laptop when triggered from the E1. Is it taking too long, perhaps?

---

<div class="post-metadata">

**Author:** ![urbanspaceman](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.electra.one/urbanspaceman/32/206_2.png) [@urbanspaceman](https://forum.electra.one/u/urbanspaceman)\
**Post date:** [April 2, 2023, 4:59pm UTC](https://forum.electra.one/t/not-enough-memory-to-get-patch-names/1988/6 "2023-04-02T16:59:12Z")

</div>

I’ve been trying more times, and the requests called VMEM (the user bank) and PMEM (performance bank) seem to get cut off. Either getting nothing or only some of the bytes in `sysexBlock:getLength()`

---

<div class="post-metadata">

**Author:** ![martin](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.electra.one/martin/32/145_2.png) [@martin](https://forum.electra.one/u/martin)\
**Post date:** [April 2, 2023, 5:11pm UTC](https://forum.electra.one/t/not-enough-memory-to-get-patch-names/1988/7 "2023-04-02T17:11:05Z")

</div>

yeah, if there are leading zeros but the data is in the `sysexBlock`, it means that had to be `F0` before the zeros.

Adjusted your script a little bit:

```auto
function dump(o)
   if type(o) == 'table' then
      local s = ''
      for k,v in ipairs(o) do
         if type(k) ~= 'number' then k = '"'..k..'"' end
         s = s .. dump(v) .. ' '
         if (math.fmod(k, 32) == 0) then
            print(s)
            s = ''
         end
      end
      return s
   else
      return tostring(o)
   end
end

function midi.onSysex (midiInput, sysexBlock)
    print ("Received sysex, length = " .. sysexBlock:getLength ())
    local theBlock = {}
    for i = 1, sysexBlock:getLength () do
        local sysexByte = string.format ("%02x", sysexBlock:peek(i))
        table.insert(theBlock, string.upper(sysexByte))
    end
    print (dump(theBlock))
end

```

and send the 4104 dump to E1. This is what I get:

 ![image](https://europe1.discourse-cdn.com/flex017/uploads/electraone/original/2X/2/287d7521a369e8c91842517c4bf98894949af6fe.png)

and the tail of the data:

 ![image](https://europe1.discourse-cdn.com/flex017/uploads/electraone/original/2X/6/61367c5f1e0eaa913b535661e6cff420e916c097.png)

edit: note, intention of the change of your script was just to print the table data, not to return correct string…

---

<div class="post-metadata">

**Author:** ![urbanspaceman](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.electra.one/urbanspaceman/32/206_2.png) [@urbanspaceman](https://forum.electra.one/u/urbanspaceman)\
**Post date:** [April 2, 2023, 5:29pm UTC](https://forum.electra.one/t/not-enough-memory-to-get-patch-names/1988/8 "2023-04-02T17:29:29Z")

</div>

I’m getting some weirdness. On the right is what I get on the laptop, on the left is what is going back to the E1, both triggered from the E1.

 ![VMEM-not-working-on-E1](https://europe1.discourse-cdn.com/flex017/uploads/electraone/original/2X/a/a6a01457dcddbac072f47bf94f6ae755a36865a7.jpeg)

---

<div class="post-metadata">

**Author:** ![urbanspaceman](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.electra.one/urbanspaceman/32/206_2.png) [@urbanspaceman](https://forum.electra.one/u/urbanspaceman)\
**Post date:** [April 2, 2023, 5:32pm UTC](https://forum.electra.one/t/not-enough-memory-to-get-patch-names/1988/9 "2023-04-02T17:32:14Z")

</div>

And I get a different result every time. Sometimes fewer than 4104 bytes, sometimes more, and a different amount ‘zeroed’ out. Sometimes I get the out of memory error, sometimes nothing.

---

<div class="post-metadata">

**Author:** ![martin](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.electra.one/martin/32/145_2.png) [@martin](https://forum.electra.one/u/martin)\
**Post date:** [April 2, 2023, 5:35pm UTC](https://forum.electra.one/t/not-enough-memory-to-get-patch-names/1988/10 "2023-04-02T17:35:04Z")

</div>

Would it be possible to send me a link to the preset and a file with the SysEx data? on PM or here. Then I could run the same thing on my E1.

---

<div class="post-metadata">

**Author:** ![urbanspaceman](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.electra.one/urbanspaceman/32/206_2.png) [@urbanspaceman](https://forum.electra.one/u/urbanspaceman)\
**Post date:** [April 3, 2023, 2:23pm UTC](https://forum.electra.one/t/not-enough-memory-to-get-patch-names/1988/11 "2023-04-03T14:23:53Z")

</div>

Haven’t had any luck getting past the problem, but I’ve written a script that extracts the patch name. It works fine for the current voice, but the E1 seems to choke on the 4K size 32 voice user bank. As before, when the 4K response comes back to the device, it often won’t even run the print command. This time when I run other SysEx requests I can see that the first patch name in the user bank has been saved, but that’s it.

```auto
function dump(o)
   if type(o) == 'table' then
      local s = ''
      for k,v in ipairs(o) do
         if type(k) ~= 'number' then k = '"'..k..'"' end
         s = s .. dump(v) .. ' '
         if (math.fmod(k, 32) == 0) then
            print(s)
            s = ''
         end
      end
      return s
   else
      return tostring(o)
   end
end

-- ------------------------------------------------------

list_midiChan = controls.get (20)
patchNames = {}

function midi.onSysex (midiInput, sysexBlock)
    print ("Received sysex message " .. sysexBlock:getLength() .. " bytes long.")
    local byte = ""
    local bytes = ""
    local voice = 1
    local pointer = 1
    local chan = string.format("%02x", 0x00 + ( list_midiChan:getValue("value"):getMessage():getValue() ))
    local VMEM_header = "f0 43 " .. chan .. " 04 20 00 " -- TX81Z User Bank header
    local VCED_header = "f0 43 " .. chan .. " 03 00 5d " -- TX81Z Voice Edit header
    repeat
        byte = string.format("%02x ", sysexBlock:read()) -- read byte & convert to hex
        bytes = bytes .. byte
        pointer = pointer + 1
    until pointer > 6 -- capture header (6 bytes)
    if bytes == VMEM_header then
        for i = 1, 32 do -- loop through 32 patches in the user bank
            pointer = pointer + 57 -- skip 57 bytes (patch info before name)
            sysexBlock:seek (pointer)
            bytes = ""
            local letter = 1
            repeat
                byte = string.char(sysexBlock:read()) -- read byte & convert to ascii
                bytes = bytes .. byte
                letter = letter + 1
            until letter > 10 -- capture 10 bytes (the patch name)
            patchNames[i] = bytes
            pointer = pointer + 61 -- skip 61 bytes (patch info after name)
        end
    elseif bytes == VCED_header then
        pointer = pointer + 77 -- skip 77 bytes (patch info before name)
        sysexBlock:seek (pointer)
        bytes = ""
        local letter = 1
        repeat
            byte = string.char(sysexBlock:read()) -- read byte & convert to ascii
            bytes = bytes .. byte
            letter = letter + 1
        until letter > 10 -- capture 10 bytes (the patch name)
        patchNames[1] = bytes
    end
    print ( dump(patchNames) )
end

```

---

<div class="post-metadata">

**Author:** ![martin](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.electra.one/martin/32/145_2.png) [@martin](https://forum.electra.one/u/martin)\
**Post date:** [April 3, 2023, 6:44pm UTC](https://forum.electra.one/t/not-enough-memory-to-get-patch-names/1988/12 "2023-04-03T18:44:12Z")

</div>

I was looking at it. What I found is that MIDI IO (low level serial driver) buffer gets exhausted for messages larger than 500bytes. It is a bit strange as the same messages are processed on USB interfaces without problems and at much higher transfer speeds. I put the problem on top of my work stack.

---

<div class="post-metadata">

**Author:** ![urbanspaceman](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.electra.one/urbanspaceman/32/206_2.png) [@urbanspaceman](https://forum.electra.one/u/urbanspaceman)\
**Post date:** [April 3, 2023, 8:37pm UTC](https://forum.electra.one/t/not-enough-memory-to-get-patch-names/1988/13 "2023-04-03T20:37:55Z")

</div>

Thanks for looking into it. Hopefully it can be fixed with software!

---

<div class="post-metadata">

**Author:** ![akira](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.electra.one/akira/32/799_2.png) [@akira](https://forum.electra.one/u/akira)\
**Post date:** [April 5, 2023, 8:08pm UTC](https://forum.electra.one/t/not-enough-memory-to-get-patch-names/1988/14 "2023-04-05T20:08:34Z")

</div>

Lua is a garbage-collected language, so remember to periodically invoke its [`collectgarbage`](https://www.lua.org/manual/5.3/manual.html#pdf-collectgarbage) function, especially when working with loops. I don’t know if the firmware does it for you automatically or how often, but I’ve empirically found it can help squeeze a few more things before I get out-of-memory errors.

---

<div class="post-metadata">

**Author:** ![urbanspaceman](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.electra.one/urbanspaceman/32/206_2.png) [@urbanspaceman](https://forum.electra.one/u/urbanspaceman)\
**Post date:** [April 6, 2023, 9:02am UTC](https://forum.electra.one/t/not-enough-memory-to-get-patch-names/1988/15 "2023-04-06T09:02:08Z")

</div>

Thanks, this is a new concept to me. Where do you think the best place to put it would be in the code I posted above? At the end of midi.onSysex?

---

<div class="post-metadata">

**Author:** ![martin](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.electra.one/martin/32/145_2.png) [@martin](https://forum.electra.one/u/martin)\
**Post date:** [April 6, 2023, 9:17am UTC](https://forum.electra.one/t/not-enough-memory-to-get-patch-names/1988/16 "2023-04-06T09:17:15Z")

</div>

E1 calls garbage collector on regular basis. That takes care of anything left over from the callbacks, formatters, timers, and such. It cannot, however, fire the GC in the middle of someones Lua function. There, it is responsibility of the user to make the judgement if it is needed.

PS: my work on resolving that sysex issue (see above) is interrupted now, I have had to switch to something a bit urgent and related to MK2.

---

<div class="post-metadata">

**Author:** ![urbanspaceman](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.electra.one/urbanspaceman/32/206_2.png) [@urbanspaceman](https://forum.electra.one/u/urbanspaceman)\
**Post date:** [April 7, 2023, 4:45pm UTC](https://forum.electra.one/t/not-enough-memory-to-get-patch-names/1988/17 "2023-04-07T16:45:21Z")

</div>

I tried writing a script that would request all 32 patches one by one and collect the patch names, but I just can’t get the loop that uses `midi.sendSysex` to work with the `midi.onSysex` function. `midi.onSysex` doesn’t seem to listen until the loop is finished, so I only get the last patch name. Seems like a no go with that idea.