Memory management: a way to resolve/optimize array memory issues [resolved]

General discussion
1

Hi’, I’m running out of memory while creating the Korg 03R/W preset :unamused:

In my opinion this issue is not limited to that preset but may be encountered for every preset that has an extensive effects section with lots of parameters that need special treatment. In many cases we need each parameter to use different ranges, names , colors depending depending on the type of preset it is used in. Try it out in the effects section of the preset, it makes working with the effect a 1000 times more comprehensive.

I 've made these effects settings work by using arrays per effect type and per attribute to control. In Korg’s case, there are 25 different effect types and I need to set the color scheme, names, min and max values for 7 controls.This implies a total of around 700 array members to keep in memory.

Up until the 20th effect type it all worked, but as of the 21st the E1 runs out of memory.
Can I work around it? Can I make the preset deal with memory more efficiently?

The preset where I use those arrays is to be found here. Electra One App

To give you an idea, the arrays I made are shown below. They are not difficult, but they are extensive and contain a lot of work (incorrect SysEx docs, who’d have guessed…). Do remark I did not make them as local variables out of fear of losing speed, would that improve things? The logic where there are used is concentrated in the function called “setFx1”. Second to last one in the lua code

Any suggestions on making memory usage more efficient?

fx1Control={123,124,125,126,129,130,131,128,122,127}
fx1Param ={160,161,162,163,164,165,166,159}
fxIndirectRef= {1,1,1,1,1,1,1,1,1,2,2,2,3,3,4,5,5,5,6,6,7,7,8,9,10,10,10,11,12,13,13,14,14,15,16,16,17,18,18,19,19,20,21,22,22,23,24}
fxColorRef= {1,1,2,3,1,1,2,2,1,1,1,1,1,1,1,1,1,4,3,3}

fxNames = {{"reverb time","pre delay","earl.refl.time","high damp","eq low","eq high",0,"dry:fx bal."},{"earl.refl.time","pre delay","eq low","eq high",0,0,0,"fx balance"},{"delay left","delay right","feedback","high damp","eq low","eq high",0,"fx balance"},{"delay L","feedback L","high damp L","delay R","feedback R","high damp R",0,"fx balance"},{"delay 1","delay 2","feedback","eq low","eq high",0,0,"fx balance"},{"delay time","mod speed","mod depth","mod  waveform","eq low","eq high",0,"fx balance"},{"delay l","delay r","mod speed","mod depth","mod shape","eq low","eq high","mod speed"},{"delay l","delay r","mod speed","mod depth","fq split point",0,0,"mod speed"},{"mod depth","eq low","eq high",0,0,0,0,"fx balance"},{"delay","mod depth","mod speed","resonance","eq low","eq high",0,"mod speed"},{"blend","emphatic point","eq low","eq high",0,0,0,"fx balance"},{"density","hot spot","stereo width","delay time","eq low","eq high",0,"fx balance"},{"drive (edge)","hot spot","resonance","eq low","eq high","out level",0,"hot spot wah"},{"manual","mod speed","mod depth","feedback","waveform",0,0,"mod speed"},{"vibrato depth","acceleration","slow speed","fast speed",0,0,0,"speed"},{"waveform","mod shape","mod speed","lfo depth","eq low","eq high",0,"dry:fx bal."},{"freq low","eq low","freq mid","eq mid","width mid","freq high","eq high","freq mid"},{"ch. delay","ch.feedback","ch.mod speed","ch.mod depth","delay","feedback",0,"dry:fx bal."},{"delay","feedback","high damp","rev time","pre delay","high damp",0,"dry:fx bal."}}

ctrlMin={{0,0,0,0,-12,-12,0},{0,0,-12,-12,0,0,0},{0,0,-99,0,-12,-12,0},{0,-99,0,0,-99,0,0},{0,0,-99,-12,-12,0,0},{0,0,0,0,-12,-12,0},{0,0,1,0,-21,-12,-12},{0,0,1,0,0,0,0},{0,-12,-12,0,0,0,0},{0,0,1,-99,-12,-12,0},{-99,1,-12,-12,0,0,0},{1,1,0, 1,-12,-12,0},{1,0,0,-12,-12,0,0},{0,0,0,-99,0,0,0},{0,1,1,1,0,0,0},{0, -99, 0,0,-12,-12,0},{0, -12, 0,-12,0,0,-12},{0, -99, 1, 0,0,-99, 0},{0, -99, 0,0,0,0,-12}}

ctrlMax={{99,200,99,99,12,12,99},{70,200,12,12,200,200,200},{500,500,99,99,12,12,200},{500,99,99,500,99,99,200},{500,500,99, 12, 12, 200, 200},{200,216,99, 1, 12, 12, 0},{250,250,99, 99, 20, 12, 12},{500,500,99, 99, 18, 0, 0},{99, 12, 12, 200, 200, 200, 200},{200, 99, 99, 99, 12, 12, 200},{99, 10, 12, 12, 200, 200, 200},{99, 20, 99, 99, 12, 12, 200},{111, 99, 99, 12, 12, 99, 200},{99, 200, 99, 99, 1, 200, 200},{15, 15, 99, 99, 200, 200, 200},{1, 99, 200, 99, 12, 12, 200},{29, 12, 99, 12, 99, 29, 12},{50, 99, 99, 99, 225, 99, 200},{500, 99, 99, 97, 150, 99, 200}}

ctrlColor={{PURPLE,PURPLE,PURPLE,PURPLE,PURPLE,PURPLE,PURPLE},{WHITE,RED,PURPLE,PURPLE,PURPLE,PURPLE,PURPLE},{WHITE,WHITE,WHITE,RED,RED,RED,PURPLE},{PURPLE,PURPLE,PURPLE,PURPLE,GREEN,GREEN,PURPLE}}
2

In general, I will start with an array, but when I see a lot of repetition or simple sequences of numbering, I will rework it to use math instead. Of course there’s nothing to be done about lots of text strings, but even there, you can maybe build the strings up dynamically for common core words/abbreviations.

I assume you are indexing into the arrays in some manner. If, for example, fx1Param[] is using indexes from 1 to 8, you can do less clear/ugly things like:

fx1ParamValue = 160 + ((index % 8) - 1)

and for Control -

if index < 5 then
   fx1ControlValue = 122 + index
else if index < 8
   fx1ControlValue = 124 + index 
else if index == 8 then
   fx1ControlValue = 128
else if index == 9 then
   fx1ControlValue = 122
else if index == 10 then
   fx1ControlValue = 127
end

Again - depending on how big the array is and how complex it is, you may not be able to see a pattern immediately, but possibly rearranging an order or changing how it’s indexed may help. Martin can also answer the question of total space constraints inside the E1 – is there separate areas for code and data, is there more or less space for global data versus locel, etc.

3

Let’s see if I caught your thinking:

In short, when I can avoid an array by converting it in some conditional instruction, that may be ugly from a programming perspective, but good for memory consumption, this would worth trying. correct?

4

yes, that is my thought anyway – typically source code gets converted to a very efficient format for loading/running, so overall memory use might be better.

Isn’t there a dump to the log file that shows memory use, or is that just a static/load time thing?

5

well, you thought right. As per your suggestion I made some ugly if-statements by which I could reduce or eliminate the arrays.
As a result the code has become quite difficult to understand (it is now a combination of arrays and logic), but it solved the memory issue AND the execution is much faster.

I intuitively was expecting the contrary, so many thanks for the help. Wouldn’t have dared doing all the effort without some kind of encouragement :slight_smile:

2 Likes
6

Glad you were able to reclaim some memory and gain speed. The complexity tradeoff is unfortunate, but sometimes the only way.

2 Likes