Wednesday 26 June 2019

Storing text on Dropbox...

Sometimes amazing stuff crops up by accident. I use Dropbox, the cloud-based online storage service, to store files that I want to share between my computers. mobile, etc. Dropbox has served me well for many years, and I have rarely needed to contact their customer support.

However, a chance happening alerted me to something interesting that I hadn't considered before. Someone sent me a 'download-only' link to their Dropbox so that I could get some files, but one of those files was just a placeholder for a file that they would upload later to complete the set. All perfectly ordinary stuff that people use to do business every day.

But that placeholder was interesting. I accidentally tried to download it along with all of the other files. But it wouldn't download and I got an alert warning me that it was 'zero bytes' long. It was at this point that I got interested, and I generated some test files to learn exactly how Dropbox stored them. It seems that if you store a zero length file, you can give it a short title (a few words), and it will occupy zero bytes of your Dropbox storage quota. Now zero length files are easy to produce - I just used an ASCII editor (I did a whole blog post on the topic from a couple of months ago...) and named and saved the empty file direct to Dropbox. After a bit more experimentation, here's the resulting directory listing from my Dropbox 'Files' page (slightly edited):


As you can see, I wasn't particularly inventive in my choice of words, but just in case, I contacted Dropbox customer support to let them know that it is possible to store text on Dropbox without affecting your storage quota, and they agreed with my analysis. They also pointed out that you could also use folder names...

Now I tend not to use the browser interface to Dropbox very much, but the directory page is interesting. It gives you a timed list of your activity on Dropbox, so for my recent activity, it showed the lines from Shakespeare in order, scrolling vertically as each new line was added... 

Now, this was very cool! Dropbox effectively gives you, for free, a performance tool that lets you publish short lines of text on Dropbox, where those lines of text from the file titles are displayed on a web page, in order, and timestamped. So as each empty file is added to Dropbox from a browser or just dropped into the local Dropbox folder on your computer, the Shakespearean soliloquy gets displayed one line at a time, using the text from the title of the file, with previous lines scrolling upwards or downwards (you can control this from the directory page)... No need to refresh the page - it just works.

The only catches are: 
- the title has to be short enough (you can see that some of those lines are getting close to being truncated), and 
- you can't put any punctuation in the text, except for symbols that would be allowed in file names anyway...

Well, I didn't think just virtual thespians at this point, I also thought about song lyrics. It seems that you could potentially store song lyrics (assuming you have the rights to do so, of course), one line at a time, on Dropbox, with the right timing, and Dropbox will scroll them on the display as they are received. And this costs you nothing extra on top of whatever you normally pay for Dropbox. The files are empty, so they don't add to your storage quota! Making an app that does timed saves of empty files with titles from the individual lines in a text file is not that difficult, and Dropbox does everything else... 

I'm now wondering if there is anything else that can be done with free text storage... Subtitles, commentaries, live comments, etc. There's a lot of things o consider before designing a solution based on what Dropbox provide: the latency and jitter of update times, for example, may mean that lyrics or subtitles aren't a good application. But the interesting thing is that there's going to be at least one application that is a perfect fit for this...

I asked Dropbox Customer Support if it was okay to publish this, and they said it was fine. So now you know too! I know it's only minor and trivial really, but for me, it is fascinating to discover that something like this is possible. My analysis follows below for those who are interested in system design...

(I'm just wondering what Charlie Brooker of 'Black Mirror'-fame would make of this...)

Analysis

Here's some serious analysis of this as a system design problem. 

Whenever you design a system, there are the things you want it to do, and a security analysis will give you pointers towards things that you don't want it to do. You can do risk analysis on those unwanted features, and decide on appropriate mitigations to reduce the residual risk to acceptable levels. But there's a hole in this very conventional process, and that is the things that you didn't specify that also are not security risks. These are the unexpected features, and they can be very interesting.

Risk analysis for security has evolved over a long time, and it has a sophisticated set of processes, approaches, tools and practitioners that are all based on lots of experience of looking at systems from a very specific viewpoint - security. These days, an additional and allied parallel activity has become important because of legislation like GDPR, and that is Privacy. Once again, there are processes, etc., and practitioners who are skilled at looking at systems from that viewpoint, and again the analysis leads to risks which result in mitigations to minimise the residual risk, etc. 

But there's that interesting hole where the 'things that you didn't specify the system to do, but it does them anyway, and they don't have any security or privacy implications'. These things fly 'under the radar' and I've never seen very much in the way of any formalisation of process or approach to identifying them, assessing them, and deciding if mitigations are appropriate. Unintended consequences are probably acceptable to some level with simple, non-networked systems that aren't critical in an way - for life-support, emergency cover, critical infrastructure, etc. But as systems become more interconnected, more networked, more 'Cloud'-based, then that little word 'unintended' starts to become more significant. 

In the case outlined above, a design would look at what can happen, and would assess it based on the consequences. Having folder names and file titles outside of the storage quota may seem like a pragmatic solution to a user wanting to know how much storage they are using, but actually, the user's viewpoint of the storage differs from the actual storage that a provider like Dropbox has to provide, because the filing system itself uses storage, and users would probably not want to pay for that storage, even when it turns out that they could actually be using it themselves to store information for free.

What is more concerning are the repercussions of the unintended (or maybe, 'assessed as insignificant') features of a system when that system becomes stressed in some way. My completely uninformed suspicion is that the designers of the Dropbox system probably assumed that users would store files in folders, and that the titles of the files and folders would be insignificant in size in comparison to the actual content of the files themselves. But all it takes is someone to make an app that uses Dropbox as a filing system for song lyrics, subtitles, commentaries, etc., and things might start to escalate. 

The first protection mechanism that would probably be hit would be a limit on the number of folders or files within folders. But if I had designed this, then I would be expecting that this would be linked to a lot of storage as well. The 'user story' for lots of files of zero length isn't something that I would have thought of, and so is an 'unexpected' feature that I would have probably missed in my design. And if a designer misses a feature, then is it tested thoroughly? I'm pretty sure that the consumed storage is a key parameter used to monitor a user's account - I know this because Dropbox is very good at telling me how much storage I use, and especially when I'm getting close to using it all up. Selling me additional storage is good for me and good for Dropbox. But if lots of users were to start using a filing system that exploited the folder and file name 'free' storage feature, then the number of files and folders being used might start to become an important parameter for Dropbox, because suddenly the design rules that said that the storage required for that purpose was insignificant in comparison to the actual chargeable storage space consumed by files uploaded by users, would be wrong...

Changes in design rules when you have a system launched and operational can be awkward, and they can even be potentially expensive, or maybe catastrophic. One of the reasons that I contacted Dropbox when I found out about the zero length file storage was because I was curious about their  response. Dropbox were very open about the storage, and agreed that I could publish this blog on what I had found. It will be very interesting to see what happens next... 

Now if I was Dropbox, then I would be assessing exactly what the consequences might be if a lot of apps started loading the Dropbox system with file and folder title metadata, and those design rules would probably be revisited. Additional checks and catches might be put in place to check for lots of zero length files, and limits for the number of zero length files could be announced and policed. Processes for designing solutions might be revisited, and new checks and balances added to try and catch unintended consequences in future designs.

But actually, there's something much more interesting that Dropbox could do, in parallel to this. They now have a lead in the 'analysis of unintended consequences of plain ordinary features in large systems', and by the time they have done all of the analysis and fixes, they will be world experts in what you need to do to mitigate against this type of potentially lurking problem. This sort of 'hard-earned' practical and theoretical expertise from actually solving a real-world problem is worth a LOT of money, and the next unintended consequence in someone else's system might be something much more damaging and dangerous, particularly if it isn't something that security or privacy risk analysis would have caught (and by its nature, it very probably will be!). At this point, that Dropbox expertise could well be one of their most precious commodities...

I have to thank Dropbox customer support for their help in this. They were wonderful.

All free!

All of the analysis here on this blog is free, of course, (I am a CISSP in the real-world until October 2019, but this is pro bono because it is fascinating...) but donations are always welcome!


  


Thursday 13 June 2019

Documentation for the 'Probably' series of M4L devices - Illustrated!

The recent release of ProbablyR 0v14 reminded me that the documentation needs writing at some stage... So here's a promoted part of the blog post for that, but as a separate blog post to make it easier to find, and with added graphics to give it extra 'eye-candy'-ness. Not quite a square coffee-table book full of nice glossy photos, but you never know...

Documentation for Probably

The MIDIprobablyR(S,Z...) series is getting to the point where it needs a proper manual, but this may take some time, so, in the meantime, here are links to all of the blog posts covering the various versions of the Probably step sequencer. (and related devices) 

The Probably... step sequencer


Probably was the first (and simplest) of the 'Probably' series of step sequencers. This blog post is a good place to start.

Probably blog post                  The first in the series (four grids, limited panel colouring)


ProbablyZ blog post               The second iteration (added the time-warping 'Order grid/panel)


ProbablyS blog post               The third iteration (added the sub-sequencer 'Memory' grid/panel)


ProbablyR 0v11 blog post      The fourth iteration (added 16 step sub-seq, warp fills, and Step length)


ProbablyR 0v14 blog post      The fifth iteration (added 'Swing' panel and 'Nudge' buttons)

Latest ProbablyR on maxforlive.com




Somewhere along the line, the breathlessly-annotated 'white and light purple' advert style got stuck in my mind, and I have used it ever since. Love it or hate it, it is immediately recognisable...

Other 'Probably...' devices

After Probably, there were some other devices that added extra letters to the end. The 'step sequencer' series adds a single letter: S, Z, R - the other devices add more than one letter. The release sequence for 'Probably the step sequencer'  is: Probably, then ProbablyZ, then ProbablyS, then ProbablyR (current).


ProbablyLFO


ProbablyLFO blog post           Outputs probabilistic (variable) waveforms  


ProbablyLFO blog post2         Extra sync additions and time warping               On maxforlive.com

Most people glance at the pixelated sine wave in the 0v03 screen-shot and say something like: 'Yuk - a pixellated sine-wave!' and ignore it. What they miss is an LFO that allows you to make arbitrary waveforms that change shape probabilistically and over time (the noise-warping is really cool, even if I say so myself!). Also, the pixelation can be smoothed out, and actually, having a little bit of it sounds good! - if you were raised in the 'S&H filter modulation era' then it is like a more sophisticated version of that. First impressions are not always reliable, and I've not even mentioned really unusual features like the 'Note' mode, where the LFO waveform is 'triggered' by notes in a clip, which one user described as 'genious'!



ProbablyGEN


ProbablyGEN blog post          A poly-rhythmic drum sequencer, plus...  

     
ProbablyGEN blog post2        Extra asynchronous additions...                           On maxforlive.com

ProbablyGEN is pure timing madness. Each of the three horizontal rows is independently timed (synced or free of Live's transport) and so you can do truly free-form poly-rhythms. This is the sort of thing that really ought to be a module for a modular synth...



ProbablyCHORD


ProbablyCHORD blog post     Probabilistic chord generator ('Sergeant...')        On maxforlive.com

ProbablyCHORD is interesting because it brings together several idea into one place. The TR2gen to-step chord generator puts two chords in sequence and provides inversions, and then lets you automate and add probability to it. So it adds polyphony to the monophonic step sequencing from ProbablyR (and in fact, the variable 'step length' pop-up menu was tested here first, then added to ProbablyR 0v11). But it is also shown to its best advantage when paired with an 'articulated' sound using velocity, and you can learn more about this in two ModeAudio interviews/tutorials.


There is more on Ableton instrument racks here:

- Macro controls in Instrument Racks In Ableton Live

And an example of a more sophisticated multi-layered articulated sound (implemented in an Instrument Rack in Ableton Live) is here:


In the pipeline is a blog post on additional articulations that you can do inside Instrument Racks, which may also have other spin-offs...


ProbablyCHORD blog post2 - coming soon!


There are updates and more devices in the pipeline, of course. Watch this blog!


All free!

Everything here is free, of course, but donations are always welcome!







Tuesday 11 June 2019

Generative non-linear step sequencer - new 'Swing' panel, and more!

This was going to be a quick post to describe a new panel for my 'ProbablyR' generative, probabilistic sequencer, but it seems to have grown into a longer tome because version 14 has made rapid progress... So this post will cover:

1. The new 'Swing' panel
2. The new 'time manipulation' facilities in the 'Order' panel
3. The new 'Nudge' buttons in most of the panels

As always with a new release of ProbablyR, don't forget to send it a MIDI note (from the keyboard emulation on a laptop, or from MIDI In), click on the 'D' default button to get a clean setup, and then save the settings using the little 'floppy disk' icon in the top right hand corner so that you get a saved .adv file in your library. Then explore the new stuff!

Oh, and I suppose that I should use my new naming scheme, so this updated device is really called: 'MIDIprobablyR 0v014'.


The Swing panel

The 'Swing' panel adds the ability to set the length of each step in the sequence in 64th note intervals. Note that ProbablyR already lets you set the length of the generated note at each step in the sequence,  so the 'Swing' allows you to control the time between steps, which is somewhat unusual in step sequencers.

I'm using the term 'swing' here because that seems to be the most commonly used term in the DAW industry, and I'm very aware that 'Swing', 'Shuffle', 'Groove', and 'Nudge' are all terms that can be used for similar parameters. Now, having been using the amazingly excellent Novation Circuit for the last couple of months, I have to say that 'nudge' was my first choice, but that was already taken for a completely different new feature. (Of course, these days, maybe what I should be doing, is putting a picture up on Facebook in a DAW group that asks: ''Swing', 'Shuffle', 'Groove', or 'Nudge': Which is the right word?' and waiting for comments...)


The 'Swing' panel is the slightly purply, dark blue one (not the re-coloured teal one for 'Note Length' that used to be 'the dark blue one') on the far right of ProbablyR, and uses the same probabilistic grid that I have been using for all of the major control functionality in the Probably series. (Time usually goes horizontally, each white cell vertically is selected at random.) So, yes, the horizontal axis is time: 18 steps maximum in this version (the UI is holding me up for more steps...), whilst the vertical axis sets the number of 64th notes between each step, with fastest (1/64th) at the top of the panel, and the slowest (64/64ths) at the bottom of the panel.

Now, following my usual trend of not doing things in the conventional way, the 'swing' control presented here is rather different to the more usual percentage value from 50% (Straight or 'even' timing, where, for example, every 16th note is the same time after the preceding note - so they are evenly spaced in time) to 70%, or even 85% (Not straight timing at all, but 'swing'-timing, where (typically = there are variations!) each second note is moved in time so that it is not the same time after the preceding note as that preceding note was for the one before that.) Instead, the probability grid allows you to set the thing of any of the 16th notes backwards or forwards in increments of 1/64th notes. In the Ableton Live manual, they describe 'groove' in terms of having a piece of elastic for the timing, instead of a fixed metronomic grid. So in ProbablyR, the probability grid provides exactly that sort of timing flexibility, but it also allows probabilistic control, so the swing is not necessarily the same for each note, or each bar, or each repetition... Now this is quite an unusual ability, even these days, unless you have ever programmed something like a Roland MC500. (where each note's timing can be controlled individually!)


Since ProbablyR's timing runs on 16th notes, then the default horizontal row is the 4/64ths line, which is a 16th note. This row is highlighted in dark grey, with the rows above and below in lighter grey. If you let ProbablyR run with the default '4' setting of white cells for each step, then there is no effect: each step is 16th of a note, as usual. The very top row is coloured red to indicate that it is very fast, so fast that on some slower computers the timing might not be exactly as you would expect. This is a result of the way that ProbablyR gets timing information from Live, and I suspect that a better programmer than me (Gregory Taylor from Cycling'74, for example) would be able to improve upon it. Having been raised in a world where the limits of analogue devices were very evident (anyone who has tuned a Yamaha CS-80 will know what I mean), then I'm very gratified that digital devices  also have limitations and wrinkles, albeit sometimes very different ones. And, as they say, 'limitations are the spurs to creativity'!

First half fast (so middle note is early), and second half slow - on average!

But adding an extra white cell at the start and middle of the sequence (the left hand side and the middle of the grid in the panel) can change the timing - quite radically if you want, or reasonably subtly if you prefer. The timing is altered 'per step', so you can have different timing for each step in the sequence if you wish, although this can be quite challenging to minds and ears that are mostly acquainted with regular 4/4 at 120 bpm. The 'Swing' panel uses the same probabilistic control as the rest of ProbablyR, so if you have more than one white cell in a vertical column, then the probability of that value being used is the reciprocal of the number of cells. In other words, if you have 1 white cell, then it happens with a probability of 1/1, which is 100% of the time. If you have 2 white cells, then each cell has a probability of happening of 1/2, which is 50%. Three cells is 33% each, four cells 25% each, and so on. The mathematical term for assigning probability weights to events is 'Bayesian', so if I was a marketing person, I would be calling this a 'Bayesian step sequencer with weights of 50%, 33%, 25%, 20%...'. But you get the idea - the more white cells there are in a vertical column, the more 'spread out' the chance of each one happening becomes. So an alternative way of thinking about the white cells is that they 'spread' the chance of that value happening: one cell doesn't spread it at all, whereas 10 white cells means that each one is only going to happen 10% of the time, on average.

So what happens when we add a white cell to the default row of '4'? Well, in step 1, the length of the step will be either 3/64ths of a note, or 4/64ths of a note. Each of these will happen half of the time, but randomly - so it won't be 3 followed by 4 followed by 3 followed by 4 each time, it could be more like: 3, 4, 4, 3, 3, 3, 4, 3, 3, 4, 4, 4... but where the 3s and 4s are randomly distributed over time. If you listen for 3 minutes and count them, then each of the counts would be pretty close to being half of the total number. It's a bit like tossing a coin: long-term, it will land on 'heads' 50% of the time, but this doesn't mean that if you get 10 heads in a row the next toss will have to be tails ('to even things out' is what people tend to say). Nope, each and every time that you toss the coin it will give heads or tails independently of the previous tosses, which is why the long-term results are going to be very close to 50% heads, 50% tails. (I am aware that it is possible, with practice, for people to deliver heads or tails on demand, but I'm assuming here that the people tossing the coins are not trained specialists at coin tossing! Or dice throwers, or whatever other 'random choice' mechanism you care to use...)

The step in the middle is going to be either 4/64ths or 5/64ths of a note, so this is slightly longer in time. So the two white cells alter the 'elastic' timing of the step sequence so that the first half of the bar is up to 1/64th note faster, while the second half is slower by up to 1/64th of a note. I write 'up to' because the grid is probabilistic, and so the two white cells in a vertical column mean that the average speed-up or slow-down will be half of 1/64th note (1/128th note). In the UI, white cells above the '4' row make things go faster, whilst white cells below the '4' row make things run slower.

I'm going to stop talking about probability right here, because this is sounding increasingly like a TED talk!

Anyway, back to music and the 'elastic' time concept. The 'Swing' grid allows you to assign any time interval for each of the steps - from 1/64th to 16/64ths (which is a 1/4 (quarter) note). Now if you set all of the notes to 1/64th notes then the whole grid is going to take 16/64ths of a note for all 16 steps (assuming we are using the 16 steps per grid default setting), which is 1/4 of a note - so the whole grid runs at 4x speed and what should be a whole note plays in a quarter note's time. At the opposite extreme, setting all of the steps to 16/64ths gives 1/4 notes per step, and 16x 1/4 notes is 4 whole notes, so the grid runs at 1/4 speed. Okay, so the grid runs faster or slower, but the bars are still in time. Unfortunately, these are ideal cases...

Suppose we leave all of the white cells in the Swing grid in the default '4' position (4/64ths = 1/16th notes, one for each of the 16 steps in a 16 step sequence), except for one. Let's make that 3/64ths, which is 1/64th shorter in time. The whole grid will now take 63/64ths of a whole note to play, which means that it jumps forward in time by one step for each repetition (and so it is not in sync), and after 64 bars we will be back in sync again. Whilst this can be useful for polyrhythms, it would be nice if there was a way of knowing when the Swing grid is going to take the sequence out of sync with
Live's transport, and that is what the large number on the right hand side of the grid is used to indicate (for a 16-step sequence, it will be showing '64' when we are in sync) - the number just above the black box with a 'G' or '4' in it. When the '64' is green, then ProbablyR is going to be playing at the same speed as Live's transport - and don't forget you can always click the Grey "Resync' circle button in the 'Memory' panel if you need to get ProbablyR back in sync with Live. But when the number shows anything other than '64' in red (in the example described just now, it would be showing '63'), then you can immediately see that ProbablyR is going to be slipping or dragging in time. So the number going green is something to look out for if you want to have ProbablyR running in sync.

First quarter of the bar is fast, the second quarter is slow, the third quarter is fast again, and the fourth quarter is slow again - on average.

When you edit the Swing grid live, you will find that the number goes red or green depending on where the white cells are, and it is easy to get out of sync. To enable you to edit the grid and not lose sync, there's a special button that enables you to make changes in the background, check that they add up to '64' (for a 16 step sequence) when the number goes green, and then apply the new swing all at once. It is the 'G'/'4' black box - when you click on it so that it shows '4', then the default 4/64ths '4' setting of swing will be applied to the whole grid, regardless of what the grid actually shows. So you can click and put/remove white cells and check that you get a green number, and then click on the '4' in the black box so that it changes to 'G', and now the Grid will be active and you will get the swing settings that you have chosen applied to the grid step timing. You may need to practice this so that the process of

[ '4', then edit the grid, then check for green, then 'G' to Go with the Grid ]

is fixed in your mind (and fingers), and you will be able to change the swing during live performance without losing sync. If you have seen the fist ProbablyR video then you may not have noticed that it is a live recording and that I never stopped ProbablyR running - it just plays the whole time, live. Well, the 'G/4' button is designed to enable you to make changes to the swing whilst ProbablyR is running and in sync with Live.

Putting a speed-up just after the first note in the bar, then a slow-down just before the middle beat, then a speed-up for the third beat, and a slow-down at the end of the bar

Of course, if you do not want to stay in sync, then go for red numbers and don't bother with the 'G/4' button! I'm hoping that some people will do this live, so please let me know if you are using ProbablyR 0v14 in this way! There is another feature for people who want to do 'off sync', and this is the smaller number just lower down from the black 'G/4' box - this number shows the running average number of steps that ProbablyR has played over the last few repetitions. So if you set the 63/64ths swing grid from earlier, then this will go from 64.0 to 63.0, going through 63.9, 63.8... etc. on the way. And if you go back to 64/64ths (or click on the 'G' black box - just reminding you!), then this 'running average' will gradually go back to 64.0. This is intended to give the performer who wants to go 'off sync' a way to see when they are back in time with Live's transport, or maybe when they need to click the grey 'Resync' circle in the Memory panel to get properly back in sync. With a little practice, you can be in complete control of where ProbablyR is in relation to Live's transport.

Editing the Swing grid whilst the '4' button is active - so the timing is even across the whole bar.

Once the 'G' button is shown, the timing variation is alive. Here the middle beat is late by 1/64th of a note, and the rest of the bar is slightly faster to compensate. Note that there are no vertical columns with two white cells, so this timing is always constant and the sequencer should stay in sync...

The 'Time Manipulation' facilities in the 'Order' panel 

The 'Order' grid sets the order in which all of the other panels happen, and the default is just the lower left-to-upper-right diagonal, which gives the 'linear' time that you expect. The first step is followed  by the second step, then the third, and so on (the cursor lines move from left to right across the panels). If you click on the '-1' button, then you get the opposite diagonal (upper left to lower right) and now time runs backwards in the panels - the cursor lines go from right to left!


Now, whilst you can do a lot with the Order panel, it often requires lots of clicking to clear and set white cells, and so the first new feature is the 'Inc' control number. This sets the number of steps that the sequencer advances across the grid for each step - known as the 'Increment'. Previously this was fixed at 1, and so the default diagonal meant that the cursor moved from left to right one cell at a time. Editing the grid to change the diagonal is possible, but the 'Inc' increment button makes it very quick and easy to change the number of steps. The default is 1, as expected, but if you set it to 2 then the Order grid cursor will jump across the grid two cells at a time, and so the grid takes half the time to complete, assuming that you are using the default 16 steps per bar for the Order grid. But the interesting setting is 3, because now the cursor jumps from cell 0 to 3, then to 6, 9, 12, 15, and then to cell 2, then 5, and ending up at 14, followed by cell 1, then across again, and finally back to cell 0. So the cursor scans across the grid three times, and ends up where it started (assuming 16 steps is set!). Setting Inc to 4 goes across very fast, but 5 is another 'multiple scans for a complete bar', and all of the odd numbers do the same re-ordering of the notes in the Pitch panel, when you have 16 steps set. So why did I include the odd numbers?

Okay, the even numbers are there for when you use an odd number of steps: 15 is a good starting point. You need to change the 'Max' control number to '15' - it will no longer be green (assuming you have the Steps selector in the 'Length' panel is set to 16). The Order grid is now only 15 steps across, and so if you use an Increment (inc) setting of 2, then you now get two scans across the grid for one bar. Now, whilst the Order grid is 15 steps long, assuming that you have left the Steps pop-up menu at 16 (the default), then all the other grids will still be 16 steps long - but only the first 15 of them will now be available to the Order grid. So the Order grid will scan back and forth (depending on the Inc value) and the cursors in the other grids will follow along, but step 16 will never be reached. If you want to use step 16 then you set the 'Min' control number to 2, and Max to 16, and now the Order grid will scan from step 2 to step 16.

In other words, the Max and Min control numbers set the width of the Order grid, and this can be anywhere from 2 cels to 16 cells (as set by the 'Steps' pop-up menu in the 'Length' panel. This allows you to set up several different orders in the Order grid, and choose them by setting the Max and Min control numbers, or control the scanning across the Order grid by using the Inc control number. And I haven't mentioned that the Order grid is probabilistic yet, so that makes things even more interesting. Basically, my recommendation is that you should proceed in this way in order to get familiar with what the new features can do:

1. Play with the Order grid with the default settings of Inc=1, Min=1 and Max =16 using one white cell per vertical column.
2. Then try two or more white cells per vertical column.
3. Then try the 'Inc' increment control number.
4. Then use the 'Max' and 'Min' control numbers to choose a part of the Order grid to control the same part of the other grids. Yes, with Inc=1 and Min and Max set not to overlap, then a basic setup might have two 8 step sequencers on two different Memory slots, or four 4 step sequences, or even two 5 step sequences and an 8 step sequence (with the 'Steps' pop-up menu in the Length panel set to 18 steps). And when you have all of these sub-sequences set-up, then don't forget that the 'Inc' control number will allow you to quickly change the order that the grid plays, or changing Min or Max will change the boundaries of the sub-sequences, and you can overlap them if you want. There's a lot to explore here!

 The 'Nudge' buttons

The Order, Probability, Velocity and Length panels all gain four new buttons plus some associated control numbers, whilst the Pitch and Octave panels both gain two extra buttons. The buttons are Nudge buttons for the grids, and they allow you to move the contents of the grid up, down, left or right. For the Pitch grid this means that you can either use notes in a Clip to control the transposition, or you can do it live using the Up/Down Nudge buttons, or store the nudged grid in a Memory slot. For the Octave grid it speeds up choosing the range of the instrument sound.


But the real fun is the Nudge control numbers. These automatically click the associated Nudge buttons according to the number in them. The default '1' setting doesn't do anything, but if you set the left nudge control number to 16 (and the steps are set to 16...) then the grid will scroll to the left by one cell each 16 steps (each bar). So if you have a probability grid or velocity grid or length grid that is controlling which notes are playing and how they are being articulated, then you can move these independently in time.


I like to have a velocity grid with an up and down wiggle and set to be nudged 1 step less than the number of steps in the bar, so that the grid moves horizontally, but each note gets a different value for 16 bars, and then it repeats. (You can hear this happening on the demo on SoundCloud). If you want, you can set more than one Nudge control number, and then the grids will slide about all over the place. There's huge amounts of control to explore here! (And you can always add more automation using Live's internal facilities as well, of course!)

SoundCloud demo

I'm working on a video, but that takes a long time, so, in the meantime, there's a demo up now on SoundCloud. This is a live recording of a single instance of ProbablyR, plus a drum pattern, and has too much reverb on it. (But it IS a demo, and they always have too much reverb, so that's how you know it is a demo!) The demo has velocity nudging, swing time manipulation, and uses the modulo scanning of the Order grid to give interesting variations of the note pool in the Pitch grid. Hopefully, if I've done it well enough, it should sound like someone improvising on the Marimba, and they are slightly imperfect in their timing... But it isn't anything like that, of course. It is produced by one of those 'boring, static step sequencers that monotonously repeats exactly the same notes...,' except that ProbablyR isn't that kind of sequencer. Not at all.

The MIDIprobablyR(S,Z...) series is getting to the point where it needs a proper manual, but this may take some time, so, in the meantime, here are links to all of the blog posts covering the various versions of the Probably step sequencer. (and related devices) 

The Probably... step sequencer

Probably was the first (and simplest) of the 'Probably' series of step sequencers. This blog post is a good place to start. 

Probably blog post                  The first in the series (four grids, limited panel colouring) 
ProbablyZ blog post               The second iteration (added the time-warping 'Order grid/panel)
ProbablyS blog post               The third iteration (added the sub-sequencer 'Memory' grid/panel)
ProbablyR 0v11 blog post      The fourth iteration (added 16 step sub-sea, warp fills, and Step length)   
ProbablyR 0v14 blog post      This post! The fifth iteration (added 'Swing' panel and 'Nudge' buttons)

Latest ProbablyR on maxforlive.com

Other 'Probably...' devices

After Probably, there were some other devices that added extra letters to the end. The 'step sequencer' series adds a single letter: S, Z, R - the other devices add more than one letter. The release sequence for 'Probably the step sequencer'  is: Probably, then ProbablyZ, then ProbablyS, then ProbablyR (current).

ProbablyLFO blog post           Outputs probabilistic (variable) waveforms    
ProbablyLFO blog post2         Extra asynchronous additions... ('swing-like?')   On maxforlive.com

ProbablyGEN blog post          A poly-rhythmic drum sequencer, plus...         
ProbablyGEN blog post2        Extra asynchronous additions...                           On maxforlive.com

ProbablyCHORD blog post     Probabilistic chord generator ('Sergeant...')        On maxforlive.com
ProbablyCHORD blog post2 - coming soon!

There's now a 'coffee-book' illustrated version of this section in a separate new blog post:

    Probably documentation

Hardware Sequencing

If you make hardware sequencers and are busy taking notes from this blog as potential ideas for your next release, then perhaps we should talk. I would love to help someone make a proper advanced hardware sequencer that goes 'beyond' the current 'state of the art', and hopefully ProbablyR (and the other M4L devices in the Probably series) vividly illustrate that I may be useful to you.

Getting MIDIprobablyR 0v014

You can get MIDIprobablyR on http://www.maxforlive.com/library/device/5529/midiprobablyr

Here are the instructions for what to do with the .amxd file that you download from MaxforLive.com:

     https://synthesizerwriter.blogspot.co.uk/2017/12/where-do-i-put-downloaded-amxd.html

(In Live 10, you can also just double-click on the .amxd file, but this puts the device in the same folder as all of the factory devices...)

Oh, yes, and sometimes last-minute fixes do get added, which is why sometimes the blog post is behind the version number of MaxForLive.com...

RAM and CPU in Live

This MaxForLive device uses quite a lot of CPU and RAM, because it is doing quite a lot of work behind the scenes. This is also a very 'wide' device, but I'm cautious about doing a pop-up window (and not very good at programming them!).

Modular Equivalents

In terms of basic modular equivalents, then ProbablyR V14 would require several sequencers in series/parallel to give the same functionality, plus quite a lot of utility logic to do the linking, the probability functions and the nudging, as well as a dedicated 'clock' sequencer just to do the swing function, giving a total of about 16 ME.




Saturday 8 June 2019

Spectral Hex Ping Pong Delay in MaxForLive for Ableton Live

After playing around with various Ping Pong Delay-like devices for a while, I realised that my 'Hex' template was a good place for experimenting with multiple delays, so I took the internals of the AUDpiPOde device and put it inside the 'hex' device template (six times). The result is AUDhexPPD, where I'm abbreviating ping pong delay to PPD to save characters.

AUDhexPPD


Because AUDhexPPD is another 'wide' device, then here's just the left hand side:


Going across the left to right, the input indicator shows the incoming audio signal (and yes, this does duplicate the tiny meters in-between devices), then the Ping Pong Delay block, with the 'Note value time delay' button grid, followed by the 'Unsync' time tweaker, the 'LR swap' output channel allocator, the 'f' delay freeze button, and the Feedback rotary control (which is individual to each PPD block).

The next block across to the right is the middle block: the three-band 'spectral' filtering, which just gives three outputs via low-pass, high-pass and band-pass filters. Unlike the usual 'crossover' filters that you get in some 'spectral' devices (or loudspeakers, now that I think about it), these filters have a Resonance rotary control, which gives the device the ability to colour the sound, so maybe it is more than just six delays.

Finally, on the right hand side, there is the 'Auto-pan' block, which has an individual meter per audio processing block, the 'X' Active/Muted switch (as per all of the 'hex' devices, and a few others of mine), the new 'Pan Position and LFO Modulation' UI widget that I introduced recently (move the white 'thumb' down and horizontal for pan position, and up for LFO modulation), and finally the LFO Rate rotary control, which goes from very slow to not very quick, but then it is a modulation control and audio rate modulation of pan position is not something I'm a great fan of...

On the far right-hand side, there are the usual memory buttons (Click to recall, Shift-click to save (red when empty, grey when filled)) and the Dry/Wet rotary control, followed by three 'common' controls which affect all of the six audio processing blocks: Feedback, Resonance, and Freeze. These just make it easier to alter all of the six audio processing blocks at once!


As usual in the 'hex' devices, the right hand half of the front panel is mirrored, which looks interesting, and makes it very clear that this is a stereo device. Just repeating things twice can make it look like they are arranged in series... There isn't a perfect solution for this, but this way works for me, and it isn't too hard to flip one half if you need an unmirrored custom device...

MaxForLive

So, what's interesting in the M4L code for this device? As I have mentioned previously, I prefer stereo in and out for my devices, and this is normally done with pairs of mono in and mono out devices where the output can be panned across the stereo image... However, each of the 6 audio processing blocks in AUDhexPPD is a tapped delay with a mono input and stereo outputs, and so instead of having one output that needed to be panned, then I had two - and for a ping pong delay then these two outputs should be in the Left and Right channels (or vice-versa - which is what the 'LR Swap' button does). But with two outputs then I needed two auto-pans, and so the obvious answer was to have two auto-pans in anti-phase, so that when one output was on the Left, then the other would be on the Right. 'Dual anti-phase auto-panning' isn't a very catchy description, so I'm going to call it 'Helicopter' mode, because the two pans circle around each other rather like the blades on a helicopter...


Yes, I could have changed the balance between control values and audio signals in this - so the 'snapshot~' and 'sig~'could have been removed from the panning part, but then I would have had to add them back in to the LFO Depth and LFO Out to get control values. This way works for me...

Alternatively, I could have had two SEPARATE auto-pans for each of the two outputs: one LFO for Left, and another LFO for Right. This would have required a doubling of all of the controls in the centre 'panning' block, and would have made the device even wider! (and more complex to use) I decided to go for the simpler solution and leave the 12 pan LFO version for another time (Maybe in an 'mc' version as a way to explore this new feature of Max 8!)

The Max code just inverts the (0 to 1) control value for the multipliers so that the two inputs are affected in opposite ways. Remember that the audio inputs are called 'L' and 'R' because this is the convention for ping pong delays - the first echo comes from the Left channel, then the next from the Right channel. These are not the main incoming audio channels to the whole device - each audio processing block has a mono input and two 'stereo' outputs. So, for each audio processing block, the two inputs are coming from a tapped delay, and so these input signal will be 'bouncing' back and forth between those inputs, and this auto-panning then adds additional pan control on top of that. If you stop the LFO and fix the pan position to Left or Right, then the result is ordinary ping pong echo - where the repeats 'bounce' between Left and Right.

Note that this is the auto-pan arrangement for just one of the six parts in the Pan block. So if you have just one 'X' highlighted and the rest muted, then that ping pong delay will bounce between the Left and Right output channels, panned according to the pan position and LFO modulation as set in that block. If you have two 'X's highlighted (one in the Left half, and one in the Right half), then you have two ping pong delay in parallel - and so the outputs can bounce between the Left and Right outputs independently. If you have all six blocks active, and use the 'triangle' tweak controls to 'unsync' the time delays, then you start to move away from 'echo' and towards 'reverb'-type sounds. The more asynchronous echoes you have, then the more 'reverb'-like the result will be - which, of course, is exactly how real reverb is produced: lots of echoes with different delay times. You 'could' put several 'AUDhexPPD' devices in series, but it is probably going to take a lot of adjusting to get something even approximately like a conventional reverb sound...

Getting AUDhexPPD

You can get AUDhexPPD on http://www.maxforlive.com/library/device/5522/audhexppd

Here are the instructions for what to do with the .amxd file that you download from MaxforLive.com:

     https://synthesizerwriter.blogspot.co.uk/2017/12/where-do-i-put-downloaded-amxd.html

(In Live 10, you can also just double-click on the .amxd file, but this puts the device in the same folder as all of the factory devices...)

Oh, yes, and sometimes last-minute fixes do get added, which is why sometimes the blog post is behind the version number of MaxForLive.com...

RAM and CPU in Live

This MaxForLive device has six tapped delays, so it will use lots of RAM storage and CPU power - but it is worth it, IMHO!

Effects Rack Equivalent in Live

I haven't tried it, but it ought to be possible to get quite a lot of the functionality of AUDhexPPD with an Effects Rack in Ableton Live, although the UI is not going to be as compact, and it might even be possible to implement the spectral crossovers using the band-pass filters inside Delay (or Ping Pong Delay if you aren't using Live 10.1).


What is interesting is that the limitations of building AUDhexPPD inside Live result in something which is a nice, usable effect, but it isn't the same as the MaxForLive version. Anyway, it was a good exercise in translation, and I'm making it available for free. You can download it from here...

Modular Equivalents

In terms of basic modular equivalents, then AUDhexPPD would require two high-pass filters, two low-pass filters, two band-pass filters, six tapped time delays, six LFOs, six panner, some utility switches, and two mixers, giving a total of about 24 ME. It is interesting that this is also a 'wide' device in MaxForLive!