Monday, 16 September 2019

How I Create Electronic Music - the Tee-shirt!

I have too many tee-shirts. Many years attending various hi-tech music events, plus more than 25 years of anime conventions, has left me with a huge choice - and a full wardrobe! But there's always room for another tee-shirt, especially when I don't have one that carries the 'statement' that I want to make...

Bands

When I first started going to concerts back in the 1970s (Camel, then Tangerine Dream at the Liverpool Empire), it was easy. You had already bought the vinyl album, so you bought the tour programme and the tee-shirt at the concert. Nowadays merchandising is more sophisticated, and available all the time, online:

https://www.merchbar.com/rock-alternative/nine-inch-nails/nine-inch-nails-t-shirt-classic-logo-nine-inch-nails-t-shirt


In these days of pirating and grey market, finding a source of 'non-fake' merchandise is important, because otherwise your money isn't going to the band, so choose your source with care!

Many concerts, and hence I have lots of tee-shirts from touring bands of the last 50 years...

Brands

Then you move to brands, and once again there are now ways to get branded tee-shirts online:



Yep, I have one or two 'branded' tee-shirts from synthesizer manufacturers (plus a tour jacket from Opcode) and various other hi-tech music companies... As before, you can buy 'generic' tee-shorts with all sorts of brand names printed on them, but buying direct from the brand itself ensures that you are supporting them, and not someone else with no connection...

Original


Then I decided that I wanted to make a 'statement', and so I started getting tee-shirts that 'said
something interesting and different':



This combines neat graphics with a relevant message, and I always like to support cottage industries. Genki Gear turn up at lots of anime conventions, and have a lot of very nice tee-shirts - thoroughly recommended! And this time, you are going straight to the designers!

Viral


Sometimes the message is part of a viral community. Adam Neely is a YouTube vlogger who covers lots of current music topics from a very informed viewpoint, and is just generally an interesting musician. Not unexpectedly, he has an on-line store that sells: a tee-shirt:



Now this is a message that gets closer to what I'm looking for, and of course I bought one from Adma's store on Teespring.com. Having a classic jazz ornamentation on display says a lot, and it only really means something to people who can 'read' it, understand what it means, understand who it is a reference to, understand who you watch on YouTube, and it kind of makes you part of a viral community of people who watch Adam Neely's videos on YouTube. 'Do-be-do-be doo be dooooo...', as it says. For everyone else, it just says: 'Music', which is okay too. But two messages in one tee-shirt is just uber-cool!

My Tee-shirt!


And finally, we get to what I was looking for all this time, but never found, and so I made it myself. A tee-shirt that sends the message that I make electronic musisc, and says how I do it. I did consider using a screen-shot of a DAW, or a plug-in (one that I've written, for example!), but in the end, these were just branding for someone else, and not me and what I do. So the end result is direct and to the point:



If you are reading this blog, then you probably make electronic music, and you may well have been searching for something that 'says what you do'. Well, you can now get a tee-shirt that says what you do, for everyone to see!

Following from the online selling theme that underlies this whole post, then there's an online store for Synthesizerwriter as well:


There's one hopefully unusual aspect to this tee-shirt online selling page, which isn't immediately obvious. Notice that there are two colour options for the tee-shirt material itself: black and white, but I only produced one graphic for the printing, and that is in white. So if you select the white tee-shirt, then what you get is a white tee-shirt with white printing on it. This leaves you free to colour the tee-shirt material in any way that you like (dye it, tie-dye it, just let it get very grimy or dirty...) and the message will gradually appear... The preview that you get on the web-site kind of warns you that you don't get a white tee-shirt with black printing, of course!

Getting the 'This is how I create...' tee-shirt

Just visit this web-site. No obligation. Enjoy!

Modular Equivalents

For the first time in a long while, there isn't any easy way to implement this in a modular synth, so this gets my first 'infinite' score: this will require Infinite ME.




Tuesday, 3 September 2019

Frequency-to-Colour display in MaxForLive for Ableton Live

It all started with an enquiry on Facebook. Kovacic Matej asked about how to display 'human chakras, colors and their frequencies' in Ableton Live. After some discussion about spectrum analyser plug-ins, a link to a web-site was given and the thread went quiet...

As part of my occasional 'fix a random query on an Ableton forum' series, I have created a MaxForLive plug-in for Ableton Live called AUDcolours that allows the generic mapping of frequency to colour. To make the specific example in the thread easier, plus for musical use, there are two presets: one is the 'human chairs' that was given in the Facebook thread; the other is the musical notes  for one octave - CDEFGABC. There are seven (of course!) additional preset memories for your own mappings.

AUDcolours


There are quite a lot of controls to enable generic mapping of frequency to colour. Each of the eight vertical bars on the left side of the AUDcolours plug-in has a frequency control, Q or resonance control, an audio thru/mute switch (X) and a colour swatch. Each vertical bar can be tuned to any frequency from 10 Hz to 10 kHz, and the width can be set using the Q (resonance) control - low values give wide responses, whilst high values give narrow responses. A medium setting is probably best for initial testing. In the screenshot above, the meter is showing that a frequency of about 261 Hz is being input, and the bar's background has lit up with Red (as set by the swatch UI object). The audio thru/mute switch allows you to listen to one or more specific bars when the X is lit up with bright purple. The swatch is a user interface device that lets you set the colour of the vertical bar.

On the right hand side of AUDcolours are common controls. The vertical group of squares are the usual MaxForLive preset memory slots. Red means empty, White is the current selected preset memory, and Grey means that there are values stored inside the preset memory slot. Underneath the memory slots are the two fixed presets: Chakras and CDEFGABC. On the far right hand side there is a Wet/Dry mix control for the audio, then the Threshold rotary control that determines how loud the input needs to be to lit the vertical bar up with colour (Start high and reduce until you get the triggering just working. Too low will trigger more than one bar!). There are also common controls that can show or hide the rotary controls, the audio level meters, or the colour swatches. Finally, in the lower right hand corner, there is a common control for the audio thru/mute toggle controls (X) - this turns all the Xs on or off at once. If you set Wet/Dry to fully clockwise (100%) then you will hear only the bars which have an X lit up.

Getting AUDcolours

You can get AUDcolours on https://www.maxforlive.com/library/device/5686/audcolours

Here are the instructions for what to do with the .amxd file that you download from MaxforLive.com:

     https://synthesizerwriter.blogspot.co.uk/2017/12/where-do-i-put-downloaded-amxd.html

(In Live 10, you can also just double-click on the .amxd file, but this puts the device in the same folder as all of the factory devices...)

Oh, yes, and sometimes last-minute fixes do get added, which is why sometimes the blog post is behind the version number of MaxForLive.com...

Modular Equivalents

In terms of basic modular equivalents, then AUDcolours 0v01 would probably require eight band-pass filters, with eight Utility/Logic modules to do the threshold detection, and eight colour panels to give the colour outputs to give the same sort of functionality, plus a voltage source panel to give the common control over threshold, giving a total of about 25 ME.

And here's a link to click on if you find my writing informative:



Saturday, 27 July 2019

Several M4L variations on Random for controlling parameters in Ableton Live

I am my own worst enemy. I find it difficult to resist making comments on some Facebook posts, and I find it hard to not create MaxForLive devices when there isn't one available as part of the comment.

Which is where MIDIrandomA comes from. Andxre Andre posted in the 'Max For Live Users' group that it seemed like:

'...the max for live essentials LFO... feels like it always gets to the same values a lot'.  

 And I commented:

Random’ is a difficult topic. People have a number of expectations that do not necessarily match to a true random source. There have been published papers describing the differences between a true random source and what people perceive/expect. People seem to expect no repeated/similar values, no small differences between successive values, and no repeated similar sequences, and more... Some ‘random’ sources are actually processed so that they fit better with expectations - Apple’s iTunes playlist generation and shuffle, for example. 

And I kind of committed myself to making a less random 'random generator'...


MIDIrandomA




MIDIrandomA is the result. It isn't perfect (the repeated sequences' bit isn't included yet...), but it does provide 'constrained' random values that can have the sort of characteristics that Andxre Andre is looking for. As always with my rapid development devices, there's probably a lot that can be improved, so feel free to let me know what you think.

MIDIrandomA is a random 'control voltage'/number/value source. It has three different types of random noise, instead of the more usual single source, and so provides comprehensive control for exacting requirements.

Type A has four rotary controls, and provides detailed control over the gain, the quantisation, the smoothing (logarithmic accumulation), and the thinning (non-linear amplification).

Type B is the classical 'Random Walk' and only provides control over the maximum size of the step between one output and the next.

Type C is the opposite of B - no repeated adjacent values are allowed, and the delta size (difference between successive output values) is the minimum that is allowed.

The generation of random number values (0-127) can be triggered by a free-running LFO (Rate, =Not Synced=), or one of several MIDI note or velocity triggers, which work from notes on a piano-roll Clip in the same track.

The output can be mapped to any co-operating control in Ableton Live.

Getting MIDIrandomA 0v01

You can get MIDIrandomA on https://www.maxforlive.com/library/device/5604/midirandoma

Here are the instructions for what to do with the .amxd file that you download from MaxforLive.com:

     https://synthesizerwriter.blogspot.co.uk/2017/12/where-do-i-put-downloaded-amxd.html

(In Live 10, you can also just double-click on the .amxd file, but this puts the device in the same folder as all of the factory devices...)

Oh, yes, and sometimes last-minute fixes do get added, which is why sometimes the blog post is behind the version number of MaxForLive.com...

Modular Equivalents

In terms of basic modular equivalents, then MIDIrandomA 0v01 would probably require three different npise/random generators, plus some post-processing (non-linear amplifier, filter/accumulator, not sure about the repeated value removal...) to give the same sort of functionality, plus a little bit of switching to do the 'Type' selection and the sync options, giving a total of about  10 ME.

And here's a link to click on if you find my writing informative:


Thursday, 25 July 2019

Notes on troubleshooting video graphics cards on a Mac Pro 5,1

So there I was, working on something other than this blog, using an old Operating System (macOS Mavericks, because I needed to use an old utility), when suddenly my monitor went blank. Now this normally happens when I've not been active enough and the screen-saver (my own, written in Quartz Composer) kicks in. So, as usual, I just pressed the space bar and waited for the screen to re-appear. But it still looked like this:


It didn't.

So I pressed the space bar again. Nothing. The screen looked like this:



And then my monitor started reporting that my Mac Pro wasn't outputting video... I got a sinking feeling in my stomach...

After swapping the miniDisplayPort to HDMI cable, and outputting my laptop screen to the monitor, I had established that it wasn't the cable, or the monitor. Which reminds me:

The Synthesizerwriter Trouble-shooting Guide

1. Sometimes things just fail. It isn't malice, and it isn't deliberate, and anthropomorphising computers and musical devices is almost always a bad idea.

2. Check the power before anything else - this should be your zeroth task - before doing anything else. I always remember the ARP Odyssey Service Manual, where the section (9?) called 'Power Supply Considerations' had a line drawing of a bewildered technician with an opened-up (and very quiet) synthesiser on a bench, and way off to one side was a mains connector not plugged into a mains power socket. Trying to get a sound out of a synthesiser that isn't connected to power can be very difficult...

Note that the 'Check the power' also applies in reverse - if you are working on anywhere near mains voltages (like in the power supply), then you should not have the device plugged in. In fact some people go so far as to put electrical tape over the socket. One other thing related to safety when working with mains voltages is often forgotten - 'Always make sure that you can see the plug that isn't in the socket'. If you can't see the socket, then you don't know if the plug is in the socket, and you might be about to electrocute yourself. If you can see the socket, but you can't see the plug, then someone else may have plugged the device into another socket, and again, you are about to electrocute yourself. Only if you can see the device, the cable, the plug and the socket are you probably safe - although experienced engineers will also keep checking the the plug is actually connected to the device they are working on!

In the case of a monitor, check that it is powered up, particularly if it has very light-touch switches. In the past, I had a monitor that suffered from the Samsung Capacitor problem (more analysis), and so was prone to suddenly losing power (and here the display!), so this was my automatic first assumption - but see 4 below...

3. Check the cables first. There are many failure modes for cables these days. The casters on many 'home office' chairs can do nasty things to the insides of cables and not leave very obvious signs of the damage until you look really closely. Pulling on cables in various ways can stress the wires inside, and it doesn't matter how accidental the trip was, or how much you enjoyed it. Pulled cables can also affect sockets, and there's the famous 'wiggle' test for those.

4. Check with a known good source. So if a mains power socket isn't working, then check it with a mobile phone charger and see if your mobile phone charges. (A table lamp, a hairdryer, and many other pieces of electrical equipment can also be mis-used as a piece of test equipment...) If there's nothing appearing on a screen, then find another source - hence using the miniDisplayPort on my MacBook Pro laptop and seeing if the monitor displayed the mouse pointer as I moved it around. For amplifiers then a microphone can be a good source of sound, but remember the golden rule of amplifiers: 'Always start with the volume turned all the way down, and turn it up slowly, ready to quickly turn it back to off if necessary'. On a monitor, then pressing one of the 'setup' buttons will display the RDTN UI (Ridiculously Difficult To Navigate') and so confirm that the monitor is at least capable of displaying something. There again, on many monitors the 'Power' LED is so bright that many people cover it with Blu-Tak (or similar slightly sticky stuff) and then press their nail into it to create a tiny hole that lets only some light out - which creates an interesting problem when it gets pushed down and the hole goes away, leaving no visible LED.

5. Try turning it off and back on again. This works rather more often than you might imagine!

In my case, the monitor was connected to mains power, the LED mostly covered with Blu-Tack was just about visible, the cable was good, and the laptop and RDTN UI both showed things on the screen. But from the Mac Pro 5,1, all I got was the well-known piece of art often titled: 'Black Cat In A Coal Cellar'.

6. Google: 'Sudden <symptom> from <device>' and see what it says. In my case, 'Sudden loss of video from Mac Pro 5,1' quickly got me to a Mac Graphics Card Specialist retailer, where a few minutes of pop-up chat (usually in the lower left or lower right corner of the web-page) quickly confirmed my worst fears. 'Yep, sounds like video card failure. We've had quite a few of those recently.' Using web-site chat is a good approach to this type of diagnosis, because they don't hear you gasp when you realise that things are about to get deadly serious and potentially expensive, and secondly, you can't hear if they make whoops of delight and wild celebration when they realise that you are about to spend lots of money with them (or another retailer - I am not going to mention the really cynical method of mis-using a helpful chat person on an expensive site when you are planning to eventually buy it from a cheaper source that has naff customer service.

7. Write down what you do and what you find out as you go. Putting it off until later is never a good idea because human beings (and any other sentient organisms reading this blog) often have amazingly  unreliable memories. Of course, you could always write a blog post about what happened...

Back to the main story...


So the monitor was powered up, the cable was ok, the monitor displayed the screen from my laptop, turning it off and on again made no difference, and an external expert had confirmed the diagnosis. When I used to be a service engineer, this would be referred to with words like 'The Monitor Display appears to be Sub-Optimal'.

Now the Mac in question is a Mac Pro, of 2012 vintage, a '5,1' 'Twelve Core', and one of the last first generation towers before the second generation 'Cylindrical' Mac Pros appeared. Just recently, the topology has returned to the tower format with the third generation, which has been, imho, unkindly labelled as 'The Cheesegrater' by some commentators. All of the first generation Mac Pros came with a Radeon 5770 video graphics card as the default output option, although there were various other more-powerful cards available. In my case, it was a 5770 card whose GPU was now pushing up daisies somewhere on a different plane.

At moment like this, there perhaps ought to be a step number 8...

8. 'Optimise your options'. By which I mean that instead of just replacing the faulty part, you look to see if there are any alternatives and maybe engage in some future proofing rather than just maintenance. Your insurance may affect hat you do, of course. In the case of video graphics card, then a card from 2012 is very likely to have been superseded, and that means that the sources are going to be second hand, used cards, not new ones. There's a word that almost always describes what happens when you start to look at alternatives in these circumstances, and that word is often 'complicated'.

This is where an 'escalation sequence' comes in. You start at the simplest solution, and work up to more complex solutions, noting down the advantages, disadvantages, consequences and cost of each. In most cases, it goes something like:

a. Repair the broken part,
b. Replace with a used part,
c. Replace with a new part,
d. Replace with a better alternative part
e. Forget the part and replace the whole system with a new, better alternative

Sometimes the order isn't quite like this when you consider all the pros, cons, consequences and costs - and weighting them can be very awkward to decide (Applying weights to pros and cons, etc., sounds like Risk Analysis, and that is definitely a topic for a different blog!). Solution C is often the easiest logistically, and solution A can often be slow and expensive. Don't forget that replacing with an alternative can also require additional setup, changes and other side effects, so the 'Consequences' column can be very important.

In my case, repair didn't seem to be worthwhile, given the cost of second hand 5770 video graphics cards. New 5770 cards don't exist, so that removed solution C, and solution D would be good for when I need to upgrade to the Mohave version of macOS, but I'm still waiting for Ableton Live to be 'officially compatible'... This left solution B, and it looked like my best option was to gradually move forwards and wait until things forced me to Mohave. So I found a specialist 'Mac video graphics card supplier' that sold 5770s and did the usual on-line checking: they had more than 10 in stock, and there were plenty of FAQs and Installation information on their web-site. It also turned out that this supplier was a good potential source for solution D in the future, because I am sure that I will eventually need to go to Mohave (and beyond), and the Radeon RX 580 looks like a good contender, if a little pricey - and there were quite a lot of 'consequences' to consider s well.

I'm not going to distract you with some of the interesting 'consequences' aspects of changing video graphics cards, because they are so dependent on your hardware and operating system, but I would stress that filling in a escalation table in detail can be very useful when making a decision and planning your route forwards. In my case, the video card failing left me stuck in a very old OS, and without a video card, then your computer is quite tricky to use. Installing drivers, for example, for a replacement card is not easy when you don't have a display!

Links


For new purchases, then I always go for AppleCare. It has dug me out of bad places so many times now that it is an essential purchase component for me.

For generic Mac stuff, then I have been using Mac Upgrades for many years. They are knowledgable and helpful. (The URL comes up as '2nd Chance PC Ltd ' in some browsers, so don't panic!)

For specialist (and used) video graphics cards, then Mac Store UK have access to customised cards from experts like MacVidCards.com, but remember that I'm not a video professional and have only bought one card from them.

As always, suss out your sources carefully before parting with money!


Here's a link to a simple 'Escalation Table' spreadsheet...

And here's a link to click on if you find my writing informative:


  









Sunday, 7 July 2019

Programming some new presets for Sprike...

Every so often, I program some sounds for a synthesiser that I am totally unfamiliar with. It provides a break from revoicing new synths when I first get them (yes, I really do that!), and it is a good way to learn about user interfaces, how to present things so that they are easy-to-use, what needs to be in a user manual, and to learn about new and unusual synthesisers.

You can see a previous example of this activity here, from when I did some sounds for the Amazing Noises Pulsor synthesiser (a MaxForLive plug-in for Ableton Live), and I also wrote my own 'alternative' manual for it.

Sprike



In contrast, and as a break from all of the MaxForLive and Ableton Live stuff, this time I chose Cognitone's Sprike, a free additive virtual analogue synth plug-in that is derived and extended from Tunefish4 (and 3...) and which is available in VST format for 64-bit Windows, and VST and AU formats for 64-bit macOS. Linux isn't directly supported, but there is some compilation information for the earlier Tunefish4 project on github. Cognitone's flagship product is arguably the 'intelligent assistant' called Synfire, which is kind of 'one layer up' above a DAW - it allows sophisticated control over the composition or 'prototyping' of music via smart editing of harmony. Powerful and deep, it is a million mile away from the auto-accompaniment fake sheet and chord utilities that I remember from the recently MIDIfied 1980s and 90s.

Sprike looks like many other VSTs. The User Interface (UI) is divided up into the sort of panels that a real piece of hardware would have, has a global section at top-left, and a virtual keyboard along the lower edge. Quite a lot of the panels can be disabled, and they go dark blue when this happens. The right hand side has the modulation 'matrix' (8 sources, 29 destinations, and 8 modulation depth controls), plus an interesting effects 'stack', which allows up to 10 of the available effects to be placed in series (a great idea that I wish was more widely adopted in plug-ins).


Sprite uses an additive generator as its main sound source (it also does various flavours of noise!). There are quite a lot of controls, and these use additive synthesis to fill a wavetable, and then that wavetable is what produces the sound output from the generator. Trying to provide an intuitive and minimal-number-of-controls interface for additive synthesis is not easy, and Sprike's solution is pretty good - there are not that many controls, and the graphic underneath shows the spectrum on top, and the waveform underneath. A little bit of experimentation should rapidly get you up and running and making sounds, although getting fully to grips with the generator may take some time.  The two rows of buttons are for the amount of detune for what sounds like two wavetable players, and the octave switching. When I was programming Sprike, I left the octave buttons in the default '0' position, rather than produce 'bass' sounds by just transposing down by a couple of octaves...

There are four filters, and they seem to be connected in series, although you can switch them in and out (but not via the modulation matrix!). There are 2 LFOs, and 2 ADSR envelope generators, and finally seven effects: Flanger, Reverb, Delay, EQ, Chorus, Vowel Formant Filters, and Distortion. Basically, you get more panels than you would expect in a basic additive synth, and so the description of 'additive virtual analog' is a much better description - you get 'subtractive-synth'-style filtering and envelopes instead of the 'envelope per harmonic' controls that traditional additive synths give you.

For something which is free, and which Cognition describe as an experiment in writing small and efficient machine code, then I hesitate to criticise Sprike in any way, because my assembler skills stopped with the 56000 DSP from Motorola many years ago (and the 6502/8080 before that). However, whilst programming a few presets, I did find a few things that are useful pointers to things that anyone who makes a synthesiser plug-in should consider...

First, there isn't much in the way of user documentation (that I could find - I would love to be wrong!). There is a brief Tunefish user manual on github, and some additional material on the tunafish-synth.com web-site, but you do need to do a lot of iterative exploring of some of the controls to get a feel for what they do.

Secondly, the additive generator doesn't make it very obvious what the programming model is. There are controls that mention parameters like 'Harmonics', 'Drive' and Spread', but you may need to spend some time to learn how to exploit them to get the sort of raw generator sound you want. having said this, I am still impressed with the small number of controls. I have a Kawai K5 additive synth, and this has the aforementioned 'traditional' 'envelope per harmonic' approach, which is a huge number of controls!

Thirdly, the volume control defeated me. There is a global volume control at the top left, but towards the lower right corner there is another 'Volume' control which is associated with the 'Pan' control, and so my assumption was that this was the main output volume control for a preset. But it doesn't seem to work like this, and I'm not sure if it is connected to the global volume control. Anyway, it doesn't really stop you from programming sounds, it just means that sometimes you need to be aware that you may need to tweak the volume control - and tweaking synth controls is not exactly an unknown of unusual activity!



Finally, the modulation values aren't exactly intuitive. If you choose an LFO as the modulation source  (look at LFO 1 in the screenshot above) and then set it to modulate the 'Scale' control in the generator, then full modulation seems to happen when the mod matrix control is set to 70. Whilst useful for ova-modulating controls, it took me a while to figure it out, and I dislike 'magic' numbers in user interfaces - of course, having programmed MIDI editors for the Kawai K5, then I'm very familiar with 'special' 'secret' numbers, because the Kawai MIDI implementation uses them!

Presets


In the end, I produced 128 new presets for Sprike, and a quick search of the Interweb gave me the impression that there are not very many easily available, so I'm making mine available for free (as usual). There's a reasonably wide range of sounds, albeit without any octave switching (so you can decide which sounds you want to be 'bass' sounds!) and a strong bias towards synthetic and away from imitative. Use at you own risk (some of the effects stacks seem to overload sometimes...), but hopefully 'enjoy!'



Sprite has a simple drop-down menu for selecting presets, plus 'Prev' and 'Next' buttons, plus copying and pasting facilities, so it is obviously intended to be programmed! The presets themselves are just plain text files (which may not be as straight-forward as you imagine for some Operating Systems) with parameter names followed semi-colons then the value per line.


Getting the presets


You can get the Sprike presets from here, for free (it is a 143 Kbytes zip file). Unzip the file and put the files into a folder called 'bank1' (or higher, as you wish) in the appropriate 'Presets/Sprike' folder inside your OS's filing system - as shown above for a Mac...


Buy me a coffeeBuy me a coffee






Wednesday, 26 June 2019

Storing text on Dropbox...

Sometimes amazing stuff crops up by accident. I use Dropbox, the cloud-based online storage service, to store files that I want to share between my computers. mobile, etc. Dropbox has served me well for many years, and I have rarely needed to contact their customer support.

However, a chance happening alerted me to something interesting that I hadn't considered before. Someone sent me a 'download-only' link to their Dropbox so that I could get some files, but one of those files was just a placeholder for a file that they would upload later to complete the set. All perfectly ordinary stuff that people use to do business every day.

But that placeholder was interesting. I accidentally tried to download it along with all of the other files. But it wouldn't download and I got an alert warning me that it was 'zero bytes' long. It was at this point that I got interested, and I generated some test files to learn exactly how Dropbox stored them. It seems that if you store a zero length file, you can give it a short title (a few words), and it will occupy zero bytes of your Dropbox storage quota. Now zero length files are easy to produce - I just used an ASCII editor (I did a whole blog post on the topic from a couple of months ago...) and named and saved the empty file direct to Dropbox. After a bit more experimentation, here's the resulting directory listing from my Dropbox 'Files' page (slightly edited):


As you can see, I wasn't particularly inventive in my choice of words, but just in case, I contacted Dropbox customer support to let them know that it is possible to store text on Dropbox without affecting your storage quota, and they agreed with my analysis. They also pointed out that you could also use folder names...

Now I tend not to use the browser interface to Dropbox very much, but the directory page is interesting. It gives you a timed list of your activity on Dropbox, so for my recent activity, it showed the lines from Shakespeare in order, scrolling vertically as each new line was added... 

Now, this was very cool! Dropbox effectively gives you, for free, a performance tool that lets you publish short lines of text on Dropbox, where those lines of text from the file titles are displayed on a web page, in order, and timestamped. So as each empty file is added to Dropbox from a browser or just dropped into the local Dropbox folder on your computer, the Shakespearean soliloquy gets displayed one line at a time, using the text from the title of the file, with previous lines scrolling upwards or downwards (you can control this from the directory page)... No need to refresh the page - it just works.

The only catches are: 
- the title has to be short enough (you can see that some of those lines are getting close to being truncated), and 
- you can't put any punctuation in the text, except for symbols that would be allowed in file names anyway...

Well, I didn't think just virtual thespians at this point, I also thought about song lyrics. It seems that you could potentially store song lyrics (assuming you have the rights to do so, of course), one line at a time, on Dropbox, with the right timing, and Dropbox will scroll them on the display as they are received. And this costs you nothing extra on top of whatever you normally pay for Dropbox. The files are empty, so they don't add to your storage quota! Making an app that does timed saves of empty files with titles from the individual lines in a text file is not that difficult, and Dropbox does everything else... 

I'm now wondering if there is anything else that can be done with free text storage... Subtitles, commentaries, live comments, etc. There's a lot of things o consider before designing a solution based on what Dropbox provide: the latency and jitter of update times, for example, may mean that lyrics or subtitles aren't a good application. But the interesting thing is that there's going to be at least one application that is a perfect fit for this...

I asked Dropbox Customer Support if it was okay to publish this, and they said it was fine. So now you know too! I know it's only minor and trivial really, but for me, it is fascinating to discover that something like this is possible. My analysis follows below for those who are interested in system design...

(I'm just wondering what Charlie Brooker of 'Black Mirror'-fame would make of this...)

Analysis

Here's some serious analysis of this as a system design problem. 

Whenever you design a system, there are the things you want it to do, and a security analysis will give you pointers towards things that you don't want it to do. You can do risk analysis on those unwanted features, and decide on appropriate mitigations to reduce the residual risk to acceptable levels. But there's a hole in this very conventional process, and that is the things that you didn't specify that also are not security risks. These are the unexpected features, and they can be very interesting.

Risk analysis for security has evolved over a long time, and it has a sophisticated set of processes, approaches, tools and practitioners that are all based on lots of experience of looking at systems from a very specific viewpoint - security. These days, an additional and allied parallel activity has become important because of legislation like GDPR, and that is Privacy. Once again, there are processes, etc., and practitioners who are skilled at looking at systems from that viewpoint, and again the analysis leads to risks which result in mitigations to minimise the residual risk, etc. 

But there's that interesting hole where the 'things that you didn't specify the system to do, but it does them anyway, and they don't have any security or privacy implications'. These things fly 'under the radar' and I've never seen very much in the way of any formalisation of process or approach to identifying them, assessing them, and deciding if mitigations are appropriate. Unintended consequences are probably acceptable to some level with simple, non-networked systems that aren't critical in an way - for life-support, emergency cover, critical infrastructure, etc. But as systems become more interconnected, more networked, more 'Cloud'-based, then that little word 'unintended' starts to become more significant. 

In the case outlined above, a design would look at what can happen, and would assess it based on the consequences. Having folder names and file titles outside of the storage quota may seem like a pragmatic solution to a user wanting to know how much storage they are using, but actually, the user's viewpoint of the storage differs from the actual storage that a provider like Dropbox has to provide, because the filing system itself uses storage, and users would probably not want to pay for that storage, even when it turns out that they could actually be using it themselves to store information for free.

What is more concerning are the repercussions of the unintended (or maybe, 'assessed as insignificant') features of a system when that system becomes stressed in some way. My completely uninformed suspicion is that the designers of the Dropbox system probably assumed that users would store files in folders, and that the titles of the files and folders would be insignificant in size in comparison to the actual content of the files themselves. But all it takes is someone to make an app that uses Dropbox as a filing system for song lyrics, subtitles, commentaries, etc., and things might start to escalate. 

The first protection mechanism that would probably be hit would be a limit on the number of folders or files within folders. But if I had designed this, then I would be expecting that this would be linked to a lot of storage as well. The 'user story' for lots of files of zero length isn't something that I would have thought of, and so is an 'unexpected' feature that I would have probably missed in my design. And if a designer misses a feature, then is it tested thoroughly? I'm pretty sure that the consumed storage is a key parameter used to monitor a user's account - I know this because Dropbox is very good at telling me how much storage I use, and especially when I'm getting close to using it all up. Selling me additional storage is good for me and good for Dropbox. But if lots of users were to start using a filing system that exploited the folder and file name 'free' storage feature, then the number of files and folders being used might start to become an important parameter for Dropbox, because suddenly the design rules that said that the storage required for that purpose was insignificant in comparison to the actual chargeable storage space consumed by files uploaded by users, would be wrong...

Changes in design rules when you have a system launched and operational can be awkward, and they can even be potentially expensive, or maybe catastrophic. One of the reasons that I contacted Dropbox when I found out about the zero length file storage was because I was curious about their  response. Dropbox were very open about the storage, and agreed that I could publish this blog on what I had found. It will be very interesting to see what happens next... 

Now if I was Dropbox, then I would be assessing exactly what the consequences might be if a lot of apps started loading the Dropbox system with file and folder title metadata, and those design rules would probably be revisited. Additional checks and catches might be put in place to check for lots of zero length files, and limits for the number of zero length files could be announced and policed. Processes for designing solutions might be revisited, and new checks and balances added to try and catch unintended consequences in future designs.

But actually, there's something much more interesting that Dropbox could do, in parallel to this. They now have a lead in the 'analysis of unintended consequences of plain ordinary features in large systems', and by the time they have done all of the analysis and fixes, they will be world experts in what you need to do to mitigate against this type of potentially lurking problem. This sort of 'hard-earned' practical and theoretical expertise from actually solving a real-world problem is worth a LOT of money, and the next unintended consequence in someone else's system might be something much more damaging and dangerous, particularly if it isn't something that security or privacy risk analysis would have caught (and by its nature, it very probably will be!). At this point, that Dropbox expertise could well be one of their most precious commodities...

I have to thank Dropbox customer support for their help in this. They were wonderful.

All free!

All of the analysis here on this blog is free, of course, (I am a CISSP in the real-world until October 2019, but this is pro bono because it is fascinating...) but donations are always welcome!


  


Thursday, 13 June 2019

Documentation for the 'Probably' series of M4L devices - Illustrated!

The recent release of ProbablyR 0v14 reminded me that the documentation needs writing at some stage... So here's a promoted part of the blog post for that, but as a separate blog post to make it easier to find, and with added graphics to give it extra 'eye-candy'-ness. Not quite a square coffee-table book full of nice glossy photos, but you never know...

Documentation for Probably

The MIDIprobablyR(S,Z...) series is getting to the point where it needs a proper manual, but this may take some time, so, in the meantime, here are links to all of the blog posts covering the various versions of the Probably step sequencer. (and related devices) 

The Probably... step sequencer


Probably was the first (and simplest) of the 'Probably' series of step sequencers. This blog post is a good place to start.

Probably blog post                  The first in the series (four grids, limited panel colouring)


ProbablyZ blog post               The second iteration (added the time-warping 'Order grid/panel)


ProbablyS blog post               The third iteration (added the sub-sequencer 'Memory' grid/panel)


ProbablyR 0v11 blog post      The fourth iteration (added 16 step sub-seq, warp fills, and Step length)


ProbablyR 0v14 blog post      The fifth iteration (added 'Swing' panel and 'Nudge' buttons)

Latest ProbablyR on maxforlive.com




Somewhere along the line, the breathlessly-annotated 'white and light purple' advert style got stuck in my mind, and I have used it ever since. Love it or hate it, it is immediately recognisable...

Other 'Probably...' devices

After Probably, there were some other devices that added extra letters to the end. The 'step sequencer' series adds a single letter: S, Z, R - the other devices add more than one letter. The release sequence for 'Probably the step sequencer'  is: Probably, then ProbablyZ, then ProbablyS, then ProbablyR (current).


ProbablyLFO


ProbablyLFO blog post           Outputs probabilistic (variable) waveforms  


ProbablyLFO blog post2         Extra sync additions and time warping               On maxforlive.com

Most people glance at the pixelated sine wave in the 0v03 screen-shot and say something like: 'Yuk - a pixellated sine-wave!' and ignore it. What they miss is an LFO that allows you to make arbitrary waveforms that change shape probabilistically and over time (the noise-warping is really cool, even if I say so myself!). Also, the pixelation can be smoothed out, and actually, having a little bit of it sounds good! - if you were raised in the 'S&H filter modulation era' then it is like a more sophisticated version of that. First impressions are not always reliable, and I've not even mentioned really unusual features like the 'Note' mode, where the LFO waveform is 'triggered' by notes in a clip, which one user described as 'genious'!



ProbablyGEN


ProbablyGEN blog post          A poly-rhythmic drum sequencer, plus...  

     
ProbablyGEN blog post2        Extra asynchronous additions...                           On maxforlive.com

ProbablyGEN is pure timing madness. Each of the three horizontal rows is independently timed (synced or free of Live's transport) and so you can do truly free-form poly-rhythms. This is the sort of thing that really ought to be a module for a modular synth...



ProbablyCHORD


ProbablyCHORD blog post     Probabilistic chord generator ('Sergeant...')        On maxforlive.com

ProbablyCHORD is interesting because it brings together several idea into one place. The TR2gen to-step chord generator puts two chords in sequence and provides inversions, and then lets you automate and add probability to it. So it adds polyphony to the monophonic step sequencing from ProbablyR (and in fact, the variable 'step length' pop-up menu was tested here first, then added to ProbablyR 0v11). But it is also shown to its best advantage when paired with an 'articulated' sound using velocity, and you can learn more about this in two ModeAudio interviews/tutorials.


There is more on Ableton instrument racks here:

- Macro controls in Instrument Racks In Ableton Live

And an example of a more sophisticated multi-layered articulated sound (implemented in an Instrument Rack in Ableton Live) is here:


In the pipeline is a blog post on additional articulations that you can do inside Instrument Racks, which may also have other spin-offs...


ProbablyCHORD blog post2 - coming soon!


There are updates and more devices in the pipeline, of course. Watch this blog!


All free!

Everything here is free, of course, but donations are always welcome!