Third Year Project: Synchronisation of Music in Node-Red
October 2017 - June 2019
The goal of this project was to modify Node-RED - a visual flow-based programming environment - to allow multiple users to create a piece of music together in a real-time live coding setting, with the music from each user's machine synchronised to the same beat. Furthermore, the aim was to integrate this system with Git to allow the steps taken within the performances to be recorded and later replayed.
The motivation behind this system was to use it in schools in order to encourage more young girls to get involved in computer science by showing a collaborative, hands-on, creative use for it; this is backed up by research by Microsoft and other organisations .
Node-RED is an online, visual, flow-based programming environment. It consists of the following:
- Blocks/Nodes: Blocks are self-containted functions, such as an interface to get the response from a webpage, or to parse JSON, or in this case, to create sound.
- Connections: A connection is a visual line which is drawn between blocks/nodes; these signify transfer of data between functions and, along with special control blocks (e.g and IF block with one input and two outputs), control the order of operations at run-time.
- Flow: Blocks and connections are organised into flows. Special inject nodes are used by the user to send commands (often start/stop/reset) which control the flow.
The synchronisation and recording sections below provide more details of the solution
- How to structure a long-term (one-year) project while being flexible with plans and achieving the desired output
- How to implement clock synchronisation algorithms and extend them for a new purpose
- How to analyse and edit source files for an existing system to integrate with a different external system
- How to write an academic research paper
Synchronising the Beats
My supervisor created musical nodes to allow a user to create a piece of music within Node-RED. My aim was to modify these nodes to allow for two users to play a piece of music together. The main issue with this is synchronising the beats; this was solved by using Berkeley's Algorithm to synchronise clocks that then drive the beats.
The following blocks/nodes were created to allow synchronisation within Node-RED:
- Berkeley Master and Berkeley Slave: Each Berkeley node creates a logical application clock that can be accessed by other nodes running within the same Node-RED environment.The slave nodes can join a master's group by sending a simple request message. The master then sychronises all of the slave's clocks using Berkeley's Algorithm.
- Beat Master and Beat Slave: These are modified beat nodes (originally created by my supervisor) that control the timing of the music. These are modified to use the synchronised clocks exposed by the Berkeley Nodes
The method to create music at run-time is as follows:
- Each user starts their own version of Node-RED on their computer
- One user is chosen as the master and adds the Berkeley Master node to their flow diagram
- The other members of the group add a Berkeley Slave node to their flows and point these to the IP address of the master
- The leader adds the Beat Master node to the start of their music flow
- The other members of the group add a Beat Slave node to the start of their music flow, and point it to the IP address of the master
- Now each member can add other musical instruments to their flow and the beats will be synchronised
Recording performances
To integrate this project with Git required analysing and understanding the many files and thousands of lines of code that make up the Node-RED system. This analysis led to the modification of certain files, to intercept certain function calls and save the relevant information for beat timings to files that could then be committed to git programmatically. When replaying the performance, the users simply needed to reset their Node-RED environment and run a single-line replay command; this triggers a function that programmatically pulls the relevant performance timing information from the specified Git repository and uses this information to decide when to perform beats.
More information is provided in the report below.
Classroom Experiment
As part of the project, the solution was taken into a school and used in an after-school coding club. The instructions for how to setup the Node-RED system in this test are given here.
The results of this test can be seen in the report below.
Final Reports
This project concluded with a 20-page report, a poster, and a presentation, along with the final code files.