Creating Music Notation using LiliPond

Lilipond is a music engraving program, which produce high quality music notation and send them into PDF format. One of the most interesting thing you can do using lilipond is, you can use lilipond for your music programming. If you follow their syntax properly you can create music applications very easily.

Lets get a kick start on lilipond.
Before you begin you need to download lilipond depending on your operating system.

For this demostration I am going to show how you can create the popular song "Twinkle Twinkle Little Star" Using lili pond.

There are three main things we need to consider when creating a song.in lili Pond

1. How to create the melody ? (the treble clef notes)
2. How are we going to accompany them? (base notes)
3. Are we going to produce a midi file to our notation.

Creating a melody

In lilipond most of the pitches can be given relatively to a given octave. for example

\relative c {
  \clef bass
  c d e f
  g a b c
  d e f g
}

Above notes are relative to the middle octave. And if you want to create octaves lower o higher you can use " ,  " (comma)  or " ' ". This is called octave changing mark in lilipond. And also default time signature is 4/4 and default clef is treble clef. Sharp {#} are defined as ("is") and flats (b) are defined as ("es"). For example if you want to create a# then the notation is like "ais". Ok those are the very basic principles of lilipond but if you want to create more complicate notations please refer the use guide.

Lets start our song. Here we are not using relatives we are using standard notation. So I am going to create a lead sheet which only has the treble clef notes and the chords are indicated on top of each bar.

\score {
{
<<
\chords  {f1 c1 g1 d1:m g1 d1:m f1 c1  }
\new Staff { \time 4/4c'4 c'4 g'4 g'4 a'4 a'4 g'2 f'4 f'4 e'4 e'4 d'4 d'4 c'2 f'4 f'4 e'4 e'4 d'4 d'4 c'2 f'4 f'4 e'4 e'4 d'4 d'4 c'2 }
>>
}
 \layout { }
\midi { }
}

Score represents its a lead sheet. And to represent chords  you need the notation "\chords" here you can define what type of chords you want and the duration on that chord.
   f1 - means fmajor and it will last  4 beats (breave).
   d1:m - means its DMinor it will also last 4 beats.
Like wise you can define your codes

By using new Staff you can give music notations such as crochests, minims rest etc depending on the cleff. If you dont have a clef then the default is treble. you can also define time signature, And the duration of the notes are given by numbers. C4 - one beat C2 - two beats , C1- four beats etc.

Lets look at our twinkle twinkle song notation

Creating MIDI

 
If you want to create a midi file as well you need to put /midi() then it will create a midi file of the song you created.

Creating Piano sheets

Creating piano sheets can also be done similarly. Only difference is you need have two staves other than one.

Example ..

\relative c'' {
  \new PianoStaff <<
    \new Staff { \time 4/4 c4 e8 g8 g4, e4 e}
    \new Staff { \clef bass c,2 e4 g e2 b'4 g}
  >>
}

Creating Fret Sheets.

If you want to create guitar tabs for notation for your songs there is an option to create fretsheets. In order to do that you need to include FretBoards  liberary and give your notation as shown below.

Example...

\include "predefined-guitar-fretboards.ly"
<<
\context FretBoards {
  \chordmode {
    c1:m e
  }
}
\relative c'' {
    \new Staff { \time 4/4 c4 e g g, e2 e} 
}
>>

Adding Chords to your melody using jMusic (Adding Accompaniments)

In my last post I showed how we can easily create a song (twinkle twinkle little star), which is basically the tune of that song or in western music language we call it the melody, in this post I am going to talk about how we can accompany this melody by adding chords.

Let’s see how we can accompany our melody ..

This is in our today’s TODO list

-          Adding Chords to our song

-          Arranging the chords in choral fashion

-          Adding  guitar chords (o any other instruments)

Adding Chords to the song “Twinkle twinkle”

 

Following diagram shows the chord arrangements for the song twinkle Twinkle however this is not static you can select appropriate chord of your choice depending on the mood and the sound you want. However let’s just use a very basic chord progression to our song.

Since we already created our song in my last post I will use the same code to add chord progression.

When we were creating the notes we needed a pitch array which is an integer array to store the notes(pitch classes), likewise we need Chord array to keep Chord structure.  

A Chord represents 3  or 4 notes together playing at once.  So I am going to create array of three notes for each of our chord. 

Here we need  3 Types of  Chords

Chord	Notes
CMaj	C E G
FMaj	F A C
GMaj	G B D

Adding  Chords is bit complicated than just creating notes therefore, I will explain simple as possible. 

Here we are going to have two parts. First one is our melody line(tune) which we created earlier. And the other part is the base part (the chords) to accompany our melody.

   Phrase phr = new Phrase("Twinkle twinkle", 0.0);
        Part treblePart = new Part("PIANO-Right", PIANO, 0);
        int[] pitchArray = {C4, C4, G4, G4, A4, A4, G4, F4, F4, E4, E4, D4, D4, C4, G4, G4, F4, F4, E4, E4, D4, G4, G4, F4, F4, E4, E4, D4};
        double[] rhythmArray = {C, C, C, C, C, C, M, C, C, C, C, C, C, M, C, C, C, C, C, C, M, C, C, C, C, C, C, M};
        phr.addNoteList(pitchArray, rhythmArray);
        treblePart.add(phr);

And then we create the base part

static Part bassPart = new Part("PIANO-Left", PIANO, 0);

To create chords we need to define each type of chord we are using. First we’l creates our three types of chords.

        Note cMaj[] = {new Note(C3, M), new Note(E3, M), new Note(G3, M)};
        Note fMaj[] = {new Note(F3, M), new Note(A3, M), new Note(C3, M)};
        Note gMaj[] = {new Note(G3, M), new Note(B3, M), new Note(D3, M)};

Earlier I created a Phrase  to store the pitch classes (notes) and their durations (time slots) here we need to create a  CPhrase chord = new CPhrase(); to store our chords. 

And we add each chord to the basePart to complete the phase.

So to make this code much efficient and elegant I am going to create a separate method which adds a given chord to our basePart.

public static void addChordsPart(Note chrd[]) {
        CPhrase chord = new CPhrase();
        chord.addChord(chrd);
        bassPart.addCPhrase(chord);
    }

Getting things all together

1. Create two parts
2. Add the melody to the first part by giving the pitch classes and durations
3. Create the types of the chords
4. Add each chord to the second part by giving the chord name and their duration per each chord
5. Create the Score and add Part one and part two together
6. Play/Save the midi

package mymusicapp;

import jm.JMC;
import jm.music.data.CPhrase;
import jm.music.data.Note;
import jm.music.data.Part;
import jm.music.data.Phrase;
import jm.music.data.Score;
import jm.util.Play;
import jm.util.Write;


public class Main implements JMC {

    static Part bassPart = new Part("PIANO-Left", PIANO, 0);

    public static void main(String[] args) {
        Phrase phr = new Phrase("Twinkle twinkle", 0.0);
        Part treblePart = new Part("PIANO-Right", PIANO, 0);
        int[] pitchArray = {C4, C4, G4, G4, A4, A4, G4, F4, F4, E4, E4, D4, D4, C4, G4, G4, F4, F4, E4, E4, D4, G4, G4, F4, F4, E4, E4, D4};
        double[] rhythmArray = {C, C, C, C, C, C, M, C, C, C, C, C, C, M, C, C, C, C, C, C, M, C, C, C, C, C, C, M};
        phr.addNoteList(pitchArray, rhythmArray);
        treblePart.add(phr);

        Note cMaj[] = {new Note(C3, M), new Note(E3, M), new Note(G3, M)};
        Note fMaj[] = {new Note(F3, M), new Note(A3, M), new Note(C3, M)};
        Note gMaj[] = {new Note(G3, M), new Note(B3, M), new Note(D3, M)};

        addChordsPart(cMaj);
        addChordsPart(cMaj);
        addChordsPart(fMaj);
        addChordsPart(cMaj);
        addChordsPart(fMaj);
        addChordsPart(cMaj);
        addChordsPart(gMaj);
        addChordsPart(cMaj);
        addChordsPart(cMaj);
        addChordsPart(fMaj);
        addChordsPart(cMaj);
        addChordsPart(gMaj);
        addChordsPart(cMaj);
        addChordsPart(fMaj);
        addChordsPart(cMaj);
        addChordsPart(gMaj);

        Score score = new Score("Twinkle-Twinkle");
        score.addPart(treblePart);
        score.addPart(bassPart);

        Play.midi(score);
        Write.midi(score);

    }

    public static void addChordsPart(Note chrd[]) {
        CPhrase chord = new CPhrase();
        chord.addChord(chrd);
        bassPart.addCPhrase(chord);
    }
}

Now we added chords to our songs. You can try out the songs by simply playing the song.

 Click here to listen to our creation

Arranging  the chords.

Just playing the chord progression for a song can be lil boarding. To make the song more interesting we can arrange the base chords into different variations. 

Following diagram shows how we can arrange the chords using each note of the chord.

So for that we need to change our method a little bit by giving the notes of each chord separately and  arranging them with proper durations.

 public static void addbaseNotesPart(Note chrd[]) {
        Phrase chord = new Phrase();
        int[] pitchArray = {chrd[0].getPitch(), chrd[2].getPitch(), chrd[1].getPitch(), chrd[2].getPitch()};
        double[] rhythmArray = {Q, Q, Q, Q};
        chord.addNoteList(pitchArray, rhythmArray);
        bassPart.addPhrase(chord);
    }

We can use the same code, but instead of using addChordsPart, use addbaseNotesPart to get the styling of our chord progression.

 Click here to listen to our creation

Adding  guitar chords (o any other instruments)

This Chord Arrangements can be done using any instrument. All you need to do is change the  Part instrument to the instrument of your choice.

  static Part bassPart = new Part("PIANO-Left", GUITAR, 0);

You can also experiment by changing the chords/durations and adding new parts. 

Introduction to Computer Music Harmonization

Automatic harmonization history 

Among musical compositional systems there has been a large number of researchs carried out in the field of automatic harmonization from the early 1950s onwards. When looking at automatic harmonization history the most pioneering work in automatic harmonization is that of Rothgeb who developed a SNOBOL, program to solve the harmonization problem and to identify the voice leadings notes to accompany it. Rothgeb used rule base approach that has a set of ‘if statements’ according ot the musical domain. His main aim was not focusing on automatic harmonzation but to test the computational soundness of two bass harmonization theories from the eighteenth century (Rothgeb, 1969). Afterwards Steals in 1979 proposed a system to use constraints to create passing chords as chords that could be inserted between two given chords. These passing chords must satisfy some musical constraints, such as interval relations between the roots of the first, passing, and last chords. Further, Steals used essentially a frame system, augmented with a bread-first search (Steals, 1979).

The above evidence suggests that the first works for automatic harmonization were carried by using rule base approaches by representing set of musical rules. Furthermore, this indicates that no constraint satisfaction algorithm was used and the main concern was about mastering the combinatorial explosion, by putting more knowledge in the solver.

Automatic Harmonization – AI Techniques

There have been several approaches taken for automatic harmonization in artificial neural networks (ANN). There has been a large number of research carried out in the field of music composition using neural networks available in the literature; Todd (1989, 1991) used a feed-forward ANN with feedback for melody generation, Lewis (1991), Hild et al. (1992) used a neural network to harmonize chorales that divides harmonization in to three parts 1) harmonic skeleton 2) chord skeleton and 3) ornamentation, Mozer (1991, 1994) generated melody using ANN, Stevens and Wiles (1993), Bellgard and Tsang (1994) constructed a Boltzmann machine for harmonization which generates harmonies non-deterministically. Further, Toviainen (1995) trained neural networks for jazz improvisation and later Hörnel and Degenhardt (1997) generated harmonization for four part chorals in baroque style.

The most of the automatic harmonization used neural networks are for four part choral harmonization. Automatic harmonization using neural networks, Schwanauer the developer of MUSE that harmonize chorals, claims that there are five learning techniques learning by rote, learning from failure, learning from examples, learning by analogy and learning from discovery (Schwanauer,1993).

Even though neural network approaches are widely used among musical composition systems Papadopoulos and Wiggins discusses that there are many disadvantages in neural network approach for musical compositions, in his analysis he claims that “The representation of time cannot be dealt efficiently even with ANNs that have feedback” (Papadopoulos & Wiggins, 2007). Further, he states that neural networks cannot reproduce fully trained data set. Even if they are fully trained it is not generalized (Papadopoulos & Wiggins, 2007). Toviainen (1999) claims that neural networks fail to pick up the higher-level features of music such as phrasing or tonal functions. Further, he explains that a neural network often solves toy problems, with many simplifications when compared to knowledge based approaches (Toiviainen, 1999). However, the review reveals that the neural network approaches used for musical compositions suggests that neural networks were used extensively in the past years for musical applications and they have been relatively successful especially in domains such as perception and cognition.

Automatic harmonization – Knowledge Based Systems  

The most pioneer technique used for music composition is using knowledge base concepts. Unlike neural network that learns from examples, knowledge based systems work according to the rules. There can be three types of knowledge based systems 1) rule based expert systems, Constraint logic programming and case based reasoning. There has been several approaches on automatic harmonization using rule based approach. Ebcioglu implemented a choral harmonizing system which is a rule based expert system using Backtracking Specification Language (Ebcioglu, 1988). Tsang and Aitken (1991) and Pachet and Roy (1998) used constraint logic programming (CLP) and constraint satisfaction techniques (CSP) respectively for four part choral harmonization. However, Sabater et al also developed four part automatic harmonization for vocals using rules and cased based reasoning but his approach failed due to constructing the harmonization purely on rules. Reason for his failure was because rules don’t make music but music makes rules. The advantage of using a knowledge based approach is because each newly correctly harmonized piece can be memorized and made available as a new example to harmonize other melodies which are also known as learning by experience process (Mantaras & Arcos, 2002). When analyzing the above existing work the main advantages of rule based systems are, their ability to be stand-alone programs, and their ability to explain their choice of action; furthermore, knowledge based systems can introduce explicit structures or rules. Even though knowledge based systems can be a good candidate for musical composition Papadopoulos and Wiggins claims that following disadvantages can be seen in knowledge based systems. 1) Knowledge elicitation is difficult and time consuming, 2) gaps between expert and programmer because representation is not flexible and also knowledge based systems can be too complicated with more exceptions to the rules.

Automatic harmonization – Mathematical Model

 

Mathematical model also known as Stochastic processes is a machine learning technique which is widely used in many applications and research fields such as; data mining, speech recondition, hand writing recondition and computer vision. One of the most popular mathematical models is Hidden Markov Model; it has been used successfully in genre classification, instrument identification and key estimation in the field of music systems (Levitt, 1993). There are few approaches taken for automatic harmonization using Hidden Markov Model. Allan and William designed a harmonization model for Bach chorales using Hidden Markov Models. In this model, the visible states represent melody notes and the hidden states are chords. This model has to be provided sequences of chords as input, restricting its applicability in more general settings (Allan & Williams, 2005). Hanlon and Ledlie have developed CPU Bach an automatic choral harmonization system which harmonizes melody lines for the style of composer Bach. This system breaks the harmonization process in to two parts 1) generation of a harmonic progression and 2) realization of harmonic progression into four part melody lines. Hanlon and Ledlie use HMM to model the chord progression and derive the realization of vocal lines using constraint satisfaction method. That has been successful for a large number of melodies (Hanlon & Ledlie, 2002). Papadopoulos and Wiggins claims the main draw back of this models are one must needs to find the probablities by analyzing many data therefore, lot of data is needed and also it is difficult to capture higher or more abstract levels of music (Papadopoulos & Wiggins, 2007). However, HMM has been sucessfully used in many applications as disscussed above. Furthermore, HMM is more robust and flexible compared to other models, and also its complexity is low.

Automatic harmonization – Genetic Algorithms

There have been several researches done in automatic composition using Genetic Algorithms (GA). Horner and Ayres have successfully developed a system that generates four-part harmony using genetic algorithms. However, the chords needed to be given and the system produces the four melody lines according to the chords (Horner, 1995). Phon-Amnuaisuk and Wiggins created another harmonizing system using GA in their system, soprano information is input by the users and the GA generates the other three voices with musical domain knowledge encoded in the fitness function. The limitations of using GA are that these systems are subjective and there is no way to simulate human behavior; furthermore, the user must hear all the potential solutions in order to evaluate a population. However, Biles claims that GA has efficient search method which is good for large scale searches and it has the ability to provide multiple solutions. (Bills, 2007).

Musical composition has been the focus of computer science since the 50s, and there exists several ways of representing music using computers. There have been several AI applications for automatic music compositions from early ages. When looking at AI applications, they can be categorized according to their most prominent features; namely, 1) Neural Networks (systems which learn), 2) Knowledge Based systems, 3) Evolutionary methods, and 4) Mathematical Models. Each of these models has their own pros and cons which were discussed in the above sections.