Thing which seemed very Thingish inside you is quite different when it gets out into the open and has other people looking at it

Sunday, October 9, 2011

Adding Chords to your melody using jMusic (Adding Accompaniments)


In my last post I showed how we can easily create a song (twinkle twinkle little star), which is basically the tune of that song or in western music language we call it the melody, in this post I am going to talk about how we can accompany this melody by adding chords.
Let’s see how we can accompany our melody ..

This is in our today’s TODO list
-          Adding Chords to our song
-          Arranging the chords in choral fashion
-          Adding  guitar chords (o any other instruments)

Adding Chords to the song “Twinkle twinkle”


 
Following diagram shows the chord arrangements for the song twinkle Twinkle however this is not static you can select appropriate chord of your choice depending on the mood and the sound you want. However let’s just use a very basic chord progression to our song.

Since we already created our song in my last post I will use the same code to add chord progression.

When we were creating the notes we needed a pitch array which is an integer array to store the notes(pitch classes), likewise we need Chord array to keep Chord structure.  

A Chord represents 3  or 4 notes together playing at once.  So I am going to create array of three notes for each of our chord. 

Here we need  3 Types of  Chords
Chord Notes
CMaj C E G
FMaj F A C
GMaj G B D

Adding  Chords is bit complicated than just creating notes therefore, I will explain simple as possible. 

Here we are going to have two parts. First one is our melody line(tune) which we created earlier. And the other part is the base part (the chords) to accompany our melody.

   Phrase phr = new Phrase("Twinkle twinkle", 0.0);
        Part treblePart = new Part("PIANO-Right", PIANO, 0);
        int[] pitchArray = {C4, C4, G4, G4, A4, A4, G4, F4, F4, E4, E4, D4, D4, C4, G4, G4, F4, F4, E4, E4, D4, G4, G4, F4, F4, E4, E4, D4};
        double[] rhythmArray = {C, C, C, C, C, C, M, C, C, C, C, C, C, M, C, C, C, C, C, C, M, C, C, C, C, C, C, M};
        phr.addNoteList(pitchArray, rhythmArray);
        treblePart.add(phr);

And then we create the base part
static Part bassPart = new Part("PIANO-Left", PIANO, 0);

To create chords we need to define each type of chord we are using. First we’l creates our three types of chords.

        Note cMaj[] = {new Note(C3, M), new Note(E3, M), new Note(G3, M)};
        Note fMaj[] = {new Note(F3, M), new Note(A3, M), new Note(C3, M)};
        Note gMaj[] = {new Note(G3, M), new Note(B3, M), new Note(D3, M)};


Earlier I created a Phrase  to store the pitch classes (notes) and their durations (time slots) here we need to create a  CPhrase chord = new CPhrase(); to store our chords. 
And we add each chord to the basePart to complete the phase.

So to make this code much efficient and elegant I am going to create a separate method which adds a given chord to our basePart.

public static void addChordsPart(Note chrd[]) {
        CPhrase chord = new CPhrase();
        chord.addChord(chrd);
        bassPart.addCPhrase(chord);
    }

Getting things all together
  1. Create two parts
  2. Add the melody to the first part by giving the pitch classes and durations
  3. Create the types of the chords
  4. Add each chord to the second part by giving the chord name and their duration per each chord
  5. Create the Score and add Part one and part two together
  6. Play/Save the midi

package mymusicapp;

import jm.JMC;
import jm.music.data.CPhrase;
import jm.music.data.Note;
import jm.music.data.Part;
import jm.music.data.Phrase;
import jm.music.data.Score;
import jm.util.Play;
import jm.util.Write;


public class Main implements JMC {

    static Part bassPart = new Part("PIANO-Left", PIANO, 0);

    public static void main(String[] args) {
        Phrase phr = new Phrase("Twinkle twinkle", 0.0);
        Part treblePart = new Part("PIANO-Right", PIANO, 0);
        int[] pitchArray = {C4, C4, G4, G4, A4, A4, G4, F4, F4, E4, E4, D4, D4, C4, G4, G4, F4, F4, E4, E4, D4, G4, G4, F4, F4, E4, E4, D4};
        double[] rhythmArray = {C, C, C, C, C, C, M, C, C, C, C, C, C, M, C, C, C, C, C, C, M, C, C, C, C, C, C, M};
        phr.addNoteList(pitchArray, rhythmArray);
        treblePart.add(phr);

        Note cMaj[] = {new Note(C3, M), new Note(E3, M), new Note(G3, M)};
        Note fMaj[] = {new Note(F3, M), new Note(A3, M), new Note(C3, M)};
        Note gMaj[] = {new Note(G3, M), new Note(B3, M), new Note(D3, M)};

        addChordsPart(cMaj);
        addChordsPart(cMaj);
        addChordsPart(fMaj);
        addChordsPart(cMaj);
        addChordsPart(fMaj);
        addChordsPart(cMaj);
        addChordsPart(gMaj);
        addChordsPart(cMaj);
        addChordsPart(cMaj);
        addChordsPart(fMaj);
        addChordsPart(cMaj);
        addChordsPart(gMaj);
        addChordsPart(cMaj);
        addChordsPart(fMaj);
        addChordsPart(cMaj);
        addChordsPart(gMaj);

        Score score = new Score("Twinkle-Twinkle");
        score.addPart(treblePart);
        score.addPart(bassPart);

        Play.midi(score);
        Write.midi(score);

    }

    public static void addChordsPart(Note chrd[]) {
        CPhrase chord = new CPhrase();
        chord.addChord(chrd);
        bassPart.addCPhrase(chord);
    }
}


Now we added chords to our songs. You can try out the songs by simply playing the song.

Arranging  the chords.

Just playing the chord progression for a song can be lil boarding. To make the song more interesting we can arrange the base chords into different variations. 

Following diagram shows how we can arrange the chords using each note of the chord.

So for that we need to change our method a little bit by giving the notes of each chord separately and  arranging them with proper durations.

 public static void addbaseNotesPart(Note chrd[]) {
        Phrase chord = new Phrase();
        int[] pitchArray = {chrd[0].getPitch(), chrd[2].getPitch(), chrd[1].getPitch(), chrd[2].getPitch()};
        double[] rhythmArray = {Q, Q, Q, Q};
        chord.addNoteList(pitchArray, rhythmArray);
        bassPart.addPhrase(chord);
    }

We can use the same code, but instead of using addChordsPart, use addbaseNotesPart to get the styling of our chord progression.


Adding  guitar chords (o any other instruments)

This Chord Arrangements can be done using any instrument. All you need to do is change the  Part instrument to the instrument of your choice.
  static Part bassPart = new Part("PIANO-Left", GUITAR, 0);

You can also experiment by changing the chords/durations and adding new parts. 







Thursday, October 6, 2011

Steve Jobs: tributes to the Apple co-founder and a very Inspirational man!!!

"3 Apples changed the World. 1st one seduced Eve, 2nd fell on Newton and 3rd was offered to the World half bitten by Steve Jobs"


Steve Jobs : You are a great inspiration to everyone ... your work is un-beleivable your creativity is endless and your imagination is real which makes our day today life much more exiting and easy... You sure change the world and touched many people's hearts. You are a true entrepreneur showed technology is something which should be easy and it sure need good taste!!!! You enjoy the process as much as the success!!!

These are the two most favourite quotes which inspired my life ...

“Your work is going to fill a large part of your life, and the only way to be truly satisfied is to do what you believe is great work. And the only way to do great work is to love what you do. If you haven’t found it yet, keep looking. Don’t settle. As with all matters of the heart, you’ll know when you find it. And, like any great relationship, it just gets better and better as the years roll on. So keep looking until you find it. Don’t settle.”-Steve Jobs

"No one wants to die. Even people who want to go to heaven don't want to die to get there. And yet, death is the destination we all share. No one has ever escaped it, and that is how it should be, because death is very likely the single best invention of life. It's life's change agent. It clears out the old to make way for the new." - Steve Jobs '


Steve Jobs : We are going to miss you alot and May you Attain Nibbana...


Tuesday, September 27, 2011

Kick Start on Music Programming – Create your first music program in java

In this tutorial I am going to talk about basics of music programming and music technology. Music programming is an interesting but a very vast area of learning and applying however, jMusic project makes music programmer’s life much easier and makes music programming more effective. jMusic library is an API for Java music programming and provides tools to for music compositional and audio processing.

Today I am going to talk about the fundamentals of music programming using jMusisc library and compose, and process and monitor simple music.

Today's TODO list ...
  1. Create a Simple Song
  2. Add Instrumentals
  3. Save and notate
Create Your First Song 

Creating your first song is crucial :) If you know the music notation to a particular song you can create almost any song using this technique (make sure you select your favorite song for this task) . Since most people know twinkle twinkle little star and its quite catchy I am going to select that song.

Before we start we need to make sure we have jMusic library, you need to download and import this library to your java class path.
First look at the manuscript notation of our songs.

For those who are not fluent with music notations and western music theory this is how it looks likes in abc format. (More like c,d,e,f,g format :) )


Now let's create this song in java.
package mymusicapp; 
import jm.JMC; 
import jm.music.data.Note;  
import jm.music.data.Part; import jm.music.data.Phrase; import jm.music.data.Score; import jm.util.Play;
import jm.util.Write;
public class Main implements JMC {
public static void main(String[] args) { 
Phrase phr = new Phrase("Twinkle twinkle", 0.0); int[] pitchArray = {C4,C4,G4,G4,A4,A4,G4,F4,F4,E4,E4,D4,D4,C4,G4,G4,F4,F4,E4,E4,D4,G4,G4,F4,F4,E4,E4,D4};
double[] rhythmArray = {C, C, C, C, C, C, M, C, C, C, C, C, C, M, C, C, C, C, C, C, M, C, C, C, C, C, C, M}; phr.addNoteList(pitchArray, rhythmArray);
Play.midi(phr);}
}
}
As you can see it is very easy to create music using jMusic library. All you need to do is add the notes and the pitch classes and put them together. If you look at it closely, Notes are given in “pitchArray” int array of pitch classes in C,D,E,F,G manner number 4 represents the fourth octave. And the duration of each pitch classes are given Crochet ( C ), Quaver (Q), Semi Quaver (SQ), Minim (M) ect.
Following table list duration of each note.


Musical Notation Name Duration jMusic Notation
Semibreve

Whole Note
4 Crotchets
4.0 SB
Minim

Half Note
2 Crotchets
2.0 M


Crotchet

Quarter Note
1 Crotchets
1.0 C
Quaver

Eight Note
½ a Crotchet
0.5 Q
Semi-Quaver

16th note
¼ a Crotchet
0.25 SQ


Add Instrumentals

Changing the instrument is easier than creating the song all you need to do is map your phrase to a part and play the part as shown below.

phr = new Phrase("Twinkle twinkle", 0.0);
Part p = new Part("FLUTE", FLUTE, 0);  
phr.addNoteList(pitchArray, rhythmArray);
p.add(phr);
Play.midi(p);

You can try and experiment with different musical instruments such as guitar violin ect. :)


Save and notate

You can save your creation as a midi file by simply adding the following code.
Write.midi(phr,"twinkle.mid");


Click here to listen to our creation

To view the manuscript notation you can use View.notate method and you can view the manuscript notation of your music creation.

View.notate(phr);

Saturday, September 24, 2011

Extracting RDF Data using WSO2 Data Services. (How to extract aircraft information from NASA rdf data sources)

In my last post I explained how to expose your data in cloud as RDF resources. Today I am going to explain how we can query RDF resources on the web/cloud and expose extracted data as a service. First of all as usual to expose our data as a service we need to create a data service using wso2 data services server.

To demonstrate RDF data extraction I am going to use a popular RDF data source which stores interesting information about NASA aircraft details. And we are going to extract aircraft information according to the agency. Following diagram shows how NASA keep their aircraft information as a data source.


If you click on RDF/XML on the top right hand side corner link you can view the RDF source of the the data.


To create data services you need to either download and install wso2 data services server or you can straight away create data services using Stratoslive cloud platform.

Once you login to data services server, click on create under Web Services -> Data Services -> add. Give an appropriate name and click on next.

Once you login to data services server, click on create under Web Services -> Data Services -> add. Give an appropriate name and click on next.

Then you need to create an RDF data source using our NASA rdf datasource. Give the DataSource Id, Data Source Type, and RDF File Location.

RDF File Location - http://nasa.dataincubator.org/~search.rdf?query=all


Click on next and create new query to create a new query. In order to query RDF data we need to write queries using SPARQL query language which is similar to SQL.The following SPARQL query is used to extract aircraft information

PREFIX space: <http://purl.org/net/schemas/space/> 

PREFIX relevance: <http://a9.com/-/opensearch/extensions/relevance/1.0/>

PREFIX foaf: <http://xmlns.com/foaf/0.1/>

PREFIX dc: <http://purl.org/dc/elements/1.1/>

SELECT ?homepage ?name ?alternateName ?internationalDesignator ?mass ?score ?launch ?agency ?description

WHERE {

?craft foaf:homepage ?homepage.

?craft foaf:name ?name.

?craft space:alternateName ?alternateName.

?craft space:internationalDesignator ?internationalDesignator.

?craft space:mass ?mass.

?craft relevance:score ?score.

?craft space:launch ?launch.

?craft space:agency ?agency.

?craft dc:description ?description.

}

Click on add Input mapping to give input mapping parameters. Since we are going to give agency as a input parameter we need to specify it in our rdfQuery. And go to main configuration.

Give the Query information and the SPARQL query as shown below.

Now we need to arrange how we display the results in the Results (output mapping section).

  • Output type – xml
  • Grouped by element - Aircrafts
  • Row name – Aircraft

Click on add new output mappings to add output mapping elements. Give the mapping type, output field name and data source column name as shown in the table below.

Mapping TypeData Source TypeData Source Column Name
elementhomepagehomepage
elementnamename
elementalternateNamealternateName
elementinternationalDesignatorinternationalDesignator
elementmassmass
elementscorescore
elementlaunchlaunch
elementagencyagency
elementdescriptiondescription


Click on save to save the query. Now we'l add an operation to get our extracted data called getAircrafts. Click next -> add New Operation. And give the query name and the operation name. Click on finish to finish creating the data service.

Click on finish to deploy the data service. Once you click on finish you can see your deployed data service under service list as shown below.

To try this service click on our try-it feature. Lets test our service by giving “United States” as the input of our service. You can see we can get all the aircraft details coming from United States agency.

Friday, September 23, 2011

Expose your cloud data as RDF Resources

Since its all about semantic web 3.0 and RDF Data linking, I am going to explain about RDF data and exposing RDF data in the cloud space in 5 to 10 mins :) Just by using WSO2 Stratos Data Services Server.

The Resource Description Framework (RDF) is one of the most powerful technique to expose and interlink data(knowledge) in the decentralized world. It is also the latest trend in publishing and consuming linked data on the cloud therefore, lets discuss how we can expose our data as a RDF resource in the cloud using WSO2 Stratos Data Services Server.

1) use the RDF data model to publish structured data on the Web

RDF data model consist of set of statements which has a way of publishing link data on the web as triplets (with the use of subject predicate and object). In simple terms RDF model is a way of representing machine understandable data on the web as shown in the diagram below.

2. use RDF links to interlink data from different data sources
All things described by RDF are called resources, RDF links represents the linkage between one resource to another which is mainly done by the use of URIs.

-------------------Simple RDF file --------------
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns# xmlns:cd="http://www.product.fake/cd#">
<rdf:Description rdf:about="http://www.product.fake/cd/S10_1678 ">
<cd:productCode>S10_1678</cd:productCode>
<cd:productName>1969 Harley Davidson Ultimate Chopper</cd:productName>
<cd:productLine>Motorcycles</cd:productLine>
<cd:quantityInStock>7933</cd:quantityInStock>
<cd:buyPrice>48.81</cd:buyPrice>
</rdf:Description>
</rdf:RDF>

Now that we have a brief understanding on RDF and the importance of RDF data, lets see how we can generate RDF data source from a Google spread sheet.

First you need to create a google spread sheet of your choice which has some sensible information. To get the full usage of RDF you need to create several rdf

resource for the linking purposes however, for clarity purposes I will demonstrate how to create a single RDF resource and link it with an existing RDF resources.

Lets expose a google spreadsheet with product information on vehicle sales.


    Product – Describe the currently available products in a car sale vendor.

    IDModelClassificationQty
    S10_16781996 Moto Guzzi 1100iMotorcycles12
    S10_19492003 Harley-Davidson Eagle Drag BikeClassic Cars23
    S10_20161972 Alfa Romeo GTAMotorcycles18
    S10_46981962 LanciaA Delta 16VMotorcycles15
    S10_47571968 Ford MustangClassic Cars13
    S10_49622001 Ferrari EnzoClassic Cars12
    S12_10991968 Ford MustangClassic Cars4
    S12_11082001 Ferrari EnzoClassic Cars10



Lets assume we have another set of RDF resources on product line ( which has information on each product line type) ie http://productLines/car , http://productLines/cycle, http://productLines/bus

Now lets create a data service to expose our Spreadsheet data as a rdf resource. In order to expose these data in the cloud you need to have a stratoslive account. Once
you create your stratos live account you can access set of stratos services such as Enterprise Service Bus, Application Server, Data Services etc (to try out stratos services you can easily create a demo account for free )

After creating your stratos account you can easily logged into your tenant domain and start working in the cloud!!!!
Now lets go back to exposing spreadsheet data as a service ... In order to do that we need to use WSO2 Stratos Data Services Server which provides a
powerful set of feature to expose data as a service and set of service utility methods. To access data services go to stratos live manager home page and click on wso2 stratos Data services.

To create a data service go to the left side menu bar and click on create under webservices->Add->Data Services. Then you will get a wizard as shown below. Give a proper data service name and click on next.


Once you click on next you will be directed to add data source page. And give information regarding the google spreadsheet you created along with your credentials


You can click on test connection to confirm your connection.

Click on next to go to the Query page. Query page describe the extracting algorithm to extract your data from the data source (google spreadsheet). Lets extract ProductID, Model,Classification and Qty.

Since our output is RDF result set, we need to specify our output type as RDF. RDF Base URI is the format of rdf:about URI which uniquely identifies each resource.

We will give RDF base URI as http://www.product/cd/{1}; this takes the Spreadsheet column 1(which is the ID) value for each row and replaces it for the RDF about attribute inside rdf:Description element

Output Type – RDF
RDF Base URI :- http://www.product/cd/{1}
Row namespace :- http://www.product/cd#

To generate the response in RDF format click on "Add New Output Mappings" button. There are two mapping types in RDF Output mapping. 1) as a element, 2) as a resource.When mapping an element as a resource, you need to give the resource URI along with the column name which needs to be mapped in curly brackets as shown below. This way we can link two RDF resources together and create a relationship between each other.

Lets map ID, Model and Qty as elements and Classification as a resouce, Lets link classification column to the productline resouces as i mention earlier ( http://productLines/car , http://productLines/cycle, http://productLines/bus )

Mappings of RDF resource

Resource URI http://productLines/{3} (as you can see we put the column 3 to get each classification type of the product).

Resouce Field Name - Classification

Mappings of RDF element

Following diagram shows the output mappings which we mapped from google spreadsheet to RDF resource.

Once we create the the query click on next to add Resources. Since we are exposing data as RDF resource we need to create a resource to expose the data. Lets give our query information when creating the resource.


Resouce Path – Products
Resource Method – Get
Query ID – RDFQuery

Click on finish to deploy the data service. Once you click on finish you can see your deployed data service under service list as shown below.


Now that we created our RDF resource we can test it by accessing it as a rest call or by using the try its feature.

Rest URL https://data.stratoslive.wso2.com/services/t/amani123.com/RDFDataSource/_getproducts (replace the tenant name amani123.com with your tenant domain)

You can validate this RDF resource by using the online RDF validator by copy pasting the rdf resource (right click on the page and view page source copy paste it inside the validation)

Now we exposed our spreadsheet data in the cloud space just within 10 mins :) you can create more rdf data sources using the same manner with different data sources (csv/excel/rdbms) and expose those data as RDF data sources. I will further explain how we can extract RDF data using SPARQL in my next blog post :)

Thursday, September 22, 2011

Extracting Web information using WSO2 Data Services

This blog explain how to scrape web data using wso2 data services web harvesting feature. In this tutorial I am going to extract Top rated books from Top Rated Books - Book Movement web page along with their authors and expose those data as a data service.

Before I begin lets look at how scraping works.

When you scrape a web page you need to identify the html/xml pattern. If we look at Top Rated Books - Book Movement page you can see list of books are listed along with their details. Now if we look at the page source we can see several html tags are repeated in the same manner.


If you look at it closely you can see a wrapper element which is

<div class=”rgLayoutCenter”>

and inside that wrapper element you have some thing like this.

<div class="rgLayoutTitle">First They Killed My Father: A Daughter of Cambodia Remembers (P.S.)</div>  

<div class="rgLayoutAuthor">by Loung Ung</div>

Now our basic requirement is to extract all the books along with their authors. To do that we need to find the pattern of the book title as wel as the author.

<div class="rgLayoutTitle">First They Killed My Father: A Daughter of Cambodia Remembers (P.S.)</div>  

<div class="rgLayoutAuthor">by Loung Ung</div>

we can easily write an xslt template to extract these information

<?xml version="1.0" encoding="ISO-8859-1"?>

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

<xsl:output method="xml" omit-xml-declaration="yes" indent="yes"/>

<xsl:template match="/">

<BookInfo>

<xsl:for-each select="//div[@class='rgLayoutCenter']">

<Book>

<Title><xsl:value-of select="div[@class='rgLayoutTitle']"/></Title>

<Author><xsl:value-of select="div[@class='rgLayoutAuthor']"/></Author>

</Book>

</xsl:for-each>

</BookInfo>

</xsl:template>

</xsl:stylesheet>

Above xslt template describe to go inside each

<div class=”rgLayoutCenter”> 
element and extract value of
<div class="rgLayoutTitle">
and
<div class="rgLayoutAuthor">
and assign it to the XML elements Title and Author inside the wrapper Book element.

Creating the web harvest data service

Now we learnt the basic concepts on web scraping/web harvesting, we will straight away create the data services using WSO2 Data services server.

To install WSO2 data services download and unzip the zip file and go to $DS_HOME/bin and start up the server from the command prompt, run bin/wso2server.bat{sh}


When the server startup is complete, access http://localhost:9443/carbon in your browser. Sign in to the server using the default credentials (username=admin, password=admin) in the right hand side corner. It will redirect you to the management console page.

To create the data service click on create in the left hand side menu under Web Services -> Add -> Create

Once you click on it you will have to fill the data service information as shown below. Lets name our data service as WebHarvestDS.


Once you click on next you will be redirected to create data-source page click on add new datasource to add our web datasource.

For the web datasource you need to have a configuration file along with the web scraping url, the scraperVariable and the HTTP method to extract data.

Also we need to provide our template.xslt file location we wrote earlier.

<?xml version="1.0" encoding="UTF-8"?>

<config>

<var-def name='bookInfo'>

<xslt>

<xml>

<html-to-xml>

<http method='get' url='http://www.bookmovement.com/app/readingguide/memberRecommendations.php'/>

</html-to-xml>

</xml>

<stylesheet>

<file path="/media/ntfs/web/template.xsl"/>

</stylesheet>

</xslt>

</var-def>

</config>

Place the above configuration file inside the inline configuration section (or you can save the above configuration in your local machine and give it as a web harvest config file path)

Lets give our scraper variable as bookInfo and HTTP method as get and also our template location.


Creating the Query

Click on next to add a query. Give a queryId, scraper variable as webQuery and bookInfo.

To populate the data properly we need to give the group by element and row-name. This is to tell the web service how our result format should be. We also need to give output mapping to arrange our extracted data.

Query information.

  • QueryId – webQuery
  • Data Source – web
  • Scraper Variable-bookInfo
  • Grouped by element – Books
  • Row name – Book

output mappings

  • Element – Title
  • Element – Author


Click on Next to add operation and select the query name and give a name to our operation.


Click on finish to finish creating the data service. Go to webservices list and you can see our webservice is successfully deployed.



You can invoke the service using the try-it tool.


You can also invoke the data service using a rest call by simply typing http://localhost:9763/services/WebScraping/getBooks/ on ur browser.