Thing which seemed very Thingish inside you is quite different when it gets out into the open and has other people looking at it

Tuesday, September 27, 2011

Kick Start on Music Programming – Create your first music program in java

In this tutorial I am going to talk about basics of music programming and music technology. Music programming is an interesting but a very vast area of learning and applying however, jMusic project makes music programmer’s life much easier and makes music programming more effective. jMusic library is an API for Java music programming and provides tools to for music compositional and audio processing.

Today I am going to talk about the fundamentals of music programming using jMusisc library and compose, and process and monitor simple music.

Today's TODO list ...
  1. Create a Simple Song
  2. Add Instrumentals
  3. Save and notate
Create Your First Song 

Creating your first song is crucial :) If you know the music notation to a particular song you can create almost any song using this technique (make sure you select your favorite song for this task) . Since most people know twinkle twinkle little star and its quite catchy I am going to select that song.

Before we start we need to make sure we have jMusic library, you need to download and import this library to your java class path.
First look at the manuscript notation of our songs.

For those who are not fluent with music notations and western music theory this is how it looks likes in abc format. (More like c,d,e,f,g format :) )


Now let's create this song in java.
package mymusicapp; 
import jm.JMC; 
import jm.music.data.Note;  
import jm.music.data.Part; import jm.music.data.Phrase; import jm.music.data.Score; import jm.util.Play;
import jm.util.Write;
public class Main implements JMC {
public static void main(String[] args) { 
Phrase phr = new Phrase("Twinkle twinkle", 0.0); int[] pitchArray = {C4,C4,G4,G4,A4,A4,G4,F4,F4,E4,E4,D4,D4,C4,G4,G4,F4,F4,E4,E4,D4,G4,G4,F4,F4,E4,E4,D4};
double[] rhythmArray = {C, C, C, C, C, C, M, C, C, C, C, C, C, M, C, C, C, C, C, C, M, C, C, C, C, C, C, M}; phr.addNoteList(pitchArray, rhythmArray);
Play.midi(phr);}
}
}
As you can see it is very easy to create music using jMusic library. All you need to do is add the notes and the pitch classes and put them together. If you look at it closely, Notes are given in “pitchArray” int array of pitch classes in C,D,E,F,G manner number 4 represents the fourth octave. And the duration of each pitch classes are given Crochet ( C ), Quaver (Q), Semi Quaver (SQ), Minim (M) ect.
Following table list duration of each note.


Musical Notation Name Duration jMusic Notation
Semibreve

Whole Note
4 Crotchets
4.0 SB
Minim

Half Note
2 Crotchets
2.0 M


Crotchet

Quarter Note
1 Crotchets
1.0 C
Quaver

Eight Note
½ a Crotchet
0.5 Q
Semi-Quaver

16th note
¼ a Crotchet
0.25 SQ


Add Instrumentals

Changing the instrument is easier than creating the song all you need to do is map your phrase to a part and play the part as shown below.

phr = new Phrase("Twinkle twinkle", 0.0);
Part p = new Part("FLUTE", FLUTE, 0);  
phr.addNoteList(pitchArray, rhythmArray);
p.add(phr);
Play.midi(p);

You can try and experiment with different musical instruments such as guitar violin ect. :)


Save and notate

You can save your creation as a midi file by simply adding the following code.
Write.midi(phr,"twinkle.mid");


Click here to listen to our creation

To view the manuscript notation you can use View.notate method and you can view the manuscript notation of your music creation.

View.notate(phr);

Saturday, September 24, 2011

Extracting RDF Data using WSO2 Data Services. (How to extract aircraft information from NASA rdf data sources)

In my last post I explained how to expose your data in cloud as RDF resources. Today I am going to explain how we can query RDF resources on the web/cloud and expose extracted data as a service. First of all as usual to expose our data as a service we need to create a data service using wso2 data services server.

To demonstrate RDF data extraction I am going to use a popular RDF data source which stores interesting information about NASA aircraft details. And we are going to extract aircraft information according to the agency. Following diagram shows how NASA keep their aircraft information as a data source.


If you click on RDF/XML on the top right hand side corner link you can view the RDF source of the the data.


To create data services you need to either download and install wso2 data services server or you can straight away create data services using Stratoslive cloud platform.

Once you login to data services server, click on create under Web Services -> Data Services -> add. Give an appropriate name and click on next.

Once you login to data services server, click on create under Web Services -> Data Services -> add. Give an appropriate name and click on next.

Then you need to create an RDF data source using our NASA rdf datasource. Give the DataSource Id, Data Source Type, and RDF File Location.

RDF File Location - http://nasa.dataincubator.org/~search.rdf?query=all


Click on next and create new query to create a new query. In order to query RDF data we need to write queries using SPARQL query language which is similar to SQL.The following SPARQL query is used to extract aircraft information

PREFIX space: <http://purl.org/net/schemas/space/> 

PREFIX relevance: <http://a9.com/-/opensearch/extensions/relevance/1.0/>

PREFIX foaf: <http://xmlns.com/foaf/0.1/>

PREFIX dc: <http://purl.org/dc/elements/1.1/>

SELECT ?homepage ?name ?alternateName ?internationalDesignator ?mass ?score ?launch ?agency ?description

WHERE {

?craft foaf:homepage ?homepage.

?craft foaf:name ?name.

?craft space:alternateName ?alternateName.

?craft space:internationalDesignator ?internationalDesignator.

?craft space:mass ?mass.

?craft relevance:score ?score.

?craft space:launch ?launch.

?craft space:agency ?agency.

?craft dc:description ?description.

}

Click on add Input mapping to give input mapping parameters. Since we are going to give agency as a input parameter we need to specify it in our rdfQuery. And go to main configuration.

Give the Query information and the SPARQL query as shown below.

Now we need to arrange how we display the results in the Results (output mapping section).

  • Output type – xml
  • Grouped by element - Aircrafts
  • Row name – Aircraft

Click on add new output mappings to add output mapping elements. Give the mapping type, output field name and data source column name as shown in the table below.

Mapping TypeData Source TypeData Source Column Name
elementhomepagehomepage
elementnamename
elementalternateNamealternateName
elementinternationalDesignatorinternationalDesignator
elementmassmass
elementscorescore
elementlaunchlaunch
elementagencyagency
elementdescriptiondescription


Click on save to save the query. Now we'l add an operation to get our extracted data called getAircrafts. Click next -> add New Operation. And give the query name and the operation name. Click on finish to finish creating the data service.

Click on finish to deploy the data service. Once you click on finish you can see your deployed data service under service list as shown below.

To try this service click on our try-it feature. Lets test our service by giving “United States” as the input of our service. You can see we can get all the aircraft details coming from United States agency.

Friday, September 23, 2011

Expose your cloud data as RDF Resources

Since its all about semantic web 3.0 and RDF Data linking, I am going to explain about RDF data and exposing RDF data in the cloud space in 5 to 10 mins :) Just by using WSO2 Stratos Data Services Server.

The Resource Description Framework (RDF) is one of the most powerful technique to expose and interlink data(knowledge) in the decentralized world. It is also the latest trend in publishing and consuming linked data on the cloud therefore, lets discuss how we can expose our data as a RDF resource in the cloud using WSO2 Stratos Data Services Server.

1) use the RDF data model to publish structured data on the Web

RDF data model consist of set of statements which has a way of publishing link data on the web as triplets (with the use of subject predicate and object). In simple terms RDF model is a way of representing machine understandable data on the web as shown in the diagram below.

2. use RDF links to interlink data from different data sources
All things described by RDF are called resources, RDF links represents the linkage between one resource to another which is mainly done by the use of URIs.

-------------------Simple RDF file --------------
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns# xmlns:cd="http://www.product.fake/cd#">
<rdf:Description rdf:about="http://www.product.fake/cd/S10_1678 ">
<cd:productCode>S10_1678</cd:productCode>
<cd:productName>1969 Harley Davidson Ultimate Chopper</cd:productName>
<cd:productLine>Motorcycles</cd:productLine>
<cd:quantityInStock>7933</cd:quantityInStock>
<cd:buyPrice>48.81</cd:buyPrice>
</rdf:Description>
</rdf:RDF>

Now that we have a brief understanding on RDF and the importance of RDF data, lets see how we can generate RDF data source from a Google spread sheet.

First you need to create a google spread sheet of your choice which has some sensible information. To get the full usage of RDF you need to create several rdf

resource for the linking purposes however, for clarity purposes I will demonstrate how to create a single RDF resource and link it with an existing RDF resources.

Lets expose a google spreadsheet with product information on vehicle sales.


    Product – Describe the currently available products in a car sale vendor.

    IDModelClassificationQty
    S10_16781996 Moto Guzzi 1100iMotorcycles12
    S10_19492003 Harley-Davidson Eagle Drag BikeClassic Cars23
    S10_20161972 Alfa Romeo GTAMotorcycles18
    S10_46981962 LanciaA Delta 16VMotorcycles15
    S10_47571968 Ford MustangClassic Cars13
    S10_49622001 Ferrari EnzoClassic Cars12
    S12_10991968 Ford MustangClassic Cars4
    S12_11082001 Ferrari EnzoClassic Cars10



Lets assume we have another set of RDF resources on product line ( which has information on each product line type) ie http://productLines/car , http://productLines/cycle, http://productLines/bus

Now lets create a data service to expose our Spreadsheet data as a rdf resource. In order to expose these data in the cloud you need to have a stratoslive account. Once
you create your stratos live account you can access set of stratos services such as Enterprise Service Bus, Application Server, Data Services etc (to try out stratos services you can easily create a demo account for free )

After creating your stratos account you can easily logged into your tenant domain and start working in the cloud!!!!
Now lets go back to exposing spreadsheet data as a service ... In order to do that we need to use WSO2 Stratos Data Services Server which provides a
powerful set of feature to expose data as a service and set of service utility methods. To access data services go to stratos live manager home page and click on wso2 stratos Data services.

To create a data service go to the left side menu bar and click on create under webservices->Add->Data Services. Then you will get a wizard as shown below. Give a proper data service name and click on next.


Once you click on next you will be directed to add data source page. And give information regarding the google spreadsheet you created along with your credentials


You can click on test connection to confirm your connection.

Click on next to go to the Query page. Query page describe the extracting algorithm to extract your data from the data source (google spreadsheet). Lets extract ProductID, Model,Classification and Qty.

Since our output is RDF result set, we need to specify our output type as RDF. RDF Base URI is the format of rdf:about URI which uniquely identifies each resource.

We will give RDF base URI as http://www.product/cd/{1}; this takes the Spreadsheet column 1(which is the ID) value for each row and replaces it for the RDF about attribute inside rdf:Description element

Output Type – RDF
RDF Base URI :- http://www.product/cd/{1}
Row namespace :- http://www.product/cd#

To generate the response in RDF format click on "Add New Output Mappings" button. There are two mapping types in RDF Output mapping. 1) as a element, 2) as a resource.When mapping an element as a resource, you need to give the resource URI along with the column name which needs to be mapped in curly brackets as shown below. This way we can link two RDF resources together and create a relationship between each other.

Lets map ID, Model and Qty as elements and Classification as a resouce, Lets link classification column to the productline resouces as i mention earlier ( http://productLines/car , http://productLines/cycle, http://productLines/bus )

Mappings of RDF resource

Resource URI http://productLines/{3} (as you can see we put the column 3 to get each classification type of the product).

Resouce Field Name - Classification

Mappings of RDF element

Following diagram shows the output mappings which we mapped from google spreadsheet to RDF resource.

Once we create the the query click on next to add Resources. Since we are exposing data as RDF resource we need to create a resource to expose the data. Lets give our query information when creating the resource.


Resouce Path – Products
Resource Method – Get
Query ID – RDFQuery

Click on finish to deploy the data service. Once you click on finish you can see your deployed data service under service list as shown below.


Now that we created our RDF resource we can test it by accessing it as a rest call or by using the try its feature.

Rest URL https://data.stratoslive.wso2.com/services/t/amani123.com/RDFDataSource/_getproducts (replace the tenant name amani123.com with your tenant domain)

You can validate this RDF resource by using the online RDF validator by copy pasting the rdf resource (right click on the page and view page source copy paste it inside the validation)

Now we exposed our spreadsheet data in the cloud space just within 10 mins :) you can create more rdf data sources using the same manner with different data sources (csv/excel/rdbms) and expose those data as RDF data sources. I will further explain how we can extract RDF data using SPARQL in my next blog post :)

Thursday, September 22, 2011

Extracting Web information using WSO2 Data Services

This blog explain how to scrape web data using wso2 data services web harvesting feature. In this tutorial I am going to extract Top rated books from Top Rated Books - Book Movement web page along with their authors and expose those data as a data service.

Before I begin lets look at how scraping works.

When you scrape a web page you need to identify the html/xml pattern. If we look at Top Rated Books - Book Movement page you can see list of books are listed along with their details. Now if we look at the page source we can see several html tags are repeated in the same manner.


If you look at it closely you can see a wrapper element which is

<div class=”rgLayoutCenter”>

and inside that wrapper element you have some thing like this.

<div class="rgLayoutTitle">First They Killed My Father: A Daughter of Cambodia Remembers (P.S.)</div>  

<div class="rgLayoutAuthor">by Loung Ung</div>

Now our basic requirement is to extract all the books along with their authors. To do that we need to find the pattern of the book title as wel as the author.

<div class="rgLayoutTitle">First They Killed My Father: A Daughter of Cambodia Remembers (P.S.)</div>  

<div class="rgLayoutAuthor">by Loung Ung</div>

we can easily write an xslt template to extract these information

<?xml version="1.0" encoding="ISO-8859-1"?>

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

<xsl:output method="xml" omit-xml-declaration="yes" indent="yes"/>

<xsl:template match="/">

<BookInfo>

<xsl:for-each select="//div[@class='rgLayoutCenter']">

<Book>

<Title><xsl:value-of select="div[@class='rgLayoutTitle']"/></Title>

<Author><xsl:value-of select="div[@class='rgLayoutAuthor']"/></Author>

</Book>

</xsl:for-each>

</BookInfo>

</xsl:template>

</xsl:stylesheet>

Above xslt template describe to go inside each

<div class=”rgLayoutCenter”> 
element and extract value of
<div class="rgLayoutTitle">
and
<div class="rgLayoutAuthor">
and assign it to the XML elements Title and Author inside the wrapper Book element.

Creating the web harvest data service

Now we learnt the basic concepts on web scraping/web harvesting, we will straight away create the data services using WSO2 Data services server.

To install WSO2 data services download and unzip the zip file and go to $DS_HOME/bin and start up the server from the command prompt, run bin/wso2server.bat{sh}


When the server startup is complete, access http://localhost:9443/carbon in your browser. Sign in to the server using the default credentials (username=admin, password=admin) in the right hand side corner. It will redirect you to the management console page.

To create the data service click on create in the left hand side menu under Web Services -> Add -> Create

Once you click on it you will have to fill the data service information as shown below. Lets name our data service as WebHarvestDS.


Once you click on next you will be redirected to create data-source page click on add new datasource to add our web datasource.

For the web datasource you need to have a configuration file along with the web scraping url, the scraperVariable and the HTTP method to extract data.

Also we need to provide our template.xslt file location we wrote earlier.

<?xml version="1.0" encoding="UTF-8"?>

<config>

<var-def name='bookInfo'>

<xslt>

<xml>

<html-to-xml>

<http method='get' url='http://www.bookmovement.com/app/readingguide/memberRecommendations.php'/>

</html-to-xml>

</xml>

<stylesheet>

<file path="/media/ntfs/web/template.xsl"/>

</stylesheet>

</xslt>

</var-def>

</config>

Place the above configuration file inside the inline configuration section (or you can save the above configuration in your local machine and give it as a web harvest config file path)

Lets give our scraper variable as bookInfo and HTTP method as get and also our template location.


Creating the Query

Click on next to add a query. Give a queryId, scraper variable as webQuery and bookInfo.

To populate the data properly we need to give the group by element and row-name. This is to tell the web service how our result format should be. We also need to give output mapping to arrange our extracted data.

Query information.

  • QueryId – webQuery
  • Data Source – web
  • Scraper Variable-bookInfo
  • Grouped by element – Books
  • Row name – Book

output mappings

  • Element – Title
  • Element – Author


Click on Next to add operation and select the query name and give a name to our operation.


Click on finish to finish creating the data service. Go to webservices list and you can see our webservice is successfully deployed.



You can invoke the service using the try-it tool.


You can also invoke the data service using a rest call by simply typing http://localhost:9763/services/WebScraping/getBooks/ on ur browser.

Saturday, September 17, 2011

Kick start on WSO2 Stratoslive


Since we are going to see lots and lots of WSO2 Stratos related demonstrations and tutorials and powerful capabilities of cloud computing and stratos, its very important to know how to get started in WSO2 Stratos .

Getting started easy!!! all you need to do is create a StratosLive account (which is TOTALY FREE) and you can start using wso2 stratos live services right away.

Lets see how we can create a stratoslive account.
First go to the stratoslive homepage.
   

To create a new account click on “Get Started Now for Free! ” . It will redirect to a page where you need to give your Domain Information, Usage plan Information, and Contact Details.


To created a domain (a user account/ tenant) you need to give a unique tenant domuain. You can test
the uniqueness of your domain by clicking “check availability” button.
Once you give your domain name you need to select the usage plan for your domain ... there are four types of usage plans you can click on pricing details to get ll be allocated for you. For exploring StratosLive features and understanding the power of cloud computing you can select the demo version (Which is the free version).
After that you need to give your contact details for the registration purposes. Give your personal information along with your tenant-admin user name password and the verification code.
Once everything is filled properly submit the document and you will be redirected to a success page.


To activate your tenant/domain you will get a email with infomation on how you can activate your account. Once you activate you can login to your account simply by giving the username@tenantdomain and the tenant admin password you gave while registering your tenant.

When you login it will redirect to Stratos Manager home page which has all the links to StratosLive services.

To get more information regarding these services click on help it will give a brief description about each service.



To try out services by clicking on each service and have fun with cloud computing!!


Friday, September 2, 2011

Expose your databases as a service in two steps!!!



WSO2-Data Services Server provides the feature to expose your data as a service right away!!!. All you need to give is your database information and then generate data services to expose your database tables/schemas as per your need.

If you are using wso2 products for the first time ...

You need to download wso2 dataservices server.
Start up the server -> go to DS_HOME/bin/wso2server.bat | wso2server.sh (DS_HOME is where your set up is located)
Once the server is up and running open a web browser and navigate to https://localhost:9443/carbon.
Login it the server using the default credentials (username=admin, password=admin).

Also note that you need to provide the related jdbc driver for the database and put in repository/component/lib in order for this feature to work.

Step 1 :- Create a data source

In order to expose your database you need to create a data source using wso2 data services server.
To create data source click on Configure -> Data Sources



Click on add data source and give your data base information.


Once you create your data source you can test your connection by giving a validation query (ie Select 1) to make sure your connection is successful.

Step 2 Generating data services

You can start by clicking on the generate link in the main menue. Once you click on generate you will be directed to a wizard which will ask for the datasource and the database name which will be needed to expose.

Once you create you give your database name you will get the list of schemas/tables to select for your data service.




Once you select the tables of your choice then you need to decide wether you need a single service for each table or multiple services per table.

Click on next to view the created data services
Click on finish to view the deployed services.
You can try your service by clicking on try this sample for database transactions :) You can also generate axis2 clients by clicking on the data service.
Try your service