About me

Paolo Iannelli Picture

Paolo Iannelli

Big Daddy at Mega Labs

Amsterdam Area, Netherlands
Information Technology and Services
C, Python, Big Data, Scalability, High Availability, Performance
Expert Software Engineer with more than 12 years of experience.
Strong in critical thinking, problem solving and high performance architectures.
Paolo Iannelli Labs Rss

Added new page for Disco Map Reduce with Bootstrap

Posted on : 03-02-2013 | By : Paolo Iannelli | In : BigData, Python, Software Development

Tags: , , , , , , ,



I just added a new page to my bootstrap implementation of Disco (http://www.discoproject.org) Map/Reduce Framework.

The reason why I’m implementing this nice dashboard (although I’m not a Frontend Developer and I know just enough about this field) is to increase awareness of this amazing tool and increase the overall visibility of the project in the Interwebs.

Source is my fork at GitHub, but be aware that many things might not work or look very crappy : it is a “in-development” work, so if you have any issues blame yourself :)

Here are few screenshots :


Disco Dashboard with Bootstrap

Disco Dashboard with Bootstrap

Disco Failed Job with Bootstrap

Disco Failed Job with Bootstrap


Save job results directly to DDFS with Disco

Posted on : 10-05-2012 | By : Paolo Iannelli | In : BigData

Tags: , , , , , , , ,


If you are looking for a way to save directly the processed data with disco in a DDFS tag, you only need to add this extra parameter:


to you run method.

Example :

After the execution of the job you will be able to find already stored in DDFS a tag named disco:job:results:myNiceJob@xxxx:xxxx:xxxx that you can use as an input for other jobs.

WARNING: is not documented anywhere, but using this option will add a read/write token to the result tag that will limit access to it. If you need to re-use the content of such tag, you will always have to specify the token “job” (that’s the automatically assigned name) to access it. It took me couple of hours of deep investigation to understand why I was getting the error :

Unable to access resource (http://localhost:8989/ddfs/tag/disco:job:results:MyNiceJob@538:9c3fb:b9f17): Incorrect or missing token. (401)

Have fun !