Recommendation engine on top hdp-hadoop-cluster and mahout

[Samuel Cozannet]

Hi All,

Following the announcement of the latest HDP Hadoop bundle by LazyPower and
Amir, I built a demo to run a recommendation engine on top of it.

You can find the code here :
https://github.com/SaMnCo/juju-hadoop-recommendation-engine

Notes:
* The script will install Mahout and Snappy compression lib. This will in
the end be included in the charm.
* It has to run once to learn the recommendation from the source file
* The output is a big matrix and there is a small python script provided to
extract meaningful results
* Wait until the YARN resource manager is started before installing and
running
* Tested on AWS only
* The initial workload uses 100k movie recommendations but there are 1M and
10M files available (see config in the demo script)
* The inspiration for this was built for Hadoop Vanilla. Most things can be
ported to that environment as well.

If anyone is willing to help on a small web GUI, that could make a nice
demo for our beloved SEs.

Hope you like it!
Best,
Sam

-- 
Samuel Cozannet
Cloud, Big Data and IoT Strategy Team
Strategic Program Manager
Changing the Future of Cloud
Ubuntu <http://ubuntu.com> / Canonical <http://canonical.com> UK LTD
samuel.cozannet at canonical.com
+33 616 702 389
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju/attachments/20140925/2829993e/attachment.html>