|
[
Permlink
| « Hide
]
Filip Pacanowski added a comment - 27/Jun/09 10:27 PM
I've written a simple test using root square mean as a measure of algorithm accuracy.
Cool! I have just run it with the following result:
$ python run_test_recommendation_engine.sh RMSE for alg1: 1.57456966323 RMSE for alg2: 2.03362124647 Now... what does it mean? :> > Now... what does it mean? :>
Well, it's just average error of algorithm. Instead of suggested arithmetic mean, I used quadric mean, as it's used to rate solutions in Netflix Prize. It also means that alg1 is more accurate than alg2. OK, makes sense. Alg2 was based on "normalized" (not really, but that's how we called it) ratings and it looked worse from the beginning.
If you think this task is finished, feel free to resolve it and check in the changes into SVN. Tested on prod, thus closing the issue.
Retested based on current ratings. With more ratings old algorithm got better, but the new one by Jakub Tlalka beats it badly:
$ python run_test_recommendation_engine.py RMSE for alg1: 1.54905903957 RMSE for alg2: 1.40973588554 ( checked in as http://bitbucket.org/filmaster/filmaster-test/changeset/3207fbf7bff1 ) With the updated version of the new algorithm, the RMSEs look as follows:
RMSE for alg1: ~1.55 RMSE for alg2: ~1.30 Getting better! |
|||||||||||||||||||||||||||||||||||||||||||||||