erlang_js - Awesome
===================
erlang_js
---------
If you haven't heard, the Basho team released [1] erlang_js [2] today. Its a
linked in driver that provides a Spidermonkey JavaScript context to run JS code
for Erlang. This is interesting to me because it avoids the stdio overhead
incurred by the current Map/Reduce system that CouchDB uses. So I did what any
bored hacker would do: threw erlang_js into the CouchDB build system and hacked
the view generation code to use the in-VM contexts.
Numbers
-------
These times are for the "mega view" reported in seconds from raindrop-perf.py
found here [3].
Run Trunk erlang_js
--- ----- ---------
1 13.63 6.89
2 11.16 6.94
3 11.82 6.80
Code
----
Can be found at [4].
Next Up
-------
The communication between Erlang and JS is unnecessarily converting Erlang ->
JSON -> Spidermonkey Objects. I've written the code to go from the external
Erlang representation to Spidermonkey objects directly so I plan on integrating
that in the next couple days to see how these numbers change.
Code Might be Nice
------------------
Just thought that maybe people would be interested in the code that's used to
talk to erlang_js. Its pretty straight forward, though not very elegant on my
side.
% From couch_query_servers.erl
start_doc_map(_Lang, Functions) ->
{ok, Port} = js_driver:new(),
ok = js_driver:define_js(
Port, <<"map_support.js">>, map_support(), 5000
),
lists:foreach(fun(FuncSource) ->
Source = <<"map_funs.push(", FuncSource/binary, ");">>,
ok = js_driver:define_js(Port, Source)
end, Functions),
{ok, Port}.
map_docs(Port, Docs) ->
Results = lists:map(
fun(Doc) ->
Json = couch_doc:to_json_obj(Doc, []),
{ok, Results} = js:call(Port, <<"map_doc">>, [Json]),
lists:map(
fun(FunRs) ->
[list_to_tuple(FunResult) || FunResult <- FunRs]
end,
Results)
end,
Docs),
{ok, Results}.
stop_doc_map(nil) ->
ok;
stop_doc_map(Port) ->
js_driver:destroy(Port).
% EOF
And map_support.js I wrote to avoid having to think to hard on the Erlang side
of things:
// src/couchdb/priv/map_support.js
var map_funs = [];
var results = [];
var emit = function(key, value) {
results.push([key, value]);
};
var map_doc = function(doc) {
var ret = [];
map_funs.forEach(function(func) {
results = [];
func(doc);
ret.push(results);
});
return ret;
};
// EOF
Shoutout
--------
Go, Basho [5], Go!
References
----------
[1]: http://twitter.com/justinsheehy/status/7950307815
[2]: http://bitbucket.org/basho/erlang_js/
[3]: /scripts/2010-01-19-raindrop-perf.py
[4]: http://github.com/davisp/couchdb/tree/erlang_js
[5]: http://basho.com
Copyright Notice
----------------
Copyright 2008-2010 Paul Joseph Davis
License
-------
http://creativecommons.org/licenses/by/3.0/