Re:Show HN: We built an open source tool for journalists to ...
by crazy2be on Tuesday November 18, 2014 @12:06PM
(#8624863)
This is awesome! I've wanted to build something like this for a while, but never gotten the chance :). Better tools like this are essential for helping users accept and reject the correct edits, and making it easier to investigate when reversions might be unjustified.
The only problem I noticed is that clicking the footnotes does not work, which makes it harder to evaluate the legitimacy of an edit (well, it just means I have to have the current article open as well).
Re:Show HN: We built an open source tool for journalists to ...
by maaaats on Tuesday November 18, 2014 @10:21AM
(#8624170)
A tip on the diffing:
We have done something similar when diffing HTML, e.g. replacing the HTML with single unicodes. And then we run the diff and get several diff-segments (EQUALS, INS, DEL). What we have done, is then to scan those for tags, and split them into a new type.
So an insert like INS(something \xE000 else) would become three changes. E.g. INS(something ) INS_TAG(\xE000) INS( else). So the INS_TAG shouldn't be wrapped in <ins> when converting this back to HTML.
Re:Show HN: We built an open source tool for journalists to ...
by xxxyy on Tuesday November 18, 2014 @12:46PM
(#8625117)
I'm not sure if Wikipedia is OK with that, considering the load it generates [1]. I am currently doing some research on Wikipedia, and for my purposes I use the official dumps site at https://dumps.wikimedia.org/
Re:Show HN: We built an open source tool for journalists to ...
by psobot on Tuesday November 18, 2014 @12:51PM
(#8625148)
Load management is a big problem that we're looking to fix in the future. Wikipedia's API definitely struggles when rendering old revisions of medium-to-large pages, but we try our best to respect their own API etiquette guidelines: http://www.mediawiki.org/wiki/API:Etiquette.
In the future, we'd love to combine a mix of the Wikipedia API for real-time edit updates with a copy of the Wikipedia history database stored locally (all 2+ TB of it) to improve performance and decrease load on their servers.
Re:Show HN: We built an open source tool for journalists to ...
by davidgerard on Tuesday November 18, 2014 @06:53PM
(#8627208)
In my experience (press volunteer for Wikimedia), just making journalists aware of the "history" tab is revelatory. It's like they don't see it on the page. So this may well be more effective than you think.
Re:Show HN: We built an open source tool for journalists to ...
by icebraining on Tuesday November 18, 2014 @10:57AM
(#8624414)
The monitoring was already there from the start (it's just being displayed differently), and this application is not part of Wikipedia's model, it's a third-party tool.
If you're complaining that edits are recorded and displayed publicly, you're thirteen years late. That was always the model of any Wiki.
The only problem I noticed is that clicking the footnotes does not work, which makes it harder to evaluate the legitimacy of an edit (well, it just means I have to have the current article open as well).
Hadn't noticed that footnotes issue, but I'll add an issue to the GitHub repo: https://github.com/twg/wikiwash.
We have done something similar when diffing HTML, e.g. replacing the HTML with single unicodes. And then we run the diff and get several diff-segments (EQUALS, INS, DEL). What we have done, is then to scan those for tags, and split them into a new type.
So an insert like INS(something \xE000 else) would become three changes. E.g. INS(something ) INS_TAG(\xE000) INS( else). So the INS_TAG shouldn't be wrapped in <ins> when converting this back to HTML.
[1] http://en.wikipedia.org/robots.txt
In the future, we'd love to combine a mix of the Wikipedia API for real-time edit updates with a copy of the Wikipedia history database stored locally (all 2+ TB of it) to improve performance and decrease load on their servers.
:(
EDIT: Should be coping with the load nicely now.
If you're complaining that edits are recorded and displayed publicly, you're thirteen years late. That was always the model of any Wiki.